New logging endpoints: Kafka and Kinesis
Senior Manager, Product Management, Fastly
We’re proud of our real-time logging capabilities. They provide complete visibility to help you make split-second decisions when needed and can help drive data-driven business decisions about your apps and services. We’re even prouder of what our partners and customers do with them though — like our recent post about how Adobe’s Project Helix is using them.
Today, we’re excited to announce two new logging endpoints that we’re adding to our ever-growing list.
Apache’s Kafka, a distributed streaming platform typically used for real-time data feeds, log aggregation, and stream processing, is now in general availability. We also added three new capabilities based on customer requests:
Log batch size configuration allows you to control the maximum bytes of a log batch.
Key-value pair parsing makes it easier for you to scale architecture by using the key to determine which partition the message gets appended.
Simple Authentication and Security Layer is a framework for authentication and data security in internet protocols that unlocks the ability to use our Kafka endpoint to send logs to Microsoft Azure Event Hubs.
Also, Amazon’s Kinesis Data Streams endpoint is now in limited availability, enabling you to capture gigabytes of data per second. You can then build Kinesis applications that continuously process this data, generate metrics, and power live dashboards. To enable this limited availability endpoint, contact support@fastly.com.
We’ll continue to add new endpoints to serve your needs in building the modern trustworthy internet. Keep us posted on what works well or integrations you’d like to see.
