Scale Flink Apps using Amazon Managed Service for Apache Flink | S02EP28 | Lets Talk About Data Show
Learn how Amazon Managed Service for Apache Flink simplifies scaling with seamless infrastructure management and robust performance monitoring.
Lydia Ray
Amazon Employee
Published Jul 25, 2024
In this episode, Lydia, Umesh, and Anand discuss the benefits and features of running Apache Flink on the AWS Managed Service for Apache Flink. They cover the basics of Apache Flink, a distributed real-time stream processing framework, and explain how the managed service simplifies the deployment and management of Flink applications. Umesh and Anand demonstrate two approaches to building Flink applications - using the SQL API with a Zeppelin-based notebook, and the Data Stream API with Java code. They highlight the monitoring and scaling capabilities of the managed service, as well as the integration with other AWS services like Kinesis and Redis. The discussion provides a comprehensive overview of how to get started with Apache Flink on AWS and the advantages of using the managed service.
Some of the key highlights are:
- Overview of Apache Flink and its key features, including stateful stream processing and exactly-once processing guarantees.
- Explanation of the AWS Managed Service for Apache Flink and how it simplifies deployment and management of Flink applications.
- Demo of using the SQL API and Zeppelin notebook to build a streaming application that processes events and updates a leaderboard in Redis.
- Demo of using the Data Stream API and Java code to build a basic Flink application that reads from and writes to Kinesis Data Streams.
- Exploration of monitoring and scaling capabilities, including CPU utilization, checkpointing, and parallelism settings.
- Discussion of best practices for building Flink applications, such as fault tolerance, versioning, and state management.
Loading...
Hosts of the show 🎤
Lydia Ray - Sr Analytics Solutions Architect @ AWS
Guests 🎤
Umesh Chaudhari - Streaming Specialist Solution Architect @ AWS
Anand Shah - Sr Big Data & Analytics Architect @ AWS
Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.