Select your cookie preferences

We use essential cookies and similar tools that are necessary to provide our site and services. We use performance cookies to collect anonymous statistics, so we can understand how customers use our site and make improvements. Essential cookies cannot be deactivated, but you can choose “Customize” or “Decline” to decline performance cookies.

If you agree, AWS and approved third parties will also use cookies to provide useful site features, remember your preferences, and display relevant content, including relevant advertising. To accept or decline all non-essential cookies, choose “Accept” or “Decline.” To make more detailed choices, choose “Customize.”

AWS Logo
Menu

Data lake with dms, glue & hudi | S01 E11 | Lets Talk About Data Show

In this show we focus on the AWS Analytics services like Glue , Hudi and how it can be used to build data lake. We focus on the ingestion sources and staging the data in the data lake. We have a detailed demo in the second half of the show focusing on building data lake and using these services.

awsdatatwitchlets-talk-about-data
Tony Mullen
Amazon Employee
Published Oct 23, 2023
Last Modified Apr 22, 2024
Comments
Image not found
In this episode of Lets Talk About Data we focus on the AWS Analytics services like Glue , Hudi and how it can be used to build data lake. We focus on the ingestion sources and staging the data in the data lake. We have a detailed demo in the second half of the show focusing on building data lake and using these services.
Check out the recording here:
To view this Twitch stream, please accept cookies.
**Roni Vered *, Senior Database Solutions Architect @ AWS

Guests

Kate Gawron, Leader in Cloud Databases
Lydia Ray, Data, Analytics, AIML @AWS

Links from today's episode

  • How get started with Apache Hudi? More info about Hudi. https://hudi.apache.org/docs/overview
  • How to get started with AWS Glue with Hudi?
  • AWS Doc - Hudi Script - https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-format-hudi.html
  • AWS Doc - Hudi in Glue Studio (low code) - https://docs.aws.amazon.com/glue/latest/ug/gs-data-lake-formats-hudi.html
  • Latest AWS blog about Glue with Hudi:
    • https://aws.amazon.com/blogs/big-data/automate-schema-evolution-at-scale-with-apache-hudi-in-aws-glue/
    • https://aws.amazon.com/blogs/big-data/how-infomedia-built-a-serverless-data-pipeline-with-change-data-capture-using-aws-glue-and-apache-hudi/
  • OTF Comparison (Hudi, Iceberg, Delta Lake) - https://aws.amazon.com/blogs/big-data/choosing-an-open-table-format-for-your-transactional-data-lake-on-aws/

Reach out to the hosts and guests:

  • Roni Vered: (https://www.linkedin.com/in/ronivered/)
  • Kate Gawron: (https://www.linkedin.com/in/katehollow/)
  • Lydia Ray: (https://www.linkedin.com/in/lydia-ray/)

Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.

Comments

Comments

Log in to comment