AWS open source newsletter, #183
A weekly round up of the latest open source news, projects, and events that every open source developer should know about.
- Best practices for scaling AWS CDK adoption within your organization is a super interesting read around how to maximise the benefits of infrastructure as code tools like AWS CDK, to empower your internal developers and increase deployment velocity whilst keeping things secure
- Securely connect to Amazon RDS for PostgreSQL with AWS Session Manager and IAM authentication looks at how to connect to RDS databases using IAM authentication and Session Manager with the remote port forwarding capability, and how this compares to the traditional database connection mechanisms you might be using [hands on]
- Connect to MySQL and MariaDB from Amazon Aurora and Amazon RDS for PostgreSQL using the mysql_fdw extension demonstrates the steps involved in setting up and using the mysql_fdw PostgreSQL extension on Amazon RDS for PostgreSQL, enabling you to interact with your MySQL database data [hands on]
- Supercharge your MySQL database on AWS provides essential reading for those of you using MySQL, or perhaps interested in using MySQL
- Introducing Amazon MQ cross-Region data replication for ActiveMQ brokers explains how cross-Region data replication works in Amazon MQ, how to set up a cross-Region replica broker, and how to test and promote the replica broker [hands on]
- Orchestrate Amazon EMR Serverless Spark jobs with Amazon MWAA, and data validation using Amazon Athena looks at how you can use Amazon MWAA, EMR Serverless, and Athena to build a complete end-to-end data processing pipeline [hands on]
- Building Reliable and Scalable Generative AI Infrastructure on AWS with Ray and Anyscale looks at some of the challenges implementing generative AI workloads introduces, and how open source tools like Ray, can help
- What’s new in AWS Amplify: Features to Build and Scale Fullstack Apps is a nice summary of the recent updates and new features that developers can use with AWS Amplify to help them build Fullstack applications
- Amazon EKS announces the general availability of upgrade insights, a new feature that surfaces insights about issues that may impact your ability to successfully upgrade a cluster to newer versions of Kubernetes. Upgrade insights are built upon best practices learned by EKS over the course of managing hundreds of thousands of Kubernetes clusters. You can use the EKS APIs and Console to check for upgrade readiness issues detected in your environment at any time against all future Kubernetes versions supported by EKS.
- Amazon Elastic Kubernetes Service (EKS) now surfaces cluster-related health issues in the EKS console and API, providing administrators enhanced visibility into the health of their clusters. Cluster health status information helps customers to quickly diagnose, troubleshoot, and remedy issues with their clusters, enabling them to run more up-to-date and secure application environments.
- Amazon EKS customers can now leverage EC2 security groups to secure applications in clusters using Internet Protocol version 6(IPv6) address space. Amazon EKS supports IPv6 enabling customers to scale containerised applications on Kubernetes beyond limits of private IPv4 address space. Kubernetes cluster administrators often have to operate in environments where IPv6 and IPv4 networks coexist. Until today, they could use network security rules that span pod to pod and pod to external Amazon Web Services service traffic defined in a single place with EC2 security groups, and applied to individual pods in IPv4 clusters. With this launch, customers can apply EC2 security groups for pods in both IPv4 and IPv6 clusters.
- Amazon EKS now supports simplified configuration of AWS Identity and Access Management (IAM) users and roles with Kubernetes clusters, through a new set of APIs that tightly integrate IAM identities with Kubernetes authentication and authorization controls. EKS already supports IAM identity authentication to Kubernetes clusters, removing the burden from cluster administrators of having to maintain and integrate a separate identity provider. This integration enables administrators to leverage IAM security features such as audit logging and multi-factor authentication. EKS access management controls introduced today simplify the process of mapping IAM to Kubernetes identities, by allowing administrators to fully define authorised IAM principals and their associated Kubernetes permissions directly through an EKS API during or after cluster creation. The IAM identity used to create a EKS cluster can have its Kubernetes permissions removed or scoped down to comply with security requirements, and control of a cluster can always be restored to an AWS account administrator. Other AWS services can use EKS access management controls to automatically obtain permissions to run applications on EKS clusters. EKS access management controls simplify the amount of work administrators need to do in order to create and manage clusters that are shared by multiple users and other AWS services. Dive deeper by reading this post, Amazon EKS Pod Identity: a new way for applications on EKS to obtain IAM credentials where George John, Ashok Srirama, and Hemanth AVS show you how you can use the Amazon EKS Pod Identity feature to securely grant IAM permissions to Kubernetes applications running in your EKS clusters.
- Amazon Aurora for PostgreSQL now supports h3-pg for geospatial indexing - Amazon Aurora PostgreSQL-Compatible Edition now supports the h3-pg extension, which provides an API to H3, an open-source hexagonal, hierarchical geospatial indexing system. With this extension, you can perform different kinds of spatial analysis over large datasets, including efficient indexing and lookups, modeling flow through a grid, and applying machine learning models over your geospatial data stored in Aurora PostgreSQL. The H3 library provides an invariant set of hexagonal map tiles over multiple layers of resolution. This allows the h3-pg extension to index your geospatial data so you can efficiently query data on your maps. For example, a retailer planning new outlets may want to create a heatmap visualisation using traffic, mobility, demographic, and other geospatial datasets to identify locations best suited for their customers. You can also use H3 and PostGIS together to perform different geospatial analyses. The h3-pg extension is available on Aurora PostgreSQL 15.5, 14.10, 13.13, 12.17 and higher.
- Amazon Aurora PostgreSQL now supports HypoPG extension for creating hypothetical indexes - Amazon Aurora PostgreSQL-Compatible Edition now supports the HypoPG extension for creating hypothetical indexes, which lets you test the performance impact of an index on query plans before you build it. Indexes are a way to accelerate queries in PostgreSQL, however, building indexes in a production system requires additional storage and may not necessarily improve the performance of slow-running queries. The HypoPG extension lets you test the impact of adding an index without having to build it, and lets you determine if you should build an index before using CPU and storage resources. HypoPG helps to provide you insight into whether the PostgreSQL query planner will choose to use an index and any performance benefits that result from the use of the index. The HypoPG extension is available on Aurora PostgreSQL 15.5, 14.10, 13.13, 12.17 and higher.
- Amazon Aurora for PostgreSQL now supports delegating extension management to lower privileged users- Amazon Aurora PostgreSQL-Compatible Edition now supports the rdsextension database role which provides a cluster administrator the ability to delegate create, alter, or drop extension operations to lower privileged users. The cluster administrator can use the new rds.alloweddelegatedextensions DB parameter to limit which extensions a member of the rdsextension role can manage. To get started, take a look at our Working with extensions and foreign data wrappers page. Delegation of extension management using the rds_extension role is available on Aurora PostgreSQL 15.5, 14.10, 13.13, 12.17
- Amazon Aurora supports PostgreSQL 15.5, 14.10, 13.13, 12.17 - Amazon Aurora PostgreSQL-Compatible Edition now supports PostgreSQL versions 15.5, 14.10, 13.13, 12.17. These releases contain product improvements and bug fixes made by the PostgreSQL community, along with Aurora-specific improvements. This release also contains new features and improvements such as group role authentication support using AWS Directory Service for Microsoft Active Directory with the new pgadmapping extension, Amazon Bedrock integration for Generative AI , and Babelfish for Aurora PostgreSQL version 3.4. As a reminder, if you are running any version of Amazon Aurora. PostgreSQL 11, you must upgrade to a newer major version by February 29, 2024.
- AWS CloudShell has migrated to Amazon Linux 2023 (AL2023) - AWS CloudShell, previously based on Amazon Linux 2 (AL2), has migrated to Amazon Linux 2023 (AL2023). You can continue to use your existing CloudShell environment with all its pre-installed tools.
- AWS Cloud9 now supports Amazon Linux 2023 - You can now launch AWS Cloud9 environments with Amazon Linux 2023 (AL2023). AL2023 is the next generation of Amazon Linux, a Linux server operating system from Amazon Web Services (AWS). AL2023 provides a secure, stable, and high performance execution environment to develop and run cloud and enterprise applications.
- Scaling strategies to help you meet your scaling and cost requirements, ranging from strict all-or-nothing, that avoid launching idle instances, to best-effort, which prioritises throughput scaling over avoiding idle instances.
- Support for mounting user-defined file systems in AWS to the /home directory of the head node, helping you simplify cluster storage management.
- Custom Munge key to help you simplify integration of additional cluster resources that require the Scheduler Munge key.
Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.