Deploy Cortex on EKS
Horizontally scalable, highly available, multi-tenant, long term storage for Prometheus.
Published Feb 8, 2024
Cortex is mainly used to store Prometheus metrics for a longer time period, and by utilizing the multi tenancy feature of Cortex, multiple Prometheus instances can be configured to send metrics to Cortex side with unique tenant names, so that Cortex will store those metrics separately in the main storage. Grafana can be used to visualize the metrics by querying the data directly from Cortex API endpoint. Cortex supports PromQL queries, so that Prometheus type data sources can be configured in Grafana.
More detail on Cortex Architecture is explained in Cortex Documentation. Refer the service section of the document to get more understanding about the components/services of Cortex. If you are planning to deploy Cortex in Production, please read the tips from the Cortex developers also.
In this article, I’m going to explain how to deploy Cortex v1.9 using helm chart with a block storage engine, I’m creating a S3 bucket as the block storage using Terraform codes. As a pre-requisite Cortex require a Key-value store to store the hash ring, Consul will be used for this requirement.
The consul-values.yaml can be found in the GitHub repository. Replica count and resource limits can be changed as per the requirement. Persistent volume capacity should increased if the Cortex rings have considerable amount of data.
I’m using an AWS S3 bucket for this, because I’m deploying Cortex in an EKS cluster. I’m going to create S3 bucket and provide the cortex service account with the read-write access to it using an IAM role. I’m going to use terraform modules to create these necessary components. I have already authenticated to AWS account, so that credentials are not mentioned in the terraform provider.
If there are no errors, terraform should create above 5 components. Mainly, S3 bucket and IAM role should be created and configured properly. Terraform state file is stored locally here, but it is recommended to store it in a centralized location, like in a S3 bucket.
Cortex v1.9 is installed using the v0.6.0 Cortex helm chart. That is the reason why the version is defined in the helm install command. I’ll write on how to upgrade/install latest version of Cortex in a future article. This cortex-values.yaml can be found in the GitHub repository. I have explained most of the important/custom configurations in the values file to give you a clear understanding. There are more parameters in the default values file which you can check in the Cortex-helm-chart GitHub repository.
