Monitoring
CloudOps for Kubernetes can deploy Prometheus. Prometheus is a monitoring tool and a time-series database for various metrics.
Enable Prometheus
To enable Prometheus on your cluster:
Download the Docker Compose configuration file for your cluster.
tip
If you followed the recommended process for bootstrapping your cluster, the file name is
docker-compose.override.yml
.Add the following settings to your Docker Compose configuration file.
# enables the Prometheus Helm chart TF_VAR_enable_prometheus: "true"
Accessing Prometheus
After you deploy the Prometheus server, you can access it by completing the following steps:
Run the following command to get the pod name of the Prometheus server:
export POD_NAME=$(kubectl get pods \ --namespace prometheus \ -l "app=prometheus,component=server" \ -o jsonpath="{.items[0].metadata.name}")
Run the following command to port-forward the Prometheus server to port 9090 on your local machine:
kubectl --namespace prometheus port-forward $POD_NAME 9090
Go to
localhost:9090
in your browser.
Example Queries
The following section includes a list of example Prometheus queries that are available to help you.
Total CPU cores used by a type of container
This query allows you to see the total number of CPU cores used by all instances of a container with the same name. For example, you would use the following query for cortex
:
sum (
rate (
container_cpu_usage_seconds_total{container_name="cortex"}[5m]
)
)
Total memory used by a type of container
This query allows you to see the total amount of memory in bytes used by all instances of a container with the same name. For example, you would use the following query for cortex
:
sum (
container_memory_usage_bytes{container_name="cortex"}
)
Number of replicas in a deployment
This query allows you to see the number of replicas for a given Kubernetes Deployment. For example, you would use the following query for ep-cortex-deployment
:
kube_deployment_status_replicas_available{deployment="ep-cortex-deployment"}
Percentage of requested CPU cores in use
This query allows you to see the percentage of requested CPU cores that are actively being used by all instances of a container with the same name. For example, you would use the following query for cortex
:
sum (
rate (
container_cpu_usage_seconds_total{container_name="cortex"}[5m]
)
)
/
sum (
kube_pod_container_resource_requests_cpu_cores{container="cortex"}
)
*
100
For example, if in a moment of time:
- There is one pod with one container named
cortex
- It requests 2 CPU cores
- It is actively using 1 core
- Prometheus would show 50% utilization at this moment
In another example, in another moment of time:
- There are 2 pods with one container each named
cortex
- They each request 2 CPU cores
- Both are actively using 0.1 of a core
- Prometheus would show 5% utilization at this moment
Percentage of requested memory in use
This query allows you to see the percentage of requested memory that is actively being used by all instances of a container with the same name. For example, you would use the following query for cortex
:
sum (
container_memory_usage_bytes{container_name="cortex"}
)
/
sum (
kube_pod_container_resource_requests_memory_bytes{container="cortex"}
)
*
100