Update EKS Node Groups · CloudOps for Kubernetes

The compute infrastructure of the Amazon EKS Kubernetes cluster is an EKS node group. The individual nodes that make up the node group are Amazon EC2 instances. You may occasionally want to update the EC2 instances in the EKS node group for the following reasons:

To apply operating system patches to the EC2 instances.
To change the type of the EC2 instances.

note

Most new versions of CloudOps for Kubernetes include upgrades to new Kubernetes versions. When you apply a CloudOps for Kubernetes update, the upgrade process will also update the EKS cluster’s node group. For more information about upgrading Cloudops for Kubernetes, see Upgrading CloudOps for Kubernetes.

Overview of Updating the EKS Nodegroup

The following occurs when you perform the process to rebuild the node groups:

New replacement node groups are created, consisting of new EC2 instances. Both the old node groups and new node groups will exist until the next steps are completed.
The Kubernetes pods for all deployments are rescheduled to run on the new node group. Pods are stopped on the old nodes and new pods start on the new nodes.
When deployments are no longer running on the old node groups, the old node groups and EC2 instance are deleted.

warning

There will likely be a Self Managed Commerce service outage while the EKS node groups are rebuilt. A Self Managed Commerce service outage duration of up to 15 minutes has been observed in our testing.

The Jenkins server will be unavailable for a period of time while the EKS node groups are rebuilt. Any Jenkins jobs running during this time will fail.

Procedure

note

Perform the following steps on the operations workstation, ideally using the workstation that you used for the initial setup. For more information, see the Operations Workstation documentation.

Complete the following steps to rebuild the node groups.

Confirm that you have checked out the correct branch of the cloud-ops-kubernetes Git project on your operations workstation.
Obtain the docker-compose.override.yml file that you saved when you setup the cluster or last updated the configuration, and copy it into the root of the cloud-ops-kubernetes Git project.
Set the parameter TF_VAR_bootstrap_mode in the docker-compose.override.yml file to setup.
[Optional] If you want to change your EC2 instance type, set parameter aws_eks_instance_type in the docker-compose.override.yml file accordingly.
Set the parameter TF_VAR_rebuild_nodegroups in the docker-compose.override.yml file to true.
Run the following Docker Compose command to build the Docker image and rebuild the EKS node groups:
```
docker-compose up --build
```

The changes will be applied to the cluster configuration. It may take a long time for the procedure to complete. The exact amount of time varies, but it can take 20-60 minutes.

For general information about updating your cluster configuration, see the Updating Cluster Configuration documentation.

tip

We recommend setting TF_VAR_rebuild_nodegroups to false in the docker-compose.override.yml file after the procedure is complete. This is to avoid accidentally rebuilding the node group if you make other cluster configuration changes in the future.

Save Important Files

Save the docker-compose.override.yml file and any dependencies, such as TLS keys, in a safe place.

warning

Do not commit these files into Git. The docker-compose.override.yml file contains secrets and should be stored somewhere securely.

important

You will need the docker-compose.override.yml file and any dependencies again to perform the following:

Update the cluster
Show the current state of the cluster
Create local Terraform configuration
Clean up the cluster