Managing CloudOps for Kubernetes
CloudOps for Kubernetes uses a Docker container to manage a Kubernetes cluster and supporting services needed to build and deploy Elastic Path Commerce and Account Management. For information about the Kubernetes cluster infrastructure, see the CloudOps for Kubernetes Cluster Infrastructure section.
The Docker container that manages CloudOps for Kubernetes can do the following:
- Initially bootstrap the cluster
- Update the cluster
- Show the current state of the cluster
- Create Terraform configuration files so you can work with the Terraform state locally
- Clean up the cluster
Initially Bootstrap the Cluster
note
Ensure that all of the required preparations are complete before starting the initial bootstrap process.
To initialize the CloudOps for Kubernetes cluster for the first time:
Clone the CloudOps for Kubernetes code.
Copy the Docker Compose configuration file
docker-compose.yml
to a new file nameddocker-compose.override.yml
.Set the Docker Compose parameter
TF_VAR_bootstrap_mode
in the filedocker-compose.override.yml
tosetup
.Read through the
docker-compose.override.yml
file and fill out the rest of the configuration.tip
For more information about the parameters, see the comments inside the file.
Reduce the configuration you need to manage by deleting any parameters in the override file with default values that you do not change.
Reduce the size of the override file by deleting the comments. You can always reference the original file.
Run the Docker Compose command
build
to build the Docker image:docker-compose build
Run the Docker Compose command
up
to initialize the CloudOps for Kubernetes cluster:docker-compose up
Save the
docker-compose.override.yml
file and any dependencies, such as TLS keys, in a safe place.important
You will need the
docker-compose.override.yml
file and any dependencies again to perform the following:- Update the cluster
- Show the current state of the cluster
- Create local Terraform configuration
- Clean up the cluster
warning
Do not commit the
docker-compose.override.yml
file and any dependencies into source code. It will contain secrets and should be stored somewhere securely.Proceed to the Post Bootstrap steps.
Update the Cluster
important
Only update CloudOps for Kubernetes one minor version at a time.
To consume updates, bug fixes, or customizations, you might need to update the CloudOps for Kubernetes cluster.
Before you upgrade to CloudOps for Kubernetes 2.4 or higher, you must ensure that you upgrade incrementally by version. For example, if you upgrade from version 2.2, you must upgrade to version 2.3 before you upgrade to 2.4. Complete the upgrade to the CloudOps for Kubernetes 2.3 code level by following the Update CloudOps for Kubernetes from version 2.2.x before you begin the upgrade to CloudOps for Kubernetes 2.4 or higher.
If you are updating from CloudOps for Kubernetes 2.3.0, see Update from 2.3.0.
If you are updating from CloudOps for Kubernetes 2.5.0, see Update from version 2.5.0.
For most updates, including CloudOps for Kubernetes versions that were not covered above, follow the steps described to update the CloudOps for Kubernetes cluster:
Checkout the new version of CloudOps for Kubernetes that can be upgraded to from your current version.
Copy the
docker-compose.override.yml
file that you saved when initially bootstraping the cluster.Set the Docker Compose parameter
TF_VAR_bootstrap_mode
in the filedocker-compose.override.yml
tosetup
.Read through the
docker-compose.yml
file in the new version and update your configuration as required.tip
Compare the old
docker-compose.yml
with the new version to see if any default values have changed, if any parameters have been deleted, or if new parameters have been added.warning
Some Docker Compose configuration parameters cannot be updated. Ensure that you leave these parameters unchanged. Changing these parameters may have unintended consequences. For more information on which parameters cannot be updated, see the documentation in the comments of the
docker-compose.yml
file.Run the Docker Compose command
build
to build the Docker image:docker-compose build
Run the Docker Compose command
up
to update the CloudOps for Kubernetes cluster:docker-compose up
Save the updated
docker-compose.override.yml
file and any dependencies, such as TLS keys, in a safe place.important
You will need the
docker-compose.override.yml
file and any dependencies again to perform the following:- Update the cluster
- Show the current state of the cluster
- Create local Terraform configuration
- Clean up the cluster
warning
Do not commit the
docker-compose.override.yml
file and any dependencies into source code. It will contain secrets and should be stored somewhere securely.
Show the Current State of the Cluster
To show the current state of the CloudOps for Kubernetes cluster:
Checkout the version of CloudOps for Kubernetes currently in use.
Copy the
docker-compose.override.yml
file that you saved when initially bootstrapping or updating the cluster.Set the Docker Compose parameter
TF_VAR_bootstrap_mode
in thedocker-compose.override.yml
file toshow
.Run the Docker Compose command
build
to build the Docker image:docker-compose build
Run the Docker Compose command
up
to show the current state of the cluster:docker-compose up
Create Local Terraform Configuration
note
You may need to install dependencies before manipulating the CloudOps for Kubernetes Terraform state locally. You will also need to update the current context of Kubectl to the Kubernetes cluster created by the CloudOps for Kubernetes Docker image.
To create Terraform backend configuration files locally so you can locally manipulate the CloudOps for Kubernetes Terraform state:
Checkout the version of CloudOps for Kubernetes currently in use.
Copy the
docker-compose.override.yml
file that you saved when initially bootstrapping or updating the cluster.Set the Docker Compose parameter
TF_VAR_bootstrap_mode
in thedocker-compose.override.yml
file tocreate-terraform-files
.Update the volumes section of the Docker Compose configuration with the path to the CloudOps for Kubernetes code.
Run the Docker Compose command
build
to build the Docker image:docker-compose build
Run the Docker Compose command
up
to create the Terraform backend configuration files locally:docker-compose up
In the CloudOps for Kubernetes code, go to the folder
bootstrap/terraform/
:cd bootstrap/terraform
Initialize Terraform:
terraform init
note
If you have created Terraform configuration files previously for a different cluster, you may need to clean up any state left behind.
To manipulate the CloudOps for Kubernetes cluster select the Terraform workspace bootstrap:
terraform workspace select bootstrap
Test your Terraform configuration by refreshing your Terraform state:
terraform refresh -var-file=bootstrap.tfvars
Clean Up the Cluster
warning
This will destroy the Kubernetes cluster, Docker repositories and their images, DNS zones and other important infrastructure.
To clean up the CloudOps for Kubernetes cluster:
Checkout the version of CloudOps for Kubernetes currently in use.
Copy the
docker-compose.override.yml
file that you saved when initially bootstrapping or updating the cluster.Set the Docker Compose parameter
TF_VAR_bootstrap_mode
in thedocker-compose.override.yml
file tocleanup
.Run the Docker Compose command
build
to build the Docker image:docker-compose build
Run the Docker Compose command
up
to clean up the cluster:docker-compose up
Force Clean Up the Cluster
warning
The clean up action destroys the following assets:
- Kubernetes cluster
- Docker repositories
- Docker images
- DNS zones
- Other important infrastructure
For Azure, this mode will destroy the node resource group and completely empty the resource group.
For AWS, this mode will attempt to destroy all existing Terraform workspaces. This can only be run with an AWS account that does not have any existing CloudOps for AWS infrastructure. This mode will then proceed to:
- Empty all Simple Cloud Storage Service (S3) buckets in your account that follow the same bucket name
- Empty all Dynamo Database (DynamoDB) tables in your account that follow the same table name
- Delete non-attached Elastic Block Store (EBS) volumes in account
- Delete all Elastic Container Registry (ECR) repositories in your account
- Delete all Domain Name Service (DNS) hosted zones in your account that match the domain name used in the
TF_VAR_domain variable
in yourdocker-compose.override.yaml
file - Delete all Amazon Certificate Manager (ACM) certificates in your account that match the domain name that you use in the
TF_VAR_domain variable
in yourdocker-compose.override.yaml
file - Delete all Ambassador Load Balancers(ALB) and Ambassador specific Target Groups and Security Groups
- Delete the Elastic File System (EFS) volume and related Security Groups for the efs-provisioner.
This mode is especially useful when the cleanup bootstrap mode fails and there are Terraform state locks existing, and so on.
To clean up the CloudOps for Kubernetes cluster:
Checkout the version of CloudOps for Kubernetes currently in use.
Copy the
docker-compose.override.yml
file that you saved when initially bootstrapping or updating the cluster.Set the Docker Compose parameter
TF_VAR_bootstrap_mode
in thedocker-compose.override.yml
file toforce-cleanup
.Set the Docker Compose parameter
TF_VAR_approve_force_cleanup
in thedocker-compose.override.yml
file totrue
.Run the Docker Compose command
up --build
to build the Docker image and force-cleanup the existing cluster resources:docker-compose up --build
CloudOps for Kubernetes Cluster Infrastructure
The CloudOps for Kubernetes platform manages the cloud and Kubernetes infrastructure listed in this section.
Azure
- Azure Kubernetes Service (AKS) cluster
- Azure Container Registry (ACR) instance
- Azure DNS Zone
- DNS A record pointing to the IP address of the AKS cluster load-balancer
note
AKS manages the deployment of the Kubernetes Cluster Autoscaler and the Kubernetes dashboard.
AWS
- Elastic Kubernetes Service (EKS) cluster
- Elastic Container Registry (ECR) repositories
- Route53 Zone
- DNS CNAME record pointing to the domain name of the EKS cluster network load-balancer
The following services are created in the EKS cluster:
- Cluster Autoscaler
- Overprovisioning
- Kubernetes dashboard
- (Optional) Fluentd CloudWatch agent
Common
The following services are deployed to Kubernetes clusters in all clouds:
- Ambassador API Gateway
- Jenkins
- Nexus
- Tiller
- (Optional) New Relic Infrastructure agent
For more information about the New Relic Infrastructure agent, see: