Database Server Options
Each Self Managed Commerce environment requires a dedicated database. This topic guides you through the database configuration options for the Self Managed Commerce database.
Introduction
CloudOps for Kubernetes includes Jenkins jobs that enable you to provision database servers and configure Self Managed Commerce stacks to use those servers, or to simply configure Self Managed Commerce stacks to use a database server that you already have. CloudOps for Kuberneets release 3.6 introduced features to make it easier to switch existing Commerce stacks to a different database type or server.
The following database options are supported.
- Provision and use a containerized MySQL database service.
- Provision and use an Amazon PostgresSQL RDS service.
- Provision and use an Amazon Aurora RDS MySQL service.
- Reuse an existing MySQL or PostgresSQL service.
- Use a MySQL or PostgresSQL service that you configure outside of CloudOps for Kubernetes.
note
MySQL 5.7 support is deprecated as of Cloudops for Kubernetes release 3.2, and will be removed in a future release. Oracle ended standard support for MySQL 5.7 in October 2023, and AWS will end standard support for MySQL 5.7 Aurora instances in October 2024.
When support for MySQL 5.7 is removed from CloudOps for Kubernetes, support for both containerized MySQL 5.7 and for Amazon RDS Aurora MySQL 5.7 will be removed.
CloudOps for Kubernetes users who are on MySQL 5.7 are encouraged to migrate to MySQL 8.4 or PostgreSQL 16. The Self-Managed Commerce application development team has shared related information at Migrating off of MySQL 5.7.
Creating Containerized MySQL Server
To create a containerized MySQL server, run the create-or-delete-mysql-container job.
note
Containerized databases are not suitable for Self Managed Commerce environments that require a resilient database service. The database and data are stored on an Amazon Elastic Block Store (EBS) volume with no redundency. This option does not support using Amazon RDS tools to perform database backup and restore.
The below types of containerized database deployments can be provisioned with the create-or-delete-mysql-container Jenkins job.
MySQL 8.4 containerized Kubernetes deployments.
important
MySQL 8.4 container support requires the following.
- CloudOps for Kubernetes release
3.6.xor higher, and - Elastic Path docker release
4.8.xor higher. - Elastic Path Self-Managed Commerce release
8.4.xor higher.
- CloudOps for Kubernetes release
MySQL 8.0 containerized Kubernetes deployments.
important
MySQL 8.0 container support requires the following.
- CloudOps for Kubernetes release
3.1.xor higher, and - Elastic Path docker release
4.5.xor higher. - Elastic Path Self-Managed Commerce release
8.3.xor higher.
- CloudOps for Kubernetes release
MySQL 5.7 containerized Kubernetes deployments (deprecated).
The Kubernetes Service created for the MySQL server is of ClusterIP type and there is no Ingress for it. By default, the MySQL container is only accessible from other containers in the same namespace.
The database name, username and password are randomly generated. You indicate which Self Managed Commerce stack will use the database by specifying the corresponding value in the kubernetesNickname Jenkins parameter when you run the Jenkins job. The necessary database information is stored in a Kubernetes Secret, in the Kubernetes namespace specified by kubernetesNickname.
See the job in your Jenkins server for more detailed usage information.
Creating a Cloud Database Server
To create an Amazon managed database server or cluster, run the create-and-manage-database-server job. The Amazon managed database services created by this job are highly available and multi-AZ, support backup and restore using RDS tooling, and are suitable for staging and production environments.
The below types of Amazon managed database services can be provisioned using the create-and-manage-database-server Jenkins job.
- Amazon RDS Aurora MySQL 8.0 cluster, with default and standby nodes in separate Availability Zones
- Amazon RDS Aurora MySQL 5.7 cluster, with default and standby nodes in separate Availability Zones (deprecated).
- Amazon PostgresSQL RDS 14.x instance, with a standby instance in a separate Availability Zone.
note
PostgresSQL and MySQL 8.0 are only supported with Self Managed Commerce version 8.3 and later.
The database instances are configured to use the private subnets that were created along with your EKS cluster. By default, each node is db.r5.xlarge. Each node has four vCPUs and 32 GB RAM. The server is configured for geo-redundant daily backups that are preserved for 14 days.
The database name, username and password are randomly generated. You indicate which Self Managed Commerce stack will use the database by specifying the corresponding value in the kubernetesNickname Jenkins parameter when you run the Jenkins job. The necessary database information is stored in a Kubernetes Secret, in the Kubernetes namespace specified by kubernetesNickname.
The job sets the server settings for Aurora MySQL RDS clusters:
character_set_server=UTF8MB4
tx_isolation=READ-COMMITTED
The job sets the server settings for PostgresSQL RDS servers:
default_transaction_isolation=read committed
See the job in your Jenkins server for more detailed usage information.
Using an Existing Database Server
To configure a Self Managed Commerce stack to use a database server that you already have, run the use-existing-database-server Jenkins job . The database server can be one that you created using the provided Jenkins jobs, or it could be one that you created outside of CloudOps for Kubernetes. The following database types are supported.
- MySQL 8.4
- MySQL 8.0
- PostgresSQL 14.x
- MySQL 5.7 (deprecated)
note
MySQL 8.4 is only supported with Self Managed Commerce version 8.4 and later.
note
PostgresSQL and MySQL 8.0 are only supported with Self Managed Commerce version 8.3 and later.
tip
Use this approach if you want to host multiple Self Managed Commerce databases on one database server or cluster.
tip
Use this approach if you want to use a database configuration that cannot be created using the Jenkins jobs provided with CloudOps for Kubernetes.
When using this option you are responsible to ensure that network connectivity is sufficient for the Self Managed Commerce services to connect to the database server.
You indicate which Self Managed Commerce stack will use the database by specifying the corresponding value in the kubernetesNickname Jenkins parameter when you run the Jenkins job. The necessary database information is stored in a Kubernetes Secret, in the Kubernetes namespace specified by kubernetesNickname.
See the job in your Jenkins server for more detailed usage information.
The Kubernetes Database Secret
The Self Managed Commerce services of a given Commerce stack use configuration information stored in a Kubernetes secret to identify which database and database server to use. Each Self Managed Commerce stack has its own database secret, in the stack's Kubernetes namespace. When you run one of the database related Jenkins jobs, that job will create (or update) the Kubernetes secret in the namespace specified by Jenkins parameter kubernetesNickname. For more information about the database secret, see Database Connectivity and Credentials.
Switching Databases
Switching databases in an existing Self-Managed Commerce environment is complicated by the fact that each of the Terrform modules that configure different database types will try to create and manage the Kubernetes database secret. If one job and Terraform module created the secret, then trying to switch the database type by running a different job and module will fail when it tries to also create and manage the secret.
CloudOps for Kubernetes release 3.6 introduced features to make it possble to switch existing Commerce stacks to a different database type or server, by
- Allowing the
use-existing-database-serverJenkins job to take over management of an existing Kubernetes database secret, and - Allowing the other jobs to stop trying to manage an existing Kubernetes database secret.
Procedure
Complete the steps below to switch database types for an existing Self-Managed Commerce environment.
Back up the existing database secret
First, back up the existing database secret in case you need to restore it. For more information about the database secret, including how to find the secet name and namespace, see Database Connectivity and Credentials.
kubectl -n <kubernetes-namespace> get secret <secret-name> -o yaml > <filename>.yaml
Create the new database instance or cluster
If you have an existing supported MySQL or PostgreSQL database instance or cluster, then you can use it. You can create and new database instance or cluster using one of the Jenkins jobs described above. Alternatively, you an create and configure the instance or cluster using external tools, such as the AWS console.
important
If you create the new database cluster using one of the Jenkins jobs described above, be sure to select parameter doNotManageDatabaseSecret. That is, check it, or set it to true.
Verify that the database secret has not been changed yet.
Migrate data to the new database
For production environments, and possibly some lower environments, you will need to ensure that data from the old database is copied and present on the new database before cutting over to the new database server. This may be a complex procedure, and will require planning and testing in lower environments before implementing in production. Work with your development team and database administrators to plan and implement this effort.
tip
If you are planning to switch from MySQL to PostgreSQL, please see MySQL to PostgreSQL Migration in the Self-Managed Commerce documentation.
Stop the Commerce applications
Shut down the Commerce application services by scaling each to zero replicas. The application outage will begin.
Update the secret
When you are ready to cut over to the new database server, run the use-existing-database-server Jenkins job to update the secret to point at the new database server.
important
When you run use-existing-database-server, be sure to select parameter adoptExistingDatabaseSecret. That is, check it, or set it to true.
Verify that the database secret is present and updated.
Optional: Run data-population
You may want to run the data-population job to create or update your database, if you did not previously migrate data or if updates are required. Seek input from your development team and database administrators.
Restart the Commerce applications
Restart the Commerce applications by scaling them back up to the desired number of replicas. As they start they will pick up the updated values in the Kubernetes database secret, connect to the new database, and refresh their state.
Ensure that the applications start successfully.
Back up the database secret
Back up the database secret in case you need to restore it. For more information about the database secret, including how to find the secet name and namespace, see Database Connectivity and Credentials.
kubectl -n <kubernetes-namespace> get secret <secret-name> -o yaml > <filename>.yaml
Decommission the old database server
When ready and if needed, decommission the old database server.
important
If you decommission or delete the old database cluster using one of the Jenkins jobs described above, be sure to select parameter doNotManageDatabaseSecret. That is, check it, or set it to true.
Verify that the database secret is present and correct.