Skip to main content
Version: 3.6.x

Release Notes

3.6.3​

Bug Fixes​

  • CLOUD-3791: Updated the recommended Amazon RDS for PostgreSQL engine versions used by the create-and-manage-database-server Jenkins job. The postgres-rds option now uses PostgreSQL 14.23, and postgres-rds-16 now uses PostgreSQL 16.10. The job also fails fast with a clear validation error when a requested RDS engine version is unavailable.
  • CLOUD-3753: Ensure clean ActiveMQ broker shutdown during pod termination and rollouts, even with ActiveMQ containers that don't properly pass signals to the broker Java process. Unclean shutdowns could log "Recovering pageFile free list due to prior unclean shutdown" and risk KahaDB corruption. ActiveMQ deployments now always find the broker process and send it SIGTERM via a Kubernetes preStop hook. The pod termination grace period is also increased from the Kubernetes default of 30s to 180s so the broker can checkpoint and flush before SIGKILL. After upgrading, redeploy existing ActiveMQ containers (for example, with the create-or-delete-activemq-containerjob) so they pick up the fix. An image rebuild is not required.
  • CLOUD-3821: Fixed an issue where ActiveMQ deployments failed to become Ready when CloudWatch Application Signals Java auto-instrumentation was enabled via the instrumentation.opentelemetry.io/inject-java: "true" pod annotation. ActiveMQ liveness and readiness probe commands now unset JAVA_TOOL_OPTIONS so the health-check subprocess does not load the OpenTelemetry javaagent and probes complete within the configured timeout.
  • CLOUD-3827: Updated DaemonSets modsecurity-spoa, HAProxy Ingress and CloudWatch logging to use high pod priority classes, for elevated scheduling priority. This is to resolve intermittent issues where a cluster node may not have all DaemonSet pods running.
  • CLOUD-3841: Fixed an issue where ActiveMQ containers failed to start because JDK_JAVA_OPTIONS was set to the literal string "null". This happened when ops-spec.json (from Self-Managed Commerce or the CloudOps for Kubernetes fall-back file) lacked .activemq.jvmRuntimeArguments; eplib.parseOpsSpec passed jq's literal null into the ActiveMQ image build. Parsing now treats a missing or null value as an empty string, and the fall-back ops-spec.json defines .activemq.jvmRuntimeArguments. After upgrading, rebuild ActiveMQ images and redeploy existing ActiveMQ containers so they pick up the fix. See ops-spec.json.

New Features​

  • CLOUD-3843: Exposed the ActiveMQ container memory and CPU values as user-configurable Terraform variables, and updated the default memory requests and limits to give the ActiveMQ process more native memory. See Configure ActiveMQ Container.

3.6.2​

Bug Fixes​

  • CLOUD-3776: Specified all Jenkins plugins to be installed, including the specific versions, to ensure that compatible plugins are always selected and installed.
  • SUP-6337: Updated the Jenkins controller Java memory configuration so the maximum heap is set at 60% of available memory instead of 87.5%. This makes more memory available to non-heap components to avoid Kubernetes OOMKilled issues.

3.6.1​

Bug Fixes​

  • CLOUD-3206: Updated the bootstrap container's entrypoint.sh script to always clear the local Terraform state that manages the backend S3 bucket. This change is to avoid error message like kubernetes-bootstrap | Error: error deleting S3 Bucket (bucketname): BucketNotEmpty: The bucket you tried to delete is not empty that might occur when running docker-compose up.
  • CLOUD-3623: Updated the multi-purpose-commerce-tool Jenkins job to use a dedicated activeMqResourcingProfile parameter for ActiveMQ deployments, separate from Commerce stack resourcing. Added early validation to ensure the selected ActiveMQ profile exists in terraform/activemq/env-file, with clear error messaging when the profile is missing.
  • CLOUD-3661: Updated ActiveMQ deployments to use the Kubernetes Recreate strategy during updates. This prevents a second broker pod from being started during rollout, which avoids KahaDB lock contention.

New Features​

  • CLOUD-3688: Added support for deploying an internal Eclipse P2 caching proxy service and for configuring Jenkins builds to use it. This change is to improve Self-Managed Commerce build stability by drastically reducing external Eclipse downloads across builds. It is not enabled by default. For more information about enabling it, see Eclipse P2 Caching Proxy.

3.6.0​

New Features​

  • CLOUD-2907: Added support for sourcing JVM arguments from the ops-spec.json file when building Self Managed Commerce application images and the ActiveMQ image. This feature requires version 4.6.1 or later of the Elastic Path Docker project.
  • CLOUD-3098: Added the adoptExistingDatabaseSecret parameter to the use-existing-database-server Jenkins job, and parameter doNotManageDatabaseSecret to the other database-management jobs, to facilitate switching database servers. See Switching Databases for more information.
  • CLOUD-3163: Added the create-and-manage-bastion-instance Jenkins job, which can be used to create a bastion instance in your environment. The bastion instance includes Kubernetes management tools, and access to the Elastic Kubernetes Service (EKS) cluster nodes.
  • CLOUD-3334: Added the extensionJob field to the commerce-branch-validation Jenkins job. This field can be used to trigger an additional Jenkins job to be run after the Cortex system tests complete.
  • CLOUD-3340: Added PostgreSQL 16 support to the create-and-manage-database-server and multi-purpose-commerce-tool Jenkins jobs. The postgres-rds-16 selection for databaseType will deploy an Amazon RDS for PostgreSQL 16.6 database.
  • CLOUD-3383: Updated the default MySQL version from 5.7 to 8.0 in the following Jenkins jobs:
    • build-selected-docker-images
    • build-core-images
    • multi-purpose-commerce-tool
    • build-mysql
    • commerce-test-and-deploy
    • commerce-branch-validation
  • CLOUD-3531: Added support to build and run MySQL 8.4 container images. Using MySQL 8.4 containers requires Self-Managed Commerce release 8.4.x or higher and Elastic Path Docker release 4.8.x or higher. An option to select MySQL 8.4 has been added to mysqlVersion parameter in the following jobs:
    • build-selected-docker-images
    • build-core-images
    • multi-purpose-commerce-tool
    • build-mysql
    • commerce-test-and-deploy
    • commerce-branch-validation
  • CLOUD-3387: Replaced the EFS Provisioner deployment with the AWS Elastic File System (EFS) Container Storage Interface (CSI) Driver add-on for Elastic Kubernetes Service. The operational behaviour when dynamically provisioning EFS volumes is unchanged.
  • CLOUD-3401: Added the MAXIMUM_CONNECTIONS variable to the ActiveMQ resource profiles. This variable allows specifying the maximum simultaneous incoming connections accepted by ActiveMQ. See Configure ActiveMQ Container for more information.
    • If you previously customized the ActiveMQ maximum connection count then update your implementation to use this resource profile approach.
  • CLOUD-3450: Added the EP_CONTAINER_MEM_ACTIVEMQ variable to the ActiveMQ resource profiles. This variable allows you to specify the ActiveMQ JVM's heap size. Both the initial heap size and maximum heap size are set to this value. See Configure ActiveMQ Container for more information.
    • If you previously customized the ActiveMQ heap size then update your implementation to use this resource profile approach.
  • CLOUD-3403: Updated the MySQL configurations to use the newer JDBC driver classname com.mysql.cj.jdbc.Driver instead of the deprecated com.mysql.jdbc.Driver. The JDBC driver change will take effect for newly configured databases and Commerce application stacks, and when the configurations of existing databases are updated using the Jenkins jobs and Commerce applications are restarted. Also updated the JDBC configuration defined by job create-and-manage-database-server to use TLSv1.3 instead of TLSv1.2 when creating or configuring Aurora MySQL 8.0 (Aurora version 3) clusters.
  • CLOUD-3406: Added Resource Profiles to the create-or-delete-activemq-container Jenkins job. The ActiveMQ Resource Profile allows specifying additional Java Virtual Machine (JVM) parameters with the EP_ACTIVEMQ_X_JVM_ARGS variable. See Configure ActiveMQ Container for more information.
  • CLOUD-3407: Updated validation code to allow IPv6 addresses when defining "allowed CIDRs" in the create-or-delete-activemq-container, deploy-or-delete-commerce-stack, create-additional-kubernetes-cluster, and multi-purpose-commerce-tool Jenkins jobs.
  • CLOUD-3416: Added variables to docker-compose.yml for configuring HAProxy connection timeout values. If these variables are not defined in docker-compose.override.yml, the default values shipped with HAProxy will be used.
  • CLOUD-3418: Updated the Jenkins version from 2.497-jdk17 to 2.504-jdk21. There are no plugin or configuration changes included in this update.
  • CLOUD-3427: Updated the Jenkins configuration to install the latest version of all plugins.
  • CLOUD-3439: Improved the clean-up of EKS add-ons to programatically discover add-ons and to better handle error messages.
  • CLOUD-3448: Updated the Maven command in Jenkins job run-cortex-system-tests to exclude setup-local-integration-test, for compatibility with the newest Self-Managed Commerce releases.
  • CLOUD-3449: Added a variable to docker-compose.yml for disabling version 1 of the AWS Instance Metadata Service on the EKS nodes. The default value for disable_imds_v1 is false, which keeps the behaviour consistent with previous versions of CloudOps for Kubernetes. Setting disable_imds_v1 to true would leave Instance Metadata Service version 2 enabled, which requires the use of a temporary token when accessing the metadata service. Out of the box functionality is compatible with both version 1 and version 2.
  • CLOUD-3459: Added the generic-webhook-trigger plugin to the default Jenkins deployment. This plugin enables a highly-compatible method for triggering Jenkins builds remotely.
  • CLOUD-3469: Added -Dsun.net.client.defaultConnectTimeout=10000 -Dsun.net.client.defaultReadTimeout=10000 to MAVEN_OPTS in the build-deployment-package, run-cortex-system-tests, run-recursive-commerce-tests, and run-select-commerce-tests Jenkins jobs. These timeout values should cause Maven HTTP transactions that become unresponsive to fail quickly rather than only failing when the Jenkins job timeout is reached.
  • CLOUD-3455: Added the stopServicesDuringDataPop parameter to the run-data-pop-tool Jenkins job. Setting this parameter to true will stop all Elastic Path Commerce application services before running the Data Population database operations. Selecting this option will cause an application outage.
  • CLOUD-3539: Increased the cortex endpoint payload ceiling from about 14KB to 40KB when using the provided optional ModSecurity WAF. With this change, requests exceeding 40KB will be blocked. The change also disables body-inspection for payloads greater than 8KB, to reduce false positives on complex JSON structures. The change introduces a new custom-rules.conf file where client-specific and environment-specific ModSecurity rule customizations can be made without modifying core WAF configuration. This change has no effect on users who are not using ModSecurity WAF.
  • CLOUD-3540: Fixed a bug in the update-waf Jenkins job which, when disabling the WAF with the deleteWAF parameter, introduced a breaking character in the configmap that caused new haproxy-ingress pods to fail to start.
  • CLOUD-3553: Jenkins authorization strategy configuration is now sourced from an external file. Select the file using TF_VAR_jenkins_authorization_configuration_file. This makes it easier to switch or customize authorization without modifying out-of-the-box files. The default is default-role-based-authorization.yaml. To use the legacy matrix approach, set it to default-matrix-authorization.yaml. For more information, see Login Credentials.
  • CLOUD-3572: Added options to enable and configure the AWS Elastic Kubernetes Service (EKS) CloudWatch Observability Add-On. With this add-on you can enable CloudWatch Container Insights, CloudWatch Application Signals or CloudWatch Logs. See CloudWatch Observability for more information.
  • CLOUD-3579: Updated the Kubernetes version to 1.34. EKS version 1.32 standard support ends March 2026 ( https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html#kubernetes-release-calendar ). Additional component version updates to support this change are as follows:
    • The kube-proxy add-on was updated to v1.34.1-eksbuild.2
    • The kubectl version was updated to v1.34.0
    • The coredns add-on was updated to v1.12.4-eksbuild.1
    • The eksctl version was updated to v0.221.0
    • The metrics server add-on was updated to v0.8.0-eksbuild.6
    • The Helm 3 version was updated to v3.19.0
    • The cert manager Helm chart was updated to v1.19.2
    • The cilium Helm chart was updated to v1.18.5
    • The cluster autoscaler version was updated to v1.34.2
    • The Amazon Elastic Block Store (EBS) CSI driver Helm chart was updated to v2.54.1
    • The Amazon Elastic File System (EFS) CSI driver add-on was updated to v2.2.0-eksbuild.1
    • The fluentd version was updated to v1.19.1-debian-cloudwatch-amd64-1.0
    • The overprovisioning pause version was updated to v1.34.1-eks-1-34-10
  • CLOUD-3591 : Added a fast fail validation check that is required when setting up CloudOps for Kubernetes in an empty AWS account. The setup process better validates that TF_VAR_rebuild_nodegroups is set to true, and will and exit quickly if it is not. This is required to ensure cilium properly propagates across all cluster nodes.
  • CLOUD-3597 : Added a new Maven pod template named maven-10gb-2core-1container, with 10Gb memory and 2 CPU cores, to support the new resource requirements for run-selected-commerce-tests Jenkins job with Self-Managed Commerce release 8.7.x.
  • CLOUD-3605: Removed the createPostgresDatabaseResources parameter from the create-and-manage-database-server Jenkins job. The parameter is removed because it is no longer of value; the job itself creates the PostgreSQL database without needing to set that parameter, and the data-pop-tool creates the PostgreSQL schema.
  • SUP-5050: Added a checksum annotation to the Jenkins Helm values file, which will trigger a Jenkins pod redeployment during docker-compose up --force-recreate if a variable value is changed. The checksum value is based on the Jenkins-specific variables defined in docker-compose.override.yml.

Bug Fixes​

  • CLOUD-3499: Updated the build-activemq Jenkins job to better support the internal testing of Elastic Path Docker project pre-release versions.
  • CLOUD-3511: Updated several Jenkins jobs and related Groovy functions to properly pass and accept the epCommerceCredentialID parameter, to ensure that child jobs use the Git credentials specified by the caller. This resolves an issue where image build jobs could fail to obtain the ops-spec.json file from the ep-commerce project and then incorrectly use the fall-back file. Also refined some Jenkins job parameter descriptions.
  • CLOUD-3516: Reduced the compute resources used by Jenkins jobs commerce-branch-validation, commerce-test-and-deploy and multi-purpose-commerce-tool. These are orchestration jobs that do little work on their own, and primarily only invoke child jobs. We reduced their resource allocations from 4GB and 2 CPU cores each, to 1GB and 1 CPU core each.
  • CLOUD-3519: Fixed an issue where the force-cleanup bootstrap mode would fail if the EKS cluster did not exist or had previously been removed.
  • CLOUD-3534: Aligned the ECR cross-account permissions policy with the current Identity and Access Management (IAM) version (2012-10-17). This is a best-practice update with no change in functionality.
  • CLOUD-3573: Update the Dockerfiles for both the maven-agent image and the bootstrap image to use Rocky Linux 10.1 as the base image. Among other things, this updates the Python version from 3.9 to 3.11, to remain compatible with the boto3 Python library.
  • CLOUD-3586: Fixed an issue where the force-cleanup bootstrap mode was failing to remove the NodeGroupPolicy AWS IAM Policy.
  • CLOUD-3592: Fixed an issue where EFS mount targets would be recreated by Terraform when unrelated cluster changes were made. This was fixed by sorting the output of an AWS CLI command so subnets are always returned in the same order.
  • SUP-4974: Resolved an issue with the commerce-branch-validation Jenkins job where setting the build description failed when it is triggered by a custom pipeline.
  • SUP-4999: Changed the ActiveMQ pod liveness and readiness probes to use a status query command rather than a TCP connection test. In addition to providing a more complete healthcheck, this eliminates EOFException log messages caused by the previous TCP check.
  • SUP-5067: Updated the build display name in Jenkins job commerce-branch-validation to use the pattern #<build-number> - <triggering user>. This fixes an issue where the job number wasn't displayed and scheduled builds showed the triggering user as null. Scheduled builds now list the user as automation, making it obvious when the job ran manually versus by schedule.
  • SUP-5090: Added a ModSecurity Web Application Firewall rule resolving an intermittent JSON parsing issue with incoming Jenkins webhook events. This rule only applies to webhooks configured with the generic-webhook-trigger plugin.
  • SUP-5823: Changed the container image build tool from img to Docker-in-Docker, to resolve runc-related errors building container images. As part of this change, the docker-agent base image is changed to docker:29.1.2-dind, and the pull-docker-images Jenkins job is updated to use docker instead of podman.

Deprecations & Removals​

  • CLOUD-3640: Remove the Kubernetes Dashboard from CloudOps for Kubernetes. The Kubernetes Dashboard project has been retired and archived, and is no longer being maintained. For more information, see the Kubernetes Dashboard project page.

Upgrade Instructions​

For upgrade instructions, see Upgrading CloudOps for Kubernetes.