Updating Author and Live Environments

Overview

Elastic Path provides an automated process with Jenkins jobs to deploy new versions of Elastic Path Containers, because manual deployment is tedious when you run database updates in a live environment concurrently. With Jenkins job, the process is repeatable and less prone to errors.

Note: This update process is to update a deployment that is not under active load. Contact Elastic Path Support for more information about updating a deployment that is under active load.

Redeployment Workflow

1. Build and tag a new set of images in the CloudOps Jenkins console, and run the image builder jobs for the Docker images that you want to redeploy
2. Create a redeployment staging script by running the UpdateEnvironment_1_StageUpdates job. The script is stored in Amazon S3, and it’s name is a concatenation of the stack name and the image tag that you use.
3. Go to the Staging directory in your Amazon S3 bucket for CloudOps, and find the script that is automatically generated by the UpdateEnvironment_1_StageUpdates job. Review the AWS (Amazon Web Services) CLI commands that is run
4. To run the staging file, run the UpdateEnvironment_2_ApplyUpdates job pointing at the staging file in Amazon S3.
5. Confirm that the changes are made successfully in the event log for your stack in the CloudFormation console

Redeployment methods

Elastic Path recommends the cfUpdateNoDownTime redeployment method. UpdateEnvironment_1_StageUpdates job calls the Update-Stack-Services.sh script using the cfUpdateNoDownTime redeployment method. You can also change the redeployment method to taskDefUpdate or taskDefUpdateNoDownTime.

cfUpdateNoDownTime

The cfUpdateNoDownTime redeployment method implements zero-downtime redeployments using the update-stack AWS CLI command to update the version of the Docker containers that is used in that stack. The update-stack command runs a redeployment with a minimum healthy percent of 100 and maximum of 200. This command starts the new containers before tearing down the old ones. For this, the ECS (Elastic Container Services) cluster for any container service in the stack must have enough additional EC2 instances to host the new containers during the redeployment.

During normal operations, most of the ECS clusters do not contain additional container hosts, so before the update-stack command is run, the size of any autoscaling groups in the stack is doubled, which is scaled down later. Scaling up the ECS cluster triggers a scale down policy in some stacks. To avoid this, turn off autoscaling from alarm notifications before doubling the cluster size, and turn them on after the cluster size is scaled down.

If this redeployment is run with database changes, the changes are applied before the docker containers are updated. New and old applications make calls to the database at the same time, so the database changes must be backwards compatible.

taskDefUpdate

The taskDefUpdate redeployment method updates the task definition for each service in the stack instead of updating the stack itself, and uses a minimum healthy percent of 0 and maximum of 100. All existing containers come down before the new ones start, resulting a functional outage or downtime. However redeployment method simpler with no scaling up and down of the cluster. Elastic Path recommends this methods for non-production systems, when database changes are not backwards compatible, or when trying to maintain consistent performance for testing.

Database changes for this redeployment method are run while the applications are down, so that it works for database changes that are not backwards compatible. To minimize downtime, the data population instance is brought up and configured before running the task definition update. This minimizes downtime, and validates that deployment package being used to update the database exists.

Data Changes (Amazon RDS)

You can run redeployments with database changes. When you run redeployment with database changes, the redeployment scripts are run on the data population instance, before or after the data population script runs. This limits downtime. However, you must redeploy multi-RDS (Relational Database Service) environments in steps. For an Author-Live Environment, you must run the redeployment jobs once for the author environment, and then again for live environment.