Scheduling Jobs Using Quartz Scheduler

Elastic Path Commerce uses the Quartz Scheduler to execute scheduled jobs. Quartz is an open source job scheduling service embedded within Spring. It can also be used as a standalone service with any Java application. Scheduled jobs are configured in Spring via the quartz.xml configuration files. There is one Quartz configuration for the Search Server web application and another for the Commerce Manager web application.

In Quartz, Jobs can be any Java class that implements the Quartz Job interface. To execute a job, you must set Triggers that detail when the job is to occur. When a trigger activates, the scheduler will trigger corresponding listeners that will execute the job. As Jobs are completed, they return a JobCompletionCode which informs the scheduler of success or failure. The JobCompletionCode also instructs the scheduler of any further action to perform depending on the job’s results.

Quartz has built in JMX support. By using JMX, you can monitor and control your Quartz jobs.

Adding a new scheduled job

To add a scheduled job to either the Batch Server or Search server, do the following:

1. Open the appropriate quartz.xml file:

   • The Batch server quartz.xml file is located at batch\ep-batch\src\main\resources\spring\scheduling\quartz.xml
   • The Search server quartz.xml file is located at search\ep-search\src\main\resources\spring\scheduling\quartz.xml

2. In quartz.xml, define a job bean as shown below:

   <bean id="newJob" class="com.elasticpath.batch.hds.HDSMethodInvokingJobDetailFactoryBean">
       <property name="targetObject" ref="newJobService"/>
       <property name="targetMethod" value="executeMethod"/>
       <property name="concurrent" value="false"/>
   </bean>
   

3. Replace the following values:

   • newJob - the name of the bean.
   • newJobService - the class that contains the logic for the scheduled job.
   • executeMethod - the name of the method to execute in the class.

4. In quartz.xml, define a trigger bean as shown below:

   <bean id="newJobTrigger" class="org.springframework.scheduling.quartz.CronTriggerFactoryBean">
       <property name="jobDetail" ref="newJob"/>
       <property name="cronExpression" value="0 0 0/1 * * ?"/>
       <property name="timeZone" value="#{systemProperties['quartz.timezone']}"/>
   </bean>
   

5. Replace the following properties:

   • newJob - the name of the job bean.
   • cronExpression - the cron expression that sets how often the job should be executed. For more information on cron expressions, see Quartz Cron Configuration.

6. Open the appropriate quartz-setup.xml file:

   • The Batch Server quartz-setup.xml file is located at batch\ep-batch\src\main\filtered-resources\spring\scheduling\quartz-setup.xml
   • The Search server quartz-setup.xml file is located at search\ep-search\src\main\filtered-resources\spring\scheduling\quartz-setup.xml

7. Add the new trigger to the list of triggers. For the Batch server, it will look similar to the code below:

   <util:list id="schedulingTriggers">
       <ref bean="cleanupOrderLocksTrigger"/>
       <ref bean="processImportJobTrigger"/>
       <ref bean="importJobCleanupTrigger"/>
       <ref bean="staleImportJobTrigger"/>
       <ref bean="cleanupSessionsTrigger"/>
       <ref bean="cleanupAbandonedCartsTrigger"/>
       <ref bean="cleanupFailedOrdersTrigger"/>
       <ref bean="cleanupFailedOrdersTrigger"/>
   
       <!-- add the reference to the new trigger here -->
       <ref bean="newJobTrigger" />
   </util:list>
   

Configuring Quartz

Quartz File Structure

The Quartz configuration file contains three types of bean definitions:

• The schedulerFactory bean, which contains the list of trigger beans that it needs to execute

• The trigger beans (xxxTrigger), which configure the triggers for each job

  This is usually either a SimpleTriggerBean, which runs a recurring job at an interval (specified in milliseconds), or a CronTriggerBean, which runs a job once at a specified time.

• The job beans (xxxJob), which specify the class and method that will be called for each job as well as any arguments that need to be passed

There is generally only one factory bean definition per quartz.xml file. Each job has a job bean definition and a trigger bean definition.

note

For more information about Quartz see the Quartz website.

Quartz Cron Configuration

The time/trigger to execute the scheduled job can be set with the cronExpression property. The cron expression contains six required components and one optional component. A cron expression is written on a single line and each component is separated from the next by space. Only the last, or rightmost, component is optional. The table below describes the cron components in detail.

Components of a Cron Expression:

Position	Meaning	Allowed Special Characters
1	Seconds (0-59)	, - * /
2	Minutes (0-59)	, - * /
3	Hours (0-23)	, - * /
4	Day of month (1-31)	, - * / ? L C
5	Month (either JAN-DEC or 1-12)	, - * /
6	Day of week (either SUN-SAT or 1-7)	, - * / ? L C #
7	Year (optional, 1970-2099), when empty, full range is assumed	, - * /

Each component accepts the typical range of values that you would expect, such as 0-59 for seconds and minutes and 1-31 for day of the month. For the month and day of the week components, you can use numbers, such as 1-7 for day of the week, or text such as SUN-SAT.

Each field also accepts a given set of special symbols, so placing a * in the hours component means every hour, and using an expression such as 6L in the day-of-the-week component means last Friday of the month. The list below describes cron wildcards and special symbols in detail.

• *

  Any value. This special character can be used in any field to indicate that the value should not be checked. Therefore, our example cron expression will be fired on any day of the month, any month, and any day of the week between 1970 and 2099.

• ?

  No specific value. This special character is usually used with other specific values to indicate that a value must be present but will not be checked.

• -

  Range. For example 10-12 in the Hours field means hours 10, 11, and 12.

• ,

  List separator. Allows you to specify a list of values, such as MON, TUE, WED in the Day of week field.

• /

  Increments. This character specifies increments of a value. For example 0/1 in the Minute field in our example means every 1-minute increment of the minute field, starting from 0.

• L

  L is an abbreviation for Last.

  The meaning is a bit different in Day of month and Day of week. When used in the Day of month field, it means the last day of the month (31st of March, 28th or 29th of February, and so on).

  When used in Day of week, it has the same value as 7 (Saturday). The L special character is most useful when you use it with a specific Day of week value. For example, 6L in the Day of week field means the last Friday of each month.

• #

  This value is allowed only for the Day of week field and it specifies the nth day in a month. For example 1#2 means the first Monday of each month.

• C[PD:*]

  The Calendar value is allowed for the Day of month and Day of week fields.

  The values of days are calculated against a specified calendar. Specifying 20C in the Day of month field fires the trigger on the first day included in the calendar on or after the 20th. Specifying 6C in the Day of week field is interpreted as the first day included in the calendar on or after Friday.

• [PD:*]C

  At the time of writing, support for the C special character and specifying both Day of week and Day of month values has not been not completed

Elastic Path Quartz Jobs

Elastic Path Commerce provides a number of Quartz jobs out of the box for both the Batch Server and the Search Server. They are configurable from a number of different locations. For more information on configuring Quartz jobs, see the following:

• Overriding Spring Configuration
• Configuring System Settings
• Configuring Batch Server Scheduled Jobs
• Configuring Search Server Scheduled Jobs

Batch Server Quartz Jobs

important

All batch jobs are configured in the commerce-engine/ep-batch module, in the quartz.xml file.

abandonedCartEventsJob

Triggers the abandoned shopping cart events.

• Default Recurrence: Daily at midnight.

• Configuration Locations:

  • COMMERCE/SYSTEM/ABANDONEDCARTEVENTS/maxHistory controls how many days since the shopping cart was last updated before it will be cleared. Defaults to 60.
  • COMMERCE/SYSTEM/ABANDONEDCARTEVENTS/batchSize controls the maximum number of records that should be processed in each job execution. Defaults to 1000.

cleanupAbandonedCartsJob

Purges abandoned shopping carts. Removes TSHOPPINGCART and dependent records.

• Default Recurrence: Once an hour, at the 30 minute mark.

• Configuration Locations:

  • COMMERCE/SYSTEM/ABANDONEDCARTCLEANUP/maxHistory controls how many days since the shopping cart was last updated before it will be cleared. Defaults to 60.
  • COMMERCE/SYSTEM/ABANDONEDCARTCLEANUP/batchSize controls the maximum number of records that should be processed in each job execution. Defaults to 1000.

cleanupAnonymousCustomerJob

Purges anonymous customers. Removes TCUSTOMER and dependent records that are anonymous and have no associated orders.

• Default Recurrence: Once an hour, at the 5 minute mark.

• Configuration Locations:

  • COMMERCE/SYSTEM/ANONYMOUSCUSTOMERCLEANUP/maxHistory controls how many days since the customer was last updated before it will be cleared. Defaults to 60.
  • COMMERCE/SYSTEM/ANONYMOUSCUSTOMERCLEANUP/batchSize controls the maximum number of records that should be processed in each job execution. Defaults to 1000.

cleanupExpiredOAuth2TokensJob

Purges expired OAuth tokens. Removes TOAUTHACCESSTOKEN records that have expired.

• Default Recurrence: Daily at midnight

• Configuration Locations: COMMERCE/SYSTEM/EXPIREDOAUTHTOKENCLEANUP/batchSize controls the maximum number of records that should be processed in each job execution. Defaults to 1000.

cleanupFailedOrdersJob

Purges failed orders. Removes TORDER and dependent records with status of FAILED.

• Default Recurrence: Once an hour, at the 45 minute mark

• Configuration Locations:

  • COMMERCE/SYSTEM/FAILEDORDERCLEANUP/maxHistory controls how many days since the order was created before it will be cleared. Defaults to 60.
  • COMMERCE/SYSTEM/FAILEDORDERCLEANUP/batchSize controls the maximum number of records that should be processed in each job execution. Defaults to 1000.

cleanupInactiveCartsJob

Cleans up inactive carts. Removes TSHOPPINGCART and dependent records in INACTIVE state, when the cart has been purchased.

• Default Recurrence: Once an hour, at the 15 minute mark.

• Configuration Locations:

  • COMMERCE/SYSTEM/INACTIVECARTSCLEANUPJOB/minAge controls the minimum number of minutes since the shopping cart was last updated before it will be cleared. Defaults to 60.
  • COMMERCE/SYSTEM/ABANDONEDCARTCLEANUP/batchSize controls the maximum number of records that should be processed in each job execution. Defaults to 1000.

cleanupOrderLocksJob

Cleans up order locks. This job removes TORDERLOCK records created by Commerce Manager when an order is edited.

• Default Recurrence: Once an hour, at the start of the hour (i.e. 1:00pm)

cleanupOrphanedOrderPaymentGuidsJob

Searches for order payment records on orders older than a configured age, and sets the payment instrument GUID reference to null (UPDATE TORDERPAYMENT SET PAYMENT_INSTRUMENT_GUID = null WHERE UIDPK IN <list>). This reference is only used for showing the "display name" (usually last 4 digits of the card number) when viewing order history in Commerce Manager.

• Default Recurrence: Daily at midnight (if enabled).

• Configuration Locations:

  • COMMERCE/SYSTEM/ORDERPAYMENTCLEANUP/enable controls whether this job is enabled. Defaults to false.
  • COMMERCE/SYSTEM/ORDERPAYMENTCLEANUP/maxHistory controls how many days old an order should be before its order payment references to payment instruments should be cleared. Defaults to 365.
  • COMMERCE/SYSTEM/ORDERPAYMENTCLEANUP/batchSize controls the maximum number of records that should be processed in each job execution. Defaults to 1000.

cleanupOrphanedOrderPaymentInstrumentsJob

Deletes order payment instrument records on orders older than a configured age (DELETE TORDERPAYMENTINSTRUMENT WHERE UIDPK IN <list>). These records are only needed for operations that require doing a payment reservation using the existing payment method, such as order modification.

• Default Recurrence: Daily at midnight (if enabled).

• Configuration Locations:

  • COMMERCE/SYSTEM/ORDERPAYMENTCLEANUP/enable controls whether this job is enabled. Defaults to false.
  • COMMERCE/SYSTEM/ORDERPAYMENTCLEANUP/maxHistory controls how many days old an order should be before its order payment references to payment instruments should be cleared. Defaults to 365.
  • COMMERCE/SYSTEM/ORDERPAYMENTCLEANUP/batchSize controls the maximum number of records that should be processed in each job execution. Defaults to 1000.

cleanupOrphanedPaymentInstrumentsJob

Deletes payment instrument records that are orphaned (have no incoming references from TORDERPAYMENTINSTRUMENT, TORDERPAYMENT, TCARTORDERPAYMENTINSTRUMENT, and TCUSTOMERPAYMENTINSTRUMENT).

• Default Recurrence: Daily at midnight.

• Configuration Locations:

  • COMMERCE/SYSTEM/PAYMENTINSTRUMENTCLEANUP/batchSize controls the maximum number of records that should be processed in each job execution.

cleanupChangesetsJob

Deletes changesets that are in the FINALIZED state and published more than a configured number of days ago.

• Default Recurrence: Daily at midnight (if enabled).

• Configuration Locations:

  • COMMERCE/SYSTEM/CHANGESETCLEANUP/enable controls whether this job is enabled. Defaults to false.
  • COMMERCE/SYSTEM/CHANGESETCLEANUP/maxHistory controls the minimum number of days to consider a changeset since it was published. Defaults to 60.
  • COMMERCE/SYSTEM/CHANGESETCLEANUP/batchSize controls the maximum number of records that should be processed in each job execution. Defaults to 1000.

dataPointRevokedConsentsJob

Removes data point values for which consent is revoked.

• Default Recurrence: Daily at 2:00am.

encryptCustomerPasswordsJob

Ensures that all registered customer records are upgraded from SHA256 password hashing to BCRYPT password hashing. Looks at the TCUSTOMERAUTHENTICATION.ENCRYPTION_TYPE field to determine the current encryption mechanism, and if it's set to SHA-256, then the existing hash is re-hashed using BCRYPT.

• Default Recurrence: Daily at 2:00am (if enabled).

• Configuration Locations:

  • COMMERCE/SYSTEM/ENCRYPTCUSTOMERPASSWORDS/enabled controls whether this job is enabled. Defaults to true.
  • COMMERCE/SYSTEM/ENCRYPTCUSTOMERPASSWORDS/batchSize controls the maximum number of records that should be processed in each job execution. Defaults to 1000.

expiredDataPointValuesJob

Removes data point values for which the retention period is expired.

• Default Recurrence: Daily at 1:00am.

expiredCompletedOrdersCleanupJob

Removes completed orders that were last modified more than a configured number of days ago.

• Default Recurrence: Every hour, at 30 minutes past the hour (if enabled).

• Configuration Locations:

  • COMMERCE/SYSTEM/ORDERCLEANUP/enable controls whether this job is enabled. Defaults to false.
  • COMMERCE/SYSTEM/ORDERCLEANUP/maxHistory controls the minimum number of days to consider an order since it last updated. Defaults to 730 days (2 years).
  • COMMERCE/SYSTEM/ORDERCLEANUP/batchSize controls the maximum number of records that should be processed in each job execution. Defaults to 1000.

importJobCleanupProcessorJob

Removes completed import jobs from the database based on their age. This job removes the TIMPORTNOTIFICATION records in the PROCESSED or QUEUED_FOR_VALIDATION state.

• Default timeout: 5 minutes.

• Default Recurrence: Every 72 hours, with a 30 second delay at startup.

• Configuration Locations:

  • COMMERCE/SYSTEM/IMPORT/importJobMaxAge controls maximum age (in seconds) a completed import job will remain in the database before it is deleted. Defaults to 172800 seconds (48 hours).
  • COMMERCE/SYSTEM/IMPORT/staleImportJobTimeout controls period of time (in minutes) before an import job is considered stale. Defaults to 5 minutes.

inventoryJournalRollupJob

Rolls up inventory journal. Sums allocated quantity delta and quantity on hand delta in TINVENTORYJOURNAL table for the given InventoryKey, then deletes records TINVENTORYJOURNAL.

• Default Recurrence: Daily with a 30 second delay at startup.

processImportJob

Processes import jobs.

• Default Recurrence: Every 5 seconds, with a 10 second delay at startup.

• Configuration Locations: commerce-manager-client/com.elasticpath.cmclient.core module: import-jobs.xml file

releaseShipmentsJob

Moves physical shipments from the INVENTORY_ASSIGNED state to the RELEASED state once the warehouse pick delay has passed. This allows customer service representatives to modify the physical shipment for a short period of time before the shipment is sent to fulfillment. Once all shipments are released, the order will change to the IN_PROGRESS state.

• Default Recurrence: Once an hour, at the start of the hour (i.e. 1:00pm).

• Configuration Locations:

  • COMMERCE/SYSTEM/RELEASESHIPMENTSJOB/batchSize controls the maximum number of records that should be processed in each job execution. Defaults to 1000.

searchTermsAggregatorJob

Aggregates search term activity into the TSEARCHTERMSACTIVITYSUMMARY table.

• Default Recurrence: Every 30 seconds, with a 30 second delay at startup.

• Configuration Locations:

  • commerce-engine module: pom.xml file. searchTermsAggregatorJob properties are generally set during the build process, not runtime.

staleImportJobProcessorJob

Processing stale import jobs. This job removes the TIMPORTNOTIFICATION records in the IN_PROCESS state.

• Default Recurrence: Every minute, with a 30 second delay at startup.

Search Server Quartz Jobs

catalogPromoMonitorJob

Checks for changes to catalog promotion rules and notifies the search server if any are found.

• Default Recurrence: Every 5 seconds, with a 10 second delay at startup.

• Configuration Locations:

  • commerce-engine/ep-search module: quartz.xml file
  • ep.index.build.interval property in the ep.properties file

categoryIndexBuildJob

Rebuilds the category index.

• Default Recurrence: Every 5 seconds, with a 10 second delay at startup.

• Configuration Locations:

  • commerce-engine/ep-search module: quartz.xml file
  • ep.index.build.interval property in the ep.properties file

cmUserIndexBuildJob

Rebuilds the Commerce Manager user index.

• Default Recurrence: Every 5 seconds, with a 10 second delay at startup.

• Configuration Locations:

  • commerce-engine/ep-search module: quartz.xml file
  • ep.index.build.interval property in the ep.properties file

customerIndexBuildJob

Rebuilds the customer index.

• Default Recurrence: Every 5 seconds, with a 10 second delay at startup.

• Configuration Locations:

  • commerce-engine/ep-search module: quartz.xml file
  • ep.index.build.interval property in the ep.properties file

optimizeSearchIndicesJob

Optimizes the search indices.

• Default Recurrence: Every 5 seconds, with a 2 minute delay at startup.

• Configuration Locations: commerce-engine/ep-search module: quartz.xml file. For more information on optimizing search indices, see Search Index Optimization Job

productIndexBuildJob

Rebuilds the product index.

• Default Recurrence: Every 5 seconds, with a 10 second delay at startup.

• Configuration Locations:

  • commerce-engine/ep-search module: quartz.xml file
  • ep.index.build.interval property in the ep.properties file

promotionIndexBuildJob

Rebuilds the promotion index.

• Default Recurrence: Every 5 seconds, with a 10 second delay at startup.

• Configuration Locations:

  • commerce-engine/ep-search module: quartz.xml file
  • ep.index.build.interval property in the ep.properties file

rulebaseCompileJob

Recompiles the rule base, and stores it in the database.

• Default Recurrence: Every 5 minutes, with a 5 minute delay at startup.

• Configuration Locations: commerce-engine/ep-search module: quartz.xml file

skuIndexBuildJob

Rebuilds the SKU index.

• Default Recurrence: Every 5 seconds, with a 10 second delay at startup.

• Configuration Locations:

  • commerce-engine/ep-search module: quartz.xml file
  • ep.index.build.interval property in the ep.properties file

Batch Job Implementation

Each future batch job must extend the AbstractBatchJob class to implement the correct workflow and achieve optimal performance. Future batch jobs need to implement only a few methods in the AbstractBatchJob contract.

Batch Java Persistence Query Language (JPQL) queries will be stored in separate Object Relational Mapping (ORM) files (per job) in the ep-batch module.

The AbstractBatchProcessor class is a transactional unit responsible for processing one batch of records. This class implements the logic for the optimal processing of a single batch and provides the abstract methods to be implemented by extensions.

The extension classes must implement the executeBulkOperations method. The preProcessBatch method is optional and is only required if a batch needs to be pre-processed. For example, some entries are filtered out or additionally modified:

    if (shipment.getCreatedDate().before(warehousePickDelayTimestamp)) {
                shipment.getOrder().setModifiedBy(eventOriginatorHelper.getSystemOriginator());
            }

• Only shipments that meet certain datetime criteria are processed in executeBulkOperations.