Scheduling Jobs Using Quartz Scheduler

Elastic Path Commerce uses the Quartz Scheduler version 2.1.7 to execute scheduled jobs such as computing product recommendations or building a Lucene index. Quartz is an open source job scheduling service embedded within Spring. It can also be used as a standalone service with any Java application. Scheduled jobs are configured in Spring via the quartz.xml configuration files. There is one Quartz configuration for the Search Server web application and another for the Commerce Manager web application.

In Quartz, Jobs can be any Java class that implements the Quartz Job interface. To execute a job, you must set Triggers that detail when the job is to occur. When a trigger activates, the scheduler will trigger corresponding listeners that will execute the job. As Jobs are completed, they return a JobCompletionCode which informs the scheduler of success or failure. The JobCompletionCode also instructs the scheduler of any further action to perform depending on the job’s results.

Quartz version 2.1.7 has built in JMX support. By using JMX, you can monitor and control some of your Quartz jobs.

Adding a new scheduled job

To add a scheduled job to either the Batch Server or Search server, do the following:

1. Open the appropriate quartz.xml file:

   • The Batch server quartz.xml file is located at batch\ep-batch\src\main\resources\spring\scheduling\quartz.xml

   • The Search server quartz.xml file is located at search\ep-search\src\main\resources\spring\scheduling\quartz.xml

2. In quartz.xml, define a job bean as shown below:

   <bean id="newJob"
       class="org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean">
   <property name="targetObject">
       <ref bean="newJobService"/>
   </property>
   <property name="targetMethod">
       <value>executeMethod</value>
   </property>
   <property name="concurrent">
       <value>false</value>
   </property>
   </bean>
   

3. Replace the following values:

   • newJob - the name of the bean
   • newJobService - the class that contains the logic for the scheduled job
   • executeMethod - the name of the method to execute in the class
   • concurrent - set to false to prevent jobs from executing concurrently

4. In quartz.xml, define a trigger bean as shown below:

   <bean id="newJobTrigger"
       class="org.springframework.scheduling.quartz.CronTriggerBean">
   <property name="jobDetail">
       <ref bean="newJob"/>
   </property>
   <property name="cronExpression">
       <value>0 0 0/1 * * ?</value>
   </property>
   </bean>
   

5. Replace the following properties:

   • newJob - the name of the job bean

   • cronExpression - the cron expression that sets how often the job should be executed

     For more information on cron expressions, see the Quartz Cron Configuration

6. Open the appropriate quartz-setup.xml file:

   • The Batch Server quartz-setup.xml file is located at batch\ep-batch\src\main\filtered-resources\spring\scheduling\quartz-setup.xml

   • The Search server quartz-setup.xml file is located at search\ep-search\src\main\filtered-resources\spring\scheduling\quartz-setup.xml

7. Add the new trigger to the list of triggers. For the Batch server, it will look similar to the code below:

   <util:list id="schedulingTriggers">
       <ref bean="cleanupOrderLocksTrigger"/>
       <ref bean="processImportJobTrigger"/>
       <ref bean="importJobCleanupTrigger"/>
       <ref bean="staleImportJobTrigger"/>
       <ref bean="cleanupSessionsTrigger"/>
       <ref bean="cleanupAbandonedCartsTrigger"/>
       <ref bean="cleanupFailedOrdersTrigger"/>
       <ref bean="cleanupFailedOrdersTrigger"/>
   
       <!-- add the reference to the new trigger here -->
       <ref bean="newJobTrigger" />
   </util:list>
   

Configuring Quartz

Quartz File Structure

The Quartz configuration file contains three types of bean definitions:

• The schedulerFactory bean, which contains the list of trigger beans that it needs to execute

• The trigger beans (xxxTrigger), which configure the triggers for each job

  This is usually either a SimpleTriggerBean, which runs a recurring job at an interval (specified in milliseconds), or a CronTriggerBean, which runs a job once at a specified time.

• The job beans (xxxJob), which specify the class and method that will be called for each job as well as any arguments that need to be passed

There is generally only one factory bean definition per quartz.xml file. Each job has a job bean definition and a trigger bean definition.

note

For more information about Quartz see the Quartz website.

Quartz Cron Configuration

The time/trigger to execute the scheduled job can be set with the cronExpression property. The cron expression contains six required components and one optional component. A cron expression is written on a single line and each component is separated from the next by space. Only the last, or rightmost, component is optional. The table below describes the cron components in detail.

Components of a Cron Expression:

Position	Meaning	Allowed Special Characters
1	Seconds (0-59)	, - * /
2	Minutes (0-59)	, - * /
3	Hours (0-23)	, - * /
4	Day of month (1-31)	, - * / ? L C
5	Month (either JAN-DEC or 1-12)	, - * /
6	Day of week (either SUN-SAT or 1-7)	, - * / ? L C #
7	Year (optional, 1970-2099), when empty, full range is assumed	, - * /

Each component accepts the typical range of values that you would expect, such as 0-59 for seconds and minutes and 1-31 for day of the month. For the month and day of the week components, you can use numbers, such as 1-7 for day of the week, or text such as SUN-SAT.

Each field also accepts a given set of special symbols, so placing a * in the hours component means every hour, and using an expression such as 6L in the day-of-the-week component means last Friday of the month. The list below describes cron wildcards and special symbols in detail.

• *

  Any value. This special character can be used in any field to indicate that the value should not be checked. Therefore, our example cron expression will be fired on any day of the month, any month, and any day of the week between 1970 and 2099.

• ?

  No specific value. This special character is usually used with other specific values to indicate that a value must be present but will not be checked.

• -

  Range. For example 10-12 in the Hours field means hours 10, 11, and 12.

• ,

  List separator. Allows you to specify a list of values, such as MON, TUE, WED in the Day of week field.

• /

  Increments. This character specifies increments of a value. For example 0/1 in the Minute field in our example means every 1-minute increment of the minute field, starting from 0.

• L

  L is an abbreviation for Last.

  The meaning is a bit different in Day of month and Day of week. When used in the Day of month field, it means the last day of the month (31st of March, 28th or 29th of February, and so on).

  When used in Day of week, it has the same value as 7 (Saturday). The L special character is most useful when you use it with a specific Day of week value. For example, 6L in the Day of week field means the last Friday of each month.

• #

  This value is allowed only for the Day of week field and it specifies the nth day in a month. For example 1#2 means the first Monday of each month.

• C[PD:*]

  The Calendar value is allowed for the Day of month and Day of week fields.

  The values of days are calculated against a specified calendar. Specifying 20C in the Day of month field fires the trigger on the first day included in the calendar on or after the 20th. Specifying 6C in the Day of week field is interpreted as the first day included in the calendar on or after Friday.

• [PD:*]C

  At the time of writing, support for the C special character and specifying both Day of week and Day of month values has not been not completed

Elastic Path Quartz Jobs

Elastic Path Commerce provides a number of Quartz jobs out of the box for both the Batch Server and the Search Server. They are configurable from a number of different locations. For more information on configuring Quartz jobs, see the following:

• Overriding Spring Configuration
• Configuring System Settings
• Configuring Batch Server Scheduled Jobs
• Configuring Search Server Scheduled Jobs

Batch Server Quartz Jobs

important

All batch jobs are configured in the commerce-engine/ep-batch module, in the quartz.xml file.

cleanupAbandonedCartsJob

Purges abandoned shopping carts. Removes TSHOPPINGCART and dependent records.

• Default Recurrence: Once an hour, at the 30 minute mark

• Configuration Locations:

  • COMMERCE/STORE/ABANDONEDCARTCLEANUP/maxHistory
  • COMMERCE/SYSTEM/ABANDONEDCARTCLEANUP/batchSize

cleanupAnonymousCustomerJob

Purges anonymous customers. Removes TCUSTOMER and dependent records that are anonymous and have no associated orders.

• Default Recurrence: Once an hour, at the 5 minute mark

• Configuration Locations:

  • COMMERCE/STORE/ANONYMOUSCUSTOMERCLEANUP/maxHistory
  • COMMERCE/SYSTEM/ANONYMOUSCUSTOMERCLEANUP/batchSize

cleanupExpiredOAuth2TokensJob

Purges expired OAuth tokens. Removes TOAUTHACCESSTOKEN records that have expired.

• Default Recurrence: Daily at midnight

• Configuration Locations: COMMERCE/SYSTEM/EXPIREDOAUTHTOKENCLEANUP/batchSize

cleanupFailedOrdersJob

Purges failed orders. Removes TORDER and dependent records with status of FAILED.

• Default Recurrence: Once an hour, at the 45 minute mark

• Configuration Locations:

  • COMMERCE/SYSTEM/FAILEDORDERCLEANUP/maxHistory
  • COMMERCE/SYSTEM/FAILEDORDERCLEANUP/batchSize

cleanupInactiveCartsJob

Cleans up inactive carts. Removes TSHOPPINGCART and dependent records in INACTIVE state, when the cart has been purchased.

• Default Recurrence: Once an hour, at the 15 minute mark

• Configuration Locations:

  • COMMERCE/SYSTEM/ABANDONEDCARTCLEANUP/batchSize
  • COMMERCE/SYSTEM/ANONYMOUSCUSTOMERCLEANUP/batchSize

cleanupOrderLocksJob

Cleans up order locks. This job removes TORDERLOCK records created by Commerce Manager when an order is edited.

• Default Recurrence: Once an hour, at the start of the hour (i.e. 1:00pm)

dataPointRevokedConsentsJob

Removes data point values for which consent is revoked.

• Default Recurrence: Every 24 hours, at 2:00am

expiredDataPointValuesJob

Removes data point values for which the retention period is expired.

• Default Recurrence: Every 24 hours, at 1:00am

importJobCleanupProcessorJob

Removes completed import jobs from the database based on their age. This job removes the TIMPORTNOTIFICATION records in the PROCESSED or QUEUED_FOR_VALIDATION state.

• Default maximum age: 172800 seconds

• Default timeout: 5 minutes

• Default Recurrence: Every 72 hours, with a 30 second delay at startup

• Configuration Locations:

  • COMMERCE/SYSTEM/IMPORT/importJobMaxAge
  • COMMERCE/SYSTEM/IMPORT/staleImportJobTimeout

inventoryJournalRollupJob

Rolls up inventory journal. Sums allocated quantity delta and quantity on hand delta in TINVENTORYJOURNAL table for the given InventoryKey, then deletes records TINVENTORYJOURNAL.

• Default Recurrence: Every 24 hours, with a 30 second delay at startup

processImportJob

Processes import jobs.

• Default Recurrence: Every 5 seconds, with a 10 second delay at startup

• Configuration Locations: commerce-manager-client/com.elasticpath.cmclient.core module: import-jobs.xml file

productRecommendationJob

Re-computes product recommendations.

• Default Recurrence: Daily, at midnight

• Configuration Locations: For more information on configuring product recommendations, see Computing Product Recommendations

releaseShipmentsJob

• Default Recurrence: Once an hour, at the start of the hour (i.e. 1:00pm)

• Configuration Locations: COMMERCE/SYSTEM/RELEASESHIPMENTSJOB/batchSize

searchTermsAggregatorJob

Aggregates search terms.

• Default Recurrence: Every 30 seconds, with a 30 second delay at startup

• Configuration Locations:

  • commerce-engine module: pom.xml file

  searchTermsAggregatorJob properties are generally set during the build process, not runtime

staleImportJobProcessorJob

Processing stale import jobs. This job removes the TIMPORTNOTIFICATION records in the IN_PROCESS state.

• Default Recurrence: Every minute, with a 30 second delay at startup

topSellerJob

Determines the latest top selling products for testing and demos.

• Default Recurrence: Every 30 seconds, with a 30 second delay at startup

• Configuration Locations:

  • commerce-engine module: pom.xml file

  topSellerJob properties are generally set during the build process, not runtime. For more information, see Computing the Top Sellers

Search Server Quartz Jobs

catalogPromoMonitorJob

Checks for changes to catalog promotion rules and notifies the search server if any are found.

• Default Recurrence: Every 5 seconds, with a 10 second delay at startup

• Configuration Locations:

  • commerce-engine/ep-search module: quartz.xml file
  • ep.index.build.interval property in the ep.properties file

categoryIndexBuildJob

Rebuilds the category index.

• Default Recurrence: Every 5 seconds, with a 10 second delay at startup

• Configuration Locations:

  • commerce-engine/ep-search module: quartz.xml file
  • ep.index.build.interval property in the ep.properties file

cmUserIndexBuildJob

Rebuilds the Commerce Manager user index.

• Default Recurrence: Every 5 seconds, with a 10 second delay at startup

• Configuration Locations:

  • commerce-engine/ep-search module: quartz.xml file
  • ep.index.build.interval property in the ep.properties file

customerIndexBuildJob

Rebuilds the customer index.

• Default Recurrence: Every 5 seconds, with a 10 second delay at startup

• Configuration Locations:

  • commerce-engine/ep-search module: quartz.xml file
  • ep.index.build.interval property in the ep.properties file

optimizeSearchIndicesJob

Optimizes the search indices.

• Default Recurrence: Every 5 seconds, with a 2 minute delay at startup

• Configuration Locations: commerce-engine/ep-search module: quartz.xml file. For more information on optimizing search indices, see Search Index Optimization Job

productIndexBuildJob

Rebuilds the product index.

• Default Recurrence: Every 5 seconds, with a 10 second delay at startup

• Configuration Locations:

  • commerce-engine/ep-search module: quartz.xml file
  • ep.index.build.interval property in the ep.properties file

promotionIndexBuildJob

Rebuilds the promotion index.

• Default Recurrence: Every 5 seconds, with a 10 second delay at startup

• Configuration Locations:

  • commerce-engine/ep-search module: quartz.xml file
  • ep.index.build.interval property in the ep.properties file

rulebaseCompileJob

Recompiles the rule base, and stores it in the database.

• Default Recurrence: Every 5 minutes, with a 5 minute delay at startup

• Configuration Locations: commerce-engine/ep-search module: quartz.xml file

skuIndexBuildJob

Rebuilds the SKU index.

• Default Recurrence: Every 5 seconds, with a 10 second delay at startup

• Configuration Locations:

  • commerce-engine/ep-search module: quartz.xml file
  • ep.index.build.interval property in the ep.properties file

Batch Job Implementation

Each future batch job must extend the AbstractBatchJob class to implement the correct workflow and achieve optimal performance. Future batch jobs need to implement only a few methods in the AbstractBatchJob contract.

Batch Java Persistence Query Language (JPQL) queries will be stored in separate Object Relational Mapping (ORM) files (per job) in the ep-batch module.

The AbstractBatchProcessor class is a transactional unit responsible for processing one batch of records. This class implements the logic for the optimal processing of a single batch and provides the abstract methods to be implemented by extensions.

The extension classes must implement the executeBulkOperations method. The preProcessBatch method is optional and is only required if a batch needs to be pre-processed. For example, some entries are filtered out or additionally modified:

    if (shipment.getCreatedDate().before(warehousePickDelayTimestamp)) {
                shipment.getOrder().setModifiedBy(eventOriginatorHelper.getSystemOriginator());
            }

• Only shipments that meet certain datetime criteria are processed in executeBulkOperations.