Core Request-Scoped Caching

Overview

Core Request-Scoped Caching is a mechanism to cache JPA named query execution.

Similar to Cortex request-scoped cache, Core Request-Scoped Caching is also request/thread scoped and its efficiency depends on the number of duplicate JPA queries with the same parameter values being called in the same thread. For that reason, it has a small memory footprint, and adds no risk of "dirty reads" after database modifications.

By default, this cache is enabled in Cortex, Search Server, and Integration Server. It can be disabled by specifying the -Dep.core.request.scoped.cache.enabled=false JVM parameter.

The Core Request-Scoped Caching operational status is indicated in the logs, with Core Request-Scoped Caching is enabled or Core Request-Scoped Caching is disabled.

With respect to performance, the gain may vary depending on the code path being executed. Some code paths make more JPA calls than others and in the case when the same JPA query is executed two or more times in the same thread, the results will be pulled from Core Request-Scoped Caching. This potentially saves many needless database round-trips.

To put it in perspective, the number of database calls in Cortex checkout is reduced by ~60% with Core Request-Scoped Caching enabled. Certain Integration Server routes are improved by 30-60%, while others by ~10%. The Search Server threads for document creation, entity loading and indexing are improved by 6-10%.

Core Request-Scoped Caching usage statistics can be found in the logs by enabling TRACE level for the com.elasticpath.cache.request.CoreRequestScopeCache logger.

Architecture

Cortex

Each HTTP request passes through the core request listener, responsible for Core Request-Scoped Caching initialization and invalidation. The core request listener plays a significant role as an entry/exit point since it is mandatory to clean up each thread before it is returned to a thread pool and reused later on. Failure to clean a thread may cause cross-contamination of the threads, security issues, reading false data etc.

Core Request-Scoped Caching wraps JPA query execution, where results are obtained from the database. As a cache key, Core Request-Scoped Caching uses the JPA named-query name and query parameters. Example: PRODUCT_SELECT_BY_CATEGORY_UID;20001

It is also important to invalidate a cache entry whenever an entity is modified, in case a write operation is immediately followed by a read operation. A good example is an address creation: a new address record is created (write) and immediately requested (read) in the same thread. Without the invalidation, previously cached results would be used and Cortex could display out-of-date data.

The sequence diagram of a typical API request flow is shown below:

Integration Server and Search Server

The Search Server and Integration Server architectures are similar to the Cortex architecture. The main difference is in the way that the cache is initialized and invalidated. See the Implementation section below for details.

Implementation

Core Request-Scoped Caching is implemented as a thread-local weak hash map.

To check the cache efficiency, hit/miss counters are implemented and made visible if the TRACE logging is enabled. Due to the nature of Core Request-Scoped Caching the counters are intended for debugging purposes only, and this logging should not be enabled in production. Additionally, unlike the application cache, with application scope, the Core Request-Scoped Caching can’t be monitored via JMX console due to its use of a thread-local architecture.

The CoreJPANamedQueryCacheImpl contains an "entity name" --> "relevant JPA query names" cache, used to determine which JPA query caches need invalidation when a particular JPA entity is modified.

Core Request-Scoped Caching is used in the AbstractQueryExecutor class executeSingleResultQueryWithEntityManager and executeMultiResultQueryWithEntityManager methods. These are the central execution points for all JPA queries.

With respect to cache invalidation, there are 2 events that trigger the invalidation:

1. When starting a thread
2. When the entity referenced in the JPA query is modified

The first event is straightforward as every application has an "entry" and "exit" point where Core Request-Scoped Caching invalidation can be invoked:

• In Cortex this is done automatically by the CRSCSupportRequestListener#requestDestroyed method.
• In Search Server this is done automatically by the AbstractIndexServiceImpl#buildIndexJobRunner and AbstractIndexingStage.LogWrappedIndexingTask#run methods.
• In Integration Server, it is important to ensure the following:
  • All Camel route builders that need Core Request-Scoped Caching support should extend CRSCEnabledRouteBuilder instead of RouteBuilder.
  • All Camel processors that need Core Request-Scoped Caching support should extend CRSCEnabledCamelProcessor instead of Processor.

The second event occurs when an entity is modified, as a result of a create, update or delete operation. In that case, only selected cached results will be evicted from the cache. The selection is based on the JPA queries containing the modified entity.