Data Access Layer in Elastic Path Commerce

Overview

The data access layer is responsible for saving and retrieving data from persistent storage. The majority of persistent data in Elastic Path is stored in the database using the OpenJPA implementation of the Java Persistence API (JPA). A small number of configuration files are persisted directly to the file system using XML and properties files. Objects that are aware of persistence implementation details such as file formats or whether data exists in a database are called Data Access Objects (DAO).

Separation of persistence API and OpenJPA specific Implementation

The ep-persistence-api module contains the generic Elastic Path persistence interfaces, abstract superclasses and persistence-related utility classes. OpenJPA specific implementations of these can be found in the ep-persistence-openjpa module. This keeps the code using the persistence API decoupled from the actual implementation detail which allows for easier customizations and implementation enhancements in the future.

OpenJPA object/relational persistence

OpenJPA is a persistence service that maps objects to tables in a relational database. For more information, see the Java Persistence API (JPA) section.

Elastic Path uses Load tuners to optimize performance of OpenJPA.

Properties files

Some data is stored in properties files located in each web application’s WEB-INF/conf/resources. This includes the lists of countries and country codes used to populate drop-downs in the store. These properties are read into the system by PropertiesDaoImpl.

Database compatibility issues

This page lists database compatibility issues and other database-specific considerations.

Oracle

Empty strings are considered null

Oracle considers the empty string ("") to be a null value. If you assign the empty string to a column that is set to not-null, the database operation will fail.

Ampersand (&) cannot be used in SQL scripts

INSERT INTO TBRAND (UIDPK,CODE,GUID)
VALUES (1110,'D&H', 'D&H');

The above must be changed to the following:

INSERT INTO TBRAND (UIDPK,CODE,GUID)
VALUES (1110,concat('D', concat(CHR(38), 'H')),
        concat('D', concat(CHR(38), 'H')));

Literal date values must use the TO_DATE function

Literal date values like '2006-11-11 11:11:11' cannot be used in Oracle SQL (Structured Query Language) scripts. Instead, the TO_DATE function must be used as shown below.

TO_DATE('2005-06-10','YYYY-MM-DD')

Load Tuners

Retrieving data from persistent storage is expensive from a performance perspective. Catalog objects (products, SKUs, categories) have large object graphs and in many situations, you don’t want to load all the data associated with a particular object.

For example, to optimize page load performance, retrieve product merchandising associations only when necessary. When a shopper browses the store catalog, the associations are not required. When a shopper clicks a product, however, you might want to retrieve the associations so that you can display cross-sells, up-sells, and accessories.

Load tuners provide coarse-grained control of what data is retrieved when loading catalog objects. Many out-of-the-box load tuners are defined in the core’s ep-core/src/main/resources/spring/models/domainModel.xml. Additional application-specific load tuners are defined in the Batch Server’s ep-batch/src/main/resources/spring/service/serviceBatch.xml and Search Server’s ep-search/src/main/resources/spring/service/serviceSearch.xml.

Configuring load tuners

A load tuner consists of a set of boolean properties that specify whether to load a particular type of data when loading objects of the load tuner type. If a property value is true, the data is loaded. If it is false or if the property is not included in the bean definition, the data is not loaded.

For example, the following are two bean definitions for product type load tuners. PRODUCT_TYPE_LOAD_TUNER_ALL loads product types with all their attributes and SKU options. PRODUCT_TYPE_LOAD_TUNER_ATTRIBUTES loads product types with only their attributes.

<bean id="PRODUCT_TYPE_LOAD_TUNER_ALL"
    class="com.elasticpath.domain.catalog.impl.ProductTypeLoadTunerImpl"
    scope="singleton" parent="epDomain">
    <property name="loadingAttributes">
        <value>true</value>
    </property>
    <property name="loadingSkuOptions">
        <value>true</value>
    </property>
</bean>

<bean id="PRODUCT_TYPE_LOAD_TUNER_ATTRIBUTES"
    class="com.elasticpath.domain.catalog.impl.ProductTypeLoadTunerImpl"
    scope="singleton" parent="epDomain">
    <property name="loadingAttributes">
        <value>true</value>
    </property>
    <property name="loadingSkuOptions">
        <value>false</value>
    </property>
</bean>

Troubleshooting load tuner issues

Load tuners are wired to services at both the domain layer (ep-batch/src/main/resources/spring/service/serviceBatch.xml) and the persistence layer (ep-core/src/main/resources/spring/dataaccess/dao.xml). Check the method implementation to check the load tuner being used:

• If your code uses any of these services to retrieve catalog objects and some data is missing.
• If too much of the object graph is being loaded causing performance problems.

For example, in ProductSkuServiceImpl, the findByProductUid(long productUid) method is implemented as follows:

/* core/ep-core/src/main/java/com/elasticpath/service/catalog/impl/ProductSkuServiceImpl.java */

public List<ProductSku> findByProductUid(final long productUid) {
    fetchPlanHelper.configureProductSkuFetchPlan(this.productSkuLoadTunerMinimal);
    List<ProductSku> result = getPersistenceEngine().retrieveByNamedQuery("PRODUCTSKU_SELECT_BY_PRODUCT_UID", new Long(productUid));
    fetchPlanHelper.clearFetchPlan();
    return result;
}

In this case, the productSkuLoadTunerMinimal is used. This load tuner is defined in core’s domainModel.xml as follows:

<bean id="PRODUCT_SKU_LOAD_TUNER_MINIMAL"
    class="com.elasticpath.domain.catalog.impl.ProductSkuLoadTunerImpl"
    scope="singleton" parent="epDomain">
    <property name="loadingAttributeValue">
        <value>false</value>
    </property>
    <property name="loadingInventory">
        <value>false</value>
    </property>
    <property name="loadingOptionValue">
        <value>true</value>
    </property>
    <property name="loadingProduct">
        <value>false</value>
    </property>
    <property name="loadingDigitalAsset">
        <value>false</value>
    </property>
</bean>

So, when findByProductUid() is called, it loads the corresponding product SKUs with only their SKU option values. Attribute values and inventory data are not included.

Service-layer database transactions

Database transaction behavior is defined on a per-method basis in the service.xml Spring configuration file. Service classes with methods that need to run as transactions inherit the parent bean named txProxyTemplate in service.xml. txProxyTemplate has a property called transactionAttributes that defines the nature of transactions for methods matching particular name patterns. For example, if you include the following property in a service that inherits txProxyTemplate, all service methods whose names begin with update will be run in a transaction.

<!-- core/ep-core/src/main/resources/spring/service/service.xml -->
<prop key="update*">PROPAGATION_REQUIRED</prop>

The following transaction attributes are in use:

• PROPAGATION_REQUIRED - Supports a transaction if one already exists. If there is no transaction a new one is started
• PROPAGATION_NEVER - Does not execute as a transaction and throws an exception if one already exists
• PROPAGATION_SUPPORTS - A new transaction will not be started to run the method, however if a transaction is in progress it will propagate to include the call to this method as well

The following attributes are also available:

• PROPAGATION_MANDATORY - Throws an exception if there is no active transaction
• PROPAGATION_NOT_SUPPORTED - Executes non-transactionally and suppends any existing transaction
• PROPAGATION_REQUIRES_NEW - Always starts a new transaction. If an active transaction exists, it is suspended
• PROPAGATION_NESTED - Runs as a nested transaction if one exists, or starts a new one

Note that all methods that access the database must have names that match one of the transaction attribute properties defined in txProxyTemplate or an exception will be thrown at runtime.

Horizontal database scaling

Horizontal database scaling allows an additional increase in transaction throughput by directing read requests to a cluster of read-only database replicas rather than to the master database.