Catalog Syndication Architecture

This section provides details about the catalog syndication architecture and describes the end-to-end data flow, showing how catalog synchronization is initiated and sent to a Content Management System (CMS).

Architecture and Data Flow

Administrators use Syndication API for catalog synchronization. Syndication consumer receives event messages from Java Messaging Service (JMS) and fetches the changes using Syndication API and the data received in the catalog event message. The following diagram illustrates the data flow for:

• Domain event channel messaging (shown in red)
• Batch projection builder (shown in green)

Domain Event Messaging Data Flow

This data flow is triggered when a supported catalog entity is created, updated, or deleted through Import/Export, Commerce Manager, or the Data Sync Tool:

1. Elastic Path Commerce detects the change through an OpenJPA Transaction Listener, and writes a record to the TOUTBOXMESSAGE table.
2. Batch Server runs a job every second that reads all records from the TOUTBOXMESSAGE table and relays the events to the Domain Event Channel.
3. Integration Server consumes the message from the Domain Event Channel. The message contains a Global Unique Identifier (GUID) for the modified entity.
4. Integration Server reads the latest version of the entity from the database, builds a projection, and saves it to theTCATALOGPROJECTIONS table.
5. Integration Server sends a catalog event message to Catalog Event Channel.
6. A syndication consumer reads the catalog event message from their Virtual Destination Consumer of the Catalog Event Channel. The message contains a Global Unique Identifier (GUID) for the modified entity.
7. The syndication consumer uses the GUID to fetch the projection using the Syndication API.

Batch Projection Builder Data Flow

This data flow is triggered when a message is received on the Bulk Domain Event Channel:

1. Integration Server consumes the message from the Bulk Domain Event Channel. The message contains a list of Global Unique Identifiers (GUIDs) for the modified entities.
2. Integration Server reads the latest versions of the entities from the database, builds projections, and saves them to theTCATALOGPROJECTIONS table.
3. Integration Server sends catalog event messages to Catalog Event Channel.
4. A syndication consumer reads the catalog event messages from their Virtual Destination Consumer of the Catalog Event Channel. The messages contain Global Unique Identifiers (GUIDs) for each modified entity.
5. The syndication consumer uses the GUIDs to fetch the projections using the Syndication API.

Outbox Pattern to ensure Transactional Integrity

Catalog entity changes are always persisted to the Elastic Path database within a transaction. This transaction ensures that changes are written atomically, and partial database updates are rolled back if there is an error while writing to the database.

You want to ensure that domain events are only sent if the catalog entity changes are successfully written to the database. To provide this guarantee, Elastic Path Commerce leverages the Outbox Pattern.

When catalog entities are being updated, a lifecycle listener detects the change and injects an insert into the outbox table into the transaction. If the transaction rolls back, the outbox record is also rolled back. A Batch Server job polls this table every second and relays any messages to JMS.

note

Some previous versions of Elastic Path used a distributed transaction approach (XA transactions via Atomikos) to provide transactional integrity. That approach was found to add unnecessary complexity to the application, so we switched to the Outbox pattern described above.

Catalog Syndication Data Model

This section provides details of the catalog syndication data model and the type of data available for synchronization and consumption.

Data model

The following diagram illustrates the basic data model for projections, with category and offer projections partially flattened or denormalized, to simplify consumption:

Catalog Projections

When you create, edit, or delete an entity, the following projections are created for the corresponding catalog entity:

• Category: Organizes products within a catalog or within another category
• Offer: Represents any tangible or non-tangible collection of items for sale in a store
• Attribute: Represents a trait or characteristic of an offer, item or category
• Brand: Represents the brand to which an offer is associated
• Option: Represents values to distinguish between items in an offer, such as color and size
• Field metadata: Represents metadata used to describe fields that are required when adding an item to the cart