Skip to main content
RichRelevance

Offline Data Feed

What is offline data?

Offline data is merchant information that is not directly collected by Algonomy. Some examples include in-store (POS) transactions, call center transactions, marketing segments, other marketing data, and social media interactions.

Offline data powers some key elements of Algonomy omnichannel personalization:

  • Cross-channel attribution: Algonomy algorithms use a closed loop optimization cycle to determine what’s working. A recommendation or online session that is finished in an offline transaction should be considered a success, not a failure. Siloed data – this is when we don’t take offline transactions into account – may suggest failure. As an additional bonus, we provide omnichannel attribution reporting to you.
  • Omnichannel data for smarter analysis: When targeting products and making offers to your shoppers, you should consider as much data as possible. Offline data is important when selecting appropriate upsells and cross-sells. Our algorithms can better predict behavior when looking at all trends of your shoppers’ behaviors.

What are the benefits of offline data analysis?

  • Immediate: shoppers who buy offline and are recognized online receive more relevant recommendations on home and category pages. The pages are tailored for cross-sells. (Conversion)
  • Immediate: price, brand, category, size, and style preferences for recognized online shoppers become more comprehensive. (Conversion, AOV)
  • Mid-term: Algonomy predictive models that use omnichannel data are smarter by having more shopper information.
  • Long-term: personalization and loyalty are tied together.

Support for Importing Offline Data through APIs

Algonomy Omnichannel Personalization allows you to import offline data through APIs. For more information see: Omnichannel Order API.

Validating and Uploading the Feed

Before scheduling the feed for regular uploading, generate a sample feed. Validate the feed to confirm that the structure is correct. After the feed is validated, send the sample to the Algonomy integration team for processing. Once the test file has been successfully processed, the production file should be scheduled for upload on an agreed schedule, typically once per day in the early morning (between 12am and 4am).

All files for the feed will be uploaded as a single compressed file (.zip or .gz). Files should be sent to the Algonomy FTP server (your integration team will provide you with your FTP credentials).

Bundling Multiple Files

Multiple files are to be delivered in a single .zip file. In this case, name the bundle with a consistent prefix and the _SITENAMEONEWORD_SITEID_YYYY_MM_DD_vNUM postfix, but the files within should retain the standard names above.

For example, a zip file with the name offline_orders_myshop_123_2013_01_12_v1.zip may contain the files

  • orders_myshop_123_2013_01_12_v1.txt
  • order_line_items_myshop_123_2013_01_12_v1.txt
  • returned_orders_myshop_123_2013_01_12_v1.txt
  • returned_line_items_myshop_123_2013_01_12_v1.txt

Coordinate with your Algonomy team to choose the prefix you’ll use.

Standard Naming Convention and Formats by File Type

You will receive a site ID for your site(s) during the setup process. Each filename should include both your site name and your site ID.* 

Additionally, if you plan to upload multiple zip files each day, it is a good idea to use versioning in the filenames. For feed types which overwrite data (e.g., store data), the version signals to the system that we’re progressing. For feed types which can be appended incrementally through the day (e.g., orders where the first file is for purchases from midnight-11 AM, and the second is from 11 AM to midnight) the versions signal incrementally. If a version goes backwards, the system will archive the file and flag an error.

For example:

  • MyShop.com is assigned SiteID 123
  • MyShop desires to upload an order feed
  • The order feed uses the filename format: orders_SITENAMEONEWORD_SITEID_YYYY_MM_DD_vNUM.zip

Compressed* feed file:        purchases_myshop_123_2013_01_12_v1.zip

Inside of the compressed file:         purchases_myshop_123_2013_01_12_v1.txt

*Note: The file can be compressed as .zip or .gz, with the extension matching the type of compression.

Below are the standard feed file names for each standard type. Every file should end with the short site name, Site ID, Date, and version fragment, e.g. for MyShop.com, site 123 and January 12, 2013:      _myshop_123_2013_01_12_v1.txt

The version fragment (‘v1′ above) can be a timestamp or other increasing value instead of a number. The important part of the version is distinguishing the order of arrival of multiple feeds in a single day.

*Please note that the 'site name' is technically your ftp login name.  If there has been any changes to your ftp login name, please use that instead of
"site name".

Feed Files

Edit section
Feed File Details Filename pattern
Order Feed A list and description of every order made over the decided time period. orders_sitename_siteID_YYYY_MM_DD_v#.txt
Order Line Item Feed Line item details for each order in the Order Feed. order_line_items_sitename_siteID_YYYY_MM_DD_v#.txt
Returned Order Feed A list and description of every return made over the decided time period. returned_orders_sitename_siteID_YYYY_MM_DD_v#.txt
Returned Line Item Feed Line item details for each return in the Returned Order Feed. returned_line_items_sitename_siteID_YYYY_MM_DD_v#.txt
Shopper GUID Mapping Feed A mapping between disparate shopper guids to a central, unifying shopper id that you specify. shopper_guid_mapping_sitename_siteID_YYYY_MM_DD_v#.txt

Order and Return Feeds

Orders and returns are separate feeds with different data. Offline purchases are broken down into orders, individual receipts, or transactions.

Order data consists of the purchase date, shopper information, and the total cost of all products for that transaction.

Orders are line items that represent different products purchased by the shopper. Think of it as an itemized receipt: the order is the receipt, and a line item is one product.

Order Feed: this describes entire orders.

Line Item Feed: this describes individual line items.

Order Feed

Filename: orders_sitename_siteID_YYYY_MM_DD_v#.txt required

Name Type Required? Definition
order_id ASCII Yes Numeric value for each order. At least unique by channel.
channel_id ENUM DEPENDS The identifier for the channel of this shopper mapping, example, “Store”.  If you are mapping website shopper guids to shopper IDs, the channel ID should be provided that matches the channel ID in the Shopper Guid mapping file. 
shopper_guid text DEPENDS
Required only if shopper_id is not provided.

Shopper guid that maps to a shopper ID in a given channel. Refer Shopper GUID Mapping Feeds, below. This only required if shopper ID is not provided. 

Important: A separate user mapping feed must be provided. 

Note: Either shopper_guid or shopper_id are required.  If both are blank, the row will be ignored. If neither can be provided, but the retailer still wants the data included, a unique arbitrary ID can be provided.

shopper_id text Depends
Required only if shopper_guid is not provided.

Global Unique Shopper ID. Shopper IDs in the offline feed must match shopper IDs received through site instrumentation.
 

Note: Either shopper_guid or shopper_id are required.  If both are blank, the row will be ignored. If neither can be provided, but the retailer still wants the data included, a unique arbitrary ID can be provided.

date_time ISO DATETIME Yes A date and time of the transaction in ISO 8601 compatible format. 

Example: 2007-04-25T14:30:00+09:00
email_hash text No A hash of the customer’s email address, if POS captures email at transaction time. Hashed out for PII standards.
cc_hash text No A fully obfuscated credit card identifier (See Data Obfuscation discussion above).
payment_method ENUM No For example, cash, credit, check, digital, other. Useful for segmentation and targeting
store_id text No A store in which the transaction occurred.
region_id Alphanumeric No The region_id in which the transaction occurred, as specified in the Region Full Feed
segment_id text No

A unique identifier for the segment the shopper belongs to. A segment is a group of shoppers with a common demographic or demonstrated pattern to which promotions can be targeted based on affinities and patterns specific to that group.

Note: Multiple segments can be sent in this field. The standard delimiter is a comma.

segment_name text No

A human-friendly name for the segment.

Note: Multiple segment names can be sent in this field. The standard delimiter is a comma.

currency ENUM No

The ISO currency code representing the currency used for the order. 

Note: if not specified, this defaults to US currency (USD).

Don't add the currency column in the header if you don't want to send the appropriate ISO currency code against each row. By default, system sets the currency to USD. If you add the currency column in the header, the default currency won't be applied.

coupon_code text No Was a coupon applied to the entire order? If so, you can record the coupon code here.
coupon_value NUMBER No Monetary value of the coupon applied to the entire order.

Order Line Item Feed

Edit section

Filename: order_line_items_sitename_siteID_YYYY_MM_DD_v#.txt required

Name Type Required? Definition
order_id ASCII Yes As defined above. Required to associate the line item data with the order data above.
channel_id ENUM No The identifier for the channel of this shopper mapping, e.g. “Store”.  If you are mapping website shopper IDs to shopper GUIDs, then the value of the channel ID is determined by the “Omnichannel Web Channel” site configuration value.
product_id text Yes Unique ID for a product. This should be a key to look up into the catalog.
sku_id text No SKU for the line item. Should correspond to a value provided in the SKU Feed
quantity INTEGER Yes Quantity of the items purchased.  This value needs to be an integer.
unit_price NUMBER Yes Monetary value paid per unit
coupon_code text No Was a coupon code used for this individual purchase/line? If so, you can record the coupon code here.
coupon_value NUMBER No Monetary value of the coupon applied to this individual line item.

Returned Order Feed

Filename: returned_orders_sitename_siteID_YYYY_MM_DD_v#.txt

Field Name Type Required? Definition
order_id ASCII Yes As defined above.
return_id text Yes As defined above. Required if no Order ID available.
channel_id ENUM DEPENDS The identifier for the channel of this shopper mapping, e.g. “Store”.  If you are mapping website shopper guids to shopper IDs, the channel id should be provided that matches the channel id in the Shopper Guid mapping file. 
shopper_guid text

DEPENDS
Yes, but only if shopper_id is not provided

Important: A separate user mapping feed must be provided. 

Shopper GUID as defined above in the orders feed.
shopper_id text DEPENDS As defined above in the orders feed. 
date_time ISO DATETIME Yes

A date and time of the transaction in ISO 8601 compatible format. 

Example: 2007-04-25T14:30:00+09:00

Note:- The date_time format should match the ISP 8601 compatible format.

email_hash text No A hash of the customer’s email address, if POS captures email at transaction time.
store_id text Yes A store in which the transaction occurred
segment_id text No A unique identifier for the segment the shopper belongs to. A segment is a group of shoppers with a common demographic or demonstrated pattern to which promotions can be targeted based on affinities and patterns specific to that group.
segment_name text No A human-friendly name for the segment.
return_value NUMBER Yes Monetary value of the return.
currency ENUM No

The ISO currency code representing the currency used for return_value

Note: if not specified, this defaults to US currency (USD).

Don't add the currency column in the header if you don't want to send the appropriate ISO currency code against each row. By default, system sets the currency to USD. If you add the currency column in the header, the default currency won't be applied.

Returned Line Item Feed

Edit section

Filename: returned_line_items_sitename_siteID_YYYY_MM_DD_v#.txt

Name Type Required? Definition
order_id ASCII Yes As defined above.
return_id text Yes Unique Return ID. Required if no Order ID avaliable.
channel_id ENUM Yes The identifier for the channel of this shopper mapping, e.g. “Store”.  If you are mapping website shopper IDs to shopper GUIDs, then the value of the channel ID is determined by the “Omnichannel Web Channel” site configuration value.
product_id text Yes Unique ID for a product. This should be a key to look up into the catalog.
sku_id text No SKU for the line item. Should correspond to a value provided in the SKU Feed
quantity INTEGER Yes Quantity of the item(s) returned. This value needs to be an integer.
unit_value NUMBER Yes Monetary value refunded per unit returned.
credit_type ENUM No Did this result in chargeback, cash back, store credit, etc.

Orders and Returns Events

Orders and returns in the offline world are events that occur at a specific time and place, and can be tracked and ingested in real-time, or as part of a feed. Unlike the feeds below, orders and returns are things that have happened, rather than descriptions of things that exist, such as products, shoppers, or stores.

At Algonomy, we  refer to data about things that have happened event data and have a globally distributed event ingestion system for online events. We call data about things that exist as entity data. Currently, we accept up to 14 days' worth of Omnichannel event data daily through batch data feeds. In the future, we anticipate that Omnichannel event data will be consumed via real-time data ingestion instead of batch data feeds.

Note: The platform will process only offline orders and returns data that was created within the past 14 days from the import date.

Shopper GUID Mapping Feed

Edit section

Filename: shopper_guid_mapping_sitename_siteID_YYYY_MM_DD_v#.txt

If the merchant does not include shopper IDs in the offline feed, the transaction information will be used to create the global omnichannel strategies. The feed needs to have shopper IDs for mapping to be able to present personalized omnichannel strategies.

Ideally, a merchant would have a globally-unique shopper identifier that functions across channels. Realistically, merchants have some systems and channels operating with such a shared identifier, and other systems and channels with their own shopper identifiers.

A single shopper GUID mapping feed can map shopper GUIDs from various channels to the Shopper ID, where this mapping is known. This is the simplest way to connect users across channel when possible.

You can use this mapping feed to map disparate shopper GUIDs to a single, known shopper ID. Each line in this file maps to one shopper ID, and multiple lines can be used to map multiple shopper GUIDs to the same shopper ID.

Name Type Required? Definition
shopper_guid ASCII Yes The per channel shopper GUID. 
channel_id ENUM Yes The identifier for the channel of this shopper mapping, e.g. “Store”.  
shopper_id text Yes The globally unique shopper ID.

Offline Feed Format, Packaging, and Delivery

A single feed is represented by a delimited text file in UTF-8 encoding without BOM (byte order mark). The file contains a single header row with the header labels for each column. Duplicate column names are not allowed, and data is matched to columns by name. The required fields must be included. Fields may be given in any order and unused ‘optional’ fields simply omitted.

The file has rows delimited by the ASCII newline character, and columns delimited by the ASCII pipe character. By default there is no quoting or escaping of values, so any fields that contain newline or pipe characters must first remove or substitute these characters. Alternatively the line and column delimiters can be configured to be non-printable ASCII characters if necessary, making it easy to remove any occurrences of these characters without visibly altering field text.

For example, the first two lines of an Order Feed might look like:

order_id|shopper_guid|channel_id|store_id|date_time|coupon_code|shopper_id
abc123xyz|987654321ab|store|s555-333|234.99|2007-04-25T14:30:00Z|jan_promo5|null

One-Time Historical Transaction Upload

Clients can provide a one-time historical transaction upload with more than 14 days of transactions. The only difference: the one-time filename starts with “historical_”. If you have already sent daily feeds, this one-time file must exclude transactions already delivered. They will not be de-duplicated.

Important: The one-time historical transaction upload runs once weekly on Sunday. 

Example filename: historical_offline_orders_myshop_123_2013_01_12_v1.zip.

Upload this file to the same FTP directory. Everything else remains the same.

Note: The sub files do not require the “historical_” prefix.

  • orders_myshop_123_2013_01_12_v1.txt
  • order_line_items_myshop_123_2013_01_12_v1.txt
  • returned_orders_myshop_123_2013_01_12_v1.txt
  • returned_line_items_myshop_123_2013_01_12_v1.txt
  • Was this article helpful?