Scoped Action
Introduction
The objective is ensure data integrity and automatically update or replace items and delete "orphan" items that are not updated or replaced.
Deletions are done in batches of 1,000 while updates or replacements are in batches of 50. That will be reflected in the status for each service that is involved like the streaming-engine, legacy catalog and streaming-find-indexer. There is a new service, the streaming-scoped-action which will provide a summary of items updated, replaced and deleted.
There are currently four types of Scoped Actions which are part of the snapshot service. A summary is provided below:
Sync Legacy Catalog to the View Store (snapshot)
The first two types of scoped actions ensure data integrity for the legacy catalog (portal). Currently the legacy catalog is not snapshot aware, which means that products and categories from previous snapshots may still be in the legacy catalog and show up in the catalog. Items from either complete or active snapshots are accepted by the legacy catalog and because it is not snapshot aware the product information may not be in sync with the current snapshot. There could be products from older snapshots that are no longer required.
There are two options:
Full Scoped Action
Sync up both products and categories from the view store (specific snapshot) to the legacy catalog to items in a specific snapshot. The Full Scoped Action replaces all items from the snapshot into the legacy catalog and keeps track of which products and categories were "touched". The scoped action will close itself once it finishes replacing all items in the legacy catalog and will then delete any "untouched" items in the legacy catalog. Please note for large catalogs, this scoped action can take time and has a high overhead. Please consult with your Algonomy consultant or customer success if your catalog has over 1 million products.
Property Sync Scoped Action
If properties seem out of sync between the legacy catalog and the view store, it is possible to "update" specific properties and sync them up. This type of scope action has low overhead.
Scoped Action to Sync a Subset of a Snapshot to an incoming Set of Products during Ingestion and delete "untouched" products in the snapshot.
The third type of scoped action happens within one snapshot. It is when one wants to provide a new set of products for a subset of the current snapshot and have the scoped action determine which items are orphans and delete them from the snapshot as well as update products. If a user only want to update or replace products, a scoped action is not required as the streaming catalog already compares incoming items to the current items in the snapshot and sends the deltas to engine.out.
This type of scoped action is only available for a"subset" of the products identified by an attribute. The attribute must be defined in the product property definition collection with the "scopedActionSettings" parameter.
One use case is for a B2B who may have stores who are supplying them with a full catalog and they do not know which items are no longer in the current snapshot. They could have an attribute for "store" and send in the the full catalog for that store. The scoped action will "touch" each item it updates in the snapshot. When the ingestion is finished, a "close" command is executed and the service then checks for any "untouched" items in the snapshot and deletes them.
See more details on the steps for a scoped action within one snapshot.
Sync Legacy Catalog to the View Store (snapshot)
Full Scoped Action to sync the Legacy Catalog
The first step to sync the Legacy Catalog to the products and categories for a specific snapshot, is to open a scoped action referencing the snapshot and provide a "scopedActionDefinition" in the body of the call.
A scopedActionDefinition provides instructions and is required for both product and category items. Both products and categories and their attributes need to be updated in order to keep the legacy catalog and snapshot in sync.
Once the scoped action is "open", it will then replace all the legacy catalog's products and categories from the snapshot. It keeps track of items touched. Once the replacement of all items is completed, the scope action automatically will close the scope action and delete any "untouched" items that are still in the legacy catalog.
There will be status messages from the streaming-engine, the legacy catalog adaptor service as well as from the streaming-scoped-action-service. The the streaming-scoped-action service details when a scoped action is open up to when it is complete.
For this scoped action there is only one "open" command to execute.
Note:
- Only one scoped action to sync a Legacy Catalog can run at one time.
- Deletions are done in batches of 1,000 while replacements are in batches of 50.
- Each scoped action has different requirements for the scopedActionDefinition
Scoped Action "open" call and scopedActionDefinition
The scopedActionDefinition requires specific parameters for action by item type. Both "product" and "category" itemTypes are mandatory and "referenceConntent" is optional.
onOpen - specify the type of scoped action which is "allLegacyCatalog". The Legacy Catalog is the target and the action is "replaceAll". If the onOpen is not provided the items will be not be replaced but a comparison is still done and items with the same "id" will be touched. It is recommended to replaceAll.
onClose - after the products and categories have been updated, the scoped action will automatically close and delete "untouched" orphans that remain in the Legacy Catalog.
Syntax for scopedActionDefinition which is the body of call is below.
{
"<itemType>" : {
"onOpen": {
"onExit": "complete",
"scope": {
"type": "allLegacyCatalog"
},
"action": {
"type": "replaceAll"
}
},
"onClose" : {
"scope" : {
"type" : "allLegacyCatalog"
},
"action" : {
"type" : "deleteUntouched"
}
}
}
}
The API Call to "open" a scoped action:
POST https://<host>/streaming-snapshot/v1/<apiKey>/action/<snapshotId>/<snapshotType>/open
Below is an example with the required body's scopedActionDefinition. In this case the apiKey is 1234, snapshot ID is 8000, and it is a product snapshot.
POST https://<host>/streaming-snapshot/v1/1234/action/8000/product/open
body
{
"product": {
"onOpen": {
"onExit": "complete",
"scope": {
"type": "allLegacyCatalog"
},
"action": {
"type": "replaceAll"
}
},
"onClose": {
"scope": {
"type": "allLegacyCatalog"
},
"action": {
"type": "deleteUntouched"
}
}
},
"category": {
"onOpen": {
"onExit": "complete",
"scope": {
"type": "allLegacyCatalog"
},
"action": {
"type": "replaceAll"
}
},
"onClose": {
"scope": {
"type": "allLegacyCatalog"
},
"action": {
"type": "deleteUntouched"
}
}
}
}
Response:
The response includes a scopedActionId, trackingId, state of the scoped action and repeats the body of the call.
In this case the siteId is 1234, snapshotID is 8000 and the state is "Opening_started". The scopedActionId is: 1045
{
"statusTracker": {
"trackingId": "9df65ae2-bba4-11ea-9929-eb0f60272db0",
"trackingInstant": "2020-07-01T14:10:28.742013Z"
},
"scopedAction": {
"id": 1045,
"siteId": 1234,
"snapshotId": 8000,
"trackingId": "9df65ae2-bba4-11ea-9929-eb0f60272db0",
"state": "Opening_started",
"definition": {
"product": {
"onOpen": {
"onExit": "complete",
"scope": {
"type": "allLegacyCatalog"
},
"action": {
"type": "replaceAll"
}
},
"onClose": {
"scope": {
"type": "allLegacyCatalog"
},
"action": {
"type": "deleteUntouched"
}
}
},
"category": {
"onOpen": {
"onExit": "complete",
"scope": {
"type": "allLegacyCatalog"
},
"action": {
"type": "replaceAll"
}
},
"onClose": {
"scope": {
"type": "allLegacyCatalog"
},
"action": {
"type": "deleteUntouched"
}
}
}
},
"lastModified": "2020-07-01T14:10:28.721655Z",
"trackingInstant": "2020-07-01T14:10:28.742013Z"
}
}
Check State of Scoped Action for Legacy Catalog Sync
Check the state of a scoped action through the streaming-snapshot service. You can find out if it is completed.
GET https://<host>/streaming-snapshot/v1/<apiKey>/action/<snapshotId>/<snapshotType>/<scopedActionId>
Example when scoped action is complete:
apiKey: 1234, snapshotId: 8000 and scopedActionID: 1045
GET https://<host>/streaming-snapshot/v1/1234/action/8000/product/1045
Response
{
"id": 1045,
"siteId": 1234,
"snapshotId": 8000,
"trackingId": "9df65ae2-bba4-11ea-9929-eb0f60272db0",
"state": "Complete",
"definition": {
"product": {
"onOpen": {
"onExit": "complete",
"scope": {
"type": "allLegacyCatalog"
},
"action": {
"type": "replaceAll"
}
},
"onClose": {
"scope": {
"type": "allLegacyCatalog"
},
"action": {
"type": "deleteUntouched"
}
}
},
"category": {
"onOpen": {
"onExit": "complete",
"scope": {
"type": "allLegacyCatalog"
},
"action": {
"type": "replaceAll"
}
},
"onClose": {
"scope": {
"type": "allLegacyCatalog"
},
"action": {
"type": "deleteUntouched"
}
}
}
},
"lastModified": "2020-07-01T14:10:38.195171Z",
"trackingInstant": "2020-07-01T14:10:28.742013Z"
}
Check Status for Full Scoped Action's replacements and deletions
A scoped action will send all updates through the streaming engine which will go to the legacy catalog. This includes both replacements and deletions. Each service will record a status as it does when using the streaming-ingest service.
The streaming-scoped-action service provides a summary of actions taken.
To see the status messages, use the standard streaming-status endpoints. Below are two examples but further information can be found in the streaming-status section of this manual.
See all status messages, including legacy catalog, streaming engine and streaming-scoped-action service, based on the trackingId for the scoped action.
GET https://<host>/streaming-status/v1/<apiKey>/trackingID?<trackingId>
Another option is to return the status messages for one specific service, like the legacy catalog (streaming-consumer-legacy-catalog). However note that this is not based on trackingId but rather date and you will see all status messages from that day and not just the status messages for the scoped action.
You can restrict it to the streaming-scoped-action to see just the scoped action updates.
GET https://<host>/streaming-snapshot/v1/<apiKey>/daySnapshotService/<snapshotId>?date=yyyy-mm-dd&service=<streaming service
Example of using the streaming-scoped-action service and date:
There is a status from when the scoped action was opened up to where it is done.
apiKey: 1234, snapshotId: 8000
GET https://<host>/streaming-status/v1/1234/daySnapshotService/8000?date=2020-07-01&service=streaming-scoped-action
Response:
[
{
"siteId": 1234,
"snapshotId": 8000,
"statusId": "a39b0e39-bba4-11ea-b9e0-73b56c2e6889",
"trackingId": "9df65ae2-bba4-11ea-9929-eb0f60272db0",
"source": "streaming-scoped-action",
"statusType": "ScopedActionEvent",
"message": "Done Closing Scoped Action 1045",
"level": "SUMMARY",
"statusData": {
"stats": {
"product": {
"untouched": 8000,
"touched": 12247,
"total": 20247
},
"category": {
"untouched": 7,
"touched": 542,
"total": 549
}
},
"scopedActionId": 1045
},
"datacenter": "qa",
"trackingInstant": "2020-07-01T14:10:28.742013Z",
"msSinceRequest": 9468,
"statusInstant": "2020-07-01T14:10:38.210002500Z"
},
{
"siteId": 1234,
"snapshotId": 8000,
"statusId": "a0a35249-bba4-11ea-9aa2-f9a2aaf3c470",
"trackingId": "9df65ae2-bba4-11ea-9929-eb0f60272db0",
"source": "streaming-scoped-action",
"statusType": "ScopedActionEvent",
"message": "CLOSING ScopedAction 1045",
"level": "SUMMARY",
"statusData": {},
"datacenter": "qa",
"trackingInstant": "2020-07-01T14:10:28.742013Z",
"msSinceRequest": 4489,
"statusInstant": "2020-07-01T14:10:33.231008900Z"
},
{
"siteId": 1234,
"snapshotId": 8000,
"statusId": "a0985572-bba4-11ea-b9e0-a55756661850",
"trackingId": "9df65ae2-bba4-11ea-9929-eb0f60272db0",
"source": "streaming-scoped-action",
"statusType": "ScopedActionEvent",
"message": "Sent Closing_started Scoped Action 1045",
"level": "SUMMARY",
"statusData": {},
"datacenter": "qa",
"trackingInstant": "2020-07-01T14:10:28.742013Z",
"msSinceRequest": 4417,
"statusInstant": "2020-07-01T14:10:33.159000200Z"
},
{
"siteId": 1234,
"snapshotId": 8000,
"statusId": "a096a8bf-bba4-11ea-b9e0-6507fd0e4f45",
"trackingId": "9df65ae2-bba4-11ea-9929-eb0f60272db0",
"source": "streaming-scoped-action",
"statusType": "ScopedActionEvent",
"message": "Sent Open Scoped Action 1045",
"level": "SUMMARY",
"statusData": {
"stats": {
"product": {
"total": 12247,
"replaced": 12247
},
"category": {
"total": 542,
"replaced": 542
}
},
"scopedActionId": 1045
},
"datacenter": "qa",
"trackingInstant": "2020-07-01T14:10:28.742013Z",
"msSinceRequest": 4406,
"statusInstant": "2020-07-01T14:10:33.148025500Z"
},
{
"siteId": 1234,
"snapshotId": 8000,
"statusId": "9e030513-bba4-11ea-9929-035efe94f10c",
"trackingId": "9df65ae2-bba4-11ea-9929-eb0f60272db0",
"source": "streaming-scoped-action",
"statusType": "ScopedAction",
"message": "ScopedAction 1045 OPENING_STARTED",
"level": "SUMMARY",
"statusData": {
"scopedAction": "Opening_started"
},
"datacenter": "qa",
"trackingInstant": "2020-07-01T14:10:28.742013Z",
"msSinceRequest": 83,
"statusInstant": "2020-07-01T14:10:28.825013100Z"
}
]
Example of using the legacy catalog service and date:
apiKey: 1234, snapshotId: 8000
GET https://<host>/streaming-status/v1/1234/daySnapshotService/8000?date=2020-07-01&service=streaming-consumer-legacy-catalog
Partial response to show replacements and deletions. Deletions are sent in batchs of 1,000.
The order below is the default, the most recent to the older status messages.
[
{
"siteId": 1234,
"snapshotId": 8000,
"statusId": "cae187dc-bba4-11ea-bdb4-fdc35df300ac",
"trackingId": "9df65ae2-bba4-11ea-9929-eb0f60272db0",
"source": "streaming-consumer-legacy-catalog",
"statusType": "InternalEnvelope",
"message": "Received delete action",
"kafkaSource": {
"topic": "streaming.engine.out",
"partition": 6,
"offset": 3050978
},
"level": "SUMMARY",
"statusData": {
"stats": {
"product": {
"total": 1000,
"delete": 1000
}
}
},
"datacenter": "qa",
"trackingInstant": "2020-07-01T14:10:28.742013Z",
"msSinceRequest": 75361,
"statusInstant": "2020-07-01T14:11:44.103010800Z"
},{
"siteId": 1234,
"snapshotId": 8000,
"statusId": "cadcccc8-bba4-11ea-bdb4-79da9b1a189e",
"trackingId": "9df65ae2-bba4-11ea-9929-eb0f60272db0",
"source": "streaming-consumer-legacy-catalog",
"statusType": "InternalEnvelope",
"message": "Received replace action",
"kafkaSource": {
"topic": "streaming.engine.out",
"partition": 6,
"offset": 3050964
},
"level": "SUMMARY",
"statusData": {
"stats": {
"category": {
"total": 2,
"replace": 2
}
}
},
"datacenter": "qa",
"trackingInstant": "2020-07-01T14:10:28.742013Z",
"msSinceRequest": 75330,
"statusInstant": "2020-07-01T14:11:44.072007200Z"
},
{
{
"siteId": 1234,
"snapshotId": 8000,
"statusId": "cadc7ea2-bba4-11ea-bdb4-ededc8264fc0",
"trackingId": "9df65ae2-bba4-11ea-9929-eb0f60272db0",
"source": "streaming-consumer-legacy-catalog",
"statusType": "InternalEnvelope",
"message": "Received replace action",
"kafkaSource": {
"topic": "streaming.engine.out",
"partition": 6,
"offset": 3050962
},
"level": "SUMMARY",
"statusData": {
"stats": {
"product": {
"total": 22,
"replace": 22
}
}
},
"datacenter": "qa",
"trackingInstant": "2020-07-01T14:10:28.742013Z",
"msSinceRequest": 75328,
"statusInstant": "2020-07-01T14:11:44.070006600Z"
},
{
"siteId": 1234,
"snapshotId": 8000,
"statusId": "ca730e6f-bba4-11ea-bdb4-f7aba951abf4",
"trackingId": "9df65ae2-bba4-11ea-9929-eb0f60272db0",
"source": "streaming-consumer-legacy-catalog",
"statusType": "InternalEnvelope",
"message": "Received replace action",
"kafkaSource": {
"topic": "streaming.engine.out",
"partition": 6,
"offset": 3050961
},
"level": "SUMMARY",
"statusData": {
"stats": {
"product": {
"total": 29,
"replace": 29
}
}
},
......
Sync Product Properties in the Legacy Catalog with the View Store
The view store is the source of truth for product and category properties. It will always have the most up to date information for a specific snapshot. If the properties in the legacy catalog (Recommend catalog) seem out of sync and may have not been updated it is possible to sync up specific properties with the view store.
This scoped action is cost-effective, thus, can be used by clients of all sizes but it is recommended to discuss with your Algonomy consultant or support to ensure there aren't other issues.
Note: Always use the active snapshot for this scoped action.
Methods: POST
End Point: https://<host>/streamingsnapshot/v1/<apiKey>/action/<snapshotID>/product/open
Scoped Action Property Sync Open Call
The following example shows how to update the Price and Sale Price of the products in the Legacy Catalog with the View Store. In this case the snapshotId is 9758
POST: http://<host>/streaming-snapshot/v1/<apiKey>/action/9758/product/open
Request Body
{
"product": {
"onOpen": {
"onExit": "complete",
"scope": {
"type": "allLegacyCatalog"
},
"action": {
"type": "updateAll",
"properties": ["sale_price", "price"]
}
}
}
}
Response Body
{
"statusTracker": {
"trackingId": "7a767e85-7563-11eb-9657-0f6ec3418f91",
"trackingInstant": "2021-02-22T23:12:48.173018100Z"
},
"scopedAction": {
"id": 2593,
"siteId": 608,
"snapshotId": 9758,
"trackingId": "7a767e85-7563-11eb-9657-0f6ec3418f91",
"state": "Opening_started",
"definition": {
"product": {
"onOpen": {
"onExit": "complete",
"scope": {
"type": "allLegacyCatalog"
},
"action": {
"type": "updateAll",
"properties": [
"sale_price",
"price"
]
}
}
}
},
"lastModified": "2021-02-22T23:12:48.184593Z",
"closingScope": {},
"trackingInstant": "2021-02-22T23:12:48.173018100Z",
"selfClosing": true
}
}
Check Status Service
Check the Status Service using Tracking ID, check:
- Batches with a finished status.
- The legacy catalog consumer and the scoped action services.
Tracking ID: 7a767e85-7563-11eb-9657-0f6ec3418f91
GET https://<host>/streaming-status/v1/<apiKey>/trackingId/7a767e85-7563-11eb-9657-0f6ec3418f91
[
{
"siteId": 608,
"snapshotId": 9758,
"statusId": "7df74028-7563-11eb-b54e-d1cd4ac0e2f6",
"trackingId": "7a767e85-7563-11eb-9657-0f6ec3418f91",
"source": "streaming-consumer-legacy-catalog",
"statusType": "ScopedActionEvent",
"message": "Done Opening scoped action 2593",
"kafkaSource": {
"topic": "streaming.engine.out",
"partition": 7,
"offset": 122812
},
"level": "SUMMARY",
"statusData": {},
"datacenter": "qa[Algonomy2] ",
"statusInstant": "2021-02-22T23:12:54.050000800Z",
"trackingInstant": "2021-02-22T23:12:48.173018100Z",
"msSinceRequest": 5877
},
{
"siteId": 608,
"snapshotId": 9758,
"statusId": "7df6caf7-7563-11eb-b54e-21fcdaa9984f",
"trackingId": "7a767e85-7563-11eb-9657-0f6ec3418f91",
"source": "streaming-consumer-legacy-catalog",
"statusType": "InternalEnvelope",
"message": "Finished InternalEnvelope",
"kafkaSource": {
"topic": "streaming.engine.out",
"partition": 7,
"offset": 122811
},
"level": "SUMMARY",
"statusData": {
"cursor": {
"batch": 2,
"finished": true
}
},
"datacenter": "qa",
"statusInstant": "2021-02-22T23:12:54.047000700Z",
"trackingInstant": "2021-02-22T23:12:48.173018100Z",
"msSinceRequest": 5874
},
{
"siteId": 608,
"snapshotId": 9758,
"statusId": "7df607a5-7563-11eb-b54e-3bf550836285",
"trackingId": "7a767e85-7563-11eb-9657-0f6ec3418f91",
"source": "streaming-consumer-legacy-catalog",
"statusType": "InternalEnvelope",
"message": "Finished InternalEnvelope",
"kafkaSource": {
"topic": "streaming.engine.out",
"partition": 7,
"offset": 122810
},
"level": "SUMMARY",
"statusData": {
"cursor": {
"batch": 1,
"finished": false
}
},
"datacenter": "qa",
"statusInstant": "2021-02-22T23:12:54.042000500Z",
"trackingInstant": "2021-02-22T23:12:48.173018100Z",
"msSinceRequest": 5869
},
{
"siteId": 608,
"snapshotId": 9758,
"statusId": "7ad45542-7563-11eb-9571-67d51845af27",
"trackingId": "7a767e85-7563-11eb-9657-0f6ec3418f91",
"source": "streaming-scoped-action",
"statusType": "ScopedActionEvent",
"message": "Done Closing Scoped Action 2593",
"level": "SUMMARY",
"statusData": {
"cursor": {
"batch": 1,
"finished": true
},
"stats": {},
"scopedActionId": 2593
},
"datacenter": "qa",
"statusInstant": "2021-02-22T23:12:48.788000200Z",
"trackingInstant": "2021-02-22T23:12:48.173018100Z",
"msSinceRequest": 615
},
{
"siteId": 608,
"snapshotId": 9758,
"statusId": "7ad42f0b-7563-11eb-ae8f-abf6c89fec9c",
"trackingId": "7a767e85-7563-11eb-9657-0f6ec3418f91",
"source": "streaming-scoped-action",
"statusType": "ScopedActionEvent",
"message": "CLOSING ScopedAction 2593",
"level": "SUMMARY",
"statusData": {},
"datacenter": "qa",
"statusInstant": "2021-02-22T23:12:48.787021900Z",
"trackingInstant": "2021-02-22T23:12:48.173018100Z",
"msSinceRequest": 614
},
{
"siteId": 608,
"snapshotId": 9758,
"statusId": "7ad391f1-7563-11eb-9571-5dba55a7b8a1",
"trackingId": "7a767e85-7563-11eb-9657-0f6ec3418f91",
"source": "streaming-engine-sidekick",
"statusType": "ScopedActionEvent",
"message": "Received ScopedActionEvent 2593 CLOSING",
"kafkaSource": {
"topic": "streaming.engine.out",
"partition": 7,
"offset": 122813
},
"level": "SUMMARY",
"statusData": {},
"datacenter": "qa",
"statusInstant": "2021-02-22T23:12:48.783000100Z",
"trackingInstant": "2021-02-22T23:12:48.173018100Z",
"msSinceRequest": 610
},
{
"siteId": 608,
"snapshotId": 9758,
"statusId": "7ad20b54-7563-11eb-b54e-2fbb06c3e704",
"trackingId": "7a767e85-7563-11eb-9657-0f6ec3418f91",
"source": "streaming-consumer-legacy-catalog",
"statusType": "InternalEnvelope",
"message": "Received update action",
"kafkaSource": {
"topic": "streaming.engine.out",
"partition": 7,
"offset": 122810
},
"level": "SUMMARY",
"statusData": {
"cursor": {
"batch": 1,
"finished": false
},
"stats": {
"product": {
"total": 10,
"update": 10
}
}
},
"datacenter": "qa",
"statusInstant": "2021-02-22T23:12:48.773000400Z",
"trackingInstant": "2021-02-22T23:12:48.173018100Z",
"msSinceRequest": 600
},
{
"siteId": 608,
"snapshotId": 9758,
"statusId": "7ad0d2d0-7563-11eb-9571-43f487e9f166",
"trackingId": "7a767e85-7563-11eb-9657-0f6ec3418f91",
"source": "streaming-scoped-action",
"statusType": "ScopedActionEvent",
"message": "Sent Closing_started Scoped Action 2593",
"level": "SUMMARY",
"statusData": {},
"datacenter": "qa",
"statusInstant": "2021-02-22T23:12:48.765Z",
"trackingInstant": "2021-02-22T23:12:48.173018100Z",
"msSinceRequest": 592
},
{
"siteId": 608,
"snapshotId": 9758,
"statusId": "7acfe96e-7563-11eb-9571-bdfd49ee59f8",
"trackingId": "7a767e85-7563-11eb-9657-0f6ec3418f91",
"source": "streaming-scoped action",
"statusType": "ScopedActionEvent",
"message": "Done Opening 2593",
"level": "SUMMARY",
"statusData": {},
"datacenter": "qa",
"statusInstant": "2021-02-22T23:12:48.759025400Z",
"trackingInstant": "2021-02-22T23:12:48.173018100Z",
"msSinceRequest": 586
},
{
"siteId": 608,
"snapshotId": 9758,
"statusId": "7acfc25d-7563-11eb-9571-6d0fb0b63a1d",
"trackingId": "7a767e85-7563-11eb-9657-0f6ec3418f91",
"source": "streaming-engine-sidekick",
"statusType": "ScopedActionEvent",
"message": "Received ScopedActionEvent 2593 OPEN",
"kafkaSource": {
"topic": "streaming.engine.out",
"partition": 7,
"offset": 122812
},
"level": "SUMMARY",
"statusData": {},
"datacenter": "qa",
"statusInstant": "2021-02-22T23:12:48.758025300Z",
"trackingInstant": "2021-02-22T23:12:48.173018100Z",
"msSinceRequest": 585
},
{
"siteId": 608,
"snapshotId": 9758,
"statusId": "7acfc25c-7563-11eb-9571-4dcc289e679f",
"trackingId": "7a767e85-7563-11eb-9657-0f6ec3418f91",
"source": "streaming-scoped-action",
"statusType": "ScopedActionEvent",
"message": "Sent Open Scoped Action 2593",
"level": "SUMMARY",
"statusData": {
"cursor": {
"batch": 2,
"finished": true
},
"stats": {
"product": {
"update-product_pricecents": 10,
"update-product_saleprice_cents": 2
}
},
"scopedActionId": 2593
},
"datacenter": "qa",
"statusInstant": "2021-02-22T23:12:48.758025200Z",
"trackingInstant": "2021-02-22T23:12:48.173018100Z",
"msSinceRequest": 585
},
{
"siteId": 608,
"snapshotId": 9758,
"statusId": "7aacf81a-7563-11eb-9571-4181a1353770",
"trackingId": "7a767e85-7563-11eb-9657-0f6ec3418f91",
"source": "streaming-engine-sidekick",
"statusType": "ScopedActionEvent",
"message": "Received ScopedActionEvent 2593 OPENING_STARTED",
"kafkaSource": {
"topic": "streaming.engine.out",
"partition": 7,
"offset": 122809
},
"level": "SUMMARY",
"statusData": {},
"datacenter": "qa",
"statusInstant": "2021-02-22T23:12:48.530025Z",
"trackingInstant": "2021-02-22T23:12:48.173018100Z",
"msSinceRequest": 357
},
{
"siteId": 608,
"snapshotId": 9758,
"statusId": "7a94dbf6-7563-11eb-9657-43d300d2dfa5",
"trackingId": "7a767e85-7563-11eb-9657-0f6ec3418f91",
"source": "streaming-scoped-action",
"statusType": "ScopedAction",
"message": "ScopedAction 2593 OPENING_STARTED",
"level": "SUMMARY",
"statusData": {
"scopedAction": "Opening_started"
},
"datacenter": "qa",
"statusInstant": "2021-02-22T23:12:48.372018200Z",
"trackingInstant": "2021-02-22T23:12:48.173018100Z",
"msSinceRequest": 199
}
]
Example body for Scoped Action with more properties
The following request body provides a longer list of product properties.
Request Body
{
"product": {
"onOpen": {
"onExit": "complete",
"scope": {
"type": "allLegacyCatalog"
},
"action": {
"type": "updateAll",
"properties": [
"sale_price",
"price",
"product_effectiveprice_cents",
"name",
"findable",
"image_url",
"ispreorder",
"stock_level"
"installment",
"inventory_status",
"recommendable",
"discountinued",
"have_stock"
]
}
}
}
}
Sync a Subset of a Snapshot to an incoming set of products during Ingestion
This scoped action has a specific use case. It is used when one wants to provide a new set of products for a subset of the current snapshot and have the scoped action determine which items are orphans and delete them from the snapshot as well as update the products and categories.
One use case is for a B2B who may have stores who are supplying them with a full catalog and they do not know which items are no longer in the current snapshot. They could have an attribute for "store" and send in the the full catalog for that store. The scoped action will "touch" each item it updates in the snapshot. When the ingestion is finished, a "close" command is executed and the service then checks for any "untouched" items in the snapshot and deletes them.
The snapshot can be in “creating”, “complete” or “active” states. If a snapshot is archived and there is an “open” scopedAction, it will be canceled. Note that a snapshot in "creating" state does not update Find or the Legacy Catalog.
Multiple scoped actions on one snapshot are supported but they must all reference the same attribute that defines the subset of items.
Property Definition Collection: scopedActionSettings
The first step is to define the attribute that defines a subset of items within a snapshot in the product property definition collection. Only properties that are String, Boolean, Integer, and Long are supported in this type of scoped action.
In the "scopedActionSettings" set the "propertyMatch" parameter to true. The property definition needs to be published. This needs to be done before creating a snapshot and ingesting items. For more information on how to create a property definition collection please see the Streaming Property Service manual.
Below is an example of a property called "store" in the product property definition collection.
},
"store": {
"dataType": "string",
"origin": "custom",
"propertyValueStyle": "baseOnly",
"required": false,
"lengthLimit": 1024,
"scopedActionSettings": {
"propertyMatch": true
}
},
Open a Scoped Action
It is assumed that the property definition collection was published and a snapshot created. It is also assumed that this snapshot has had items ingested into it. The customer may now have a subset of data to be ingested into the snapshot. The streaming-ingest by itself will update and handle new items but it doesn't figure out which items in the snapshot need to deleted as they are not in the incoming new ingestion of items but there is a requirement to have the scoped action update as well as delete "untouched" items.
The next step is to open an scoped action for a specific snapshot with a scopedActionDefinition in the body of the call. The response will include the scopedActionId
Below is the API endpoint syntax:
POST https://<host>/streaming-snapshot/v1/<apiKey>/action/<snapshotId>/<snapshotType>/open
The scopedActionDefinition requires specific parameters by item type. It is required to provide a propertyName and it's value to define the subset of items. The deletions will be restricted within that subset of items. Updates stream through the streaming-ingest as usual but the scoped action service does touch each item so it can later determine which ones are untouched and are deleted.
onClose - Type is "propertyMarch" and the action is "deleteUntouched"
Syntax for scopedActionDefinition, which is the body of the open call is below.
{
"<itemType>" : {
"onClose" : {
"scope" : {
"type" : "propertyMatch",
"propertyName" : "<propertyName>",
"propertyValue" : "propertyValue"
},
"action" : {
"type" : "deleteUntouched"
}
}
}
}
Below is an example open scoped action with the body and response.
apiKey: 1234, snapshotId: 8000, propertyName "store", and propertyValue "one"
POST https://<host>/streaming-snapshot/v1/1234/8000/product/open
Body
{
"product" : {
"onClose" : {
"scope" : {
"type" : "propertyMatch",
"propertyName" : "store",
"propertyValue" : "one"
},
"action" : {
"type" : "deleteUntouched"
}
}
}
}
Response
{
"statusTracker": {
"trackingId": "b3a6f40a-5fd4-11ea-833b-7b4db4ad81b4",
"trackingInstant": "2020-03-06T18:02:53.999002600Z"
},
"scopedAction": {
"id": 180,
"siteId": 1234,
"snapshotId": 8000,
"trackingId": "b3a6f40a-5fd4-11ea-833b-7b4db4ad81b4",
"state": "Open",
"definition": {
"product": {
"onClose": {
"scope": {
"type": "propertyMatch",
"propertyName": "store",
"propertyValue": "one"
},
"action": {
"type": "deleteUntouched"
}
}
}
},
"lastModified": "2020-03-06T18:02:53.997718Z",
"trackingInstant": "2020-03-06T18:02:53.999002600Z"
}
Ingest items into snapshot while the Scoped Action is open.
Now that the scoped action is open for a snapshot, it is time to ingest the items that represent the catalog for a subset of the snapshot. In our example it could be one store who has updated their catalog and may have removed items and don't want to have to figure out what needs to be deleted. The new ingested catalog is the source of truth.
The streaming-ingest service is used to ingest items with a reference to the scopedActionId. It is important that the scopeActionId is included in the streaming-ingest call, otherwise the scoped action won't know to keep track of what is "touched" and could delete everything on the close.
The API endpoint syntax is as follows:
PUT https://<host>/streaming-ingest/v1/<apiKey>/<itemType>?snapshotId=<snapshotId>&scopedActionId=<scopedActionId>
"Body" will be the regular JSONL for products or categories. More information on the streaming-ingest can be found in the streaming-ingest-service manual.
Can do as many streaming-ingest calls with the scopeActionId as required to ingest all items. All items will updated as usual through the streaming-engine, legacy catalog and streaming-find-indexer. Once the ingestion is finished, the the scoped action can be closed.
Close the Scoped Action
POST https://<host>/streaming-snapshot/v1/<apiKey>/action/<snapshotId>/<snapshotType>/close/<scopedActionId>
Check status for “done”.
When a scoped update is “closed”, the “streaming-scoped-action” service provides the status with the stats. The stats includes how many items were “touched” and “untouched”. This service then sends the deletes to the streaming-engine which sends them to other services such as streaming-view, Find, Legacy Catalog to be processed. Each service produces their own status.
Here is an example of a status for a completed closed streaming-scoped-action.
[
{
"siteId": 123,
"snapshotId": 3393,
"statusId": "01f49eb7-5fd6-11ea-8023-ab1a944371c2",
"trackingId": "01d30c8c-5fd6-11ea-833b-a134d8882a64",
"source": "streaming-scoped-action",
"statusType": "ScopedActionEvent",
"message": "Done Closing Scoped Action 180",
"level": "SUMMARY",
"statusData": {
"stats": {
"ALL": {
"untouched": 1,
"total": 2,
"touched": 1
}
}
},
Check streaming-view and other services.
Note: All “untouched Items” will be deleted.
Here is an example of a status that is returned when checking on the status of a closed scoped action will include other services.
{
"siteId": 123,
"snapshotId": 3393,
"statusId": "01f317c7-5fd6-11ea-a01e-d3ca3f50419d",
"trackingId": "01d30c8c-5fd6-11ea-833b-a134d8882a64",
"source": "streaming-consumer-view",
"statusType": "InternalEnvelope",
"message": "Finished InternalEnvelope",
"level": "SUMMARY",
"statusData": {
"stats": {
"product": {
"total": 1,
"delete": 1
}
}
Scoped Action Endpoints
Base URL: https://<host>/streaming-snapshot/v1/
Full URL: https://<host>/streaming-snapshot/v1/<apiKey>/<action>/<snapshotId>/<snapshotType>
Note: Both streaming-snapshot and streaming-ingest services are leveraged for a scopedAction
| Action | REST API for scoped action within one snapshot |
|---|---|
|
Open a new scopedAction Same call for both scoped actions but they have different bodies
The response will include a scopedActionId.
|
POST https://<host>/streaming-snapshot/v1/<apiKey>/action/<snapshotId>/<snapshotType>/open
Body: (scopedActionDefinition for scoped action within a snapshot)
{
"<itemType>" : {
"onClose" : {
"scope" : {
"type" : "propertyMatch",
"propertyName" : "<propertyName>",
"propertyValue" : "<propertyValue>"
},
"action" : {
"type" : "deleteUntouched"
}
}
}
}
The response will include a scopedActionId. Response Codes: |
|
Ingest items Only for Scoped Action for within a snapshot. Note: If the scopedActionId is not open then the items will be processed as usual and not as part of the scopedAction |
PUT https://<host>/streaming-ingest/v1/<apiKey>/<itemType>?snapshotId=<snapshotId>&scopedActionId=<scopedActionId> Body - will have new and/or updated products. |
| Close a scopedAction |
POST https://<host>/streaming-snapshot/v1/<apiKey>/action/<snapshotId>/<snapshotType>/close/<scopedActionId> Response Codes: |
|
Cancel a scopedAction. Can only cancel an “open” scopedAction |
POST https://<host>/streaming-snapshot/v1/<apiKey>/action/<snapshotId>/<snapshotType>/cancel/<scopedActionId> Response Codes: |
| List scoped actions for a snapshot |
GET https://<host>/streaming-snapshot/v1/<apiKey>/action/<snapshotId>/<snapshotType> |
| Get details for a specific scopedAction | GET https://<host>/streaming-snapshot/v1/<apiKey>/action/<snapshotId>/<snapshotType>/scopedActionId |
Parameters
| Name | Description | Additional Information & Options |
|---|---|---|
|
<apiKey> |
Unique identifier for customer’s environment. For example, if a customer has multiple environments in production or staging, each environment would have a unique apiKey. There can be many apiKeys associated to a client_ID. |
Provided by Algonomy to customer. |
|
<snapshotId> |
Unique identifier for a snapshot. |
snapshotId is provided in the response to a POST, when creating the snapshot. |
|
<snapshotType> |
There are currently two types of snapshots: product: Requires product and category property definitions. Product items can have region and language over-rides. place: Required for Regions. All updates to regions are through the Place snapshot type. The Place snapshot should be "active" and requires the Region property definition. |
product place |
|
Scoped Action Commands |
The following actions are used for creating a scoped action, closing, cancelling, and deleting a scoped action. |
open, close, cancel, delete |
|
scopedActionDefinition |
The definition provided in the body when opening a scoped action. Require to define each item type. Example for Scoped Action within a Snapshot
{
"<itemType>" : {
"onClose" : {
"scope" : {
"type" : "propertyMatch",
"propertyName" : "<propertyName>",
"propertyValue" : "<propertyValue>"
},
"action" : {
"type" : "deleteUntouched"
}
}
}
}
Example with multiple item types. |
|
Detailed Example: syncing a subset of a snapshot to an ingestion of products
Get existing Scoped Action for a specific snapshot
GET https://<host>/streaming-snapshot/v1/123/action/3393/product
where apiKey=123, snapshotId=3393
Response:
[
{
"id": 152,
"siteId": 123,
"snapshotId": 3393,
"trackingId": "f0af2321-5749-11ea-a353-b1118ac84a32",
"state": "Complete",
"definition": {
"product": {
"onClose": {
"scope": {
"type": "propertyMatch",
"propertyName": "store",
"propertyValue": "one"
},
"action": {
"type": "deleteUntouched"
}
}
}
},
"lastModified": "2020-02-24T21:16:16.434831Z",
"trackingInstant": "2020-02-24T21:09:27.046019300Z"
},
{
"id": 153,
"siteId": 123,
"snapshotId": 3393,
"trackingId": "49f15c25-5761-11ea-a353-33d040339ebe",
"state": "Complete",
"definition": {
"product": {
"onClose": {
"scope": {
"type": "propertyMatch",
"propertyName": "store",
"propertyValue": "one"
},
"action": {
"type": "deleteUntouched"
}
}
}
},
"lastModified": "2020-02-25T00:01:55.755933Z",
"trackingInstant": "2020-02-24T23:56:35.222019700Z"
}
Open, Ingest and then close scopedAction
POST https://<host>/streaming-snapshot/v1/<apiKey>/action/3393/product/open
Response: The snapshotId is 158
{
"statusTracker": {
"trackingId": "7b5fbaed-58ce-11ea-8f38-c19a9fad48ee",
"trackingInstant": "2020-02-26T19:30:44.460004500Z"
},
"scopedAction": {
"id": 158,
"siteId": 123,
"snapshotId": 3393,
"trackingId": "7b5fbaed-58ce-11ea-8f38-c19a9fad48ee",
"state": "Open",
"definition": {
"product": {
"onClose": {
"scope": {
"type": "propertyMatch",
"propertyName": "store",
"propertyValue": "one"
},
"action": {
"type": "deleteUntouched"
}
}
}
},
"lastModified": "2020-02-26T19:30:44.460768Z",
"trackingInstant": "2020-02-26T19:30:44.460004500Z"
}
}
Ingest a new product for store “one”
Current view store - 2 products, one (test_ms_3) for store “two” and one (test_ms_4) for store “one”
GET http://10.3.252.38/streaming-view/v1/<apiLey>/3393/product
{
"items": [
{
"itemType": "product",
"internalId": "101729812602",
"externalId": "test_ms_3",
"trackingId": "4bba6584-5748-11ea-817a-971ac1658db4",
"item": {
"properties": {
"num_reviews": "10",
"image_url": "https://image.migros.ch/fm-xl/452fccb597ac1b6965dbffc5aa1ac952133232e8/maurten-drink-mix-320-getraenkepulver-mit-natriumchlorid_1.jpg(16 kB)",
"name": "Drink Mix test_1",
"product_recommendable": true,
"product_description": "added product 463027402900_test_1",
"description": "added product 463027402900_test_1",
"in_stock": true,
"price": "1001",
"product_external_id": "test_0923_1020_1",
"store": "two",
"product_image_id": "https://image.migros.ch/fm-xl/452fccb597ac1b6965dbffc5aa1ac952133232e8/maurten-drink-mix-320-getraenkepulver-mit-natriumchlorid_1.jpg(16 kB)",
"brand": "BrandOneStoreTwo",
"product_num_reviews": 10,
"product_link_id": "/de/p/463027402900/maurten-drink-mix-320?selectVariant_test_1",
"rating": "1",
"product_name": "Drink Mix test_1",
"product_rating": 1.0,
"link_url": "/de/p/463027402900/maurten-drink-mix-320?selectVariant_test_1",
"recommendable": "true",
"product_pricecents": 100100,
"product_brand": "BrandOneStoreTwo"
}
}
},
{
"itemType": "product",
"internalId": "101730595788",
"externalId": "test_ms_4",
"trackingId": "4fb064d0-581c-11ea-817a-5569c49e1494",
"item": {
"properties": {
"num_reviews": "10",
"image_url": "https://image.migros.ch/fm-xl/452fccb597ac1b6965dbffc5aa1ac952133232e8/maurten-drink-mix-320-getraenkepulver-mit-natriumchlorid_1.jpg(16 kB)",
"name": "Drink Mix test_1",
"product_recommendable": true,
"product_description": "added product 463027402900_test_1",
"description": "added product 463027402900_test_1",
"in_stock": true,
"price": "100",
"product_external_id": "test_0923_1020_1",
"store": "one",
"product_image_id": "https://image.migros.ch/fm-xl/452fccb597ac1b6965dbffc5aa1ac952133232e8/maurten-drink-mix-320-getraenkepulver-mit-natriumchlorid_1.jpg(16 kB)",
"brand": "ScopedAction",
"product_num_reviews": 10,
"product_link_id": "/de/p/463027402900/maurten-drink-mix-320?selectVariant_test_1",
"rating": "1",
"product_name": "Drink Mix test_1",
"product_rating": 1.0,
"link_url": "/de/p/463027402900/maurten-drink-mix-320?selectVariant_test_1",
"recommendable": "true",
"product_pricecents": 10000,
"product_brand": "ScopedAction"
}
}
}
],
"numFound": 2,
"nextCursorMark": "AoEucDoxMDE3MzA1OTU3ODg=",
"errors": []
Ingest a new product #8 for store one
Make sure the scopedActionId is included or else the item won’t be counted as “touched” for the scopedAction.
snapshotId: 3393
scopedActionId: 158
PUT https://<host>/streaming-ingest/v1/123/product?snapshotId=3393&scopedActionId=158
Body:
{"test_ms_8":{"properties":{"brand":"ScopedBrand","recommendable":true,"price":10,"store":"one","rating":1,
"image_url":"https://image.migros.ch/fm-xl/452fccb597ac1b6965dbffc5aa1ac952133232e8/maurten-drink-mix-320-getraenkepulver-mit-natriumchlorid_1.jpg(16 kB)",
"product_external_id":"test_0923_1020_1","name":"Drink Mix test_1","num_reviews":10,"in_stock":true,
"link_url":"/de/p/463027402900/maurten-drink-mix-320?selectVariant_test_1",
"description":"added product 463027402900_test_1"}}}
Response:
{
"statusTracker": {
"trackingId": "2b7b2087-58d1-11ea-8a19-dd50b48edb01",
"trackingInstant": "2020-02-26T19:49:58.912013500Z"
}
}
Check Status with trackingId
GET http://10.3.252.38/streaming-status/...9-dd50b48edb01
[
{
"siteId": 123,
"snapshotId": 3393,
"statusId": "2c4c5380-58d1-11ea-b89c-7fb08cb0548c",
"trackingId": "2b7b2087-58d1-11ea-8a19-dd50b48edb01",
"source": "streaming-engine",
"statusType": "ExternalEnvelope",
"message": "ItemEvent",
"level": "SUMMARY",
"statusData": {
"stats": {
"product": {
"total": 1,
"update": 1
}
}
},
"datacenter": "qa",
"statusInstant": "2020-02-26T19:50:00.283020800Z",
"trackingInstant": "2020-02-26T19:49:58.912013500Z",
"msSinceRequest": 1371
},
{
"siteId": 123,
"snapshotId": 3393,
"statusId": "2ba082e5-58d1-11ea-9778-ff1c509552e2",
"trackingId": "2b7b2087-58d1-11ea-8a19-dd50b48edb01",
"source": "find-index-deployer",
"statusType": "StreamingEvent",
"message": "Streaming item not processed correctly",
"level": "WARNING",
"statusData": {
"stats": {
"product": {
"total": 1,
"product ignored": 1
}
}
},
"datacenter": "qa",
"statusInstant": "2020-02-26T19:49:59.157014900Z",
"trackingInstant": "2020-02-26T19:49:58.912013500Z",
"msSinceRequest": 245
},
{
"siteId": 123,
"snapshotId": 3393,
"statusId": "2b9c8ba9-58d1-11ea-82ca-b756c0072cd9",
"trackingId": "2b7b2087-58d1-11ea-8a19-dd50b48edb01",
"source": "streaming-consumer-view",
"statusType": "InternalEnvelope",
"message": "Finished InternalEnvelope",
"level": "SUMMARY",
"statusData": {
"stats": {
"product": {
"total": 1,
"update": 1
}
}
},
"datacenter": "qa",
"statusInstant": "2020-02-26T19:49:59.131024900Z",
"trackingInstant": "2020-02-26T19:49:58.912013500Z",
"msSinceRequest": 219
},
{
"siteId": 123,
"snapshotId": 3393,
"statusId": "2b93d918-58d1-11ea-82ca-4d753fff94b1",
"trackingId": "2b7b2087-58d1-11ea-8a19-dd50b48edb01",
"source": "streaming-consumer-view",
"statusType": "InternalEnvelope",
"message": "Received InternalEnvelope",
"level": "SUMMARY",
"statusData": null,
"datacenter": "qa",
"statusInstant": "2020-02-26T19:49:59.074024800Z",
"trackingInstant": "2020-02-26T19:49:58.912013500Z",
"msSinceRequest": 162
},
{
"siteId": 123,
"snapshotId": 3393,
"statusId": "2b7c590a-58d1-11ea-8a19-510187ebb49e",
"trackingId": "2b7b2087-58d1-11ea-8a19-dd50b48edb01",
"source": "streaming-ingest",
"statusType": "item",
"message": "Finished REPLACE action",
"level": "SUMMARY",
"statusData": {
"stats": {
"product": {
"total": 1,
"replace": 1
}
}
},
"datacenter": "qa",
"statusInstant": "2020-02-26T19:49:58.920013800Z",
"trackingInstant": "2020-02-26T19:49:58.912013500Z",
"msSinceRequest": 8
},
{
"siteId": 123,
"snapshotId": 3393,
"statusId": "2b7b95b9-58d1-11ea-8a19-f717d0f8a868",
"trackingId": "2b7b2087-58d1-11ea-8a19-dd50b48edb01",
"source": "streaming-ingest",
"statusType": "item",
"message": "Batch sending 1 item(s)",
"level": "SUMMARY",
"statusData": {
"stats": {
"product": {
"total": 1,
"replace": 1
}
}
},
"datacenter": "qa",
"statusInstant": "2020-02-26T19:49:58.915013700Z",
"trackingInstant": "2020-02-26T19:49:58.912013500Z",
"msSinceRequest": 3
},
{
"siteId": 123,
"snapshotId": 3393,
"statusId": "2b7b2088-58d1-11ea-8a19-b3147c9a56a9",
"trackingId": "2b7b2087-58d1-11ea-8a19-dd50b48edb01",
"source": "streaming-ingest",
"statusType": "item",
"message": "Received REPLACE action",
"level": "SUMMARY",
"statusData": null,
"datacenter": "qa",
"statusInstant": "2020-02-26T19:49:58.912013600Z",
"trackingInstant": "2020-02-26T19:49:58.912013500Z",
"msSinceRequest": 0
}
]
Check View Store, New Product is Available
GET http://10.3.252.38/streaming-view/v1/123/3393/product
{
"items": [
{
"itemType": "product",
"internalId": "101729812602",
"externalId": "test_ms_3",
"trackingId": "4bba6584-5748-11ea-817a-971ac1658db4",
"item": {
"properties": {
"num_reviews": "10",
"image_url": "https://image.migros.ch/fm-xl/452fccb597ac1b6965dbffc5aa1ac952133232e8/maurten-drink-mix-320-getraenkepulver-mit-natriumchlorid_1.jpg(16 kB)",
"name": "Drink Mix test_1",
"product_recommendable": true,
"product_description": "added product 463027402900_test_1",
"description": "added product 463027402900_test_1",
"in_stock": true,
"price": "1001",
"product_external_id": "test_0923_1020_1",
"store": "two",
"product_image_id": "https://image.migros.ch/fm-xl/452fccb597ac1b6965dbffc5aa1ac952133232e8/maurten-drink-mix-320-getraenkepulver-mit-natriumchlorid_1.jpg(16 kB)",
"brand": "BrandOneStoreTwo",
"product_num_reviews": 10,
"product_link_id": "/de/p/463027402900/maurten-drink-mix-320?selectVariant_test_1",
"rating": "1",
"product_name": "Drink Mix test_1",
"product_rating": 1.0,
"link_url": "/de/p/463027402900/maurten-drink-mix-320?selectVariant_test_1",
"recommendable": "true",
"product_pricecents": 100100,
"product_brand": "BrandOneStoreTwo"
}
}
},
{
"itemType": "product",
"internalId": "101730595788",
"externalId": "test_ms_4",
"trackingId": "4fb064d0-581c-11ea-817a-5569c49e1494",
"item": {
"properties": {
"num_reviews": "10",
"image_url": "https://image.migros.ch/fm-xl/452fccb597ac1b6965dbffc5aa1ac952133232e8/maurten-drink-mix-320-getraenkepulver-mit-natriumchlorid_1.jpg(16 kB)",
"name": "Drink Mix test_1",
"product_recommendable": true,
"product_description": "added product 463027402900_test_1",
"description": "added product 463027402900_test_1",
"in_stock": true,
"price": "100",
"product_external_id": "test_0923_1020_1",
"store": "one",
"product_image_id": "https://image.migros.ch/fm-xl/452fccb597ac1b6965dbffc5aa1ac952133232e8/maurten-drink-mix-320-getraenkepulver-mit-natriumchlorid_1.jpg(16 kB)",
"brand": "ScopedAction",
"product_num_reviews": 10,
"product_link_id": "/de/p/463027402900/maurten-drink-mix-320?selectVariant_test_1",
"rating": "1",
"product_name": "Drink Mix test_1",
"product_rating": 1.0,
"link_url": "/de/p/463027402900/maurten-drink-mix-320?selectVariant_test_1",
"recommendable": "true",
"product_pricecents": 10000,
"product_brand": "ScopedAction"
}
}
},
{
"itemType": "product",
"internalId": "101732217999",
"externalId": "test_ms_8",
"trackingId": "2b7b2087-58d1-11ea-8a19-dd50b48edb01",
"item": {
"properties": {
"num_reviews": "10",
"image_url": "https://image.migros.ch/fm-xl/452fccb597ac1b6965dbffc5aa1ac952133232e8/maurten-drink-mix-320-getraenkepulver-mit-natriumchlorid_1.jpg(16 kB)",
"name": "Drink Mix test_1",
"product_recommendable": true,
"product_description": "added product 463027402900_test_1",
"description": "added product 463027402900_test_1",
"in_stock": true,
"price": "10",
"product_external_id": "test_0923_1020_1",
"store": "one",
"product_image_id": "https://image.migros.ch/fm-xl/452fccb597ac1b6965dbffc5aa1ac952133232e8/maurten-drink-mix-320-getraenkepulver-mit-natriumchlorid_1.jpg(16 kB)",
"brand": "ScopedBrand",
"product_num_reviews": 10,
"product_link_id": "/de/p/463027402900/maurten-drink-mix-320?selectVariant_test_1",
"rating": "1",
"product_name": "Drink Mix test_1",
"product_rating": 1.0,
"link_url": "/de/p/463027402900/maurten-drink-mix-320?selectVariant_test_1",
"recommendable": "true",
"product_pricecents": 1000,
"product_brand": "ScopedBrand"
}
}
}
],
"numFound": 3,
"nextCursorMark": "AoEucDoxMDE3MzIyMTc5OTk=",
"errors": []
Close scopedAction
apiKey: 123, snapshotId: 3393, scopedActionId: 158
POST https://<host>/streaming-snapshot/v1/123/action/3393/product/close/158
Response:
{
"statusTracker": {
"trackingId": "144129d6-58d3-11ea-8f38-f5b421a98d65",
"trackingInstant": "2020-02-26T20:03:38.938005400Z"
},
"scopedAction": {
"id": 158,
"siteId": 123,
"snapshotId": 3393,
"trackingId": "7b5fbaed-58ce-11ea-8f38-c19a9fad48ee",
"state": "Closing_started",
"definition": {
"product": {
"onClose": {
"scope": {
"type": "propertyMatch",
"propertyName": "store",
"propertyValue": "one"
},
"action": {
"type": "deleteUntouched"
}
}
}
},
"lastModified": "2020-02-26T20:03:38.938939Z",
"trackingInstant": "2020-02-26T19:30:44.460004500Z"
}
}
Check Status, Use the trackingId Found Under “statusTracker”
When the scopedAction is complete will see the following status message:
[
{
"siteId": 123,
"snapshotId": 3393,
"statusId": "0905ada9-5762-11ea-9927-f1c2f7ffeaed",
"trackingId": "49f15c25-5761-11ea-a353-33d040339ebe",
"source": "streaming-scoped-action",
"statusType": "ScopedAction",
"message": "COMPLETE ScopedAction 158",
"level": "SUMMARY",
"statusData": null,
"datacenter": "qa",
"statusInstant": "2020-02-25T00:01:55.800004100Z",
"trackingInstant": "2020-02-24T23:56:35.222019700Z",
"msSinceRequest": 320578
},
{
"siteId": 123,
"snapshotId": 3393,
"statusId": "08f9c6c6-5762-11ea-9927-d799a38b2424",
"trackingId": "49f15c25-5761-11ea-a353-33d040339ebe",
"source": "streaming-scoped-action",
"statusType": "ScopedAction",
"message": "CLOSING ScopedAction 158",
"level": "SUMMARY",
"statusData": null,
"datacenter": "qa",
"statusInstant": "2020-02-25T00:01:55.722003800Z",
"trackingInstant": "2020-02-24T23:56:35.222019700Z",
"msSinceRequest": 320500
},
{
"siteId": 123,
"snapshotId": 3393,
"statusId": "08f4be58-5762-11ea-a353-956b33f0d960",
"trackingId": "49f15c25-5761-11ea-a353-33d040339ebe",
"source": "streaming-scoped-action",
"statusType": "ScopedAction",
"message": "ScopedAction 158 CLOSING_STARTED",
"level": "SUMMARY",
"statusData": {
"scopedAction": "Closing_started"
},
"datacenter": "qa",
"statusInstant": "2020-02-25T00:01:55.689020Z",
"trackingInstant": "2020-02-24T23:56:35.222019700Z",
"msSinceRequest": 320467
}