Batch Delta Feed File Upload
Overview
Using the FTP upload command, you can upload the delta file segments feed. This allows processing of the delta feed for the segment feed, enabling a subset of users and their segments to be updated in the UPS from the delta file. In this upload method, the batch delta file is processed and only users with modifications/updates in their profiles/segments are updated in the UPS instead of all the users from the file. This is in contrast to the full file upload in which all the users in the feed file are uploaded.
Note: Processing a delta feed file is quite similar to a full feed file with a slight difference.
Scope
Presently delta feed files are processed for modified/updated users only. To remove users, you may still need to use full feed files as users cannot be removed using delta feed files.
Assumptions
The client is expected to send a full feed file to update all users information in one go and the incremental delta feed files on a periodic basis.
Processing Delta Feed File
What is a full feed file? The full feed file consists of all users' details and each user will have full segments and non-segments batch data, user attributes, user preferences and/or user linking details in a single file.
What is a delta feed file? The delta feed file consists of only updated users' details and each user will have full segments and non-segments batch data, user attributes, user preferences and/or user linking details in a single file.
What is a user linking file? The user linking file is a delimited text file and it has a special Unicode character as a delimiter.
Preparing Delta Feed File for Upload
- Syntax and payload structure in delta feed files is similar to full feed files.
- Delta feed files should include only updated users.
- When preparing batch segment delta files, only updated and/or new users' segment details are added.
- When preparing batch non-segment files and user linking files, only updated users' and/or new users' details are added.
Other details with respect to the processing of these files are provided in the below table.
Delta Feed File vs. Full Feed File
|
Full Feed File |
Delta Feed File |
---|---|---|
Data Processing |
|
Note: The delta feed files are uploaded in the same way as mentioned in the Full Feed File column. |
Data Restore | The client sends a full feed file to sync UPS data with client data. | The client sends the full feed file whenever UPS data needs to be synced with client data or to restore the missing data. |
UPS Reader |
|
|
Data Compaction |
For the same API Key and attribute type, if multiple files are uploaded using the FTP command, the compactor removes the stalled versioned users' data from the database. |
|
Sample Data Files
Batch Segment File
rr-segments
|
Batch User-attributes File
rr-userattributes
|
Syntaxes and Commands to Process and Upload Batch Files
Process and Upload File Commands' Syntax
The following syntaxes are used to process the batch files. Uploading Full and Delta Feed File by Client Client uploads full and delta feed files to the FTP server, using the following FTP command:
Processing Batch Files Batch Segments Full Feed File After uploading the full feed file, run the following command to process the batch segments full feed file:
Batch Segments Delta Feed File After uploading the delta feed file, run the following command to process the batch segments delta feed file:
User Linkings Full Feed File After uploading the full feed file, run the following command to process the user linkings full feed file:
User Linkings Delta Feed File After uploading the delta feed file, run the following command to process the user linkings delta feed file:
|
Upload Commands
Run the following commands to upload and process batch files with the required changes in file names:
|
FAQs
What happens when a customer uploads a delta feed file without sending the full feed file at the very first time?
BuildFTP is smart enough to process the incoming delta feed file and all users' data persist in the database. However, It is not recommended to send the delta feed file at first.
Note: It is recommended that you send the delta feed file on top of the full feed file.
What happens when a customer uploads a full feed file as a delta feed file?
All uploaded users' data persist in the database. However, it leads to data inconsistency (All the stalled versions of the users' data should be compacted if you place a full feed file. Because, you have placed it as a delta feed file, there will be no stalled versioned users' data and compaction).
To resolve the above data inconsistency, the customer should resend the actual full feed file.
What happens when a customer uploads a delta feed file as a full feed file?
All uploaded users' data persist in the database. However, it leads to another sort of data inconsistency (No users' data should to be compacted if you place a delta feed file. Because, you have placed it as a full feed file, the previous full and all delta versioned users' data becomes a stalled version and compacted).
To resolve the above data inconsistency, the customer should resend the actual full feed file and then all delta feed files in the same order again if any. Finally, the actual delta feed file should be uploaded.
Does the BuildFTP process support Bi-directional(Grouped) user linking delta feed file?
Yes. The BuildFTP process supports Bi-directional (Grouped) as well as Uni-directional (explicit) mapping with user linking delta feed file.
How does UPS system data sync with customer data?
The customer should send a full feed file so that all user's data will be updated and sync with customer data.