Data processing queue

In order to be able to submit multiple requests at once and honor the limits set up by the API, all app requests are processed in batches and queued in the log system. The following points explain in detail how changes are submitted.

Export

• Changes made via the UI are immediately created as at least one upload batch. Multiple batches will be created if an update affects multiple models—e.g. sponsored products and sponsored brands, or both product targets and keywords.

• Every minute the system checks for the next available bulk upload. If there is one in the queue, it is translated into an update batch.

• Every minute the system checks for the next available update batch. If there is one in queue, the system will send up to 500 records in that batch to amazon, with certain amazon-imposed limitations:

  • Only 100 records are sent at a time if the batch includes sponsored brands or sponsored display records, per amazon’s limitation.

  • Only 10 records are sent at a time if the batch includes sponsored brand campaigns.

• If the next batch has fewer unprocessed records than the limits (500, 100, or 10 respectively), all remaining records will be sent in this batch.

Import

Several jobs constantly take snapshots/reports from the queue, requesting data from Amazon and later retrieving it.

• Snapshots (dimensions)

  • The snapshots handle all parameters/attributes referred to as “dimensions”: campaign name, state, budget, etc.

  • Brand snapshots are queued for import at midnight* for active accounts (advertising=true).

  • Portfolio, and all SP, SB, and SD snapshots are queued at 1 am* for active accounts (advertising=true).

  • Accounts marked advertising=false are queued weekly, to prioritize active accounts over dormant accounts.

• Reports (metrics)

  • The reports refer to all fields considered “metrics”: impressions, clicks, spend, etc, imported more regularly to ensure maximum accuracy.

  • SP, SB, and SD reports are queued daily at 1 am*, 7 am*, 1 pm*, 7 pm*. The app requests data for the most recent 14 days, as well as the 30th and 60th most recent day. Amazon updates metric data constantly, so the 60th day is pulled to ensure reporting the latest data before Amazon stops providing it.

  • Accounts marked advertising=false are queued weekly, to prioritize active accounts over dormant accounts.

*All times are expressed in PST.