Skip to main content
Use Bulk Data Ingest to send historical or batch usage to Metronome in JSONL files. Bulk Data Ingest uses the same event structure as /ingest, but processes files asynchronously.
Bulk Data Ingest is currently in private preview. Contact your Metronome representative for access.

​
When to use Bulk Data Ingest

Use Bulk Data Ingest for usage that does not fit a real-time ingestion workflow:
  • Migrate to Metronome: Import historical usage from a previous billing or metering system.
  • Recover missing usage: Backfill events that were not delivered because of a pipeline failure or reporting issue.
  • Report usage on a schedule: Submit usage monthly, quarterly, or on another schedule when events are not available in real time.
  • Support self-hosted and air-gapped deployments: Import usage from environments that cannot connect to Metronome’s /ingest endpoint.
Use the /ingest API for ongoing, real-time usage reporting.

​
Before you upload

Create the relevant customers, ingest aliases, and billable metrics in Metronome before uploading usage. Prepare an uncompressed JSONL file with one event per line. Each event must follow the /ingest event schema:
Only uncompressed .jsonl files are supported. Keep each file no larger than 5 GB. Split larger datasets across multiple files. Do not overwrite a file after you upload it. If a file fails validation, fix the reported errors and upload a new file with a different name. Bulk Data Ingest cannot change an event that Metronome has already processed.

​
Validate files locally

Validate each file before uploading it. Confirm that:
  • Each line contains one valid JSON object.
  • Every event includes transaction_id, customer_id, timestamp, and event_type.
  • Timestamps use RFC 3339 format.
  • Timestamps are not in the future and are no more than one year old.
  • Each customer_id matches a Metronome customer ID or ingest alias.
  • Property keys and values are strings, including numeric values.
  • The file does not contain duplicate transaction_id values.
  • The file is uncompressed JSONL and no larger than 5 GB.
Metronome validates the complete file before processing any events. If any event fails validation, Metronome rejects the complete file.

​
Use idempotent transaction IDs

Use a deterministic transaction_id for each event. If you resend an event, keep its transaction_id and timestamp unchanged. Do not reuse a transaction ID for a different event. Metronome uses the transaction_id to identify duplicate events. For periodic measurements such as heartbeats, you can construct the transaction ID from the source system, entity, and measurement period:
For example, billing_node42_2025031008 identifies a heartbeat from node 42 for the 8:00 AM measurement period on March 10, 2025. Duplicate transaction IDs within one file cause the complete file to fail validation. Across files, Metronome ignores duplicate events with the same transaction_id. Metronome checks for duplicates against events with a timestamp in the past year, regardless of when those events were uploaded. Do not rely on upload order to determine which event Metronome processes.

​
Use /ingest and Bulk Data Ingest together

You can use /ingest and Bulk Data Ingest for the same customer. Bulk Data Ingest accepts events of any age up to one year. Metronome selects the processing pipeline based on each event’s timestamp: events within /ingest’s 34-day backdating window use standard ingestion, and earlier events use historical ingestion. If a bulk file contains both, Metronome routes each event automatically. Historical ingestion checks each transaction_id against events previously received through /ingest. If Metronome finds a match, it ignores the event from the bulk file. If you send the same event through both methods, keep its transaction_id and timestamp unchanged. The /ingest endpoint does not check historical events for duplicates. It accepts events within its 34-day backdating window, while historical ingestion handles events from before that window.

​
Processing and verification

Bulk Data Ingest processes files asynchronously. Events do not appear immediately after you upload a file. Wait for processing to finish before reviewing usage or generating an invoice that depends on the imported events. You cannot currently check upload status through the API or the Metronome application. After processing finishes, events appear in:
  • Event search
  • Usage graphs and breakdowns
  • Billing calculations for invoices that have not been finalized
  • Error usage reporting
Bulk events do not appear when event search is filtered to a specific billable metric.
Bulk ingestion can affect alerts as events are processed. Review your alert configuration before uploading a large historical dataset.

​
How imported usage affects invoices

Imported usage affects invoices that have not been finalized. It does not automatically update an invoice that has already been finalized. If a finalized invoice needs to include the imported usage, void and regenerate the invoice after processing finishes.

​
Limits and requirements

  • Supported format: Uncompressed JSONL
  • Maximum file size: 5 GB
  • Event timestamps: Up to one year before upload
  • Future events: Not supported
  • Processing: Asynchronous
  • File validation: Metronome rejects the complete file if any event fails validation
  • Duplicate transaction IDs: Metronome rejects a file containing duplicate transaction_id values
  • Finalized invoices: Not updated automatically