Integrate.io logo

Integrate.io Release Notes & Product Updates

An overview of the latest updates and product improvements. Got a product update or feature you'd like to see supported? Let us know - support@integrate.io

August 2026: New Tools for Schema Changes and Large-Scale Loads

ELT & CDC

August's release cycle went into Oracle replication depth and keeping destination schemas current as sources change. Security hardening and large-volume load reliability took the rest of the month.

Automatic Column Detection Keeps Destination Schemas Current

A column added at the source stayed invisible until someone noticed a report was missing it, worked out which pipeline was responsible, and edited the schema by hand. Singer-based connectors detect new source fields and carry them into the destination on the next sync, so the warehouse tracks the source instead of drifting from it. The gap between a change upstream and a change in your tables becomes one sync interval instead of however long it takes someone to notice. Past rows keep an empty value for a new column, so a backfill still needs an explicit resync.

Parallel Chunk Workers and History Mode Deepen Oracle Replication

A large Oracle initial sync ran as one extraction stream, and a record's earlier state vanished the moment it changed. Splitting extraction across parallel chunk workers shortens a large first load, and a failed setup releases its database sessions instead of leaving them for a DBA to clean up. History Mode keeps prior versions of a row with the source's change times, so a point-in-time question about what a record looked like last quarter has an answer in the warehouse instead of in a backup. Replication also drops rolled-back changes and replays partially acknowledged ones after an interruption, so updates and deletes stop going missing quietly.

New Features

  • Configurable Salesforce Lookback: Each Salesforce pipeline can re-check recent changes on a window from thirty minutes up to twenty-eight days. Records that land after their original sync come through on the re-check instead of going missing, and the setting stays off until you enable it.
  • Initial Sync Only Override: A pipeline can complete its initial snapshot, load the staged data into the warehouse, and stop before continuous replication starts, configured per pipeline by our team. One-time migrations finish without a CDC stream you then have to turn off.
  • Validation Overrides: Pipeline creation and edits can proceed without validating the source and destination, and our team can apply an account-specific override for an overly strict Snowflake check. Bypassed checks stay visible and auditable, while connection, credential, and provisioning failures still block deployment.

Improvements

  • TikTok Ads syncs ingest reservation campaigns, including TopView and Reach & Frequency buys, alongside the auction campaigns already covered.
  • TikTok Ads tracks report progress separately for each advertiser, so a stalled advertiser stops hiding missing data elsewhere.
  • TikTok Ads fetches partially complete recent report windows after a resume or a restart, so the newest data is not skipped.
  • TikTok Ads handles more error responses from the provider rather than failing the sync outright.
  • Oracle destinations support SCD Type 2, so a dimension table keeps its change history instead of overwriting a row in place.
  • Singer source validation messages carry the underlying provider error, such as a missing permission, instead of a generic credentials message.
  • The Snowflake connector runs on a newer Snowflake SDK driver version that carries the provider's latest fixes.
  • Resumed and retried initial syncs report steadily advancing progress, and BigQuery receives completion notifications during large initial loads.
  • Chunk workers wait until the schema record is fully applied before they start, so chunked uploads complete more consistently.
  • Large pipelines wait for destination preparation to finish, and the platform validates timing and retry settings before use.
  • DB2 setup queries double-quote database, schema, and table names consistently on the generated setup SQL.
  • Alert setup consistently rejects malformed, unauthorized, or excessive notification-hook references when an alert is created or edited.

Fixes

  • Snowflake operations quote reserved-word columns and names containing double quotes without a pipeline change.
  • Chunked COPY statements on Snowflake run in sequence, and chunk stream keys resolve before they leave the copy handler.
  • The TikTok Ads pixel_instant_page_events stream completes on parameter combinations the provider does not support.
  • DB2 CDC setup SQL runs on the DB2 CLP, and it handles SAP-style identifiers, unusual punctuation, and repeated execution.
  • Initial-sync progress reports current and total in the right order, and null fields return a result rather than a server error.
  • Auto restart completes without firing a false error event hook.
  • Oracle pipelines handle a duplicate record on an index column instead of failing.
  • User invitations stay scoped to the correct organization, and moving an existing account requires the invited user to accept.
  • Observability alert validation on create and edit checks object ownership.

Performance Enhancements

  • Snowflake imports of more than 1,000 files complete instead of failing partway, so a high-volume load no longer needs splitting into smaller batches.
  • TikTok Ads backfills use thirty-day windows and larger pages, so a historical load needs fewer requests and still fetches the latest partial range.

Tell us how the Oracle changes hold up on your largest tables, and which sources should get column detection next.

about 1 month ago