DB Diff Pro

Cross-engine diff, without guessing

Compare tables between PostgreSQL, MySQL and ClickHouse. Pro reduces every value to a canonical form before it is hashed, so a boolean, a decimal or a timestamp means the same thing on both sides.

PostgreSQL MySQL ClickHouse
pro demo
Any pairing

PostgreSQL, MySQL and ClickHouse, compared in either direction.

Canonical values

Booleans, decimals, floats and timestamps hash the same on both sides.

Honest refusals

A type that cannot be compared is named, not approximated.

Use cases

Check that a replica, a restored backup, or a logical replication target holds the same rows as its source. Equal tables are answered by checksums alone, so nothing but a handful of hashes crosses the wire.

Try it yourself:

$ db-diff --source="$PRIMARY" --target="$REPLICA" --table=orders

Prove a migration moved the data and not just the schema. When a table does not match, DB Diff reads only the segments that differ and names the exact primary keys, so you get the keys that moved rather than a yes or no.

Try it yourself:

$ db-diff --source="$BEFORE" --target="$AFTER" --table=line_items

Timestamps and counters change on every write and will always differ. Include the columns that carry meaning and exclude the ones that only carry noise, and the comparison stays about the data you care about.

Try it yourself:

$ db-diff --source="$SRC" --target="$DST" --table=line_items --exclude=quantity

Get started

Free

0 for ever

One binary, every engine and every flag. Comparisons run within one engine.

  • PostgreSQL, MySQL and ClickHouse
  • Server side checksums, one pass per segment
  • Drill down to the differing primary keys
  • Include and exclude columns
  • Tunable segment size and CI exit codes
Install it

Requirements

Engines: PostgreSQL 14 or newer, MySQL 5.7 or newer, ClickHouse 20.1 or newer. Pro accepts any pairing of the three.

Table: one integer primary key named id, with the same table name and column order on both sides.

Run: a differing segment is read whole, and the comparison is not a transaction. Pin the session timezone and charset on both connections.

Algorithm notes

DB Diff does the comparison inside the database. The wire carries checksums when the tables match, and rows only when they do not.

  1. 01

    Segment the table

    The table is cut into ranges of primary keys, 10,000 by default. Both sides can address the same range without moving a row.

  2. 02

    Checksum each segment

    Each side hashes every compared column on its own, folds the digests with XOR, and returns one 64-bit checksum. Equal checksums skip the segment.

  3. 03

    Read only what differs

    A differing segment is read row by row and compared column by column. The output is the primary keys that changed.

Row checksum
md5( null_bitmap || md5(COALESCE(col_a, '')) || md5(COALESCE(col_b, '')) || ... ) -> first 8 bytes
segment = XOR(row checksums)

Across engines, the checksum needs a canonical form: PostgreSQL writes a decimal as 1.50 and ClickHouse writes it as 1.5. Pro reduces every value to a form all three engines agree on before it is hashed. A NULL bitmap sits in front of the column digests, so a NULL and an empty string cannot hash alike.

Read the full algorithm notes