What's the faster way to compare around 700,000 rows from 2 different databases using Perl? What's the faster way to compare around 700,000 rows from 2 different databases using Perl? database database

What's the faster way to compare around 700,000 rows from 2 different databases using Perl?


Load each table, sorted, in to Perl in its entirety, then run Algorithm:Diff on the two lists. In the end you'll get a nice list of rows to delete, and rows to insert. Some rows may be deleted and reinserted (if you have foreign keys hanging of those rows, you'll need to do an update rather than an delete/insert).

700,000 rows is not a lot of data on modern machines, nor a lot of memory.

If you only need existence of rows (i.e. the rows exists or it doesn't rather than actual row changes), you can just do a diff on the keys, then fetch the rows you need from there.