Slow running MySql restore - 10 times slower than backup speed and still going Slow running MySql restore - 10 times slower than backup speed and still going database database

Slow running MySql restore - 10 times slower than backup speed and still going


Very large imports are notoriously hard to make fast. It sounds like your import is slowing down--processing fewer rows per second--as it progresses. That probably means MySQL is checking each new row to see whether it has key-conflicts with the rows already inserted.

A few things you can do:

Before starting, disable key checking.

   SET FOREIGN_KEY_CHECKS = 0;   SET UNIQUE_CHECKS = 0;

After ending restore your key checking.

  SET UNIQUE_CHECKS = 1;  SET FOREIGN_KEY_CHECKS = 1;

And, if you can wrap every few thousand lines of INSERT operations in

   START TRANSACTION;   INSERT ...   INSERT ...   ...   COMMIT;

you'll save a lot of disk churning.

Notice that this only matters for tables with many thousands of rows or more.

mysqldump can be made to create a dump with that disables keys. https://dev.mysql.com/doc/refman/5.7/en/mysqldump.html#option_mysqldump_disable-keys

mysqldump --disable-keys

Similarly,

mysqldump --extended-insert --no-autocommit

will make the dumped sql file contain a variant of my suggestion about using transactions.

In your case if you had used --opts --no-autocommit you probably would have gotten an optimal dump file. You already used --opts.


I changed my.ini and got some improvements while also using mysqldump --extended-insert --no-autocommit

my.ini for 16GB RAM on Windows 10 mysql 7.4

# Comment the following if you are using InnoDB tables#skip-innodbinnodb_data_home_dir="C:/xampp74/mysql/data"innodb_data_file_path=ibdata1:10M:autoextendinnodb_log_group_home_dir="C:/xampp74/mysql/data"#innodb_log_arch_dir = "C:/xampp74/mysql/data"## You can set .._buffer_pool_size up to 50 - 80 %## of RAM but beware of setting memory usage too high#innodb_buffer_pool_size=16Minnodb_buffer_pool_size=8G## Set .._log_file_size to 25 % of buffer pool size#innodb_log_file_size=5Minnodb_log_file_size=2Ginnodb_log_buffer_size=8M#innodb_flush_log_at_trx_commit=1#Use for restore onlyinnodb_flush_log_at_trx_commit=2innodb_lock_wait_timeout=50