How does Amazon RDS backup/snapshot actually work?

We also run several RDS instances, in addition to MySQL on some machines that we manage ourselves. I can't comment specifically, as I'm not an Amazon engineer, but several things I've learned that might explain what you're seeing:

• Although Amazon does not share the backend details 100%, we strongly suspect that they are using their EBS system to back RDS databases.

• This article helps explain EBS limitations and snapshot functionality http://blog.rightscale.com/2008/08/20/amazon-ebs-explained/ Again, while it's not explicit, it would make sense for Amazon to be using this infrastructure to provide RDS services.

• Typically, a MySQL backup, in contrast to a snapshot, involves using a tool like mysqldump to create a file of SQL statements that will then reproduce the database. The database does not need to be frozen to do this. With an EBS backend, the best practice is to freeze the database (pause all transactions) while you are snapshotting to avoid data corruption.

• The spikes you're seeing at the ends of the backup window. If replication is paused by Amazon during the snapshot of your replica, the replica would then need to "catch up" on transactions when the snapshot was complete. This would cause a latency spike.

• Replication across a multi-AZ deployment is inherently slower then a single AZ deployment. The price you pay for better redundancy.

Amazon revealed the basic architecture that they use in Multi AZ deployments. This may help people to take decisions

https://aws.amazon.com/blogs/database/amazon-rds-under-the-hood-multi-az/