Is it viable to handle MySQL backups with git?

Normally you don't keep every backup (or snapshot) forever. A git repository does keep every checkin you ever make. If you ever decide to prune old revisions (say month-old revisions down to once a week, year old to once a month, etc) you will have to do it with git filter-branch which will rewrite the entire history. Then git gc to remove the unwanted revisions.

Given that git's strengths are distributed version control and complex patch/branch workflows (neither of which apply to snapshots or backups) I'd consider using a different VCS with a more malleable history.

This approach sounds fine to me. I use Git for backing up my own important data.

Note that you are not storing diffs -- Git effectively stores snapshots of the directory state with each commit. You can generate the diff of two commits, but the actual storage mechanism has nothing to do with diff.

In theory this will work, but you will start to have problems when the database dumps get large.

Git doesn't have any hard file size limits, but it will diff the contents of your latest dump with the one previously stored in the repository, which will require at least as much memory as the sizes of both of those files added together - so I would imagine it will start to get very slow, very quickly with files over 100MB (or even 10MB).

Git wasn't made for dealing with files of this type (i.e. big data files instead of source code), so I think this is fundamentally a bad idea. You could, however, use something like Dropbox to store the dumps - which will still save the version history for you, but is more tailored towards files which can't effectively be diffed.