Django override_settings won't work for running multiple tests
Well, there is a warning about this very situation.
Warning
The settings file contains some settings that are only consulted during initialization of Django internals. If you change them with override_settings, the setting is changed if you access it via the django.conf.settings module, however, Django’s internals access it differently. Effectively, using override_settings() or modify_settings() with these settings is probably not going to do what you expect it to do.
The first time you run the tests, you are running a specific test case so the override takes effect. The second time you run the test, you are running a whole suite and your specific testcase probably isn't the first one that's being run. So the above happens.