Commit manually in Django data migration

The best workaround I found is manually exiting the atomic scope before running the data migration:

def modify_data(apps, schema_editor):    schema_editor.atomic.__exit__(None, None, None)    # [...]

In contrast to resetting connection.in_atomic_block manually this allows using atomic context manager inside the migration. There doesn't seem to be a much saner way.

One can contain the (admittedly messy) transaction break out logic in a decorator to be used with the RunPython operation:

def non_atomic_migration(func):  """  Close a transaction from within code that is marked atomic. This is  required to break out of a transaction scope that is automatically wrapped  around each migration by the schema editor. This should only be used when  committing manually inside a data migration. Note that it doesn't re-enter  the atomic block afterwards.  """  @wraps(func)  def wrapper(apps, schema_editor):      if schema_editor.connection.in_atomic_block:          schema_editor.atomic.__exit__(None, None, None)      return func(apps, schema_editor)  return wrapper

Update

Django 1.10 will support non-atomic migrations.

From the documentation about RunPython:

By default, RunPython will run its contents inside a transaction on databases that do not support DDL transactions (for example, MySQL and Oracle). This should be safe, but may cause a crash if you attempt to use the schema_editor provided on these backends; in this case, pass atomic=False to the RunPython operation.

So, instead of what you've got:

class Migration(migrations.Migration):  operations = [      migrations.RunPython(modify_data, atomic=False),  ]

For others coming across this. You can have both data (RunPython), in the same migration. Just make sure all the alter tables goes first. You cannot do the RunPython before any ALTER TABLE.