What is an efficient way of inserting thousands of records into an SQLite table using Django?
You want to check out django.db.transaction.commit_manually.
http://docs.djangoproject.com/en/dev/topics/db/transactions/#django-db-transaction-commit-manually
So it would be something like:
from django.db import transaction@transaction.commit_manuallydef viewfunc(request): ... for item in items: entry = Entry(a1=item.a1, a2=item.a2) entry.save() transaction.commit()Which will only commit once, instead at each save().
In django 1.3 context managers were introduced.So now you can use transaction.commit_on_success() in a similar way:
from django.db import transactiondef viewfunc(request): ... with transaction.commit_on_success(): for item in items: entry = Entry(a1=item.a1, a2=item.a2) entry.save()In django 1.4, bulk_create was added, allowing you to create lists of your model objects and then commit them all at once.
NOTE the save method will not be called when using bulk create.
>>> Entry.objects.bulk_create([... Entry(headline="Django 1.0 Released"),... Entry(headline="Django 1.1 Announced"),... Entry(headline="Breaking: Django is awesome")... ])In django 1.6, transaction.atomic was introduced, intended to replace now legacy functions commit_on_success and commit_manually.
from the django documentation on atomic:
atomic is usable both as a decorator:
from django.db import transaction@transaction.atomicdef viewfunc(request): # This code executes inside a transaction. do_stuff()and as a context manager:
from django.db import transactiondef viewfunc(request): # This code executes in autocommit mode (Django's default). do_stuff() with transaction.atomic(): # This code executes inside a transaction. do_more_stuff()