In python, how can I load a sqlite db completely to memory before connecting to it? [duplicate]
apsw is an alternate wrapper for sqlite, which enables you to backup an on-disk database to memory before doing operations.
From the docs:
###### Backup to memory#### We will copy the disk database into a memory databasememcon=apsw.Connection(":memory:")# Copy into memorywith memcon.backup("main", connection, "main") as backup: backup.step() # copy whole database in one go# There will be no disk accesses for this queryfor row in memcon.cursor().execute("select * from s"): pass
connection
above is your on-disk db.
- Get an in-memory database running (standard stuff)
- Attach the disk database (file).
- Recreate tables / indexes and copy over contents.
- Detach the disk database (file)
Here's an example (taken from here) in Tcl (could be useful for getting the general idea along):
proc loadDB {dbhandle filename} { if {$filename != ""} { #attach persistent DB to target DB $dbhandle eval "ATTACH DATABASE '$filename' AS loadfrom" #copy each table to the target DB foreach {tablename} [$dbhandle eval "SELECT name FROM loadfrom.sqlite_master WHERE type = 'table'"] { $dbhandle eval "CREATE TABLE '$tablename' AS SELECT * FROM loadfrom.'$tablename'" } #create indizes in loaded table foreach {sql_exp} [$dbhandle eval "SELECT sql FROM loadfrom.sqlite_master WHERE type = 'index'"] { $dbhandle eval $sql_exp } #detach the source DB $dbhandle eval {DETACH loadfrom} }}
If you are using Linux, you can try tmpfs which is a memory-based file system.
It's very easy to use it:
- mount tmpfs to a directory.
- copy sqlite db file to the directory.
- open it as normal sqlite db file.
Remember, anything in tmpfs will be lost after reboot. So, you may copy db file back to disk if it changed.