In python, how can I load a sqlite db completely to memory before connecting to it? [duplicate] In python, how can I load a sqlite db completely to memory before connecting to it? [duplicate] sqlite sqlite

In python, how can I load a sqlite db completely to memory before connecting to it? [duplicate]


apsw is an alternate wrapper for sqlite, which enables you to backup an on-disk database to memory before doing operations.

From the docs:

###### Backup to memory#### We will copy the disk database into a memory databasememcon=apsw.Connection(":memory:")# Copy into memorywith memcon.backup("main", connection, "main") as backup:    backup.step() # copy whole database in one go# There will be no disk accesses for this queryfor row in memcon.cursor().execute("select * from s"):    pass

connection above is your on-disk db.


  1. Get an in-memory database running (standard stuff)
  2. Attach the disk database (file).
  3. Recreate tables / indexes and copy over contents.
  4. Detach the disk database (file)

Here's an example (taken from here) in Tcl (could be useful for getting the general idea along):

proc loadDB {dbhandle filename} {    if {$filename != ""} {        #attach persistent DB to target DB        $dbhandle eval "ATTACH DATABASE '$filename' AS loadfrom"        #copy each table to the target DB        foreach {tablename} [$dbhandle eval "SELECT name FROM loadfrom.sqlite_master WHERE type = 'table'"] {            $dbhandle eval "CREATE TABLE '$tablename' AS SELECT * FROM loadfrom.'$tablename'"        }        #create indizes in loaded table        foreach {sql_exp} [$dbhandle eval "SELECT sql FROM loadfrom.sqlite_master WHERE type = 'index'"] {            $dbhandle eval $sql_exp        }        #detach the source DB        $dbhandle eval {DETACH loadfrom}    }}


If you are using Linux, you can try tmpfs which is a memory-based file system.

It's very easy to use it:

  1. mount tmpfs to a directory.
  2. copy sqlite db file to the directory.
  3. open it as normal sqlite db file.

Remember, anything in tmpfs will be lost after reboot. So, you may copy db file back to disk if it changed.