Can this SQLite query be made much faster? Can this SQLite query be made much faster? sqlite sqlite

Can this SQLite query be made much faster?


You need a clustered index, or if you are using a version of SQLite which doesn't support one, a covering index.

Sqlite 3.8.2 and above

Use this in SQLite 3.8.2 and above:

create table recording (  camera_id integer references camera (id) not null,  sample_file_bytes integer not null check (sample_file_bytes > 0),  -- The starting time of the recording, in 90 kHz units since  -- 1970-01-01 00:00:00 UTC.  start_time_90k integer not null check (start_time_90k >= 0),  -- The duration of the recording, in 90 kHz units.  duration_90k integer not null      check (duration_90k >= 0 and duration_90k < 5*60*90000),  video_samples integer not null check (video_samples > 0),  video_sync_samples integer not null check (video_samples > 0),  video_sample_entry_id integer references video_sample_entry (id),  --- here is the magic  primary key (camera_id, start_time_90k)) WITHOUT ROWID;

Earlier Versions

In earlier versions of SQLite you can use this sort of thing to create a covering index. This should allow SQLite to pull the data values from the index, avoiding fetching a separate page for each row:

create index recording_camera_start on recording (     camera_id, start_time_90k,     sample_file_bytes, duration_90k, video_samples, video_sync_samples, video_sample_entry_id );

Discussion

The cost is likely to be IO (regardless of that you said it wasn't) because recall that IO requires CPU as data must be copied to and from the bus.

Without a clustered index, rows are inserted with a rowid, and may not be in any sensible order. That means that for each 26 byte row you request, the system may have to fetch a 4KB page from the SD card - which is a lot of overhead.

With a limit of 8 cameras, a simple clustered index on id to ensure they appear on disk in inserted order would probably give you about 10x speed increase by ensuring that the fetched page contains the next 10-20 rows which are going to be required.

A clustered index on both camera and time should ensure that each page fetched contains 100 or more rows.