Manual indexing over jagged arrays?

Surprisingly, this was also faster than a jagged array by around 15% for accesses

This should come as no surprise at all, because indexing a jagged array requires an additional dereference. When you write a[i][j], computer must do the following:

• Compute the location of nested array i inside the jagged array a
• Fetch the location of the nested array a[i] (first dereference)
• Compute the location of element j in a[i]
• Fetch the value at location j of a[i] (second dereference)

When you fold 2D array in a vector, the computer does only one dereference:

• Compute the location of the target element
• Fetch the value from the array (the only dereference)

Essentially, you are trading a dereference for a multiplication; multiplication is cheaper.

In addition, you get continuity of elements in memory - something that you cannot guarantee with a jagged array. This becomes important for code that is sensitive to cache hits.

Is there anything I can do to recoup the speed benefits without having to use my own manual indexing?

Using your indexing scheme is a way to go. You can hide it from viewers of your code by making a class, say, Matrix2D exposing an operator [] that takes two indexes and produces the value. This way the code that computes the offset would be hidden from readers of your program, because the a[i * 24 + j] part would look like a[i, j]