How to master in-place array modification algorithms?

About an O(n) time, O(1) space algorithm for out-shuffle

Doing an out-shuffle in O(n) time and O(1) space is possible, but it is tough. Not sure why people think it is easy and are suggesting you try something else.

The following paper has an O(n) time and O(1) space solution (though it is for in-shuffle, doing in-shuffle makes out-shuffle trivial):

http://arxiv.org/PS_cache/arxiv/pdf/0805/0805.1598v1.pdf

About a method to tackle in-place array modification algorithms

In-place modification algorithms could become very hard to handle.

Consider a couple:

• Inplace out-shuffle in linear time. Uses number theory.
• In-place merge sort, was open for a few years. An algorithm came but was too complicated to be practical. Uses very complicated bookkeeping.

Sorry, if this sounds discouraging, but there is no magic elixir that will solve all in-place algorithm problems for you. You need to work with the problem, figure out its properties, and try to exploit them (as is the case with most algorithms).

That said, for array modifications where the result is a permutation of the original array, you can try the method of following the cycles of the permutation. Basically, any permutation can be written as a disjoint set of cycles (see John's answer too). For instance the permutation:

1 4 2 5 3 6

of 1 2 3 4 5 6 can be written as

1 -> 12 -> 3 -> 5 -> 4 -> 26 -> 6.

you can read the arrow as 'goes to'.

So to permute the array 1 2 3 4 5 6 you follow the three cycles:

1 goes to 1.

6 goes to 6.

2 goes to 3, 3 goes to 5, 5 goes to 4, and 4 goes to 2.

To follow this long cycle, you can use just one temp variable. Store 3 in it. Put 2 where 3 was. Now put 3 in 5 and store 5 in the temp and so on. Since you only use constant extra temp space to follow a particular cycle, you are doing an in-place modification of the array for that cycle.

Now if I gave you a formula for computing where an element goes to, all you now need is the set of starting elements of each cycle.

A judicious choice of the starting points of the cycles can make the algorithm easy. If you come up with the starting points in O(1) space, you now have a complete in-place algorithm. This is where you might actually have to get familiar with the problem and exploit its properties.

Even if you didn't know how to compute the starting points of the cycles, but had a formula to compute the next element, you could use this method to get an O(n) time in-place algorithm in some special cases.

For instance: if you knew the array of unsigned integers held only positive integers.

You can now follow the cycles, but negate the numbers in them as an indicator of 'visited' elements. Now you can walk the array and pick the first positive number you come across and follow the cycles for that, making the elements of the cycle negative and continue to find untouched elements. In the end, you just make all the elements positive again to get the resulting permutation.

You get an O(n) time and O(1) space algorithm! Of course, we kind of 'cheated' by using the sign bits of the array integers as our personal 'visited' bitmap.

Even if the array was not necessarily integers, this method (of following the cycles, not the hack of sign bits :-)) can actually be used to tackle the two problems you state:

• The in-shuffle (or out-shuffle) problem: When 2n+1 is a power of 3, it can be shown (using number theory) that 1,3,3^2, etc are in different cycles and all cycles are covered using those. Combine this with the fact that the in-shuffle is susceptible to divide and conquer, you get an O(n) time, O(1) space algorithm (the formula is i -> 2*i modulo 2n+1). Refer to the above paper for more details.

• The cyclic shift an array problem: Cyclic shift an array of size n by k also gives a permutation of the resulting array (given by the formula i goes to i+k modulo n), and can also be solved in linear time and in-place using the following the cycle method. In fact, in terms of the number of element exchanges this following cycle method is better than the 3 reverses algorithm. Of course, following the cycle method can kill the cache because of the access patterns, and in practice, the 3 reverses algorithm might actually fare better.

As for interviews, if the interviewer is a reasonable person, they will be looking at how you think and approach the problem and not whether you actually solve it. So even if you don't solve a problem, I think you should not be discouraged.

The basic strategy with in place algorithms is to figure out the rule for moving a entry from slot N to slot M.

So, your shuffle, for instance. if A and B are cards and N is the number of chards. the rules for the first half of the deck are different than the rules for the second half of the deck

 // A is the current location, B is the new location. // this math assumes that the first card is card 0 if (A < N/2)    B = A * 2; else    B = (A - N/2) * 2 + 1;

Now we know the rule, we just have to move each card, each time we move a card, we calculate the new location, then remove the card that is currently in B. place A in slot B, then let B be A, and loop back to the top of the algorithm. Each card moved displaces the new card which becomes the next card to be moved.

I think the analysis is easier if we are 0 based rather than 1 based, so

 0 1 2 3 4 5 6 7  // before 0 4 1 5 2 6 3 7  // after

So we want to move 1->2 2->4 4->1 and that completes a cyclethen move 3->6 6->5 5->3 and that completes a cycleand we are done.

Now we know that card 0 and card N-1 don't move, so we can ignore those,so we know that we only need to swap N-2 cards in total. The only sticky bitis that there are 2 cycles, 1,2,4,1 and 3,6,5,3. when we get to card 1 thesecond time, we need to move on to card 3.

 int A = 1; int N = 8; card ary[N]; // Our array of cards card a = ary[A]; for (int i = 0; i < N/2; ++i) {     if (A < N/2)        B = A * 2;     else        B = (A - N/2) * 2 + 1;     card b = ary[B];     ary[B] = a;     a = b;     A = B;     if (A == 1)     {        A = 3;        a = ary[A];     } }   

Now this code only works for the 8 card example, because of that if test that moves us from 1 to 3 when we finish the first cycle. What we really need is a general rule to recognize the end of the cycle, and where to go to start the next one.

That rule could be mathematical if you can think of a way, or you could keep track of which places you had visited in a separate array, and when A is back to a visited place, you could then scan forward in your array looking for the first non-visited place.

For your in-place algorithm to be 0(n), the solution will need to be mathematical.

I hope this breakdown of the thinking process is helpful to you. If I was interviewing you, I would expect to see something like this on the whiteboard.

Note: As Moron points out, this doesn't work for all values of N, it's just an example of the sort of analysis that an interviewer is looking for.

Frank,

For programming with loops and arrays, nothing beats David Gries's textbook The Science of Programming. I studied it over 20 years ago, and there are ideas that I still use every day. It is very mathematical and will require real effort to master, but that effort will repay you many times over for your whole career.