Returning data from forked processes

We actually just had to handle this problem in Rails isolation testing. I posted about it some on my blog.

Basically, what you want to do is open a pipe in the parent and child, and have the child write to the pipe. Here's a simple way to run the contents of a block in a child process and get back the result:

def do_in_child  read, write = IO.pipe  pid = fork do    read.close    result = yield    Marshal.dump(result, write)    exit!(0) # skips exit handlers.  end  write.close  result = read.read  Process.wait(pid)  raise "child failed" if result.empty?  Marshal.load(result)end

Then you could run:

do_in_child do  require "some_polluting_library"  SomePollutingLibrary.some_operationend

Note that if you do a require in the child, you will not have access to that library in the parent, so you cannot return an object of that type using this method. However, you could return any type that's available in both.

Also note that a lot of the details here (read.close, Process.wait2(pid)) are mostly housekeeping details, so if you use this a lot you should probably move this out into a utility library that you can reuse.

Finally, note that this will not work on Windows or JRuby, since they don't support forking.

Thanks for all the answers, I got my solution up and running, still need to see how to handle non-forking environments, but for now it works :)

read, write = IO.pipeProcess.fork do  write.puts "test"endProcess.fork do  write.puts 'test 2'endProcess.waitProcess.waitwrite.closeputs read.readread.close

you can see it in action @ parallel_specs Rails plugin

I wrapped all the solutions I found along the way (some other problems like user exiting + piping-buffers) into ruby parallel gem. Now it is as easy as:

results = Parallel.map([1,2,3],:in_processes=>4) do |i|  execute_something(i)end

or

results = Parallel.map([1,2,3],:in_threads=>4) do |i|  execute_something(i)end