What tool to test complex multi-processes system?

You don't tell us enough about your setup to give you specific answers. It strongly depends on the language/environment you are using. You probably need a combination of one or more of the following:

• Vagrant/Chef or Puppet/Veewee/VirtualBox to automatically build VMs and configure and run them. That's assuming a Linux platform. Chef + Linux containers is a lot faster, especially if you run it on a ram disk (it is supposed to be a repeatable process, so only the artifacts need to be stored)
• A package cache with version control for all the system packages you need to install on the VMs
• Some framework to run acceptance tests, Cucumber, Robot or so. If your unit tests are good, you might be able to get away with smoke tests.
• A simulation of your system, that you can feed run-time results from the log so you know how and when to scale.

You are very likely to run into the slow tests problem as you have to populate a database, and might have to split test jobs. In general, the feedback loop to get this working is way too slow, and you'll probably need to be very creative to get an acceptable cycle time.

In different environments, there are different tools for this, and having multiple stacks is not much fun. Jenkins runs on a Java stack, Puppet & Chef on a Ruby stack, and there are similar tools for the Python, c & perl stack. You'll have to decide for yourself if you want to have best-in-class with integration problems, or a single stack and coding things yourself. Both can work, and neither is trivial. Team experience is most likely the deciding factor there.

The feedback loop in DevOps is a lot worse than that of a modern CI development environment, so you'll have to do much more planning & research up front. The number of experiments you can run in a day is much lower, at least a factor 20 or so, so making things work the first time is going to be difficult. Make sure you have experienced people doing that.

The answer is: Robot Framework (see 1 or 2).

• Supports Data Driven Tests and Behavioural Tests
• Tests are written using user-defined keywords, in a human-like language. Easy to develop, easy to read, easy to maintain.
• Lot of libraries to access different functionality (ssh connections, database, .....)
• Can be extended with custom libraries written in python, java or other languages.
• For each test a detailed log is generated

I don't know Jenkins, but I am pretty sure you can integrate Robot Framework with it. It's as simple as running a Python script.

Depending on the complexity of your system you can use a pletora of tools. Generically, you need some sort of job flow with visualization of results at the end. Instead of giving a deifnitive answer, let me point you to some tools/plugins.

Jenkins:

• The MultiJob plugin let's you specify the workflow into phases, with parallel jobs phase
• The BuildFlow plugin let's you create your own build flow with parallel and dependent jobs in a simple DSL.
• The Plot plugin to collect metrics and plot these over time
• The xUnit plugin to capture JUnit style xml or JSON from the tests, for instance generated by a little script or an xUnit framework such as py.test for Python or something more fancy.
• RobotFramework plugin related to OFrandeDiEnne's answer.

Vagrant: Vagrant is basically a command-line interface on top of VirtualBox VMs. You can use this to start multiple VMs simultaneously (can even be done from Jenkins). Use a provisioner such as Puppet, Chef, SaltStack to setup your database, configuration, etc.

Build flow tools, such as offered by Run-Time Design Automation (RTDA). Commercial. This will most likely be an overkill.