Frameworks for creating asynchronous streaming API

How about finagle? https://github.com/twitter/finagle

It doesn't cover all your needs out of the box, but it's a very nice and extensible framework and might provide a good base to build upon.

And you can see an example of doing a streaming server using finagle:https://github.com/twitter/finagle/blob/master/finagle-example/src/main/scala/com/twitter/finagle/example/stream/StreamServer.scala

You should also definitely take a look at Akka IO:http://doc.akka.io/docs/akka/snapshot/scala/io.html

I would suggest Scramjet. It will work with your local and analog cases, but it will simply shine when given a task with multiple REST API's in the pipeline - in this case it scales logarithmic while other streaming systems scale exponentially (!). Here's a benchmark scramjet-benchmark on GitHub

It can be programmed with a chain of simple lambdas and is based on node.js with all the plethora of asynchronous modules you'll be able to run all your REST services in a single line (I suggest you tried request-promise). A simple case looks like this:

const scramjet = require("scramjet");const request = require("request-promise");process.stdin.pipe(new scramjet.StringStream())    .split("\r?\n")    .map((uri) => request.get("https://example.com/" + uri))    .filter((data) => data.status === 200 && data.articleUrl)    .map((data) => request.get("http://somesite.org/" + data.articleUrl)    .accumulate((data) => database.put(data.key, data))    .then(() => console.log('finished'))

And that's it for a software that calls two API's and writes the results to database.