How to deal with streaming data in PHP?

fopen and fgets

<?php$sock = fopen('http://domain.tld/path/to/file', 'r');$data = null;while(($data = fgets($sock)) == TRUE){    echo $data;}fclose($sock);

This is by no means great (or even good) code but it should provide the functionality you need. You will need to add error handling and data parsing among other things.

I'm pretty sure that your script will time out after ~30 seconds of listening for data on the stream. Even if it doesn't, once you get a significant server load, the sheer number of open and listening connections will bring the server to it's knees.

I would suggest you take a look at an AJAX solution that makes a call to a script that just stores a Queue of messages. I'm not sure how the Twitter API works exactly though, so I'm not sure if you can have a script run when requested to get all the tweets, or if you have to have some sort of daemon append the tweets to a Queue that PHP can read and pass back via your AJAX call.

There are libraries for this these days that make things much easier (and handle the tricky bits like reconnections, socket handling, TCP backoff, etc), ie:

http://code.google.com/p/phirehose/