Is there a good way to integrate beanstalkd with hadoop? Is there a good way to integrate beanstalkd with hadoop? hadoop hadoop

Is there a good way to integrate beanstalkd with hadoop?


Really interesting question.

Checkout Monolog and here is a great tutorial about handling logs with Fluentd.

You might want to consider a "triangle" of services, Laravel comes with Monolog, also with Beanstalkd support, and you can quickly add via composer Fluentd. So you could have a Laravel app that has workers on your tubes, and forwards to Fluentd. Fluentd has some great features like delayed log and tags. Not sure how you will handle delayed logs, or tags on the logs, but you probably already have those on your tube message.

You could use Beanstalkd Console to view your jobs and help you in development.