How to configure Flume to listen a web api http petitions How to configure Flume to listen a web api http petitions hadoop hadoop

How to configure Flume to listen a web api http petitions


The standard Flume 'HTTPSource', and its default JSONHandler, will only process an event in a specific, Flume-centric format.

That format is documented in the user manual, and also in the comments at the beginning of the JSONHandler source code.

In summary, it expects to receive a list of JSON objects, each one containing headers (key/value pairs, mapped to the Flume Event headers) and body (a simple string, mapped to the Flume Event body).

To take your example, if you send:

[{"headers": {}, "body": "{\"ano\":\"2010\"}"}]

I think you'd get what you were looking for.

If you don't have the flexibility to change what you send, then you may be able to use org.apache.flume.source.http.BLOBHandler, depending upon what processing you are trying to do (NB. there's no documentation in the manual for this, only for org.apache.flume.sink.solr.morphline.BlobHandler - they are not the same thing, but there are some notes in FLUME-2718), or you may need to provide your own implementation of Flume's HTTPSourceHandler interface instead.

Side note: the HTTP Source bind option requires a hostname or IP address. You may just be being lucky with your value being treated as the hostname, and the path being ignored.