How to map nested JSON in Log-stash HTTP Output
It is not possible to do this with the message
mapping in the http
output. That mapping can only create a single-level JSON.
What you can do, however, is to construct your JSON message before it reaches the http
output using the mutate/add_field
filter.
filter { grok { match => { "message" => "TID:%{SPACE}\[%{INT:SourceSystemId}\]%{SPACE}\[%{DATA:ProcessName}\]%{SPACE}\[%{TIMESTAMP_ISO8601:log_TimeStamp}\]%{SPACE}%{LOGLEVEL:log_MessageType}%{SPACE}{%{JAVACLASS:log_MessageTitle}}%{SPACE}-%{SPACE}%{GREEDYDATA:log_Message}" } } # add additional fields in your event here mutate { gsub => [ "log_TimeStamp", "\s", "T", "log_TimeStamp", ",", "." ] add_field => { "MessageId" => "654656" "TimeStamp" => "%{log_TimeStamp}" "CorrelationId" => "986565" "MessageType" => "%{log_MessageType}" "MessageTitle" => "%{log_MessageTitle}" "Message" => "%{log_Message}" "[MessageDetail][FieldA]" => "65656" "[MessageDetail][FieldB]" => "192.168.1.1" "[MessageDetail][FieldC]" => "sample value" } remove_field => ["@version", "@timestamp", "host", "message", "SourceSystemId", "ProcessName", "log_TimeStamp", "log_MessageType", "log_MessageTitle", "log_Message"] }}output { stdout { codec => "rubydebug" } http { url => "http://localhost:8087/messages" http_method => "post" format => "json" }}
You'll get exactly the JSON you expect posted to your HTTP endpoint
{ "MessageId": "654656", "TimeStamp": "2016-05-30T23:02:02.602", "CorrelationId": "986565", "MessageType": "INFO", "MessageTitle": "org.wso2.carbon.registry.core.jdbc.EmbeddedRegistryService", "Message": "Configured Registry in 572ms {org.wso2.carbon.registry.core.jdbc.EmbeddedRegistryService}", "MessageDetail": { "FieldA": "65656" "FieldB": "192.168.1.1" "FieldC": "sample value" }}