Parsing XML data from Filebeat using Logstash Parsing XML data from Filebeat using Logstash elasticsearch elasticsearch

Parsing XML data from Filebeat using Logstash


Can you try editing the xpath configuration in the filter as below:

filter {    xml     {        source => "ticket"        xpath =>         [            "/IDH_Ticket/text()", "ticketId",            "/CodeBus/text()", "codeBus",            "/CodeCh/text()", "codeCh",            "/CodeConv/text()", "codeConv",            "/Codeligne/text()", "codeLigne",            "/Date/text()", "date",            "/Heur/text()", "heure",            "/NomFR1/text()", "nomFR1",            "/NomAR1/text()", "nomAR1",            "/NomFR2/text()", "nomFR2",            "/NomAR2/text()", "nomAR2",            "/Prix/text()", "prix",            "/IDTicket/text()", "idTicket",            "/CodeRoute/text()", "codeRoute",            "/origine/text()", "origine",            "/Distination/text()", "destination",            "/Num/text()", "num",            "/Ligne/text()", "ligne",            "/requisition/text()", "requisition",            "/voyage/text()", "voyage",            "/faveur/text()", "faveur"        ]        store_xml => true        target => "doc"    }}


The xml filter won't work since the source configuration points to a field that does not exist.
There are no field ticket in your document:

{    "message" => "\t<H_Ticket>\r\n\t\t<IDH_Ticket>1</IDH_Ticket>\r\n\t\t<CodeBus>186</CodeBus>\r\n\t\t<CodeCh>5531</CodeCh>\r\n\t\t<CodeConv>5531</CodeConv>\r\n\t\t<Codeligne>12</Codeligne>\r\n\t\t<Date>20150903</Date>\r\n\t\t<Heur>1101</Heur>\r\n\t\t<NomFR1>SOUK AHAD</NomFR1>\r\n\t\t<NomAR1>??? ?????</NomAR1>\r\n\t\t<NomFR2>SOVIVA </NomFR2>\r\n\t\t<NomAR2>??????</NomAR2>\r\n\t\t<Prix>0.66</Prix>\r\n\t\t<IDTicket>1</IDTicket>\r\n\t\t<CodeRoute>107</CodeRoute>\r\n\t\t<origine>01</origine>\r\n\t\t<Distination>07</Distination>\r\n\t\t<Num>3</Num>\r\n\t\t<Ligne>107</Ligne>\r\n\t\t<requisition> </requisition>\r\n\t\t<voyage>0</voyage>\r\n\t\t<faveur> </faveur>\r\n\t</H_Ticket>",    "@version" => "1",    "@timestamp" => "2016-07-03T12:13:28.892Z",    "input_type" => "log",    "source" => "C:\\busesdata\\ticket2.xml",    "offset" => 125,    "type" => "ticket",    "count" => 1,    "fields" => nil,    "beat" => {        "hostname" => "hp-pavillion-g6",        "name" => "hp-pavillion-g6"    },    "host" => "hp-pavillion-g6",    "tags" => [        [0] "beats_input_codec_plain_applied"    ]}

You should change the xml filter to:

 xml {        source => "message"        ... }