Parsing XML data from Filebeat using Logstash
Can you try editing the xpath
configuration in the filter
as below:
filter { xml { source => "ticket" xpath => [ "/IDH_Ticket/text()", "ticketId", "/CodeBus/text()", "codeBus", "/CodeCh/text()", "codeCh", "/CodeConv/text()", "codeConv", "/Codeligne/text()", "codeLigne", "/Date/text()", "date", "/Heur/text()", "heure", "/NomFR1/text()", "nomFR1", "/NomAR1/text()", "nomAR1", "/NomFR2/text()", "nomFR2", "/NomAR2/text()", "nomAR2", "/Prix/text()", "prix", "/IDTicket/text()", "idTicket", "/CodeRoute/text()", "codeRoute", "/origine/text()", "origine", "/Distination/text()", "destination", "/Num/text()", "num", "/Ligne/text()", "ligne", "/requisition/text()", "requisition", "/voyage/text()", "voyage", "/faveur/text()", "faveur" ] store_xml => true target => "doc" }}
The xml filter won't work since the source configuration points to a field that does not exist.
There are no field ticket
in your document:
{ "message" => "\t<H_Ticket>\r\n\t\t<IDH_Ticket>1</IDH_Ticket>\r\n\t\t<CodeBus>186</CodeBus>\r\n\t\t<CodeCh>5531</CodeCh>\r\n\t\t<CodeConv>5531</CodeConv>\r\n\t\t<Codeligne>12</Codeligne>\r\n\t\t<Date>20150903</Date>\r\n\t\t<Heur>1101</Heur>\r\n\t\t<NomFR1>SOUK AHAD</NomFR1>\r\n\t\t<NomAR1>??? ?????</NomAR1>\r\n\t\t<NomFR2>SOVIVA </NomFR2>\r\n\t\t<NomAR2>??????</NomAR2>\r\n\t\t<Prix>0.66</Prix>\r\n\t\t<IDTicket>1</IDTicket>\r\n\t\t<CodeRoute>107</CodeRoute>\r\n\t\t<origine>01</origine>\r\n\t\t<Distination>07</Distination>\r\n\t\t<Num>3</Num>\r\n\t\t<Ligne>107</Ligne>\r\n\t\t<requisition> </requisition>\r\n\t\t<voyage>0</voyage>\r\n\t\t<faveur> </faveur>\r\n\t</H_Ticket>", "@version" => "1", "@timestamp" => "2016-07-03T12:13:28.892Z", "input_type" => "log", "source" => "C:\\busesdata\\ticket2.xml", "offset" => 125, "type" => "ticket", "count" => 1, "fields" => nil, "beat" => { "hostname" => "hp-pavillion-g6", "name" => "hp-pavillion-g6" }, "host" => "hp-pavillion-g6", "tags" => [ [0] "beats_input_codec_plain_applied" ]}
You should change the xml filter to:
xml { source => "message" ... }