How to parse XML to R data frame How to parse XML to R data frame xml xml

How to parse XML to R data frame


Data in XML format are rarely organized in a way that would allow the xmlToDataFrame function to work. You're better off extracting everything in lists and then binding the lists together in a data frame:

require(XML)data <- xmlParse("http://forecast.weather.gov/MapClick.php?lat=29.803&lon=-82.411&FcstType=digitalDWML")xml_data <- xmlToList(data)

In the case of your example data, getting location and start time is fairly straightforward:

location <- as.list(xml_data[["data"]][["location"]][["point"]])start_time <- unlist(xml_data[["data"]][["time-layout"]][    names(xml_data[["data"]][["time-layout"]]) == "start-valid-time"])

Temperature data is a bit more complicated. First you need to get to the node that contains the temperature lists. Then you need extract both the lists, look within each one, and pick the one that has "hourly" as one of its values. Then you need to select only that list but only keep the values that have the "value" label:

temps <- xml_data[["data"]][["parameters"]]temps <- temps[names(temps) == "temperature"]temps <- temps[sapply(temps, function(x) any(unlist(x) == "hourly"))]temps <- unlist(temps[[1]][sapply(temps, names) == "value"])out <- data.frame(  as.list(location),  "start_valid_time" = start_time,  "hourly_temperature" = temps)head(out)  latitude longitude          start_valid_time hourly_temperature1    29.81    -82.42 2013-06-19T16:00:00-04:00                 912    29.81    -82.42 2013-06-19T17:00:00-04:00                 903    29.81    -82.42 2013-06-19T18:00:00-04:00                 894    29.81    -82.42 2013-06-19T19:00:00-04:00                 855    29.81    -82.42 2013-06-19T20:00:00-04:00                 836    29.81    -82.42 2013-06-19T21:00:00-04:00                 80


Use xpath more directly for both performance and clarity.

time_path <- "//start-valid-time"temp_path <- "//temperature[@type='hourly']/value"df <- data.frame(    latitude=data[["number(//point/@latitude)"]],    longitude=data[["number(//point/@longitude)"]],    start_valid_time=sapply(data[time_path], xmlValue),    hourly_temperature=as.integer(sapply(data[temp_path], as, "integer"))

leading to

> head(df, 2)  latitude longitude          start_valid_time hourly_temperature1    29.81    -82.42 2014-02-14T18:00:00-05:00                 602    29.81    -82.42 2014-02-14T19:00:00-05:00                 55


Here's a partial solution using xml2. Breaking the solution up into smaller pieces generally makes it easier to ensure everything is lined up:

library(xml2)data <- read_xml("http://forecast.weather.gov/MapClick.php?lat=29.803&lon=-82.411&FcstType=digitalDWML")# Point locationspoint <- data %>% xml_find_all("//point")point %>% xml_attr("latitude") %>% as.numeric()point %>% xml_attr("longitude") %>% as.numeric()# Start timedata %>%   xml_find_all("//start-valid-time") %>%   xml_text()# Temperaturedata %>%   xml_find_all("//temperature[@type='hourly']/value") %>%   xml_text() %>%   as.integer()