R - How to make a click on webpage using rvest or rcurl R - How to make a click on webpage using rvest or rcurl r r

R - How to make a click on webpage using rvest or rcurl


Sometimes it's better to attack the problem at the ajax web-request level. For this site, you can use Chrome's dev tools and watch the requests. To build the table (the whole table, too) it makes a POST to the site with various ajax-y parameters. Just replicate that, do a bit of data-munging of the response and you're good to go:

library(httr)library(rvest)library(dplyr)res <- POST("http://www.tradingeconomics.com/",            encode="form",            user_agent("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.50 Safari/537.36"),            add_headers(`Referer`="http://www.tradingeconomics.com/",                        `X-MicrosoftAjax`="Delta=true"),            body=list(              `ctl00$AjaxScriptManager1$ScriptManager1`="ctl00$ContentPlaceHolder1$defaultUC1$CurrencyMatrixAllCountries1$UpdatePanel1|ctl00$ContentPlaceHolder1$defaultUC1$CurrencyMatrixAllCountries1$LinkButton1",              `__EVENTTARGET`="ctl00$ContentPlaceHolder1$defaultUC1$CurrencyMatrixAllCountries1$LinkButton1",              `srch-term`="",              `ctl00$ContentPlaceHolder1$defaultUC1$CurrencyMatrixAllCountries1$GridView1$ctl01$DropDownListCountry`="top",              `ctl00$ContentPlaceHolder1$defaultUC1$CurrencyMatrixAllCountries1$ParameterContinent`="",              `__ASYNCPOST`="false"))res_t <- content(res, as="text")res_h <- paste0(unlist(strsplit(res_t, "\r\n"))[-1], sep="", collapse="\n")css <- "#ctl00_ContentPlaceHolder1_defaultUC1_CurrencyMatrixAllCountries1_GridView1"tab <- html(res_h) %>%   html_nodes(css) %>%  html_table() tab[[1]]$COUNTRIESWORLDAMERICAEUROPEASIAAUSTRALIAAFRICAglimpse(tab[[1]]

Another alternative would have been to use RSelenium to go to the page, click the "+" and then scrape the resultant table.