Getting attribute's value in Nokogiri to extract link URLs
html = <<HTML <div id="block"> <a href="http://google.com">link</a> </div>HTMLdoc = Nokogiri::HTML(html)doc.xpath('//div/a/@href')#=> [#<Nokogiri::XML::Attr:0x80887798 name="href" value="http://google.com">]Or if you wanna be more specific about the div:
>> doc.xpath('//div[@id="block"]/a/@href')=> [#<Nokogiri::XML::Attr:0x80887798 name="href" value="http://google.com">]>> doc.xpath('//div[@id="block"]/a/@href').first.value=> "http://google.com"
doc = Nokogiri::HTML(open("[insert URL here]"))href = doc.css('#block a')[0]["href"]The variable href is assigned to the value of the "href" attribute for the <a> element inside the element with id 'block'. The line doc.css('#block a') returns a single item array containing the attributes of #block a. [0] targets that single element, which is a hash containing all the attribute names and values. ["href"] targets the key of "href" inside that hash and returns the value, which is a string containing the url.
Having struggled with this question in various forms, I decided to write myself a tutorial disguised as an answer. It may be helpful to others.
Starting with with this snippet:
require 'rubygems'require 'nokogiri'html = <<HTML<div id="block1"> <a href="http://google.com">link1</a></div><div id="block2"> <a href="http://stackoverflow.com">link2</a> <a id="tips">just a bookmark</a></div>HTMLdoc = Nokogiri::HTML(html)extracting all the links
We can use xpath or css to find all the elements and then keep only the ones that have an href attribute:
nodeset = doc.xpath('//a') # Get all anchors via xpathnodeset.map {|element| element["href"]}.compact # => ["http://google.com", "http://stackoverflow.com"]nodeset = doc.css('a') # Get all anchors via cssnodeset.map {|element| element["href"]}.compact # => ["http://google.com", "http://stackoverflow.com"]But there's a better way: in the above cases, the .compact is necessary because the searches return the "just a bookmark" element as well. We can use a more refined search to find just the elements that contain an href attribute:
attrs = doc.xpath('//a/@href') # Get anchors w href attribute via xpathattrs.map {|attr| attr.value} # => ["http://google.com", "http://stackoverflow.com"]nodeset = doc.css('a[href]') # Get anchors w href attribute via cssnodeset.map {|element| element["href"]} # => ["http://google.com", "http://stackoverflow.com"]finding a specific link
To find a link within the <div id="block2">
nodeset = doc.xpath('//div[@id="block2"]/a/@href')nodeset.first.value # => "http://stackoverflow.com"nodeset = doc.css('div#block2 a[href]')nodeset.first['href'] # => "http://stackoverflow.com"If you know you're searching for just one link, you can use at_xpath or at_css instead:
attr = doc.at_xpath('//div[@id="block2"]/a/@href')attr.value # => "http://stackoverflow.com"element = doc.at_css('div#block2 a[href]')element['href'] # => "http://stackoverflow.com"find a link from associated text
What if you know the text associated with a link and want to find its url? A little xpath-fu (or css-fu) comes in handy:
element = doc.at_xpath('//a[text()="link2"]')element["href"] # => "http://stackoverflow.com"element = doc.at_css('a:contains("link2")')element["href"] # => "http://stackoverflow.com"find text from a link
And what if you want to find the text associated with a particular link? Not a problem:
element = doc.at_xpath('//a[@href="http://stackoverflow.com"]')element.text # => "link2"element = doc.at_css('a[href="http://stackoverflow.com"]')element.text # => "link2"useful references
In addition to the extensive Nokorigi documentation, I came across some useful links while writing this up: