python, collecting links / script values from page python, collecting links / script values from page selenium selenium

python, collecting links / script values from page


I think you pasted the wrong script of yours ;)

I'm not sure what you need exactly - there are at least two different approaches.

  • Matching all hrefs using regex
  • Matching specific tags and using getAttribute(...)

For the first one, you have to get the whole html source of the page with something like webdriver.page_source and using something like the following regex (you will have to escape either the normal or the double quotes!):

<a.+?href=['"](.*?)['"].*?/?>

If you need the hrefs of all matching links, you could use something similar to webdriver.find_elements_by_css_selector('.visit') (take care to choose find_elements_... instead of find_element_...!) to obtain a list of webelements and iterate through them to get their attributes.

This could result in code like this:

hrefs = []elements = webdriver.find_elements_by_css_selector('.visit')for element in elements:    hrefs.append(element.getAttribute('href'))

Or a one liner using list comprehension:

hrefs = [element.getAttribute('href') for element \         in webdriver.find_elements_by_css_selector('.visit')]