How to integrate Jsoup with WebDriver? How to integrate Jsoup with WebDriver? selenium selenium

How to integrate Jsoup with WebDriver?


It's quite interesting we're doing the similar approach, integrating JSoup and Selenium WebDriver. I can understand your issue especially dealing with some dynamic website based on some Javascript framework which has no stable IDs or attributes.

Our solution looks like the following, and hopefully it could be some advice for you:

  • webDriver.getPageSource() to get the current HTML source
  • use JSoup to parse this HTML source, and leverage Jsoup selector (which is much more powerful than Selenium) to locate the target element
  • get parents or siblings of this element
  • write an iteration function to get element xPath, such as //body/div[2]/form[1]/input[3]
  • webDriver.findElement(By.xpath(...)) to locate element in selenium context

EDITED

The idea of the iteration function is:

  • first check the tag of your parent node, if it is body, then iteration ends
  • if not , then use getSiblings to check the index of the node among all the nodes with same tag, e.g, the 3rd div, then equals to div[3]
  • iterate to your parent node, and do the same procedures

Once you get the xpath of the child node, and parent node, just replace parent node xpath to be empty string inside the child node xpath, finally you can get the relative xpath.


What about running findElements with xpath : .//* on your particular element? Also, look into xpath parent::* and following-sibling::*. For the particular case I understand, there is no need for Jsoup.