Scrape website with dynamic mouseover event

To trigger the mouseover event for each circle you have to induce WebDriverWait for the visibility_of_all_elements_located() and you can use the following Locator Strategies:

• Code Block:

  from selenium import webdriverfrom selenium.webdriver.common.by import Byfrom selenium.webdriver.support.ui import WebDriverWaitfrom selenium.webdriver.support import expected_conditions as ECfrom selenium.webdriver.common.action_chains import ActionChainschrome_options = webdriver.ChromeOptions()chrome_options.add_argument("start-maximized")chrome_options.add_experimental_option("excludeSwitches", ["enable-automation"])chrome_options.add_experimental_option('useAutomationExtension', False)driver = webdriver.Chrome(options=chrome_options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')driver.get("https://slushpool.com/stats/?c=btc")driver.execute_script("return arguments[0].scrollIntoView(true);", WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//h1//span[text()='Distribution']"))))elements = WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//h1//span[text()='Distribution']//following::div[1]/*[name()='svg']//*[name()='g']//*[name()='g' and @class='paper']//*[name()='circle']")))for element in elements:    ActionChains(driver).move_to_element(element).perform()

• Browser Snapshot:

This is the circle locator you mean:

.find_element_by_css_selector('._1p0PmxVw._3GzjmWLG')

But it will change because mouseover effect, to be:

.find_element_by_css_selector('._1p0PmxVw._3GzjmWLG._1suU9Mx1')

So you need wait until the element to changed for each move.

And the most important is how to inspect a hover element, then you can get the bellow:

And causes the element for get data you mean to be appear:

xpath: //div[@class="_3jGHi0co _1zbokARu" and contains(@style,"display: block")]

You can use ActionChains to perform move the element.

Finally you can try the bellow code:

browser.get('https://slushpool.com/stats/?c=btc')browser.maximize_window()#wait all circleelements = WebDriverWait(browser, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, '._1p0PmxVw._3GzjmWLG')))table = browser.find_element_by_class_name('paper')#move perform -> to tablebrowser.execute_script("arguments[0].scrollIntoView(true);", table)data = []for circle in elements:    #move perform -> to each circle    ActionChains(browser).move_to_element(circle).perform()    # wait change mouseover effect    mouseover = WebDriverWait(browser, 5).until(EC.visibility_of_element_located((By.XPATH, '//div[@class="_3jGHi0co _1zbokARu" and contains(@style,"display: block")]')))    data.append(mouseover.text)print(data[0])print(data)

Following import:

from selenium.webdriver.common.by import Byfrom selenium.webdriver.support.ui import WebDriverWaitfrom selenium.webdriver.support import expected_conditions as ECfrom selenium.webdriver import ActionChains

Console output:

• First data > data[0]
  536.9 Ph/s - 1.074 Eh/s
  User Count 2
  Average Hash Rate 546.1 Ph/s
  Group Hash Rate 1.092 Eh/s
• All data > data

[u'536.9 Ph/s - 1.074 Eh/s\nUser Count 2\nAverage Hash Rate 546.9 Ph/s\nGroup Hash Rate 1.094 Eh/s', u'67.11 Ph/s - 134.2 Ph/s\nUser Count 14\nAverage Hash Rate 91.27 Ph/s\nGroup Hash Rate 1.278 Eh/s', u'67.11 Ph/s - 134.2 Ph/s\nUser Count 14\nAverage Hash Rate 91.27 Ph/s\nGroup Hash Rate 1.278 Eh/s', u'16.78 Ph/s - 33.55 Ph/s\nUser Count 23\nAverage Hash Rate 23.36 Ph/s\nGroup Hash Rate 537.2 Ph/s', u'8.389 Ph/s - 16.78 Ph/s\nUser Count 33\nAverage Hash Rate 11.80 Ph/s\nGroup Hash Rate 389.4 Ph/s', u'4.194 Ph/s - 8.389 Ph/s\nUser Count 67\nAverage Hash Rate 5.704 Ph/s\nGroup Hash Rate 382.2 Ph/s', u'2.097 Ph/s - 4.194 Ph/s\nUser Count 137\nAverage Hash Rate 2.959 Ph/s\nGroup Hash Rate 405.3 Ph/s', u'1.049 Ph/s - 2.097 Ph/s\nUser Count 233\nAverage Hash Rate 1.475 Ph/s\nGroup Hash Rate 343.7 Ph/s', u'1.049 Ph/s - 2.097 Ph/s\nUser Count 233\nAverage Hash Rate 1.475 Ph/s\nGroup Hash Rate 343.7 Ph/s', u'524.3 Th/s - 1.049 Ph/s\nUser Count 397\nAverage Hash Rate 731.4 Th/s\nGroup Hash Rate 290.4 Ph/s', u'262.1 Th/s - 524.3 Th/s\nUser Count 745\nAverage Hash Rate 360.3 Th/s\nGroup Hash Rate 268.4 Ph/s', u'131.1 Th/s - 262.1 Th/s\nUser Count 1479\nAverage Hash Rate 182.7 Th/s\nGroup Hash Rate 270.1 Ph/s', u'65.54 Th/s - 131.1 Th/s\nUser Count 2351\nAverage Hash Rate 92.47 Th/s\nGroup Hash Rate 217.4 Ph/s', u'32.77 Th/s - 65.54 Th/s\nUser Count 3107\nAverage Hash Rate 47.23 Th/s\nGroup Hash Rate 146.8 Ph/s', u'16.38 Th/s - 32.77 Th/s\nUser Count 3380\nAverage Hash Rate 25.24 Th/s\nGroup Hash Rate 85.30 Ph/s', u'8.192 Th/s - 16.38 Th/s\nUser Count 4276\nAverage Hash Rate 13.00 Th/s\nGroup Hash Rate 55.57 Ph/s', u'4.096 Th/s - 8.192 Th/s\nUser Count 540\nAverage Hash Rate 5.953 Th/s\nGroup Hash Rate 3.215 Ph/s', u'2.048 Th/s - 4.096 Th/s\nUser Count 284\nAverage Hash Rate 3.193 Th/s\nGroup Hash Rate 906.8 Th/s', u'1.024 Th/s - 2.048 Th/s\nUser Count 226\nAverage Hash Rate 1.368 Th/s\nGroup Hash Rate 309.1 Th/s', u'512.0 Gh/s - 1.024 Th/s\nUser Count 136\nAverage Hash Rate 774.4 Gh/s\nGroup Hash Rate 105.3 Th/s', u'256.0 Gh/s - 512.0 Gh/s\nUser Count 116\nAverage Hash Rate 401.5 Gh/s\nGroup Hash Rate 46.57 Th/s', u'128.0 Gh/s - 256.0 Gh/s\nUser Count 75\nAverage Hash Rate 186.4 Gh/s\nGroup Hash Rate 13.98 Th/s', u'64.00 Gh/s - 128.0 Gh/s\nUser Count 78\nAverage Hash Rate 96.39 Gh/s\nGroup Hash Rate 7.518 Th/s', u'32.00 Gh/s - 64.00 Gh/s\nUser Count 70\nAverage Hash Rate 45.68 Gh/s\nGroup Hash Rate 3.198 Th/s', u'16.00 Gh/s - 32.00 Gh/s\nUser Count 48\nAverage Hash Rate 23.37 Gh/s\nGroup Hash Rate 1.122 Th/s', u'8.000 Gh/s - 16.00 Gh/s\nUser Count 62\nAverage Hash Rate 11.91 Gh/s\nGroup Hash Rate 738.5 Gh/s', u'4.000 Gh/s - 8.000 Gh/s\nUser Count 153\nAverage Hash Rate 3.078 Gh/s\nGroup Hash Rate 471.0 Gh/s']