python - webdriver and asyncio
If you want to use Selenium in an async fashion I would suggest using multiple instances of the Driver and a executor like this:
import asynciofrom concurrent.futures.thread import ThreadPoolExecutorfrom selenium import webdriverexecutor = ThreadPoolExecutor(10)def scrape(url, *, loop): loop.run_in_executor(executor, scraper, url)def scraper(url): driver = webdriver.Chrome("./chromedriver") driver.get(url)loop = asyncio.get_event_loop()for url in ["https://google.de"] * 2: scrape(url, loop=loop)loop.run_until_complete(asyncio.gather(*asyncio.all_tasks(loop)))Please note that you can run selenium in headless mode so you don't need to spawn the whole GUI for calling some simple url.
The problem was discussed at: https://github.com/SeleniumHQ/selenium/issues/3399
If you want to have async webdrivers, there are two libraries you can use: