Cloudflare and Chromedriver - cloudflare distinguishes between chromedriver and genuine chrome?
This error message...
...implies that the Cloudflare have detected your requests to the website as an automated bot and subsequently denying you the access to the application.
Solution
In these cases the a potential solution would be to use the undetected-chromedriver to initialize the Chrome Browsing Context.
undetected-chromedriver is an optimized Selenium Chromedriver patch which does not trigger anti-bot services like Distill Network / Imperva / DataDome / Botprotect.io. It automatically downloads the driver binary and patches it.
Code Block:
import undetected_chromedriver as ucfrom selenium import webdriverimport timeoptions = webdriver.ChromeOptions() options.add_argument("start-maximized")driver = uc.Chrome(options=options)url1 = 'https://www.fanfiction.net/s/8832472'url2 = 'https://www.fanfiction.net/s/5218118'driver.get(url1)time.sleep(5)driver.get(url2)
References
You can find a couple of relevant detailed discussions in: