Login to a website through web-scraping tool in Python Login to a website through web-scraping tool in Python selenium selenium

Login to a website through web-scraping tool in Python


I had good luck using mechanize. It's pretty straightforward and simple to use.

Here's a stripped-down version of a script I made:

from BeautifulSoup import BeautifulSoupfrom tidylib import tidy_documentimport mechanizeimport cookielibif __name__ == '__main__':  browser = mechanize.Browser()  cookiejar = cookielib.LWPCookieJar()  browser.set_cookiejar(cookiejar)  browser.set_handle_equiv(True)  browser.set_handle_redirect(True)  browser.set_handle_referer(True)  browser.set_handle_robots(False)  browser.open('https://www.example.com/')  browser.select_form(name = 'loginform')  browser['username'] = 'foo'  browser['password'] = 'bar'  browser.submit()  browser.open(browser.click_link(text = 'Link text'))  soup = BeautifulSoup(tidy_document(browser.response().read())[0])

You don't need to click on the image, really. You just need to fill out all the appropriate form details and just submit() it.

Also, if you won't be parsing anything, just get rid of the BeautifulSoup and tidylib dependencies.


You need to call the click function of the element, not the driver.

submitButton=driver.find_element_by_xpath("//input[@type='image'][@src='/images/buttons/loginnow.gif']")submitButton.click()