Extracting text from HTML file using Python

The best piece of code I found for extracting text without getting javascript or not wanted things :

from urllib.request import urlopenfrom bs4 import BeautifulSoupurl = "http://news.bbc.co.uk/2/hi/health/2284783.stm"html = urlopen(url).read()soup = BeautifulSoup(html, features="html.parser")# kill all script and style elementsfor script in soup(["script", "style"]):    script.extract()    # rip it out# get texttext = soup.get_text()# break into lines and remove leading and trailing space on eachlines = (line.strip() for line in text.splitlines())# break multi-headlines into a line eachchunks = (phrase.strip() for line in lines for phrase in line.split("  "))# drop blank linestext = '\n'.join(chunk for chunk in chunks if chunk)print(text)

You just have to install BeautifulSoup before :

pip install beautifulsoup4

html2text is a Python program that does a pretty good job at this.

NOTE: NTLK no longer supports clean_html function

Original answer below, and an alternative in the comments sections.

Use NLTK

I wasted my 4-5 hours fixing the issues with html2text. Luckily i could encounter NLTK.
It works magically.

import nltk   from urllib import urlopenurl = "http://news.bbc.co.uk/2/hi/health/2284783.stm"    html = urlopen(url).read()    raw = nltk.clean_html(html)  print(raw)