So i’m working on the markov chain final project and it is pretty much done, but i can’t write a text file because of this encoding issue:
UnicodeEncodeError: ‘ascii’ codec can’t encode character u’\u2013’ in position 18: ordinal not in range(128)
I tried to encode the string before writing in into the file which is the answer to this problem according to the internet but it just doesn’t work.
Thanks in advance.
import urllib2 from bs4 import BeautifulSoup def fetch_text(url): html = urllib2.urlopen(url).read() soup = BeautifulSoup(html, "lxml") for script in soup(["script", "style"]): script.extract() text_data = soup.get_text() return text_data text_string = fetch_text("https://de.wikipedia.org/wiki/Softwaremetrik") new_text = str(text_string).encode('utf-8') with open("string.txt", "w") as my_file: my_file.write(new_text)