Encoding issue

So i’m working on the markov chain final project and it is pretty much done, but i can’t write a text file because of this encoding issue:
UnicodeEncodeError: ‘ascii’ codec can’t encode character u’\u2013’ in position 18: ordinal not in range(128)

I tried to encode the string before writing in into the file which is the answer to this problem according to the internet but it just doesn’t work.
Thanks in advance.

import urllib2
from bs4 import BeautifulSoup




def fetch_text(url):
    html = urllib2.urlopen(url).read()
    soup = BeautifulSoup(html, "lxml")

    for script in soup(["script", "style"]):
        script.extract()

    text_data = soup.get_text()
    return text_data

text_string = fetch_text("https://de.wikipedia.org/wiki/Softwaremetrik")

new_text = str(text_string).encode('utf-8') 

with open("string.txt", "w") as my_file:
    my_file.write(new_text)



2 Likes

Nevermind i found the problem. The problem was that it wanted ASCII and not unicode while i thought it needs unicode even though it was already unicode. :tired_face:

1 Like