FAQ: Text Preprocessing - Stemming

This community-built FAQ covers the “Stemming” exercise from the lesson “Text Preprocessing”.

Paths and Courses
This exercise can be found in the following Codecademy content:

Natural Language Processing

FAQs on the exercise Stemming

There are currently no frequently asked questions associated with this exercise – that’s where you come in! You can contribute to this section by offering your own questions, answers, or clarifications on this exercise. Ask or answer a question by clicking reply (reply) below.

If you’ve had an “aha” moment about the concepts, formatting, syntax, or anything else with this exercise, consider sharing those insights! Teaching others and answering their questions is one of the best ways to learn and stay sharp.

Join the Discussion. Help a fellow learner on their journey.

Ask or answer a question about this exercise by clicking reply (reply) below!
You can also find further discussion and get answers to your questions over in Language Help.

Agree with a comment or answer? Like (like) to up-vote the contribution!

Need broader help or resources? Head to Language Help and Tips and Resources. If you are wanting feedback or inspiration for a project, check out Projects.

Looking for motivation to keep learning? Join our wider discussions in Community

Learn more about how to use this guide.

Found a bug? Report it online, or post in Bug Reporting

Have a question about your account or billing? Reach out to our customer support team!

None of the above? Find out where to ask other questions here!

It’s interesting to point out that the PorterStemmer also performs .lower on every word it will return!

1 Like

why do we initialize PorterStemmer() while we don’t do it with other built-in libraries?

hey @vlearnstocode ! Well, the logic is that you are creating a stemmer, it will iterate through your words and stem them;
Since you are creating an instance of it, one reason to do so is that you could have different stemmers in the same piece of code (multiple languages, multiple optional parameters). I think you could simply make a call to PorterStemmer().stem([‘words’, ‘list’]) as well. Haven’t tried it though!

oh! that’s very logical!

thank you, kind soul

Interesting, that PorterStemmer from NLTK is quite far from ideal. For example, it has transformed ‘Pacific’ into ‘pacif’, ‘people’ into ‘peopl’. What is common way to deal with this algorithm behaviour?

I’d consider “people” to “peopl-” to be the correct kind of stemming though Pacific to pacif- (root of e.g. pacify) is a sign of over-stemming. With this kind of algorithmic approach it won’t be perfect. There are other stemming tools you could use and they may or may not catch more edge cases (with varying computational costs) but you always have some limitations. If you’re curious have a web search for some more details.

1 Like