FAQ: Data Cleaning with Pandas - More String Parsing

This community-built FAQ covers the “More String Parsing” exercise from the lesson “Data Cleaning with Pandas”.

Paths and Courses
This exercise can be found in the following Codecademy content:

Practical Data Cleaning

FAQs on the exercise More String Parsing

There are currently no frequently asked questions associated with this exercise – that’s where you come in! You can contribute to this section by offering your own questions, answers, or clarifications on this exercise. Ask or answer a question by clicking reply (reply) below.

If you’ve had an “aha” moment about the concepts, formatting, syntax, or anything else with this exercise, consider sharing those insights! Teaching others and answering their questions is one of the best ways to learn and stay sharp.

Join the Discussion. Help a fellow learner on their journey.

Ask or answer a question about this exercise by clicking reply (reply) below!

Agree with a comment or answer? Like (like) to up-vote the contribution!

Need broader help or resources? Head here.

Looking for motivation to keep learning? Join our wider discussions.

Learn more about how to use this guide.

Found a bug? Report it!

Have a question about your account or billing? Reach out to our customer support team!

None of the above? Find out where to ask other questions here!

Step one seems to be impossible since even copying and pasting the answer from the solution does not work. I get an AttributeError and cannot continue. This issue has been plaguing me repeatedly in the pandas section as you seem to not be able to use any methods on columns

I’ve seen this error a number of times as well. This occurs whether I’m on my iMac using Firefox or Safari, or on my raspberry pi using Chromium. Sometimes refreshing the page fixes the problem. Other times, going ‘Back’ works. Other times, nothing seems to work except using the solution option. I don’t like that approach either.

Here’s my full solution:

split_grade = students['grade'].str.split('(\d+)', expand=True)
print("split_grade\n", split_grade)
students.grade = split_grade[1].replace('[\s]', '', regex=True)
print(students.dtypes)
students.grade = pd.to_numeric(students.grade)
avg_grade = students.grade.mean()
print("average grade = ", avg_grade)
print(students.dtypes)
1 Like

I am also having the same problem and its very frustrating. I have sent a message to the bot, lets see if someone gets back. In any case I am writing to ask about a question in this excercise. When we split the grade by the digits, why is it that we need to select the second item, i.e, index 1, instead of the first one. The number is at the beginning.

students.grade = students.grade.str.split(’(\d+)’, expand = True)[1]

7 Likes
df.exercise = split_df[2].replace('[\- ]', '', regex=True)

can’t understand why [2] is the exercise column if it’s must be ‘reps’ column?

12 Likes

@airens:

When we split the grade by the digits, why is it that we need to select the second item, i.e, index 1, instead of the first one. The number is at the beginning.

students.grade = students.grade.str.split(’(\d+)’, expand = True)[1]

Your question is based on what seems to be common sense, i.e., ‘The number is at the beginning of the string I want to split, so why isn’t it the first element (index 0) in the list returned by .split()?’. That common sense doesn’t work in practice, however, because of how the .split() method works. The first element actually returned from that split is an empty string: '', which if you follow the example in this exercise and create a split_df, results in an empty first column in that dataframe. You can find some more explanation and discussion about the reason for split returning an empty string at this Stack Overflow topic.

2 Likes

i don’t understand why on exercise 7 you can use :
“str_split.str.get(i)” to get the value of the split

and now we use " split_df[i] " to get the same value.

I thought maybe that’s because the last split returned a ‘serie’ and now in this exercise the split returns a ‘dataframe’ (because of expand=True) .
But then i tried to re-do the exercise 7 converting the serie into a dataframe with .reset_index() and i couldn’t reach it .

With the line students.grade = students.grade.str.split('(\d+)', expand = True)[1], why do we use [1]? I understand that in the resulting series, column 1 holds the grade, but when printing the series, column 0 and column 2 are empty. Why is that?

3 Likes

Can someone help me as to why the parentheses around the ‘\d+’ in the split change the rest. i.e. the below return different answers
students[‘grade’] = students[‘grade’].str.split(’\d+’, expand=True)[1] == 11
students[‘grade’] = students[‘grade’].str.split(’(\d+)’, expand=True)[1] == th grade

indeed printing the full return of the splits you can see that the former returns [ “”, “11”, “th grade”] while the latter returns [ “”, “th grade”]

I dont believe this is how it works for normal python string conversion (I’m unable to replicate it at least) and I dont understand it.

Note the same thing happens if I replace ‘\d+’ with simply ‘11’ and run it on the first row only meaning its nothing to do with the regex splitting to reduce one element.

Is there simply a special feature when using split on pandas.core.strings.StringMethods which means if you surround the split pattern in parentheses it returns the pattern as well as the blank?

It also doesn’t matter if I make expand true or false. If I split on ‘1’ instead of ‘11’ I get either:
[’’, ‘’, ‘th grade’]
or
[’’, ‘1’, ‘’, ‘1’, ‘th grade’]
so at least it’s pretty clear what’s happening but I’d feel better if I saw this written somewhere, why it’s different from normal string splitting and if theres a way to do it when splitting ordinary strings as its fairly useful.

1 Like

students.grade = students.grade.str.split(’(\d+)’, expand=True)[1]
why are we selecting the second index of the grade and we have grades that are only single indexed like grade 9.

1 Like

I found this in document so I deleted my previous reply. It’s just the built-in parameters that hasn’t been covered in the course

“Split string by the occurrences of pattern. If capturing parentheses are used in pattern, then the text of all groups in the pattern are also returned as part of the resulting list.”

https://docs.python.org/3/library/re.html#re.split

2 Likes

Thanks Angie. That article is for re.split rather than str.split but I guess it implies that pandas use re.split by default?

Yes I believe they are related so that we could explain :grinning:

Me neither. I believe it’s meant to be [0]?

Why do I get a different .mean() result depending on the placement of .to_numeric()?

Using .to_numeric() on a temporary split_df[0]:

split_df = students.grade.str.split("th", expand=True)

split_df = pd.to_numeric(split_df[0])
students.grade = split_df[0]

avg_grade = students.grade.mean()

returns a mean of 11.0

Using .to_numeric() on the main dataframe,students.grade:

split_df = students.grade.str.split("th", expand=True)

students.grade = split_df[0]
students.grade = pd.to_numeric(students.grade)

avg_grade = students.grade.mean()

returns a mean of 10.620445344129555

Yea I don’t get this either

Would like to ask what is the difference between using brackets and using parentheses to enclose the regex?
Since in the previous section the example is [\$,] and then in the current section the example is (\d+) .
Also, I don’t know why there is a ‘,’ in [\$,]

1 Like

Yes, I had these exact same questions! Thank you to Angie above for the indexing answer, it makes sense since we are splitting on the datum that we are trying to collect. Splitting on the “th” is a more logical solution but then I guess we couldn’t practice regex. :wink:

I too struggled to understand how the number was returned when it was defined as the mechanism to split upon. Thank you very much for your share @angie_sheng. Great find.

Cheers and God bless.

I had a different approach on how to get element from str using lambda, it returns same result:
students[‘grade’] = students.grade.apply(lambda x: x.split(’ ‘)[0][:2] if x.split(’ ‘)[0][1].isdigit() else x.split(’ ')[0][:1])