Project: US Census Data. How do I use loop on conversion into numeric

On task #14, I was making histogram for each race. I was thinking that can I use loop on these processes? I tried it on removing “%” sign and it’s good. However, I cannot do it on either converting data into numeric type or creating histogram. Is there any way to do so? Here is my code:

for race in us_census:
  race = ['Hispanic', 'White', 'Black', 'Native', 'Asian', 'Pacific']
  us_census[race] = us_census[race].replace('%', '', regex = True)

I tried to add these into the loop and I got error on both.

  us_census.race = pd.to_numeric(us_census.race)
  us_census[race] = pd.to_numeric(us_census[race])

I also tried to use loop on creating histogram and it’s also not working

for race in us_census:
  race = ['Hispanic', 'White', 'Black', 'Native', 'Asian', 'Pacific']
  plt.scatter(us_census[race], us_census['Income'])

Quite rightly so, for one very simple reason: how would you change any of those text values (e.g. hispanic, or white) into a meaningful numerical value? You can’t, because such an operation is nonsense.

If you read the documentation for the to_numeric() method, you’ll see that there is an optional parameter errors which defaults to raise. This means that when you provide it with data it cannot parse to a number, it raises an exception - this is the behaviour you’re seeing. Changing your code to us_census.race = pd.to_numeric(us_census.race, errors='coerce') will make pandas return NaN for any data which it cannot parse to a number.

What the value of this would be is dubious, though - because the entire list you have in race would evaluate to NaN, so not sure how that’ll improve what you’re trying to do.

I am not hugely familiar with pandas or matplotlib, but I don’t need to be to see that your code is wrong. You’ve got:

for race in us_census:
  race = ['Hispanic', 'White', 'Black', 'Native', 'Asian', 'Pacific']
  plt.scatter(us_census[race], us_census['Income'])

You’re declaring race as the placeholder variable for your iterator, then immediately assigning it the full list once you’re in the loop… which, again quite rightly, is throwing an error. As to whether you’re doing the plotting correctly? I don’t know, as I don’t know that much about how to use matplotlib.

Lastly, whilst perhaps not so urgent in this case as the issues are reasonably obvious, it generally helps to provide the complete text of any error messages your code is throwing rather than simply stating that you got an error message. The Traceback in Python is pretty helpful at tracking down what’s causing your code to fail. :slight_smile:

I think I am changing numbers with string type in to numeric type. The value in these columns (Hispanic, White, Black…) are percentage value in string type, for example, 6.344322%. That’s why I removed ‘%’ mark at first. The next step of this project is to transfer remaining digits into numeric type so that we can do some calculation. us_census.[‘Hispanic’] = pd.to_numeric(us_census.[‘Hispanic’]) actually works well, when it’s placed outside the loop, and I have to do it with each race, which is 7 times in total. That’s why I was trying to throw it into the loop to see can I do it only once as what I did on removing the ‘%’ mark. However, I got error from it.

The matplotlib part is also based on the same idea. The original code is as below. It creates a scatter chart with Hispanic data as x-value and Income as y-value. Again, I have to do it with each race, which is 7 times in total.

plt.scatter(us_census['Hispanic'], us_census['Income'])

I’m sure you already figured it out. Here’s what I did for the pd.to_numeric part:

us_census[[‘Hispanic’,‘White’,‘Black’,‘Native’, ‘Asian’, ‘Pacific’]]= us_census[[‘Hispanic’,‘White’,‘Black’,‘Native’, ‘Asian’, ‘Pacific’]].apply(pd.to_numeric)


any idea why if we use the for loop with:

for race in us_census:
  race = ['Hispanic', 'White', 'Black', 'Native', 'Asian', 'Pacific']
  us_census.race = us_census.race.replace('%', '', regex = True)

it doesn’t work?

I tried it at first with .race instead of [race]

I am working on this project as well and can’t get started. I’m not able to import the files and view the columns of the data set. There’s no project walk through so I’m not sure what I’m doing wrong. My code is as follows:

files = glob.glob(“file*.csv”)
us_census =
for filename in files:
data = pd.read_csv(filename)
us_census = pd.concat(us_census)

This gives me the error:

Traceback (most recent call last):
File “”, line 14, in
AttributeError: ‘list’ object has no attribute ‘columns’

Why can’t it pull any columns? Has the file not been loaded?
Thanks for the help!

  • Ian