Iterables New Teacher in Town Exercise

First of all tgrtim, thank you for taking time out of your day to help me. I do appreciate it. I do have some questions about your response that I hope you can answer for me.

As a rookie, I am going to have to look up some of the things you are talking about. You talk about containers but I find the link to the documentation very technical. Clearly I will need to use youtube and other resources.

“So the closest match to the lessons in my eyes would be to create two classes, one ClassroomOrganizer that stores and orders the data and a second class, an iterator template than can access the data within a ClassroomOrganizer instance.”

Sounds good, I keep reading…

"Container types like list or perhaps a very similar tool like collections.OrderedDict (a standard module class that acts as a container which can order it’s contents) are iterable . They are not themselves iterators , that task is handled by other types. "

→ okay s I am thinking list and OrderedDict acts as containers that sort things in alphabetical order. I continue reading…

from collections import OrderedDict
foo = iter(list())
bar = iter(OrderedDict())

# Note the types here are not list or OrderedDict but distinct iterators
print(f"{type(foo)!r}")  # <class 'list_iterator'>
print(f"{type(bar)!r}")  # <class 'odict_iterator'>

As a rookie, when I see that, if I am being honest, I don’t see a difference except that it says that list is a list_iterator and bar dictionary is a dictionary iterator. Other than that I am not quite grasping the point. I continue reading…

This is really an implementation detail but it matters because the behaviour of iterables and iterators is quite different. For a start iterables have no __next__ method (that’s reserved for actual iterators) and calling next() on an iterable will simply throw exceptions e.g.

→ iterators traverse and use the next method.
→ iterables themselves are not capable of throwing the next method because they are the things be iterated on and have been granted no such capability. in other words a list does not have a built in iterator ability so it cannot work. At least I think that is what you are saying.

" In addition to the behaviour of next() iterable container types behave differently to iterators when you call iter() on them. You can get typically get several different independent iterators from a single iterable (maybe a couple of CPython things violate this but it’s rare)"

x = [1, 2, 3,]
y = iter(x)
iter_y = iter(y)  # called on an iterator!
print(next(y))  # 1
# It's the SAME ITERATOR!
print(next(iter_y))  # 2

iterator types themselves have different behaviors when you call iter on them.
I believe you are showing y and z are not the same by having them show they are equal to iter(x) and than telling python to compare the two. When it comes back false, this is supposed to be suprising because they were both set equal to the iter(x). I am indeed suprised and do not understand why that is. I continue reading…

print(next(y), next(y)) #2. → I am guessing it traverssed the x = [1, 2, 3,] and came back 2 because next moved to the 2 value.

as for z it is only has next activated once and that makes it only 1. I may be wrong on this.

“In general if you’re defining a new container/collection then making it iterable is wise, making it an iterator probably isn’t (I won’t swear to every possible situation but following this pattern for custom container types is normally a good idea).”

in other words you are saying to making the container iterable is better in general than creating an iterator via iter() and next() is preferable.

I continue reading…

Long story short, for the requirements of this lesson (define a custom iterator) I suggest using at least two classes (an iterable and an iterator).

Okay this is where I am lost. I percieve that you took some time to explain to me that I can make the containers holding the values iterable and that making an iterator is a bad idea.

Has something been made in the code where I can make it a container and make it iterable?

Once again, thank you for taking time out of your day to help me with this. I have questions about this section. Heres to hoping I find some answers…
Okay so looking at this code

class ClassroomOrganiser:
    
    ... # rest remains the same
    
    def __iter__(self):  # this class is now ITERABLE
        return ClassroomIterator(self)


class ClassroomIterator:

    def __init__(self, target):
        self._target = target
        self._index = -1
        self._max_index = len(self._target.sorted_names) - 1
        
    def __iter__(self):  # iterator behaivour
        return self
    
    def __next__(self):  # now an ITERATOR
        if self._index < self._max_index:
            self._index += 1
            return self._target.sorted_names[self._index]
        else:
            raise StopIteration

Okay so in this code we have ClassrroomOrganiser being “converted” perhaps into an iterable? I believed that iterables were lists, dictionaries and sets. I am a little confused how a class can become one. Clearly I have misunderstood something.

I continue reading.

Another class is created, ClasrroomIterator.

def init(self, target):
self._target = target
self._index = -1
self._max_index = len(self._target.sorted_names) - 1

I see target is passed in as a parameter. There is are some attributes being set such as index and max index. perhaps you are creating a container here to hold the data? the -1 seems an interesting starting point however because I believe lists and such start at index 0. I have not grasped something it seems.

iter
return self

next
if self._index < self._max_index:
self._index += 1
return self._target.sorted_names[self._index]
else: Stop Iteration

okay this seems to be the codecademy material that I remember reviewing.

iter will return or hold whatever is being iterated.

__next is used to traverse the container or iterable. So long as the self.index is less than the max it will increment by one and than will grab the student names and than return it.

if self._index gets too big than a StopIteration is raised.

So basically this is a working iterator template that I can use for the exercise? I may have to edit the indexing?

Do I have a solid grasp of understanding this?

class ClassroomOrganiser:

    ...  # rest of class stays the same

    def __iter__(self):
        # self.sorted_names is a list
        # we can simply return iter() called on this list
        # it provides an iterator over this list
        return iter(self.sorted_names)
    
    # ClassroomOrganiser is now an iterable (not an iterator)

The general point I’m trying to make is that you can’t make the container itself an iterator without breaking the promises/protocols set forward by Python’s definition of an iterator. It’s worth noting that the general iterator pattern is a deliberate separation of the container and iterator, see e.g. Iterator pattern - Wikipedia (ideally less tightly coupled than this example but it’s a start). By and large Python’s existing types perform by having a separate type for the actual iteration, they are iterables not iterators.

I’m suggesting you do the same and use a separate iterator type of some form.


I apologise for adding any confusion but this course has changed more than once and I may be subconsciously applying instructions/guidance than no longer exists.

When the course was early days it suggested making ClassroomOrganizer a container type that ordered its contents and an iterator. I think it has been updated to remove this but they don’t explicitly say what to do any longer. In fact I can no longer work out whether the contents of the class should be ordered or whether the iterator should do the alphabetical ordering. It’s also no longer entirely to clear to me whether ClassroomOrganizer is still a container at all or whether it’s supposed to be an iterator over an existing structure :person_shrugging:.

Considering task 4 asks you to add an additonal method to ClassroomOrganizer with a special output I’ll assume ClassroomOrganizer is not a simple iterator. At a guess it’s still supposed to be a container but I could be wrong; I just cannot say for sure what the current instructions expect.

Hopefully codecademy will clarify if you ask them.


For now it looks like it might now be up to you on exactly how you solve it. I gave three possible suggestions beforehand: re-use an existing type, use a generator, or define a new class. For practice I’d suggest creating a new iterator class (defines __iter__ and __next__) that can return elements from a container based ClassroomOrganizer instance. If you want a bit more challenge perform then alphabetical ordering in the iterator and not the container.

You could try making ClassroomOrganizer an iterator that returns elements from the existing list but I dislike the idea of adding addtional methods to that iterator in task 4; if you want to make two iterators with different behaviour (or maybe even injected behaviour) that makes more sense.

I’ll try to answer your more direct questions in the next post.

I’ve tried to respond to some of your queries in this post-


No it’s nothing to do with alphabetical order (at least not in any strict sense) I was just giving examples of Python types that have their iterators implemented separately (they are iterable types but not iterators). In short imagine you create a new list type with a class List definition; I’d expect that type to be iterable but not an iterator.


Iterable just describes the behaviour of an object, quoting the Python docs glossary on iterable-

Iterable
An object capable of returning its members one at a time. Examples of iterables include all sequence types (such as list, str, and tuple) and some non-sequence types like dict, file objects, and objects of any classes you define with an __iter__() method or with a __getitem__() method that implements sequence semantics.

With classes you’re creating your own object templates and you can make those objects iterable if you wish. The main differences between new types you create and existing types is implementation but the behaviour can be more or less identical (barring certain CPython factors).


This is an ugly tightly-coupled example, you want your iterator to be keeping track of which item should come next when calling next() on it. I just relied on numerical indices based on the length of the ClassroomOrganiser items. Don’t overthink the index values I used to do this; I did warn I didn’t really check this bit (-1 is just to fudge the fact I used index += 1 before using it as a subscript).

It’s an example of how you might implement this, it’s not supposed to be a perfect solution, I just want you to understand how you could implement your own (ideally you should).

Alright I have decided to utilize the next and iter methods and have hit a snag. What exactly is the thing I need to be iterating over? Obviously student_roster holds the student data. But this data must be organized in alaphbetical order.
Per the instructions: " Define the iterator protocol for our ClassroomOrganizer class that can achieve this. Once defined, use either next() calls or a for loop on the ClassroomOrganizer object to print out the next student on the roll call."

This tells me to instantiate an object and store it in a variable, which I have done so on line 29. After that I tell it to iterate over it. And it fails. The hint for the step says: “A class member to store the index of which student within self.stored_names has been returned should be defined.”

What do they mean by the hint? Am I on the right track? How would I utilize self.sorted_names?

from roster import student_roster
import itertools

# Import modules above this line
class ClassroomOrganizer:
  def __init__(self):
    self.sorted_names = self._sort_alphabetically(student_roster)

  def _sort_alphabetically(self,students):
    names = []
    for student_info in students:
      name = student_info['name']
      names.append(name)
    return sorted(names)

  def get_students_with_subject(self, subject):
    selected_students = []
    for student in student_roster:
      if student['favorite_subject'] == subject:
        selected_students.append((student['name'], subject))
    return selected_students
  
  def __iter__(self): 
    self.index = 0
    return self
  
  def __next__(self):
    student_roster = self.stored_names
    return student_roster

classroom_roster_object = ClassroomOrganizer()
classroom_roster_object_iterator = iter(classroom_roster_object)

New Teacher In Town Exercise Link: Learn Intermediate Python 3 | Codecademy

No-reply. The above does not look right. Where is the iteration?

You will break protocols with this implementation. Resetting an iterator “index” to 0 within __iter__ is a mistake. An iterator object should return itself when called with iter(), in this case it does but it also resets its effective position. That’s a violation of Python’s requirements for iterators, the most obvious being the rule that once an iterator returns StopIteration that’s all it ever returns (you can call iter() on that “iterator” and reset it). If you want to iterate from over the same data from the start then create a new iterator object, don’t resurrect the old one.

You can make your own methods do what you like e.g. return elements one by one but also allow for resets and walking forwards / backwards but Python’s magic methods have requirements, you should follow them. If you’re violating protocols and misusing magic methods then you might find horrific things when you pass it along to things like itertools (or you might not and you’ve risking hidden bugs instead).

There’s a secondary issue in this implementation that each separate iterator necessitates a separate store of data (using sorted in the constructor means it’s not a reference to an existing list object but a new one). You don’t really want an iterator lugging around that much memory, nor should it have extra methods and pretend to be something it’s not. Keep it simple.

I’ll suggest for the last time separating the iterator from the rest of it.

I have decided to create a separate class to do the iteration as you have suggested. I have instantiated the object.

from roster import student_roster
import itertools

# Import modules above this line
class ClassroomOrganizer:
  def __init__(self):
    self.sorted_names = self._sort_alphabetically(student_roster)

  def _sort_alphabetically(self,students):
    names = []
    for student_info in students:
      name = student_info['name']
      names.append(name)
    return sorted(names)

  def get_students_with_subject(self, subject):
    selected_students = []
    for student in student_roster:
      if student['favorite_subject'] == subject:
        selected_students.append((student['name'], subject))
    return selected_students

  def __iter__(self): 
    self.index = 0
    return ClassroomIterator(self)
  
class ClassroomIterator:

    def __init__(self, target):
        self._target = target
        self._index = -1
        self._max_index = len(self._target.sorted_names) - 1
        
    def __iter__(self):  # iterator behaivour
        return self
    
    def __next__(self):  # now an ITERATOR
        if self._index < self._max_index:
            self._index += 1
            return self._target.sorted_names[self._index]
        else:
            raise StopIteration
  

classroom_roster_object = ClassroomOrganizer()
print(next(classroom_roster_object)

Output:
Error
Traceback (most recent call last):
File “classroom_organizer.py”, line 45, in
print(next(classroom_roster_object))
TypeError: ‘ClassroomOrganizer’ object is not an iterator

I guess what I am trying to do is use the next() calls or for loop iteration.

I am sorry that I did not try to separate the classes before. I was trying to do it the way the codecademy material in the lesson was telling me to do it by adding a next and iter method. But I am simply not getting anywhere.

Can you advise me on what to do next?

1 Like

Yeah it’s unfortunate that this lesson still seems to have this issue but I’m glad you’ve seen the light :laughing:; I tried to separate my opinion to make it clear that it really is an issue to make the container an iterator in this way (it will almost always break Python’s protocol and is most definitely not following the iterator patten).

I’d suggest removing anything about self.index in the ClassroomOrganizer class, I think you may have overlooked it.


Important bits-

What you’ve done with this separation is made ClassroomOrganizer instances iterable like more or less any other Python container/collection type. So you can’t call next() on it directly (in the same way you can’t call next() on a list directly).

You can however use it like you would a list so the following should be acceptable-

classroom_roster_object = ClassroomOrganizer()

for element in classroom_roster_object:
    print(element)  # as an example

roster_iterator_1 = iter(classroom_roster_object)
print(next(roster_iterator_1))

Hopefully that’s now obvious that the new type you’ve created can behave like existing types, you could swap out the classroom_roster_object for a list or tuple without changing the rest of the code (this shared behaviour/interface is a good thing).

2 Likes

THANK YOU SO MUCH IT WORKS! :slight_smile:

Can you verify my understanding really quick!?
I left some comments by the code, can you check for accuracy?

from roster import student_roster
import itertools

# Import modules above this line
class ClassroomOrganizer:
  def __init__(self):
    self.sorted_names = self._sort_alphabetically(student_roster)
  def _sort_alphabetically(self,students):
    names = []
    for student_info in students:
      name = student_info['name']
      names.append(name)
    return sorted(names)

  def get_students_with_subject(self, subject):
    selected_students = []
    for student in student_roster:
      if student['favorite_subject'] == subject:
        selected_students.append((student['name'], subject))
    return selected_students

  def __iter__(self): 
    return ClassroomIterator(self)
  
# Okay so the main idea behind this solution is to create another class that does # the iteratorting. To make this possible the first thing that you do is create an def # __iter__(self) method for the class with the data having it return itself.

class ClassroomIterator:

    def __init__(self, target):
        self._target = target
        self._index = -1
        self._max_index = len(self._target.sorted_names) - 1

# I make an init method because I have to have it to activate a class. I make the target an attribute of the class. After doing so I create an index attribute in the iterator and set it equal to -1 as the starting point. I than make a max index attribute and make it the length of my iterable data -1. The minus one accounts for the 0 indexing and makes certain the last index in the list or dictionary or whatever I am iterating does not have index's past the data. 

    def __iter__(self):  # iterator behaivour
        return self
# After that I create an iter method and return self.

    def __next__(self):  # now an ITERATOR
        if self._index < self._max_index:
            self._index += 1
            return self._target.sorted_names[self._index]
        else:
            raise StopIteration
  # Than I make a next method and have it have an self.index position of 1. And than I have it hold the data being iterated over with. 
# when it iterates over it checks to see if the index is less than the max. if so it will move the iterating position move over by one. After that it returns what data is inside of the index position it is in. If however the index is past the max index position it will raise a stop iteration. 

classroom_roster_object = ClassroomOrganizer()

# here we instatiate the object. 
for element in classroom_roster_object:
    print(element)  # as an example
 
 # this for loop grabs each value in the list and than prints it. 
roster_iterator_1 = iter(classroom_roster_object)
# # this is making the classroom iterator object iterable. Based on the next line of code it looks like it holds all of the list indexes that were iterated over. 
print(next(roster_iterator_1))

1 Like

Aye, in general terms you don’t have to explicitly use a class (using the iterator created from the underlying list is fine, using a generator is fine) but the point is the separation of iterator from container (see Iterator pattern - Wikipedia).


It doesn’t matter exactly how you do it but the iterator needs some way of tracking what should come next which normally means keeping track of state internally, initialising with __init__ is probably the easiest way to get started.

Don’t get hung up on the idea of indices (especially not the hackneyed version I showed). Iteration can be done over more or less any container. Think about non-linear data structures too e.g. trees and graphs where numeric indices have no specific meaning. If you’re unfamiliar with these imagine the directory/folder system in your computer. An iterator could allow you to return each element (each file/path) in turn despite the fact it’s not a linear structure.

It’s a form of abstraction, a neat one. You could have a variety of iterators for a single container e.g. one that goes in reverse or searches in a different order, one that returns pairs and so on. Under __iter__ for a Python iterable you should return something consistent but you can return other iterators from your own methods that do all sorts of things.


Under the hood for loops call .__iter__ anyway (barring any CPython trickery) and simply binds the name to each element returned by the iterator in turn. To use an object as the target of a for loop like this it must be iterable. If you can remember this then for loops will make more sense as will the benefits of making things iterable.

This object is already iterable under Python’s definitions. Using iter on an object calls its __iter__ method. You defined your .__iter__ method for this class to return an iterator object which is literally what this does; it returns an iterator, a separate object. In the way you’ve defined it each time you call iter on that iterable object it creates a new iterator object.

Treat it like a function call that returns something (a different object, an iterator instance) and you might find it easier to describe.


Most of your other comments are implementation details e.g. index values, the best way to check if those work when you can’t reason through them would be testing.