Using mutable class variables

Can I raise a topic on here about using mutable class variables?

I’ve noticed that in the python documentation that it is frowned upon to use a mutable instance variable such as a list and it’s plainly stated that this is wrong.

For example, for the reasons stated above a list (any mutable type) should not be used as a class variable and instead used as an instance variable.

But, I was thinking, is there not some use cases where using a mutable class variable would be acceptable?

For example, a class variable list to store each object that is created so that each object has access to all other objects that have been created. I.e a Student object that wants to talk to all other Student objects.

Could you help me understand why the documentation is worded like this. I feel I may be missing the point or possibly overthinking it and maybe the documentation is just stating the norm.

I’m no expert but I can see one possible use case… If the class variable is to be a default list or dictionary that all instances can access, as well as the class itself, but which can be modified by an instance to customize it. Will need to play with this when I get a chance to try and come up with an example. Gotta go, right now but I’ll leave this topic open for when I get back.

Disclaimer… We can do lots of things with Python that are not recommended. Use cases are by discretion.

Hi @mtf,

thanks for your insight and hopefully can revisit this topic again soon

Please provide us with a link to the specific documentation to which you are referring.

Hi,

the link I’m referring to is https://docs.python.org/3.7/tutorial/classes.html and is in section 9.3.5:

As discussed in A Word About Names and Objects, shared data can have possibly surprising effects with involving mutable objects such as lists and dictionaries. For example, the tricks list in the following code should not be used as a class variable because just a single list would be shared by all Dog instances:

The objection to the tricks class variable in the Dog example is that the actual intent was to maintain a list of tricks for each individual dog. As a class variable, it is shared by all Dog instances, so it would not serve that purpose.

However, if we wanted to maintain a comprehensive list of the Dog tricks that occur among the totality of Dog instances, then storing them in a class variable as a list would be appropriate, in my opinion. I don’t think that runs counter to the objection raised in this example.

Yeh, I totally agree but I felt as if the documentation was suggesting that class variables should never be used with mutable objects which is why I had to be sure as I could think of several use cases like yourself where having a mutable class variable would be useful.

Thanks for your help

1 Like

My point, exactly. Once it is mutated it becomes an instance variable, and the class variable is no longer accessible to that instance since it is shadowed.

Yeah, the wording there is a little unclear.

Assigning a value to a variable via the current instance reference (conventionally self) would shadow a class variable of the same name. However, mutating a class variable via a current instance reference or via the name of the class does not create an instance variable with the same name, so it does not cause shadowing.

See the following:

class Dog:
  all_tricks = set()
  def __init__(self, name):
    self.name = name
    self.my_tricks = set()
  def add_trick(self, trick):
    self.my_tricks.add(trick)
    self.all_tricks.add(trick) # same effect with Dog.all_tricks.add(trick)
  def show_my_tricks(self):
    print(f"{self.name} -> {self.my_tricks}")
  def cite_all_tricks(self):
    print(f"All Dogs -> {self.all_tricks}") # same effect with print(Dog.all_tricks)

pointer = Dog("Pointer")
rover = Dog("Rover")

pointer.add_trick("Roll over")
rover.add_trick("Operate the GPS device")

pointer.show_my_tricks()
rover.show_my_tricks()

pointer.cite_all_tricks()
rover.cite_all_tricks()

Output:

Pointer -> {'Roll over'}
Rover -> {'Operate the GPS device'}
All Dogs -> {'Roll over', 'Operate the GPS device'}
All Dogs -> {'Roll over', 'Operate the GPS device'}

Edited on July 8, 2020 to clarify the mention of the name, self.

1 Like
>>> class Foo:
	bar = 'baz'

	
>>> foo = Foo()
>>> faz = Foo()
>>> foo.bar
'baz'
>>> faz.bar
'baz'
>>> foo.bar = 'faz'
>>> foo.bar
'faz'
>>> faz.bar
'baz'
>>> 

Mind in this case the class variable is not a structure.

2 Likes

Ok, I understand…

So by assignment, the statement foo.bar = "faz", the class variable bar is now shadowed by an instance variable of the same name and now reference to the namebar is via the object’s instance variable of bar instead of the Foo class’s class variable bar.

Hence bar still exists but is only accessible via the class this time around.

>>> Foo.bar
'baz'
1 Like

It is also still accessible to other instances that have not overridden it.

1 Like

But this “shadowing effect” is only via assignment I believe, yes?

So if using a method or constructor to mutate the class variable via self.mutable_class_var.add(item) it stays a class variable and using self just accesses the class variable and doesn’t create an instance variable that shadows the class variable.

Is that right?

In the example, yes. By virtue of assigning a new value it becomes an instance variable of that instance only.

Still a little on the fence for that one. I think you have the right idea but I’ll defer to @appylpye

1 Like

Thanks for taking the time to provide the example @appylpye.

Thinking about it,

even if an instance variable was created and shadowed the class variable as expected it would make no odds anyway as both the class and instance variable would be referencing the same mutable object so any changes to the mutable would be seen in both types of variable.

Where with an immutable a completely new object would be assigned to the overshadowing instance variable meaning as we discussed they would be accessed in different ways with self.variable accessing the instance variable and Class.variable accessing the class variable.

I think I am starting to understand now, just need some time for it to sink in. I come from a Java background so the this behaviour is a little different to what I’m used to.

I am curious though, as to wether it’s actually the self or assignment that actually creates the instance variable in reality.

1 Like
>>> Foo.bar = 'baz'
>>> faz.bar
'baz'
>>> foo.bar
'faz'
>>> 

Only those instances that sill have access to the class variable will be affected by any change to it.

In Glenn’s example the object is a structure. Changes made within the structure (mutating an element, adding/deleting an element) does not change the value of the variable, which is why the example works as it does. All references to a structure see the same structure.

If we create an instance variable (via assignment) that replaces that structure, only that instance will shadow the class variable.

>>> class Foo:
	bar = ['r', 'g', 'b']

	
>>> foo = Foo()
>>> faz = Foo()
>>>
>>> foo.bar
['r', 'g', 'b']
>>> faz.bar
['r', 'g', 'b']
>>> Foo.bar += 'a'
>>> foo.bar
['r', 'g', 'b', 'a']
>>> faz.bar
['r', 'g', 'b', 'a']
>>> foo.bar += 'x'
>>> faz.bar
['r', 'g', 'b', 'a', 'x']
>>> foo.bar = foo.bar + 'y'
Traceback (most recent call last):
  File "<pyshell#59>", line 1, in <module>
    foo.bar = foo.bar + 'y'
TypeError: can only concatenate list (not "str") to list
>>> foo.bar = foo.bar + ['y']
>>> foo.bar
['r', 'g', 'b', 'a', 'x', 'y']
>>> faz.bar
['r', 'g', 'b', 'a', 'x']
>>> 
1 Like

I see ya point,

the += operator’s behaviour works by appending the string object to the original object (the list):

in contrast, the + operator:

creates a new object from the original object (the list) and the other object (the list holding the string) being added and this is then assigned to the instance variable.

Hence an instance variable with the same name - bar (which shadows the class variable bar) is created for the instance foo, which explains why there is now a difference when calling the name bar on both instances.

But the class variable IS still accessible but ONLY via the Foo class reference: **Foo**.bar.

1 Like

It is also available to other instances, such as faz, above.


Glad you caught the distinction between += and +.

Yeh,

there are so many subtle nuances,

I now find myself asking why is it that object += "x" mutates an object and object = object + ["y"] makes a new object. When they are effectively the same thing? Except in one we are appending a string and the other a list.

Maybe a topic for a new thread?

I’ve also noticed the example above doesn’t work when appending an int?