Is there a way to open a file in both read and write?

Having gone through the whole lesson, I didn’t come across a way of opening a file in both ‘read’ and ‘write’ mode (I might have missed something). Is there such a way?

if you check the documentation:

https://docs.python.org/3/tutorial/inputoutput.html#reading-and-writing-files

you will see this mode exist. there is read ®, write (w), append (a) and read + write (r+)

28 Likes

Grand, thank you! I was on that page but clearly didn’t read it through…

1 Like

Thank you, i saw it, but. I have this type of code:

with open("bad_bands.txt", "r+") as bad_bands_doc:
  bad_bands_doc.write("Grupa Tuberculyos")
  print(bad_bands_doc.read())

with open("bad_bands.txt") as bad_bands_doc:
  print(bad_bands_doc.read())

and it outputs:


Grupa Tuberculyos

There is an empty first line. I suppose that there should be a similar to the second line. But could you help me with why it is not?
Thank you.

2 Likes

ran the code on repl.it:

and went fine, maybe there was already content in the file? You could removing and creating the file again.

Is it fine though? Because what i would suppose is that there should be two identical lines of output:

Grupa Tuberculyos
Grupa Tuberculyos

Since we wrote “open("bad_bands.txt", "r+")” with "r+", i would expect we could write “bad_bands_doc.write("Grupa Tuberculyos")” to the file and then read it with “print” “print(bad_bands_doc.read())” so that we could output two identical lines.

1 Like

I think the problem is with how python handles files.

here:

  bad_bands_doc.write("Grupa Tuberculyos")
  print(bad_bands_doc.read())

the write is kept in memory (to minimize interaction with disk, which is slow). So then when you use read(), there is nothing in the file yet

4 Likes

It displays as expected when I used the .close function. According to the course we don’t need to use close() because using “with” automatically closes the file, however, I needed to explicitly close the file before I could read it.

with open(‘bad_bands.txt’, ‘r+’) as bad_bands_doc:
bad_bands_doc.write(‘I like all bands’)
bad_bands_doc.close()

with open(‘bad_bands.txt’, ‘r+’) as bad_bands_doc:
print(bad_bands_doc.read())

I’m unable to reproduce your issue on this lesson. It works fine without having to call .close(). What was your indentation here? See - How do I format code in my posts? for including proper formatting. Were those statements nested?

I think you’re saying that the .write() is saved to be executed just before the file is closed, to minimize disk interaction, so the order you write code in doesn’t matter?

So does this mean that if I want to write something to a file and then read it, it’s best to create new file objects for each task?

running programs exists in RAM (random access memory), what I assume happens is that when we open a file, the file is loaded from the disk into RAM. Then when we close the file, does the content get written to the disk

I don’t think this presents a real world scenario, in a real world scenario you write something to a disk because you need to persist the data after the program is finished running (and is “removed” from RAM).

so writing and then reading from a file right away kind of feels wrong. RAM is much faster, so then simply do something like:

with open("bad_bands.txt", "r+") as bad_bands_doc:
  to_write = "Grupa Tuberculyos"
  print("writing to file: {}".format(to_write))
  bad_bands_doc.write(to_write)

The file object and the file are different. I understand that the .write() isn’t executed to the file itself until the file is closed, to reduce interaction with the disk. But the .read() runs on the file object, not the file (I assume, because why else have a file object except to store it in memory and reduce disk interaction) so why doesn’t it .read() the rewritten file object that is already in memory, rather than the empty file?

I’m not sure how your code solves the problem. If I want to write a user’s input to a file, and then immediately read the contents of that file, how would I do that?

Not sure, you would need to delve into how the file object works in more depth. This is something very specific I do not know without doing research

is essentially the same except that you have user input? You already have the input (which you could store in a variable), which means you don’t need to read from file.

2 Likes

This is getting back to old Unix-isms. Everything you have written to the “file” are available to read back, whether it’s been flushed yet to media or not. There is no wait period. Stuff just wouldn’t work if this wasn’t honored. However, when you have just written the file, the file pointer is poised for the next write. Thus, the next time you write, you don’t overwrite existing data. If you execute a read at that point, you’ll read from the memory that has not yet been written. Closing and reopening the file does position the read pointer back to the beginning of the file, and that is why it appears to do the desired magic. What you really want to accomplish is a subset of that, though. You want to reposition the file pointer back to the start of the file to read it. That is done with a file rewind, or using what’s available in the python interface, a seek to byte zero, aka

bad_bands_doc.write("hello")
bad_bands_doc.seek(0) # your next read or write will be at the start of the file
hope_and_pray = bad_bands_doc.read()
And see if you get your data back

If you think about it, when you read, what was it you wanted to read? The last write, the whole file, etc. This specifies the “where”.
If on a non-Windows system run “man fopen” and “man fseek” for more information.

First time posting/formatting here. Be gentle.
(I’m just taking this course this week, but come from an open systems background

1 Like

I’ve never dug too deep in Python’s file objects but I believe the following gives a rough overview (I’ve tried to keep it vague to avoid inaccuracy but I can’t promise everything below is 100% accurate, higher level means ignoring implementation sometimes).

Python doesn’t read an entire file into memory by default (it could easily be a file larger than the available memory and even if it isn’t, loading the entire thing is rarely necessary). There’s a difference between the file object and a buffer (which would be all or part of the file read into memory). You can read about and change how Python uses buffers with file objects using open(). You’d typically read a file in chunks (there is a parameter for this in the .read() method) so read chunk, process chunk, repeat.

This is also the case for what you could call the write buffer. It is much more efficient to write large chunks of data to the disk at once. As such writes are often buffered so that large chunks of data can be written at once, the size of the buffer determines when these writes occur. As you mention this may be when the file is closed, it may happen after the buffer reaches a certain size or you can force a flush of the buffer (thereby writing it).

On the other hand is that balance with memory efficiency, say I’m writing a few hundred GB of video, most systems simply cannot support keeping the entire thing in memory. Both reading and writing in moderate chunks is typically a more sensible approach (it’s a bit of a balancing act between memory efficiency and read/write operations). As discussed at the link above you can control the size of these buffers to some degree although the defaults are normally decent for simple operations.

Bear in mind that your operating system also uses its own buffers so trying to guess exactly when a file is being written to is difficult; unless you spent a lot of time looking into it I’d always err on the side of caution.

As mentioned above, there’s few reasons why you’d want to do something like this if you already have the object, the safest & easiest option has already been mentioned, just store the original input. In theory you could use r+ or similar and moving the file pointer with f.seek as @net3912806482 mentions (which I believe flushes the buffer when called anyway) but I haven’t personally tried using it. I’d just be cautious working around .flush and .seek unless you spend some time looking into them (do they explicitly state that the system will write to the file) as you’re dealing with multiple layers of abstraction.

1 Like

Great, thank you!

Totally agreed!

1 Like