I was writing a bit a ruby code that used flock to write and read from a file using two different processes. I ran into some funny issues, so I’m going to try to explain the process I went through to reach my final working code.
First let’s say I have a two ruby processes, one is writing a small amount of text to a file while the other is reading from that file. For example the process doing the writing looks like:
for i in 0..1000 do
puts i.to_s
file = File.new('shared_file','w')
file.write("line1\\nline2")
file.close
end
…and the process reading the file looks like:
for i in 0..1000 do
puts i.to_s
file = File.new('shared_file','r')
lines = file.readlines
if(lines[0]!="line1\\n")
puts("got #{lines[0]} instead\\n")
exit
end
file.close
end
Unfortunately when these two chunks of code run at the same time I end up with a message something like..
file_writer.rb:3:in `initialize': Text file busy - shared_file (Errno::ETXTBSY)
This message comes from the process doing the writing. So I think I can positively say that even though I’m only writing a small amount of text, the file needs to be locked in order for both processes to run correctly.
So the next step was to test out the flock File method using a couple of irb sessions. In irb I am able to lock a file by doing the following:
irb(main):001:0> (file = File.new('shared_file','w')).flock(File::LOCK_EX)
=> 0
irb(main):002:0>
When I try to open the same file for reading in the second irb session I get:
irb(main):001:0> (file = File.new('shared_file','r')).flock(File::LOCK_EX)
…this will hang here until I unlock the file from the first irb session. So using this approach to locking the file I can update my writing code:
for i in 0..1000 do
puts i.to_s
(file = File.new('shared_file','w')).flock(File::LOCK_EX)
file.write("line1\\nline2")
file.flock(File::LOCK_UN)
file.close
end
… and my reading code …
for i in 0..1000 do
puts i.to_s
(file = File.new('shared_file','r')).flock(File::LOCK_EX)
lines = file.readlines
if(lines[0]!="line1\\n")
puts("got #{lines[0]} instead\\n")
exit
end
file.flock(File::LOCK_UN)
file.close
end
Now I would expect this to work, but it doesn’t behave quite as I’d expect. Sometimes the output of my reader is:
# ruby file_reader.rb
0
got instead
When I get this output from my reader, the writer continues on its merry way until it terminates. Now this output is saying that instead of reading “line1″ from the file, it read nothing or an empty string. This is weird since the writer is always writing the same thing and the file should always contain “line1\nline2″ Other times once I start the reader the writer spits out:
file_writer.rb:3:in `initialize': Text file busy - shared_file (Errno::ETXTBSY)
This output is also strange, since this is the sort of error I was getting before I added the file locking code. This would lead me to believe that the file is actually not getting locked even though it worked in irb.
So what’s going on? I’m not exactly sure, but I found this thread from someone else who seemed to be having a similar problem. The solution that was purposed included:
“Also, opening files for writing and truncating them may introduce subtle
problems - you cannot lock the file before it’s truncated, so you need to
try open it in read-write mode, then lock and then truncate it…”
So I tried updated my writing code:
for i in 0..1000 do
puts i.to_s
(file = File.new('shared_file','r+')).flock(File::LOCK_EX)
file.truncate 0
file.write("line1\nline2")
file.flock(File::LOCK_UN)
file.close
end
And what do you know, it works. I’m not sure why, but opening the file using “r+” (read-write) instead of just “w” (write) fixes all of the problems that I was seeing previously. I’m not sure if this is the only or best solution to this problem, but this is the only one I could come up with while still using flock to lock the file.
