flock(), fclose() and O/S buffering

[Andrew Hill]

Hi all,

I have a question about the way flock() and fclose() work in PHP.
Consider the following code, slightly modified from the flock() PHP
manual page:


$fp = fopen("/tmp/lock.txt", "w+");
if (flock($fp, LOCK_EX)) { // do an exclusive lock
fwrite($fp, "$processName\n");
flock($fp, LOCK_UN); // release the lock
} else {
echo "Couldn't lock the file !";
}
fclose($fp);


If the above code was executed by two processes, process A and process
B, one possible sequence of events is:

Process A opens the file.
Process B opens the file.
Process A obtains an exclusive lock.
Process A writes it's process name to the file.
Process A releases the exclusive lock.
Process A closes the file.
Process B obtains an exclusive lock.
Process B writes it's process name to the file.
Process B releases the exclusive lock.
Process B closes the file.

The results would be as desired - that is, as process B obtained the
lock on the file after process A, it is process B's process name that is
in the contents of the file, not process A.

However, another possible sequence of events is:

Process A opens the file.
Process B opens the file.
Process A obtains an exclusive lock.
Process A writes it's process name to the file.
Process A releases the exclusive lock.
Process B obtains an exclusive lock.
Process B writes it's process name to the file.
Process B releases the exclusive lock.
Process B closes the file.
Process A closes the file.

In this case, although process B is the second process to obtain a lock
and write its process name to the file, it is the first process to close
the file handle.

This raises the question of when the operating system actually writes
file contents to disk. Is it when PHP performs an fwrite()? Or does the
O/S buffer the file contents?

Essentially, when performing the style of concurrent programming above,
can one be certain at what point in time the file contents will be
written (on any operating system)?

My suspicion is that the answer to the above question is no, and as a
result, in order to be certain of correctly serialising the file locking
and output process, it would be necessary to use a separate lockfile,
which is opened and locked *before* the file to be written to is opened,
written, and then closed, after which the lock on the lockfile can be
released.

Can anyone please confirm that this is the case?

Thanks,

[Michael Sims]

Post by Andrew Hill
$fp = fopen("/tmp/lock.txt", "w+");
if (flock($fp, LOCK_EX)) { // do an exclusive lock
fwrite($fp, "$processName\n");
flock($fp, LOCK_UN); // release the lock
} else {
echo "Couldn't lock the file !";
}
fclose($fp);

[...]

Post by Andrew Hill
In this case, although process B is the second process to obtain a
lock and write its process name to the file, it is the first process
to close the file handle.

I've used flock() quite a bit in Perl, but not so much in PHP. I'm almost
positive that they both use the same system calls so I think the behavior
(at least under unix) is the same. In Perl, you do not have to release a
lock before closing the file...the lock is automatically released on close.
Newer versions of Perl automatically flush the disk buffers when releasing a
lock, but it's still widely recommended that you do NOT release a lock
before closing a file if you have written to it, precisely because of the
race condition you pointed out.

From:
http://www.tcp.com/~mink/CLAM/week8.html

"...Locks will dissolve when the file is closed anyway (when your process
exits, if not before). If you unlock without closing, the buffer may not
have been flushed before another process tries to read the file. In that
case, the other process will get "old" data. There are ways around this, but
the easiest is to simply not use LOCK_UN on files you are really writing
to - simply close the files and release the locks that way..."

Also:
http://groups.google.com/groups?selm=388373bd.890334%40news.skynet.be
http://groups.google.com/groups?selm=Pine.GSO.3.96.970311110353.695N-100000%
40kelly.teleport.com

Of course these all apply to Perl, but I believe they could be applied to
PHP as well. I wonder if this should be considered a documentation problem
with the given examples.

Post by Andrew Hill
My suspicion is that the answer to the above question is no, and as a
result, in order to be certain of correctly serialising the file
locking and output process, it would be necessary to use a separate
lockfile, which is opened and locked *before* the file to be written
to is opened, written, and then closed, after which the lock on the
lockfile can be released.

I think that you will still have to do this, however, not because of the
fclose() issue but because you are opening the file in write mode, which
truncates it. If you were opening in append mode you would be good by just
not releasing the lock before the fclose(), but since the fopen() in write
mode truncates the file BEFORE the lock is requested, you're in trouble
without using a separate lock file. There is a note in PHP's flock()
documentation about that:

"Note: Because flock() requires a file pointer, you may have to use a
special lock file to protect access to a file that you intend to truncate by
opening it in write mode (with a "w" or "w+" argument to fopen())."

So, I'd still go with getting an exclusive lock on a separate lock file,
writing to my actual file, then releasing the lock on the separate lock
file...

HTH