Revisiting open()

You may recall the declaration for open():

int open(const char *pathname, int flags, mode_t mode);

Earlier, we said that when opening a file for plain I/O, we could ignore the mode argument. Having seen creat(), though, you can probably guess that open() can also be used for creating files and that the mode argument is used in this case. This is indeed true.

Besides the O_RDONLY, O_WRONLY, and O_RDWR flags, additional flags may be bitwise OR'd when open() is called. The POSIX standard mandates a number of these additional flags. Table 4.7 presents the flags that are used for most mundane applications.

Table 4.7. Additional POSIX flags for open()

Flag Meaning

O_APPEND Force all writes to occur at the end of the file.

O_CREAT Create the file if it doesn't exist.

O_EXCL When used with O_CREAT, cause open() to fail if the file already exists.

O_TRUNC Truncate the file (set it to zero length) if it exists.

Given O_APPEND and O_TRUNC, you can imagine how the shell might open or create files corresponding to the >

and >> operators. For example:

int fd;

extern char *filename;

mode_t mode = S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP|S_IROTH|S_IWOTH; /* 0666 */

fd = open(filename, O_CREAT|O_WRONLY|O_TRUNC, mode); /* for > */

fd = open(filename, O_CREAT|O_WRONLY|O_APPEND, mode); /* for >> */

Note that the O_EXCL flag would not be used here, since for both > and >>, it's not an error for the file to exist.

Remember also that the system applies the umask to the requested permissions.

Also, it's easy to see that, at least conceptually, creat() could be written this easily:

int creat(const char *path, mode_t mode) {

return open(path, O_CREAT|O_WRONLY|O_TRUNC, mode);

}

Note

If a file is opened with O_APPEND, all data will be written at the end of the file, even if the current position has been reset with lseek().

Modern systems provide additional flags whose uses are more specialized. Table 4.8 describes them briefly.

Table 4.8. Additional advanced POSIX flags for open()

Flag Meaning

O_NOCTTY If the device being opened is a terminal, it does not become the process's controlling terminal. (This is a more advanced topic, discussed briefly in Section 9.2.1, page 312.)

O_NONBLOCK Disables blocking of I/O operations in certain cases (see Section 9.4.3.4, page 333).

O_DSYNC Ensure that data written to a file make it all the way to physical storage before write() returns.

O_RSYNC Ensure that any data that read() would read, which may have been written to the file being read, have made it all the way to physical storage before read() returns.

O_SYNC Like O_DSYNC, but also ensure that all file metadata, such as access times, have also been written to physical storage.

The O_DSYNC, O_RSYNC, and O_SYNC flags need some explanation. Unix systems (including Linux) maintain an internal cache of disk blocks, called the buffer cache. When the write() system call returns, the data passed to the operating system have been copied to a buffer in the buffer cache. They are not necessarily written out to the disk.

The buffer cache provides considerable performance improvement: Since disk I/O is often an order of magnitude or more slower than CPU and memory operations, programs would slow down considerably if they had to wait for every write to go all the way through to the disk. In addition, if data have recently been written to a file, a subsequent read of that same data will find the information already in the buffer cache, where it can be returned immediately instead of having to wait for an I/O operation to read it from the disk.

Unix systems also do read-ahead; since most reads are sequential, upon reading one block, the operating system will read several more consecutive disk blocks so that their information will already be in the buffer cache when a program asks for it. If multiple programs are reading the same file, they all benefit since they will all get their data from the same copy of the file's disk blocks in the buffer cache.

All of this caching is wonderful, but of course there's no free lunch. While data are in the buffer cache and before

they have been written to disk, there's a small—but very real—window in which disaster can strike; for example, if the power goes out. Modern disk drives exacerbate this problem: Many have their own internal buffers, so while data may have made it to the drive, it may not have made it onto the media when the power goes out! This can be a significant issue for small systems that aren't in a data center with controlled power or that don't have an uninterruptible power supply (UPS).^[5]

[5] If you don't have a UPS and you use your system for critical work, we highly recommend investing in one. You should also be doing regular backups.

For most applications, the chance that data in the buffer cache might be inadvertently lost is acceptably small.

However, for some applications, any such chance is not acceptable. Thus, the notion of synchronous I/O was added to Unix systems, whereby a program can be guaranteed that if a system call has returned, the data are safely written on a physical storage device.

The O_DSYNC flag guarantees data integrity; the data and any other information that the operating system needs to find the data are written to disk before write() returns. However, metadata, such as access and modification times, may not be written to disk. The O_SYNC flag requires that metadata also be written to disk before write()

returns. (Here too there is no free lunch; synchronous writes can seriously affect the performance of a program, slowing it down noticeably.)

The O_RSYNC flag is for data reads: If read() finds data in the buffer cache that were scheduled for writing to disk, then read() won't return that data until they have been written to disk. The other two flags can affect this:

In particular, O_SYNC will cause read() to wait until the file metadata have been written out as well.

Note

As of kernel version 2.4, Linux treats all three flags the same, with essentially the meaning of O_SYNC. Furthermore, Linux defines additional flags that are Linux specific and intended for specialized uses.

Check the GNU/Linux open(2) manpage for more information.