I’m writing a program that runs on both Linux and FreeBSD, and I want to make sure that the data is actually written to the file on the physical device when each write() returns, so that my data won’t get lost by accident (eg, power lost, the process is interrupted unexpected, etc.).
According to OPEN(2) man page, on Linux (higher than 2.6), O_DIRECT is synchronous but may have performance problems; on FreeBSD, O_DIRECT is not guaranteed synchronous and may also has problems.
So, on Linux, either O_DIRECT or O_SYNC guarantees synchronous write, but which one has better performance?
On FreeBSD, to guarantee synchronous write, which option has the best performance: (1) O_DIRECT + fsync() (2) O_DIRECT | O_SYNC or (3)O_SYNC alone?
Advertisement
Answer
With current harddisks, there is no assurance that a file is actually written to disk even if the disk reports the write as complete to the OS! This is due to built-in cache in the drive.
On freeBSD you can disable this by setting the kern.cam.ada.write_cache sysctl to 0. This will degrade write performance significantly. Last time I measured it (WDC WD5001ABYS-01YNA0 harddisk on an ICH-7 chipset, FreeBSD 8.1 AMD64), continuous write performance (measured with dd if=/dev/zero of=/tmp/foo bs=10M count=1000) dropped from 75,000,000 bytes/sec to 12,900,000 bytes/sec.
If you want to be absolutely sure that your files are written;
- Disable write caching with
sysctl kern.cam.ada.write_cache=0followed bycamcontrol reset <bus>:<target>:<lun>. - Open the file with the
O_SYNCoption.
Note:
- Your write perfomance (on a HDD) will now absolutely suck.
- Do not mount the partition with the
syncoption; that will cause all I/O (including reads) to be done syncronously. - Do not use
O_DIRECT. It will try to bypass the cache altogether. That will probably also influence reads.