UIO and msync: Why does msync return “invalid argument” even though the address is a multiple of PAGESIZE

Linux version: 4.19
Platform: Xilinx Ultrascale+ Zynq

In the programmable logic I’ve created a memory mapped device located at physical address 0xA0001000. I’m using uio_pdrv_genirq as my device driver. The device shows up as uio0 and I’m ready to read and write to it, using mmap. I want to be able to guarantee that any writes that I make get written to the device right away instead of waiting for Linux to flush the dirty pages on its own. For that I should use msync according to all my research. But I keep getting an error when I do that. Here’s my test program:

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <unistd.h>
#include <stdio.h>
#include <errno.h>
#include <string.h>
#include <unistd.h>

void main() {
    int fid;
    int rval;
    char *data;
    int idx;

    printf("Open UIO Device n");
    fid= open("/dev/uio0", O_RDWR | O_SYNC);

    data= mmap(NULL, 0x1000, PROT_WRITE|PROT_READ, MAP_SHARED, fid, 0);
    if(MAP_FAILED == data) {
        printf("Error code when mapping! %s", strerror(errno));
    }
    printf("addr= 0x%8Xn", data);
    printf("pagesize= 0x%4Xn", getpagesize());

    printf("Write some datan");
    data[11]= 0xDE;
    data[10]= 0xC0;
    data[ 9]= 0xDE;
    data[ 8]= 0xAD;

    rval= msync(data, 0x1000, MS_SYNC);
    if(-1 == rval) {
        printf("Error on msync! %sn", strerror(errno));
    }

    if(munmap(data, 0x1000) < 0) {
        printf("munmap error! %sn", strerror(errno));
    }

    printf("Close UIO devicen");
    rval= close(fid);
    if(rval != 0) {
        printf("UIO device close Error!n");
    }
}

And here’s the program output:

mylinux:~/test-apps$ ./a.out
Open UIO Device
addr= 0xABA05000
pagesize= 0x1000
Write some data
Error on msync! Invalid argument
Close UIO device

I’m not understanding the source of this error. The msync man pages states that this error can occur if the address is not a multiple of PAGESIZE. But as you can see from the example above, it is a multiple. Even the physical address is a multiple of the PAGESIZE.

A few other notes:

1. I get the same error even I remove O_SYNC from the flags to the open function.
2. The AXI4 bus into the FPGA fabric is 128 bits, which is then converted to 32 bits by Xilinx IP blocks. I don’t think this would have anything to do with my problem.

Thanks for any insights folks can provide.

Answer

I believe the EINVAL error is because the kernel’s handler for the msync syscall (in “mm/msync.c“) calls vfs_fsync_range:

            error = vfs_fsync_range(file, fstart, fend, 1);

and vfs_fsync_range (in “fs/sync.c“) is returning -EINVAL here:

    if (!file->f_op->fsync)
        return -EINVAL;

because the UIO driver core (in “drivers/uio/uio.c” does not set the fsync file operation handler.

---

The physical memory you have mmaped is mapped as non-cached memory in the page tables, so writes do not need to be flushed. However, you should probably use volatile accesses to the memory mapped I/O registers. The register access functions such as readb and writeb in the kernel are architecture-specific, but invariably convert the address to a pointer to a volatile integer type (e.g. volatile unsigned char *, volatile unsigned short *, or volatile unsigned int * depending on the width of the register access) before accessing the memory location.