I know there are kernel apis that device drivers can use to set the transfer direction, count etc., but I would like to understand what CPU instruction does the api use to inform cpu to initiate this DMA transfer?
Advertisement
Answer
For most cases each device has its own DMA/bus mastering abilities built into the device; and the device driver uses the device’s DMA/bus mastering (in a “device specific” manner) without any kernel API being involved. In these cases, typically the OS prevents other code from using the device’s registers (by denying access to IO ports or not mapping “memory mapped IO registers” into areas that can be used by normal code); so other code is also prevented from using the device’s DMA/bus mastering.
There are rare cases where DMA/bus mastering abilities are not built into a specific device; but are a separate device on their own. This includes the ancient “DMA controller chip” in PCs, and a few more modern devices (e.g. “Intel Quickdata IO Accelerator”). In these cases; you might have a device driver for the DMA engine that other device drivers can use via. some kind of “DMA engine specific” interface, or it might (if the hardware is extremely common) be “more integrated” into the kernel (such that the “DMA engine driver code” is built into the kernel and the “DMA engine specific interface” is more like a kernel API).
Also note that:
a) for modern computers almost all software uses virtual addresses, but hardware devices can only use physical addresses. This means that at some point virtual addresses (for one virtual address space) need to be converted into physical addresses that the device can use.
b) for modern operating systems often device drivers are transferring data directly to/from memory belonging to a process (in user-space). In these cases the device driver needs to make sure that the kernel’s virtual memory management doesn’t interfere with the transfer (e.g. ensure that kernel can’t/won’t send the contents of RAM to swap space and free the RAM while the device is transferring data). Typically a device driver does this by asking the kernel to “pin” the virtual memory before starting the DMA/bus mastering (and asking kernel to “unpin” after).
c) for modern computers there may be an IOMMU involved. If IOMMU is being used to enhance security (and not just used for virtualization or “pass through”); then a device driver may need to ask the kernel to configure the IOMMU before starting any DMA/bus mastering (and if it doesn’t, the IOMMU hardware may refuse to allow the device to transfer data). In some cases an IOMMU can also be used for IO remapping to fix/avoid a device’s limitations – e.g. if a device can only use 32-bit addresses but needs to transfer data to/from a 64-bit address, then the IO MMU can be configured to remap an area into the address range a device supports (and prevent the device driver from needing to use a “bounce buffer’ in RAM where the data is copied into that temporary buffer before the DMA transfer or copied out of the temporary buffer after the DMA transfer).
For these reasons; the kernel may provide an interface that device drivers are expected to use before starting a DMA/bus mastering transfer and after a DMA/bus mastering transfer completes; but that shouldn’t be considered part of setting up the DMA/bus mastering transfer itself.