Skip to content
Advertisement

Linux DMA: Using the DMAengine for scatter-gather transactions

I try to use the DMAengine API from a custom kernel driver to perform a scatter-gather operation. I have a contiguous memory region as source and I want to copy its data in several distributed buffers through a scatterlist structure. The DMA controller is the PL330 one that supports the DMAengine API (see PL330 DMA controller).

My test code is the following:

In my driver header file (test_driver.h):

JavaScript

In my source that contains the dma functions (dma_functions.c):

JavaScript

In my driver source file (test_driver.c):

JavaScript

However, the DMA never calls my callback function and I do not have any idea why it happens. Maybe, I am misunderstanding something…

Could anyone help me?

Thanks in advance.

Advertisement

Answer

Caveat: I don’t have a definitive solution for you, but merely some observations and suggestions on how to debug this [based on many years of experience writing/debugging linux device drivers].

I presume you believe the callback is not being done because you don’t get any printk messages. But, the callback is the only place that has them. But, is the printk level set high enough to see the messages? I’d add a dev_info to your module init, to prove it prints as expected.

Also, you [probably] won’t get a callback if dma_start doesn’t work as expected, so I’d add some dev_info calls there, too (e.g. before and after the call in step 7). I also notice that not all calls in dma_start check error returns [may be fine or void return, just mentioning in case you missed one]

At this point, it should be noted that there are really two questions here: (1) Did your DMA request start successfully [and complete]? (2) Did you get a callback?

So, I’d split off some code from dma_complete into (e.g.) dma_test_done. The latter does the same checking but only prints the “complete” message. You can call this in a poll mode to verify DMA completion.

So, if you [eventually] get a completion, then the problem reduces to why you didn’t get the callback. If, however, you don’t [even] get a completion, that’s an even more fundamental problem.

This reminds me. You didn’t show any code that calls dma_start or how you wait for the completion. I presume that if your callback were working, it would issue a wakeup of some sort that the base level would wait on. Or, the callback would do the request deallocate/cleanup (i.e. more code you’d write)

At step 7, you’re calling dma_async_issue_pending, which should call pl330_issue_pending. pl330_issue_pending will call pl330_tasklet.

pl330_tasklet is a tasklet function, but it can also be called directly [to kick off DMA when there are no active requests].

pl330_tasklet will loop on its “work” queue and move any completed items to its “completed” queue. It then tries to start new requests. It then loops on its completed queue and issues the callbacks.

pl330_tasklet grabs the callback pointer, but if it’s null it is silently ignored. You’ve set a callback, but it might be good to verify that where you set the callback is the same place [or propagates to] the place where pl330_tasklet will fetch it from.

When you make the call, everything may be busy, so there are no completed requests, no room to start a new request, so nothing to complete. In that case, pl330_tasklet will be called again later.

So, when dma_async_issue_pending returns, nothing may have happened yet. This is quite probable for your case.

pl330_tasklet tries to start new DMA by calling fill_queue. It will check that a descriptor is not [already] busy by looking at status != BUSY. So, you may wish to verify that yours has the correct value. Otherwise, you’d never get a callback [or even any DMA start].

Then, fill_queue will try to start the request via pl330_submit_req. But, that can return an error (e.g. queue already full), so, again, things are deferred.

For reference, notice the following comment at the top of pl330_submit_req:

JavaScript

What I’d do is start hacking up pl330.c and add debug messages and cross-checking. If your system is such that pl330 is servicing many other requests, you might limit the debug messages by checking that the device’s private data pointer matches yours.

In particular, you’d like to get a message when your request actually gets started, so you could add a debug message to the end of pl330_submit_req

Then, adding messages within pl330_tasklet for requests will help, too.

Those are two good starting points. But, don’t be afraid to add more printk calls as needed. You may be surprised by what gets called [or doesn’t get called] or in what order.


UPDATE:

If I install the kernel module with the blocking behaviour, everything is initialized well. However, the dma_busy_loop function shows that the DMA descriptor is always IN PROGESS and the DMA transaction never completes. For this reason, the callback function is not executed. What could be happening?

Did a little more research. Cookies are just sequence numbers that increment. For example, if you issue a request that gets broken up into [say] 10 separate scatter/gather operations [descriptors], each one gets a unique cookie value. The cookie return value is the latest/last of the bunch (e.g. 10).

When you’re calling (1) dma_async_is_tx_complete, (2) it calls chan->device->device_tx_status, (3) which is pl330_tx_status, (4) which calls dma_cookie_status

Side note/tip: When I was tracking this down, I just kept flipping back and forth between dmaengine.h and pl330.c. It was like: Look at (1), it calls (2). Where is that set? In pl330.c, I presume. So, I grepped for the string and got the name of pl330’s function (i.e. (3)). So, I go there, and see that it does (4). So … Back to dmaengine.h

However, when you make the outer call, you’re ignoring [setting to NULL] the last two arguments. These can be useful because they return the “last” and “used” cookies. So, even if you don’t get full completion, these values could change and show partial progress.

One of them should eventually be >= to the “return” cookie value. (i.e.) The entire operation should be complete. So, this will help differentiate what may be happening.

Also, note that in dmaengine.h, right below dma_async_is_tx_complete, there is dma_async_is_complete. This function is what decides whether to return DMA_COMPLETE or DMA_IN_PROGRESS, based on the cookie value you pass and the “last” and “used” cookie values. It’s passive, and not used in the code path [AFAICT], but it does show how to calculate completion yourself.

User contributions licensed under: CC BY-SA
1 People found this is helpful
Advertisement