Re: [PATCH] dma: tegra: add accurate reporting of dma state

[Dmitry Osipenko]

01.05.2019 11:58, Ben Dooks ÐÐÑÐÑ:
> On 24/04/2019 19:17, Dmitry Osipenko wrote:
>> 24.04.2019 19:23, Ben Dooks ÐÐÑÐÑ:
>>> The tx_status callback does not report the state of the transfer
>>> beyond complete segments. This causes problems with users such as
>>> ALSA when applications want to know accurately how much data has
>>> been moved.
>>>
>>> This patch addes a function tegra_dma_update_residual() to query
>>> the hardware and modify the residual information accordinly. It
>>> takes into account any hardware issues when trying to read the
>>> state, such as delays between finishing a buffer and signalling
>>> the interrupt.
>>>
>>> Signed-off-by: Ben Dooks <ben.dooks@xxxxxxxxxxxxxxx>
>>
>> Hello Ben,
>>
>> Thank you very much for keeping it up. I have couple comments, please
>> see them below.
>>
>>> Cc: Dmitry Osipenko <digetx@xxxxxxxxx>
>>> Cc: Laxman Dewangan <ldewangan@xxxxxxxxxx> (supporter:TEGRA DMA DRIVERS)
>>> Cc: Jon Hunter <jonathanh@xxxxxxxxxx> (supporter:TEGRA DMA DRIVERS)
>>> Cc: Vinod Koul <vkoul@xxxxxxxxxx> (maintainer:DMA GENERIC OFFLOAD
>>> ENGINE SUBSYSTEM)
>>> Cc: Dan Williams <dan.j.williams@xxxxxxxxx> (reviewer:ASYNCHRONOUS
>>> TRANSFERS/TRANSFORMS (IOAT) API)
>>> Cc: Thierry Reding <thierry.reding@xxxxxxxxx> (supporter:TEGRA
>>> ARCHITECTURE SUPPORT)
>>> Cc: dmaengine@xxxxxxxxxxxxxxx (open list:DMA GENERIC OFFLOAD ENGINE
>>> SUBSYSTEM)
>>> Cc: linux-tegra@xxxxxxxxxxxxxxx (open list:TEGRA ARCHITECTURE SUPPORT)
>>> Cc: linux-kernel@xxxxxxxxxxxxxxx (open list)
>>> ---
>>> Â drivers/dma/tegra20-apb-dma.c | 92 ++++++++++++++++++++++++++++++++---
>>> Â 1 file changed, 86 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/drivers/dma/tegra20-apb-dma.c
>>> b/drivers/dma/tegra20-apb-dma.c
>>> index cf462b1abc0b..544e7273e741 100644
>>> --- a/drivers/dma/tegra20-apb-dma.c
>>> +++ b/drivers/dma/tegra20-apb-dma.c
>>> @@ -808,6 +808,90 @@ static int tegra_dma_terminate_all(struct
>>> dma_chan *dc)
>>> ÂÂÂÂÂ return 0;
>>> Â }
>>> Â +static unsigned int tegra_dma_update_residual(struct
>>> tegra_dma_channel *tdc,
>>> +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ struct tegra_dma_sg_req *sg_req,
>>> +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ struct tegra_dma_desc *dma_desc,
>>> +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ unsigned int residual)
>>> +{
>>> +ÂÂÂ unsigned long status = 0x0;
>>> +ÂÂÂ unsigned long wcount;
>>> +ÂÂÂ unsigned long ahbptr;
>>> +ÂÂÂ unsigned long tmp = 0x0;
>>> +ÂÂÂ unsigned int result;
>>
>> You could pre-assign ahbptr=0xffffffff and result=residual here, then
>> you could remove all the duplicated assigns below.
> 
> ok, ta.
> 
>>> +ÂÂÂ int retries = TEGRA_APBDMA_BURST_COMPLETE_TIME * 10;
>>> +ÂÂÂ int done;
>>> +
>>> +ÂÂÂ /* if we're not the current request, then don't alter the
>>> residual */
>>> +ÂÂÂ if (sg_req != list_first_entry(&tdc->pending_sg_req,
>>> +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ struct tegra_dma_sg_req, node)) {
>>> +ÂÂÂÂÂÂÂ result = residual;
>>> +ÂÂÂÂÂÂÂ ahbptr = 0xffffffff;
>>> +ÂÂÂÂÂÂÂ goto done;
>>> +ÂÂÂ }
>>> +
>>> +ÂÂÂ /* loop until we have a reliable result for residual */
>>> +ÂÂÂ do {
>>> +ÂÂÂÂÂÂÂ ahbptr = tdc_read(tdc, TEGRA_APBDMA_CHAN_AHBPTR);
>>> +ÂÂÂÂÂÂÂ status = tdc_read(tdc, TEGRA_APBDMA_CHAN_STATUS);
>>> +ÂÂÂÂÂÂÂ tmp =Â tdc_read(tdc, 0x08);ÂÂÂ /* total count for debug */
>>
>> The "tmp" variable isn't used anywhere in the code, please remove it.
> 
> must have been left over.
> 
>>> +
>>> +ÂÂÂÂÂÂÂ /* check status, if channel isn't busy then skip */
>>> +ÂÂÂÂÂÂÂ if (!(status & TEGRA_APBDMA_STATUS_BUSY)) {
>>> +ÂÂÂÂÂÂÂÂÂÂÂ result = residual;
>>> +ÂÂÂÂÂÂÂÂÂÂÂ break;
>>> +ÂÂÂÂÂÂÂ }
>>
>> This doesn't look correct because TRM says "Busy bit gets set as soon
>> as a channel is enabled and gets cleared after transfer completes",
>> hence a cleared BUSY bit means that all transfers are completed and
>> result=residual is incorrect here. Given that there is a check for EOC
>> bit being set below, this hunk should be removed.
> 
> I'll check notes, but see below.
> 
>>> +
>>> +ÂÂÂÂÂÂÂ /* if we've got an interrupt pending on the channel, don't
>>> +ÂÂÂÂÂÂÂÂ * try and deal with the residue as the hardware has likely
>>> +ÂÂÂÂÂÂÂÂ * moved on to the next buffer. return all data moved.
>>> +ÂÂÂÂÂÂÂÂ */
>>> +ÂÂÂÂÂÂÂ if (status & TEGRA_APBDMA_STATUS_ISE_EOC) {
>>> +ÂÂÂÂÂÂÂÂÂÂÂ result = residual - sg_req->req_len;
>>> +ÂÂÂÂÂÂÂÂÂÂÂ break;
>>> +ÂÂÂÂÂÂÂ }
>>> +
>>> +ÂÂÂÂÂÂÂ if (tdc->tdma->chip_data->support_separate_wcount_reg)
>>> +ÂÂÂÂÂÂÂÂÂÂÂ wcount = tdc_read(tdc, TEGRA_APBDMA_CHAN_WORD_TRANSFER);
>>> +ÂÂÂÂÂÂÂ else
>>> +ÂÂÂÂÂÂÂÂÂÂÂ wcount = status;
>>> +
>>> +ÂÂÂÂÂÂÂ /* If the request is at the full point, then there is a
>>> +ÂÂÂÂÂÂÂÂ * chance that we have read the status register in the
>>> +ÂÂÂÂÂÂÂÂ * middle of the hardware reloading the next buffer.
>>> +ÂÂÂÂÂÂÂÂ *
>>> +ÂÂÂÂÂÂÂÂ * The sequence seems to be at the end of the buffer, to
>>> +ÂÂÂÂÂÂÂÂ * load the new word count before raising the EOC flag (or
>>> +ÂÂÂÂÂÂÂÂ * changing the ping-pong flag which could have also been
>>> + * used to determine a new buffer). This means there is a
>>> +ÂÂÂÂÂÂÂÂ * small window where we cannot determine zero-done for the
>>> +ÂÂÂÂÂÂÂÂ * current buffer, or moved to next buffer.
>>> +ÂÂÂÂÂÂÂÂ *
>>> +ÂÂÂÂÂÂÂÂ * If done shows 0, then retry the load, as it may hit the
>>> +ÂÂÂÂÂÂÂÂ * above hardware race. We will either get a new value which
>>> +ÂÂÂÂÂÂÂÂ * is from the first buffer, or we get an EOC (new buffer)
>>> +ÂÂÂÂÂÂÂÂ * or both a new value and an EOC...
>>> +ÂÂÂÂÂÂÂÂ */
>>> +ÂÂÂÂÂÂÂ done = get_current_xferred_count(tdc, sg_req, wcount);
>>> +ÂÂÂÂÂÂÂ if (done != 0) {
>>> +ÂÂÂÂÂÂÂÂÂÂÂ result = residual - done;
>>> +ÂÂÂÂÂÂÂÂÂÂÂ break;
>>> +ÂÂÂÂÂÂÂ }
>>> +
>>> +ÂÂÂÂÂÂÂ ndelay(100);
>>
>> Please use udelay(1) because there is no ndelay on arm32 and
>> ndelay(100) is getting rounded up to 1usec. AFAIK, arm64 doesn't have
>> reliable ndelay on Tegra either because timer rate changes with the
>> CPU frequency scaling.
> 
> I'll check, but last time it was implemented. This seems a backwards step.
> 
>> Secondly done=0 isn't a error case, technically this could be the case
>> when tegra_dma_update_residual() is invoked just after starting the
>> transfer. Hence I think this do-while loop and timeout checking aren't
>> needed at all since done=0 is a perfectly valid case.
> 
> this is not checking for an error, it's checking for a possible
> inaccurate reading.

If you'll change reading order of the status / words registers like I
suggested, then there won't be a case for the inaccuracy.

The EOC bit should be set atomically once transfer is finished, you
can't get wrapped around words count and EOC bit not being set.

For oneshot transfer that runs with interrupt being disabled, the words
counter will stop at 0 and the unset BUSY bit will indicate that the
transfer is completed.

>>
>> Altogether seems the tegra_dma_update_residual() could be reduced to:
>>
>> static unsigned int tegra_dma_update_residual(struct tegra_dma_channel
>> *tdc,
>> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ struct tegra_dma_sg_req *sg_req,
>> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ struct tegra_dma_desc *dma_desc,
>> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ unsigned int residual) 
>> {
>> ÂÂÂÂunsigned long status, wcount;
>>
>> ÂÂÂÂif (list_is_first(&sg_req->node, &tdc->pending_sg_req))
>> ÂÂÂÂÂÂÂ return residual;
>>
>> ÂÂÂÂif (tdc->tdma->chip_data->support_separate_wcount_reg)
>> ÂÂÂÂÂÂÂ wcount = tdc_read(tdc, TEGRA_APBDMA_CHAN_WORD_TRANSFER);
>>
>> ÂÂÂÂstatus = tdc_read(tdc, TEGRA_APBDMA_CHAN_STATUS);
>>
>> ÂÂÂÂif (!tdc->tdma->chip_data->support_separate_wcount_reg)
>> ÂÂÂÂÂÂÂ wcount = status;
>>
>> ÂÂÂÂif (status & TEGRA_APBDMA_STATUS_ISE_EOC)
>> ÂÂÂÂÂÂÂ return residual - sg_req->req_len;
>>
>> ÂÂÂÂreturn residual - get_current_xferred_count(tdc, sg_req, wcount);
>> }
> 
> I'm not sure if that will work all the time. It took days of testing to
> get reliable error data for the cases we're looking for here.

Could you please tell exactly what those cases are. I don't see when the
simplified variant could fail, but maybe I already forgot some extra
details about how APB DMA works.

I tested the variant I'm suggesting (with the fixed typos and added
check for the BUSY bit) and it works absolutely fine, audio stuttering
issue is fixed, everything else works too. Please consider to use it for
the next version of the patch if there are no objections.