Re: [PATCH] dma: tegra: add accurate reporting of dma state

[Dmitry Osipenko]

04.05.2019 19:06, Dmitry Osipenko ÐÐÑÐÑ:
> 01.05.2019 11:58, Ben Dooks ÐÐÑÐÑ:
>> On 24/04/2019 19:17, Dmitry Osipenko wrote:
>>> 24.04.2019 19:23, Ben Dooks ÐÐÑÐÑ:
>>>> The tx_status callback does not report the state of the transfer
>>>> beyond complete segments. This causes problems with users such as
>>>> ALSA when applications want to know accurately how much data has
>>>> been moved.
>>>>
>>>> This patch addes a function tegra_dma_update_residual() to query
>>>> the hardware and modify the residual information accordinly. It
>>>> takes into account any hardware issues when trying to read the
>>>> state, such as delays between finishing a buffer and signalling
>>>> the interrupt.
>>>>
>>>> Signed-off-by: Ben Dooks <ben.dooks@xxxxxxxxxxxxxxx>
>>>
>>> Hello Ben,
>>>
>>> Thank you very much for keeping it up. I have couple comments, please
>>> see them below.
>>>
>>>> Cc: Dmitry Osipenko <digetx@xxxxxxxxx>
>>>> Cc: Laxman Dewangan <ldewangan@xxxxxxxxxx> (supporter:TEGRA DMA DRIVERS)
>>>> Cc: Jon Hunter <jonathanh@xxxxxxxxxx> (supporter:TEGRA DMA DRIVERS)
>>>> Cc: Vinod Koul <vkoul@xxxxxxxxxx> (maintainer:DMA GENERIC OFFLOAD
>>>> ENGINE SUBSYSTEM)
>>>> Cc: Dan Williams <dan.j.williams@xxxxxxxxx> (reviewer:ASYNCHRONOUS
>>>> TRANSFERS/TRANSFORMS (IOAT) API)
>>>> Cc: Thierry Reding <thierry.reding@xxxxxxxxx> (supporter:TEGRA
>>>> ARCHITECTURE SUPPORT)
>>>> Cc: dmaengine@xxxxxxxxxxxxxxx (open list:DMA GENERIC OFFLOAD ENGINE
>>>> SUBSYSTEM)
>>>> Cc: linux-tegra@xxxxxxxxxxxxxxx (open list:TEGRA ARCHITECTURE SUPPORT)
>>>> Cc: linux-kernel@xxxxxxxxxxxxxxx (open list)
>>>> ---
>>>> Â drivers/dma/tegra20-apb-dma.c | 92 ++++++++++++++++++++++++++++++++---
>>>> Â 1 file changed, 86 insertions(+), 6 deletions(-)
>>>>
>>>> diff --git a/drivers/dma/tegra20-apb-dma.c
>>>> b/drivers/dma/tegra20-apb-dma.c
>>>> index cf462b1abc0b..544e7273e741 100644
>>>> --- a/drivers/dma/tegra20-apb-dma.c
>>>> +++ b/drivers/dma/tegra20-apb-dma.c
>>>> @@ -808,6 +808,90 @@ static int tegra_dma_terminate_all(struct
>>>> dma_chan *dc)
>>>> ÂÂÂÂÂ return 0;
>>>> Â }
>>>> Â +static unsigned int tegra_dma_update_residual(struct
>>>> tegra_dma_channel *tdc,
>>>> +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ struct tegra_dma_sg_req *sg_req,
>>>> +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ struct tegra_dma_desc *dma_desc,
>>>> +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ unsigned int residual)
>>>> +{
>>>> +ÂÂÂ unsigned long status = 0x0;
>>>> +ÂÂÂ unsigned long wcount;
>>>> +ÂÂÂ unsigned long ahbptr;
>>>> +ÂÂÂ unsigned long tmp = 0x0;
>>>> +ÂÂÂ unsigned int result;
>>>
>>> You could pre-assign ahbptr=0xffffffff and result=residual here, then
>>> you could remove all the duplicated assigns below.
>>
>> ok, ta.
>>
>>>> +ÂÂÂ int retries = TEGRA_APBDMA_BURST_COMPLETE_TIME * 10;
>>>> +ÂÂÂ int done;
>>>> +
>>>> +ÂÂÂ /* if we're not the current request, then don't alter the
>>>> residual */
>>>> +ÂÂÂ if (sg_req != list_first_entry(&tdc->pending_sg_req,
>>>> +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ struct tegra_dma_sg_req, node)) {
>>>> +ÂÂÂÂÂÂÂ result = residual;
>>>> +ÂÂÂÂÂÂÂ ahbptr = 0xffffffff;
>>>> +ÂÂÂÂÂÂÂ goto done;
>>>> +ÂÂÂ }
>>>> +
>>>> +ÂÂÂ /* loop until we have a reliable result for residual */
>>>> +ÂÂÂ do {
>>>> +ÂÂÂÂÂÂÂ ahbptr = tdc_read(tdc, TEGRA_APBDMA_CHAN_AHBPTR);
>>>> +ÂÂÂÂÂÂÂ status = tdc_read(tdc, TEGRA_APBDMA_CHAN_STATUS);
>>>> +ÂÂÂÂÂÂÂ tmp =Â tdc_read(tdc, 0x08);ÂÂÂ /* total count for debug */
>>>
>>> The "tmp" variable isn't used anywhere in the code, please remove it.
>>
>> must have been left over.
>>
>>>> +
>>>> +ÂÂÂÂÂÂÂ /* check status, if channel isn't busy then skip */
>>>> +ÂÂÂÂÂÂÂ if (!(status & TEGRA_APBDMA_STATUS_BUSY)) {
>>>> +ÂÂÂÂÂÂÂÂÂÂÂ result = residual;
>>>> +ÂÂÂÂÂÂÂÂÂÂÂ break;
>>>> +ÂÂÂÂÂÂÂ }
>>>
>>> This doesn't look correct because TRM says "Busy bit gets set as soon
>>> as a channel is enabled and gets cleared after transfer completes",
>>> hence a cleared BUSY bit means that all transfers are completed and
>>> result=residual is incorrect here. Given that there is a check for EOC
>>> bit being set below, this hunk should be removed.
>>
>> I'll check notes, but see below.
>>
>>>> +
>>>> +ÂÂÂÂÂÂÂ /* if we've got an interrupt pending on the channel, don't
>>>> +ÂÂÂÂÂÂÂÂ * try and deal with the residue as the hardware has likely
>>>> +ÂÂÂÂÂÂÂÂ * moved on to the next buffer. return all data moved.
>>>> +ÂÂÂÂÂÂÂÂ */
>>>> +ÂÂÂÂÂÂÂ if (status & TEGRA_APBDMA_STATUS_ISE_EOC) {
>>>> +ÂÂÂÂÂÂÂÂÂÂÂ result = residual - sg_req->req_len;
>>>> +ÂÂÂÂÂÂÂÂÂÂÂ break;
>>>> +ÂÂÂÂÂÂÂ }
>>>> +
>>>> +ÂÂÂÂÂÂÂ if (tdc->tdma->chip_data->support_separate_wcount_reg)
>>>> +ÂÂÂÂÂÂÂÂÂÂÂ wcount = tdc_read(tdc, TEGRA_APBDMA_CHAN_WORD_TRANSFER);
>>>> +ÂÂÂÂÂÂÂ else
>>>> +ÂÂÂÂÂÂÂÂÂÂÂ wcount = status;
>>>> +
>>>> +ÂÂÂÂÂÂÂ /* If the request is at the full point, then there is a
>>>> +ÂÂÂÂÂÂÂÂ * chance that we have read the status register in the
>>>> +ÂÂÂÂÂÂÂÂ * middle of the hardware reloading the next buffer.
>>>> +ÂÂÂÂÂÂÂÂ *
>>>> +ÂÂÂÂÂÂÂÂ * The sequence seems to be at the end of the buffer, to
>>>> +ÂÂÂÂÂÂÂÂ * load the new word count before raising the EOC flag (or
>>>> +ÂÂÂÂÂÂÂÂ * changing the ping-pong flag which could have also been
>>>> + * used to determine a new buffer). This means there is a
>>>> +ÂÂÂÂÂÂÂÂ * small window where we cannot determine zero-done for the
>>>> +ÂÂÂÂÂÂÂÂ * current buffer, or moved to next buffer.
>>>> +ÂÂÂÂÂÂÂÂ *
>>>> +ÂÂÂÂÂÂÂÂ * If done shows 0, then retry the load, as it may hit the
>>>> +ÂÂÂÂÂÂÂÂ * above hardware race. We will either get a new value which
>>>> +ÂÂÂÂÂÂÂÂ * is from the first buffer, or we get an EOC (new buffer)
>>>> +ÂÂÂÂÂÂÂÂ * or both a new value and an EOC...
>>>> +ÂÂÂÂÂÂÂÂ */
>>>> +ÂÂÂÂÂÂÂ done = get_current_xferred_count(tdc, sg_req, wcount);
>>>> +ÂÂÂÂÂÂÂ if (done != 0) {
>>>> +ÂÂÂÂÂÂÂÂÂÂÂ result = residual - done;
>>>> +ÂÂÂÂÂÂÂÂÂÂÂ break;
>>>> +ÂÂÂÂÂÂÂ }
>>>> +
>>>> +ÂÂÂÂÂÂÂ ndelay(100);
>>>
>>> Please use udelay(1) because there is no ndelay on arm32 and
>>> ndelay(100) is getting rounded up to 1usec. AFAIK, arm64 doesn't have
>>> reliable ndelay on Tegra either because timer rate changes with the
>>> CPU frequency scaling.
>>
>> I'll check, but last time it was implemented. This seems a backwards step.
>>
>>> Secondly done=0 isn't a error case, technically this could be the case
>>> when tegra_dma_update_residual() is invoked just after starting the
>>> transfer. Hence I think this do-while loop and timeout checking aren't
>>> needed at all since done=0 is a perfectly valid case.
>>
>> this is not checking for an error, it's checking for a possible
>> inaccurate reading.
> 
> If you'll change reading order of the status / words registers like I
> suggested, then there won't be a case for the inaccuracy.
> 
> The EOC bit should be set atomically once transfer is finished, you
> can't get wrapped around words count and EOC bit not being set.
> 
> For oneshot transfer that runs with interrupt being disabled, the words
> counter will stop at 0 and the unset BUSY bit will indicate that the
> transfer is completed.
> 
>>>
>>> Altogether seems the tegra_dma_update_residual() could be reduced to:
>>>
>>> static unsigned int tegra_dma_update_residual(struct tegra_dma_channel
>>> *tdc,
>>> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ struct tegra_dma_sg_req *sg_req,
>>> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ struct tegra_dma_desc *dma_desc,
>>> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ unsigned int residual) 
>>> {
>>> ÂÂÂÂunsigned long status, wcount;
>>>
>>> ÂÂÂÂif (list_is_first(&sg_req->node, &tdc->pending_sg_req))
>>> ÂÂÂÂÂÂÂ return residual;
>>>
>>> ÂÂÂÂif (tdc->tdma->chip_data->support_separate_wcount_reg)
>>> ÂÂÂÂÂÂÂ wcount = tdc_read(tdc, TEGRA_APBDMA_CHAN_WORD_TRANSFER);
>>>
>>> ÂÂÂÂstatus = tdc_read(tdc, TEGRA_APBDMA_CHAN_STATUS);
>>>
>>> ÂÂÂÂif (!tdc->tdma->chip_data->support_separate_wcount_reg)
>>> ÂÂÂÂÂÂÂ wcount = status;
>>>
>>> ÂÂÂÂif (status & TEGRA_APBDMA_STATUS_ISE_EOC)
>>> ÂÂÂÂÂÂÂ return residual - sg_req->req_len;
>>>
>>> ÂÂÂÂreturn residual - get_current_xferred_count(tdc, sg_req, wcount);
>>> }
>>
>> I'm not sure if that will work all the time. It took days of testing to
>> get reliable error data for the cases we're looking for here.
> 
> Could you please tell exactly what those cases are. I don't see when the
> simplified variant could fail, but maybe I already forgot some extra
> details about how APB DMA works.
> 
> I tested the variant I'm suggesting (with the fixed typos and added
> check for the BUSY bit) and it works absolutely fine, audio stuttering
> issue is fixed, everything else works too. Please consider to use it for
> the next version of the patch if there are no objections.
> 

Actually the BUSY bit checking shouldn't be needed. I think it's a bug
in the driver that it may not enable EOC interrupt and will send a patch
to fix it.