Re: [PATCH] net: ethernet: ti: davinci_cpdma: fix warning "device driver frees DMA memory with different size"

[Ivan Khoronzhuk]

Hi Grygorii,

On Wed, Dec 04, 2019 at 02:46:18PM +0200, Grygorii Strashko wrote:
> The TI CPSW(s) driver produces warning with DMA API debug options enabled:
>
> WARNING: CPU: 0 PID: 1033 at kernel/dma/debug.c:1025 check_unmap+0x4a8/0x968
> DMA-API: cpsw 48484000.ethernet: device driver frees DMA memory with different size
> [device address=0x00000000abc6aa02] [map size=64 bytes] [unmap size=42 bytes]
> CPU: 0 PID: 1033 Comm: ping Not tainted 5.3.0-dirty #41
> Hardware name: Generic DRA72X (Flattened Device Tree)
> [<c0112c60>] (unwind_backtrace) from [<c010d270>] (show_stack+0x10/0x14)
> [<c010d270>] (show_stack) from [<c09bc564>] (dump_stack+0xd8/0x110)
> [<c09bc564>] (dump_stack) from [<c013b93c>] (__warn+0xe0/0x10c)
> [<c013b93c>] (__warn) from [<c013b9ac>] (warn_slowpath_fmt+0x44/0x6c)
> [<c013b9ac>] (warn_slowpath_fmt) from [<c01e0368>] (check_unmap+0x4a8/0x968)
> [<c01e0368>] (check_unmap) from [<c01e08a8>] (debug_dma_unmap_page+0x80/0x90)
> [<c01e08a8>] (debug_dma_unmap_page) from [<c0752414>] (__cpdma_chan_free+0x114/0x16c)
> [<c0752414>] (__cpdma_chan_free) from [<c07525c4>] (__cpdma_chan_process+0x158/0x17c)
> [<c07525c4>] (__cpdma_chan_process) from [<c0753690>] (cpdma_chan_process+0x3c/0x5c)
> [<c0753690>] (cpdma_chan_process) from [<c0758660>] (cpsw_tx_mq_poll+0x48/0x94)
> [<c0758660>] (cpsw_tx_mq_poll) from [<c0803018>] (net_rx_action+0x108/0x4e4)
> [<c0803018>] (net_rx_action) from [<c010230c>] (__do_softirq+0xec/0x598)
> [<c010230c>] (__do_softirq) from [<c0143914>] (do_softirq.part.4+0x68/0x74)
> [<c0143914>] (do_softirq.part.4) from [<c0143a44>] (__local_bh_enable_ip+0x124/0x17c)
> [<c0143a44>] (__local_bh_enable_ip) from [<c0871590>] (ip_finish_output2+0x294/0xb7c)
> [<c0871590>] (ip_finish_output2) from [<c0875440>] (ip_output+0x210/0x364)
> [<c0875440>] (ip_output) from [<c0875e2c>] (ip_send_skb+0x1c/0xf8)
> [<c0875e2c>] (ip_send_skb) from [<c08a7fd4>] (raw_sendmsg+0x9a8/0xc74)
> [<c08a7fd4>] (raw_sendmsg) from [<c07d6b90>] (sock_sendmsg+0x14/0x24)
> [<c07d6b90>] (sock_sendmsg) from [<c07d8260>] (__sys_sendto+0xbc/0x100)
> [<c07d8260>] (__sys_sendto) from [<c01011ac>] (__sys_trace_return+0x0/0x14)
> Exception stack(0xea9a7fa8 to 0xea9a7ff0)
> ...
>
> The reason is that cpdma_chan_submit_si() now stores original buffer length
> (sw_len) in CPDMA descriptor instead of adjusted buffer length (hw_len)
> used to map the buffer.

It's kind not complete fix.
Seems like debug just shows it only fo dma_map, but for sync it doesn't.
The issue is not in swlen, it contains smilar size and inited to len equally as
hw_len. Problem that hw_len is inited later then swlen after below:

	if (len < ctlr->params.min_packet_size) {
		len = ctlr->params.min_packet_size;
		chan->stats.runt_transmit_buff++;
	}

Thus it potentially can have different size for lens less min_pakcet_size, that
could never happen but tracked with dma debug.

	mode = CPDMA_DESC_OWNER | CPDMA_DESC_SOP | CPDMA_DESC_EOP;
	cpdma_desc_to_port(chan, mode, si->directed);

	if (si->data_dma) {
		buffer = si->data_dma;
		dma_sync_single_for_device(ctlr->dev, buffer, len, chan->dir);
		swlen |= CPDMA_DMA_EXT_MAP;
	} else {
		buffer = dma_map_single(ctlr->dev, si->data_virt, len, chan->dir);
		ret = dma_mapping_error(ctlr->dev, buffer);
		if (ret) {
			cpdma_desc_free(ctlr->pool, desc, 1);
			return -EINVAL;
		}
	}

So, the fix you propose do it only for dma_map_single. But, even it should never
happen, it should be fixed for dma_sync() also for consistency.
Proposition is to move the following initialization:

	int				swlen = len;

after

	if (len < ctlr->params.min_packet_size) {
		len = ctlr->params.min_packet_size;
		chan->stats.runt_transmit_buff++;
	}

->     swlen = len;

And that should work for both cases, even if dma_sync uses len:

dma_sync_single_for_device(ctlr->dev, buffer, len, chan->dir);

....


> Hence, fix an issue by using adjusted buffer length (hw_len) to unmap
> packet buffer.
>
> Cc: Ivan Khoronzhuk <ivan.khoronzhuk@xxxxxxxxxx>
> Fixes: 6670acacd59e ("net: ethernet: ti: davinci_cpdma: add dma mapped submit")
> Signed-off-by: Grygorii Strashko <grygorii.strashko@xxxxxx>
> ---
> drivers/net/ethernet/ti/davinci_cpdma.c | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/net/ethernet/ti/davinci_cpdma.c b/drivers/net/ethernet/ti/davinci_cpdma.c
> index 37ba708ac781..6be5a514d5b1 100644
> --- a/drivers/net/ethernet/ti/davinci_cpdma.c
> +++ b/drivers/net/ethernet/ti/davinci_cpdma.c
> @@ -1197,12 +1197,13 @@ static void __cpdma_chan_free(struct cpdma_chan *chan,
> {
> 	struct cpdma_ctlr		*ctlr = chan->ctlr;
> 	struct cpdma_desc_pool		*pool = ctlr->pool;
> +	int				origlen, hw_len;
> 	dma_addr_t			buff_dma;
> -	int				origlen;
> 	uintptr_t			token;
>
> 	token      = desc_read(desc, sw_token);
> 	origlen    = desc_read(desc, sw_len);
> +	hw_len = desc_read(desc, hw_len);
>
> 	buff_dma   = desc_read(desc, sw_buffer);
> 	if (origlen & CPDMA_DMA_EXT_MAP) {
> @@ -1210,7 +1211,7 @@ static void __cpdma_chan_free(struct cpdma_chan *chan,
> 		dma_sync_single_for_cpu(ctlr->dev, buff_dma, origlen,
> 					chan->dir);
> 	} else {
> -		dma_unmap_single(ctlr->dev, buff_dma, origlen, chan->dir);
> +		dma_unmap_single(ctlr->dev, buff_dma, hw_len, chan->dir);
> 	}

I've tried to avod two register reads on data path. So read only one sw_len
that actually supposed to contain same size. So proposition is to fix it like
in above comment would be more correct, and leave this func unchanged I mean in:

static int cpdma_chan_submit_si(struct submit_info *si) {}

> 	cpdma_desc_free(pool, desc, 1);
> --
> 2.17.1

--
Regards,
Ivan Khoronzhuk