Re: [PATCH] ring-buffer: Fix race while reader and writer are on the same page

[Steven Rostedt]

On Fri, 24 Mar 2023 20:50:37 +0800
Zheng Yejian <zhengyejian1@xxxxxxxxxx> wrote:

> Fixes: 77ae365eca89 ("ring-buffer: make lockless")
> Signed-off-by: Zheng Yejian <zhengyejian1@xxxxxxxxxx>
> ---
>  kernel/trace/ring_buffer.c | 14 +++++++++++++-
>  1 file changed, 13 insertions(+), 1 deletion(-)
> 
> diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
> index c6f47b6cfd5f..79fd5e10ee05 100644
> --- a/kernel/trace/ring_buffer.c
> +++ b/kernel/trace/ring_buffer.c
> @@ -2942,6 +2942,13 @@ rb_update_event(struct ring_buffer_per_cpu *cpu_buffer,
>  		event->array[0] = length;
>  	} else
>  		event->type_len = DIV_ROUND_UP(length, RB_ALIGNMENT);
> +
> +	/*
> +	 * The 'event' may be reserved from the page which is reading
> +	 * by reader, make sure 'event' is completely updated before
> +	 * reader_page->page->commit being set.
> +	 */
> +	smp_wmb();

This isn't the place to put this. We only care before the commit is
updated, not at *ever* update to the event (this can be called several
times before a commit).

If we need to add a smp_wmb() it's best to be in rb_set_commit_to_write()

>  }
>  
>  static unsigned rb_calculate_event_length(unsigned length)
> @@ -4684,7 +4691,12 @@ rb_get_reader_page(struct ring_buffer_per_cpu *cpu_buffer)
>  
>  	/*
>  	 * Make sure we see any padding after the write update
> -	 * (see rb_reset_tail())
> +	 * (see rb_reset_tail()).
> +	 *
> +	 * In addition, writer may be writing on the reader page
> +	 * if the page has not been fully filled, so the read barrier
> +	 * is also needed to make sure we see the completely updated
> +	 * event that reserved by writer (see rb_update_event()).
>  	 */
>  	smp_rmb();
>  

I think we want this instead:

diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index 2d5c3caff32d..22d05cd04a3a 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -3092,6 +3092,10 @@ rb_set_commit_to_write(struct ring_buffer_per_cpu *cpu_buffer)
 		if (RB_WARN_ON(cpu_buffer,
 			       rb_is_reader_page(cpu_buffer->tail_page)))
 			return;
+		/*
+		 * No need for a memory barrier here, as the update
+		 * of the tail_page did it for this page.
+		 */
 		local_set(&cpu_buffer->commit_page->page->commit,
 			  rb_page_write(cpu_buffer->commit_page));
 		rb_inc_page(&cpu_buffer->commit_page);
@@ -3101,6 +3105,8 @@ rb_set_commit_to_write(struct ring_buffer_per_cpu *cpu_buffer)
 	while (rb_commit_index(cpu_buffer) !=
 	       rb_page_write(cpu_buffer->commit_page)) {
 
+		/* Make sure the readers see the content of what is committed. */
+		smp_wmb();
 		local_set(&cpu_buffer->commit_page->page->commit,
 			  rb_page_write(cpu_buffer->commit_page));
 		RB_WARN_ON(cpu_buffer,
@@ -4676,7 +4682,12 @@ rb_get_reader_page(struct ring_buffer_per_cpu *cpu_buffer)
 
 	/*
 	 * Make sure we see any padding after the write update
-	 * (see rb_reset_tail())
+	 * (see rb_reset_tail()).
+	 *
+	 * In addition, a writer may be writing on the reader page
+	 * if the page has not been fully filled, so the read barrier
+	 * is also needed to make sure we see the completely updated
+	 * event reserved by the writer (see rb_tail_page_update()).
 	 */
 	smp_rmb();