Re: [PATCH v3 4/9] ext4: fix slab-out-of-bounds in ext4_mb_find_good_group_avg_frag_lists()

[Jan Kara]

On Mon 18-03-24 18:09:18, Ojaswin Mujoo wrote:
> On Thu, Mar 14, 2024 at 10:09:01PM +0800, Baokun Li wrote:
> > We can trigger a slab-out-of-bounds with the following commands:
> > 
> >     mkfs.ext4 -F /dev/$disk 10G
> >     mount /dev/$disk /tmp/test
> >     echo 2147483647 > /sys/fs/ext4/$disk/mb_group_prealloc
> >     echo test > /tmp/test/file && sync
> > 
> > ==================================================================
> > BUG: KASAN: slab-out-of-bounds in ext4_mb_find_good_group_avg_frag_lists+0x8a/0x200 [ext4]
> > Read of size 8 at addr ffff888121b9d0f0 by task kworker/u2:0/11
> > CPU: 0 PID: 11 Comm: kworker/u2:0 Tainted: GL 6.7.0-next-20240118 #521
> > Call Trace:
> >  dump_stack_lvl+0x2c/0x50
> >  kasan_report+0xb6/0xf0
> >  ext4_mb_find_good_group_avg_frag_lists+0x8a/0x200 [ext4]
> >  ext4_mb_regular_allocator+0x19e9/0x2370 [ext4]
> >  ext4_mb_new_blocks+0x88a/0x1370 [ext4]
> >  ext4_ext_map_blocks+0x14f7/0x2390 [ext4]
> >  ext4_map_blocks+0x569/0xea0 [ext4]
> >  ext4_do_writepages+0x10f6/0x1bc0 [ext4]
> > [...]
> > ==================================================================
> > 
> > The flow of issue triggering is as follows:
> > 
> > // Set s_mb_group_prealloc to 2147483647 via sysfs
> > ext4_mb_new_blocks
> >   ext4_mb_normalize_request
> >     ext4_mb_normalize_group_request
> >       ac->ac_g_ex.fe_len = EXT4_SB(sb)->s_mb_group_prealloc
> >   ext4_mb_regular_allocator
> >     ext4_mb_choose_next_group
> >       ext4_mb_choose_next_group_best_avail
> >         mb_avg_fragment_size_order
> >           order = fls(len) - 2 = 29
> >         ext4_mb_find_good_group_avg_frag_lists
> >           frag_list = &sbi->s_mb_avg_fragment_size[order]
> >           if (list_empty(frag_list)) // Trigger SOOB!
> > 
> > At 4k block size, the length of the s_mb_avg_fragment_size list is 14,
> > but an oversized s_mb_group_prealloc is set, causing slab-out-of-bounds
> > to be triggered by an attempt to access an element at index 29.
> > 
> > Add a new attr_id attr_clusters_in_group with values in the range
> > [0, sbi->s_clusters_per_group] and declare mb_group_prealloc as
> > that type to fix the issue. In addition avoid returning an order
> > from mb_avg_fragment_size_order() greater than MB_NUM_ORDERS(sb)
> > and reduce some useless loops.
> > 
> > Fixes: 7e170922f06b ("ext4: Add allocation criteria 1.5 (CR1_5)")
> > CC: stable@xxxxxxxxxxxxxxx
> > Signed-off-by: Baokun Li <libaokun1@xxxxxxxxxx>
> > Reviewed-by: Jan Kara <jack@xxxxxxx>
> > ---
> >  fs/ext4/mballoc.c |  4 ++++
> >  fs/ext4/sysfs.c   | 13 ++++++++++++-
> >  2 files changed, 16 insertions(+), 1 deletion(-)
> > 
> > diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c
> > index 12b3f196010b..48afe5aa228c 100644
> > --- a/fs/ext4/mballoc.c
> > +++ b/fs/ext4/mballoc.c
> > @@ -831,6 +831,8 @@ static int mb_avg_fragment_size_order(struct super_block *sb, ext4_grpblk_t len)
> >     return 0;
> >   if (order == MB_NUM_ORDERS(sb))
> >     order--;
> > + if (WARN_ON_ONCE(order > MB_NUM_ORDERS(sb)))
> > +   order = MB_NUM_ORDERS(sb) - 1;
> 
> Hey Baokun,
> 
> Thanks for fixing this. This patch looks good to me, feel free to add:
> 
> Reviewed-by: Ojaswin Mujoo <ojaswin@xxxxxxxxxxxxx>
> 
> my comments after this are less about the patch and more about some
> thoughts on the working of average fragment lists.
> 
> So going through the v2 and this patch got me thinking about what really
> is going to happen when a user tries to allocate 32768 blocks which is also 
> the maximum value we could have in say ac->ac_g_ex.fe_len.
> 
> When this happens, ext4_mb_regular_allocator() will directly set the
> criteria as CR_GOAL_LEN_FAST. Now, we'll follow:
> 
> ext4_mb_choose_next_group_goal_fast()
>   for (i=mb_avg_fragment_size_order(); i < MB_NUM_ORDERS; i++) { .. }
> 
> Here, mb_avg_fragment_siz_order() will do something like:
> 
>   order = fls(32768) - 2 = 14
>   ...
>   if (order == MB_NUM_ORDERS(sb))
>     order--;
> 
>   return order;
> 
> And we'll look in the fragment list[13] and since none of the groups
> there would have 32768 blocks free (since we dont track it here) we'll
> unnecessarily traverse the full list before falling to CR_BEST_AVAIL_LEN
> (this will become a noop due to the way order and min_order
> are calculated) and eventually to CR_GOAL_LEN_SLOW where we might get
> something or end up splitting.

Yeah, agreed this looks a bit suboptimal. I'm just not 100% sure whether
we'll ever generate a request to allocate 32768 blocks - that would need
verification with tracing - because I have some vague recollection I once
arrived at conclusion this is not possible.

> I think something more optimal would be to:
> 
> 1. Add another entry to average fragment lists for completely empty
> groups. (As a sidenote i think we should use something like MB_NUM_FRAG_ORDER
> instead of MB_NUM_ORDERS in calculating limits related to average
> fragment lists since the NUM_ORDERS seems to be the buddy max order ie
> 8192 blocks only valid for CR_POWER2 and shouldn't really limit the
> fragment size lists)

I guess the thinking was that you can never get larger than
1<<(MB_NUM_ORDERS-1) chunk from mballoc so there's no point to keep
fragment lists of such chunks?

								Honza
-- 
Jan Kara <jack@xxxxxxxx>
SUSE Labs, CR