Re: [PATCH 2/2] vfs: grab the lock instead of blocking in __fd_install during resizing

[Eric Dumazet]

On Wed, Oct 4, 2017 at 7:58 AM, Matthew Wilcox <willy@xxxxxxxxxxxxx> wrote:
> On Wed, Oct 04, 2017 at 07:00:40AM -0700, Eric Dumazet wrote:
>> On Wed, Oct 4, 2017 at 6:55 AM, Matthew Wilcox <willy@xxxxxxxxxxxxx> wrote:
>> > Any chance you could review the patches from Sandhya that make this entire
>> > codepath obsolete?
>> >
>> > https://lkml.org/lkml/2017/4/29/20
>> >
>>
>> Hmm...
>>
>> 18 files changed, 578 insertions(+), 585 deletions(-)
>>
>> Frankly I need to be convinced with solid performance numbers before I
>> am taking a look at this series.
>
> I was hoping you'd help us get some solid performance numbers ... you
> seem to have workloads available to you that help find weaknesses in
> implementations.
>
> The number of lines inserted is a bit of a red herring.  Over 100 are in
> the test suite (you surely aren't going to review those) and another ~300
> are adding enhancements to the IDR & radix tree that should be useful
> for other users (eg I think I have a way to speed up writing out dirty
> pages by using get_tag_batch()).
>
>> I do not believe an IDR will be faster than current implementation, so
>> I am not quite convinced at this moment.
>
> I don't think it should be significantly different in performance.  Let's
> look at the layout of data for a typical bash shell (fds 0-2 and 255 open).
>
> Current implementation:
>
> files_struct -> fdt -> fd -> struct file
>
> IDR:
>
> files_struct -> radix node (shift 6) -> radix node (shift 0) -> struct file
>
> In either case, it's the same number of dependent loads.  It'll start
> to look worse for the radix tree above 4096 open fds in a given process.

I am interested in performance for process with 10,000,000 fds, and
~100 threads constantly adding/deleting/using fds.

Thanks