Re: [PATCH v2 4/6] locking/lockdep: Reuse freed chain_hlocks entries

[Waiman Long]

On 1/13/20 10:58 AM, Peter Zijlstra wrote:
> On Mon, Dec 16, 2019 at 10:15:15AM -0500, Waiman Long wrote:
>> +/*
>> + * Return offset of a chain block of the right size or -1 if not found.
>> + */
>> +static inline int alloc_chain_hlocks_from_buckets(int size)
>> +{
>> +	int prev, curr, next;
>> +
>> +	if (!nr_free_chain_hlocks)
>> +		return -1;
>> +
>> +	if (size <= MAX_CHAIN_BUCKETS) {
>> +		curr = chain_block_buckets[size - 1];
>> +		if (curr < 0)
>> +			return -1;
>> +
>> +		chain_block_buckets[size - 1] = next_chain_block(curr);
>> +		nr_free_chain_hlocks -= size;
>> +		return curr;
>> +	}
>> +
>> +	/*
>> +	 * Look for a free chain block of the given size
>> +	 *
>> +	 * It is rare to have a lock chain with depth > MAX_CHAIN_BUCKETS.
>> +	 * It is also more expensive as we may iterate the whole list
>> +	 * without finding one.
>> +	 */
>> +	prev = -1;
>> +	curr = chain_block_buckets[0];
>> +	while (curr >= 0) {
>> +		next = next_chain_block(curr);
>> +		if (chain_block_size(curr) == size) {
>> +			set_chain_block(prev, 0, next);
>> +			nr_free_chain_hlocks -= size;
>> +			nr_large_chain_blocks--;
>> +			return curr;
>> +		}
>> +		prev = curr;
>> +		curr = next;
>> +	}
>> +	return -1;
>> +}
>> +/*
>> + * The graph lock must be held before calling this function.
>> + *
>> + * Return: an offset to chain_hlocks if successful, or
>> + *	   -1 with graph lock released
>> + */
>> +static int alloc_chain_hlocks(int size)
>> +{
>> +	int curr;
>> +
>> +	if (size < 2)
>> +		size = 2;
>> +
>> +	curr = alloc_chain_hlocks_from_buckets(size);
>> +	if (curr >= 0)
>> +		return curr;
>> +
>> +	BUILD_BUG_ON((1UL << 24) <= ARRAY_SIZE(chain_hlocks));
>> +	BUILD_BUG_ON((1UL << 6)  <= ARRAY_SIZE(current->held_locks));
>> +	BUILD_BUG_ON((1UL << 8*sizeof(chain_hlocks[0])) <=
>> +		     ARRAY_SIZE(lock_classes));
>> +
>> +	/*
>> +	 * Allocate directly from chain_hlocks.
>> +	 */
>> +	if (likely(nr_chain_hlocks + size <= MAX_LOCKDEP_CHAIN_HLOCKS)) {
>> +		curr = nr_chain_hlocks;
>> +		nr_chain_hlocks += size;
>> +		return curr;
>> +	}
>> +	if (!debug_locks_off_graph_unlock())
>> +		return -1;
>> +
>> +	print_lockdep_off("BUG: MAX_LOCKDEP_CHAIN_HLOCKS too low!");
>> +	dump_stack();
>> +	return -1;
>> +}
> *groan*....
>
> That's _two_ allocators :/ And it can trivially fail, even if there's
> plenty space available.
>
> Consider nr_chain_hlocks is exhaused, and @size is empty, but size+1
> still has blocks.
>
> I'm guessing you didn't make it a single allocator because you didn't
> want to implement block splitting? why?
>
In my testing, most of the lock chains tend to be rather short (within
the 2-8 range). I don't see a lot of free blocks left in the system
after the test. So I don't see a need to implement block splitting for now.

If you think this is a feature that needs to be implemented for the
patch to be complete, I can certainly add patch to do that. My initial
thought is just to split long blocks in the unsized list for allocation
request that is no longer than 8 to make thing easier.

Cheers,
Longman