Re: [RFC PATCH] Add proc interface to set PF_MEMALLOC flags

[Damien Le Moal]

Mike,

On 2019/09/09 19:26, Mike Christie wrote:
> Forgot to cc linux-mm.
> 
> On 09/09/2019 11:28 AM, Mike Christie wrote:
>> There are several storage drivers like dm-multipath, iscsi, and nbd that
>> have userspace components that can run in the IO path. For example,
>> iscsi and nbd's userspace deamons may need to recreate a socket and/or
>> send IO on it, and dm-multipath's daemon multipathd may need to send IO
>> to figure out the state of paths and re-set them up.
>>
>> In the kernel these drivers have access to GFP_NOIO/GFP_NOFS and the
>> memalloc_*_save/restore functions to control the allocation behavior,
>> but for userspace we would end up hitting a allocation that ended up
>> writing data back to the same device we are trying to allocate for.
>>
>> This patch allows the userspace deamon to set the PF_MEMALLOC* flags
>> through procfs. It currently only supports PF_MEMALLOC_NOIO, but
>> depending on what other drivers and userspace file systems need, for
>> the final version I can add the other flags for that file or do a file
>> per flag or just do a memalloc_noio file.

Awesome. That probably will be the perfect solution for the problem we hit with
tcmu-runner a while back (please see this thread:
https://www.spinics.net/lists/linux-fsdevel/msg148912.html).

I think we definitely need nofs as well for dealing with cases where the backend
storage for the user daemon is a file.

I will give this patch a try as soon as possible (I am traveling currently).

Best regards.

>>
>> Signed-off-by: Mike Christie <mchristi@xxxxxxxxxx>
>> ---
>>  Documentation/filesystems/proc.txt |  6 ++++
>>  fs/proc/base.c                     | 53 ++++++++++++++++++++++++++++++
>>  2 files changed, 59 insertions(+)
>>
>> diff --git a/Documentation/filesystems/proc.txt b/Documentation/filesystems/proc.txt
>> index 99ca040e3f90..b5456a61a013 100644
>> --- a/Documentation/filesystems/proc.txt
>> +++ b/Documentation/filesystems/proc.txt
>> @@ -46,6 +46,7 @@ Table of Contents
>>    3.10  /proc/<pid>/timerslack_ns - Task timerslack value
>>    3.11	/proc/<pid>/patch_state - Livepatch patch operation state
>>    3.12	/proc/<pid>/arch_status - Task architecture specific information
>> +  3.13  /proc/<pid>/memalloc - Control task's memory reclaim behavior
>>  
>>    4	Configuring procfs
>>    4.1	Mount options
>> @@ -1980,6 +1981,11 @@ Example
>>   $ cat /proc/6753/arch_status
>>   AVX512_elapsed_ms:      8
>>  
>> +3.13 /proc/<pid>/memalloc - Control task's memory reclaim behavior
>> +-----------------------------------------------------------------------
>> +A value of "noio" indicates that when a task allocates memory it will not
>> +reclaim memory that requires starting phisical IO.
>> +
>>  Description
>>  -----------
>>  
>> diff --git a/fs/proc/base.c b/fs/proc/base.c
>> index ebea9501afb8..c4faa3464602 100644
>> --- a/fs/proc/base.c
>> +++ b/fs/proc/base.c
>> @@ -1223,6 +1223,57 @@ static const struct file_operations proc_oom_score_adj_operations = {
>>  	.llseek		= default_llseek,
>>  };
>>  
>> +static ssize_t memalloc_read(struct file *file, char __user *buf, size_t count,
>> +			     loff_t *ppos)
>> +{
>> +	struct task_struct *task;
>> +	ssize_t rc = 0;
>> +
>> +	task = get_proc_task(file_inode(file));
>> +	if (!task)
>> +		return -ESRCH;
>> +
>> +	if (task->flags & PF_MEMALLOC_NOIO)
>> +		rc = simple_read_from_buffer(buf, count, ppos, "noio", 4);
>> +	put_task_struct(task);
>> +	return rc;
>> +}
>> +
>> +static ssize_t memalloc_write(struct file *file, const char __user *buf,
>> +			      size_t count, loff_t *ppos)
>> +{
>> +	struct task_struct *task;
>> +	char buffer[5];
>> +	int rc = count;
>> +
>> +	memset(buffer, 0, sizeof(buffer));
>> +	if (count != sizeof(buffer) - 1)
>> +		return -EINVAL;
>> +
>> +	if (copy_from_user(buffer, buf, count))
>> +		return -EFAULT;
>> +	buffer[count] = '\0';
>> +
>> +	task = get_proc_task(file_inode(file));
>> +	if (!task)
>> +		return -ESRCH;
>> +
>> +	if (!strcmp(buffer, "noio")) {
>> +		task->flags |= PF_MEMALLOC_NOIO;
>> +	} else {
>> +		rc = -EINVAL;
>> +	}
>> +
>> +	put_task_struct(task);
>> +	return rc;
>> +}
>> +
>> +static const struct file_operations proc_memalloc_operations = {
>> +	.read		= memalloc_read,
>> +	.write		= memalloc_write,
>> +	.llseek		= default_llseek,
>> +};
>> +
>>  #ifdef CONFIG_AUDIT
>>  #define TMPBUFLEN 11
>>  static ssize_t proc_loginuid_read(struct file * file, char __user * buf,
>> @@ -3097,6 +3148,7 @@ static const struct pid_entry tgid_base_stuff[] = {
>>  #ifdef CONFIG_PROC_PID_ARCH_STATUS
>>  	ONE("arch_status", S_IRUGO, proc_pid_arch_status),
>>  #endif
>> +	REG("memalloc", S_IRUGO|S_IWUSR, proc_memalloc_operations),
>>  };
>>  
>>  static int proc_tgid_base_readdir(struct file *file, struct dir_context *ctx)
>> @@ -3487,6 +3539,7 @@ static const struct pid_entry tid_base_stuff[] = {
>>  #ifdef CONFIG_PROC_PID_ARCH_STATUS
>>  	ONE("arch_status", S_IRUGO, proc_pid_arch_status),
>>  #endif
>> +	REG("memalloc", S_IRUGO|S_IWUSR, proc_memalloc_operations),
>>  };
>>  
>>  static int proc_tid_base_readdir(struct file *file, struct dir_context *ctx)
>>
> 
> 


-- 
Damien Le Moal
Western Digital Research