Re: [PATCH v5 2/2] vfs: add FILE_DEDUPE_RANGE_REPORT_PROGRESS flag to FIDEDUPERANGE

[Darrick J. Wong]

On Sat, Sep 26, 2026 at 12:38:36PM +0800, Matthias Goergens wrote:
> Deduplication tools such as duperemove, bees and rmlint advance their
> file offsets by the bytes_deduped the kernel returns for each
> FIDEDUPERANGE call.
> 
> vfs_dedupe_file_range() passes REMAP_FILE_CAN_SHORTEN, so
> generic_remap_checks() may round the length down to a block multiple,
> but the ioctl still reports the requested length in bytes_deduped. The
> caller cannot tell that the tail of its request was left alone: rmlint
> 2.10.3 on btrfs (4 KiB blocks) deduping a 100000-byte file against a
> 250000-byte file with the same prefix is told bytes_deduped=100000
> with status SAME while only 98304 bytes were actually shared; its loop
> ends and it reports the pair fully deduplicated. duperemove and bees
> advance the same way, and jdupes advances by its own requested length
> without reading the field; all of them skip such a tail.
> 
> Add FILE_DEDUPE_RANGE_REPORT_PROGRESS for file_dedupe_range.flags:
> with it, bytes_deduped is the length the filesystem actually
> deduplicated when status is FILE_DEDUPE_RANGE_SAME, and 0 on
> FILE_DEDUPE_RANGE_DIFFERS or error. Callers advance by it as today,
> but must treat 0 as "stop or subdivide", not retry unchanged. One
> cause of SAME with 0 is a sub-block request that does not end at both
> files' EOF, which the generic range preparation shortens to nothing.
> On DIFFERS there is no sound progress or mismatch offset to report, so
> 0 leaves subdividing to the caller, as rmlint already does.
> 
> The default cannot change: commit 4a57a8400075 ("vf/remap: return the
> amount of bytes actually deduplicated") did exactly that and was
> reverted the next day; among deployed callers, duperemove re-queues a
> request while its status is 0 and would re-issue the same sub-block
> request forever.
> 
> Without the flag nothing changes.
> 
> Suggested-by: Darrick J. Wong <djwong@xxxxxxxxxx>
> Link: https://lore.kernel.org/linux-fsdevel/20260805071414.3414870-1-matthias.goergens@xxxxxxxxx/
> Signed-off-by: Matthias Goergens <matthias.goergens@xxxxxxxxx>
> ---
>  fs/remap_range.c        |  4 +++-
>  include/uapi/linux/fs.h | 11 ++++++++++-
>  2 files changed, 13 insertions(+), 2 deletions(-)
> 
> diff --git a/fs/remap_range.c b/fs/remap_range.c
> index 26afbbbfb10c..63f1b6f90c16 100644
> --- a/fs/remap_range.c
> +++ b/fs/remap_range.c
> @@ -503,7 +503,7 @@ int vfs_dedupe_file_range(struct file *file, struct file_dedupe_range *same)
>  	if (!(file->f_mode & FMODE_READ))
>  		return -EINVAL;
>  
> -	if (same->reserved1 || same->reserved2)
> +	if (same->reserved1 || (same->flags & ~FILE_DEDUPE_RANGE_REPORT_PROGRESS))
>  		return -EINVAL;
>  
>  	off = same->src_offset;
> @@ -555,6 +555,8 @@ int vfs_dedupe_file_range(struct file *file, struct file_dedupe_range *same)
>  			info->status = FILE_DEDUPE_RANGE_DIFFERS;
>  		else if (deduped < 0)
>  			info->status = deduped;
> +		else if (same->flags & FILE_DEDUPE_RANGE_REPORT_PROGRESS)
> +			info->bytes_deduped = deduped;

Very good!  I'm glad this is resolved now.

Reviewed-by: "Darrick J. Wong" <djwong@xxxxxxxxxx>

--D

>  		else
>  			info->bytes_deduped = len;
>  
> diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h
> index 34c6f219462a..c61bc98909cc 100644
> --- a/include/uapi/linux/fs.h
> +++ b/include/uapi/linux/fs.h
> @@ -178,13 +178,22 @@ struct file_dedupe_range_info {
>  	__u32 reserved;		/* must be zero */
>  };
>  
> +/* flags for struct file_dedupe_range */
> +/*
> + * Without this flag, bytes_deduped is the requested length on success,
> + * even if the filesystem deduplicated fewer bytes (e.g. after shortening
> + * the request to a block boundary).  With this flag, bytes_deduped is
> + * the number of bytes actually deduplicated.
> + */
> +#define FILE_DEDUPE_RANGE_REPORT_PROGRESS (1U << 0)
> +
>  /* from struct btrfs_ioctl_file_extent_same_args */
>  struct file_dedupe_range {
>  	__u64 src_offset;	/* in - start of extent in source */
>  	__u64 src_length;	/* in - length of extent */
>  	__u16 dest_count;	/* in - total elements in info array */
>  	__u16 reserved1;	/* must be zero */
> -	__u32 reserved2;	/* must be zero */
> +	__u32 flags;		/* in - FILE_DEDUPE_RANGE_* flags */
>  	struct file_dedupe_range_info info[];
>  };
>  
> -- 
> 2.55.0
> 
>