Re: [PATCH net 4/4] rxrpc: Fix CHALLENGE packet overqueuing and simplify RESPONSE generation

[Paolo Abeni]

On 7/2/26 4:49 PM, David Howells wrote:
> Currently, when a CHALLENGE packet comes in, it's queued in an OOB queue on
> the AF_RXRPC socket that generated one of the calls on that connection for
> the application (which might be in userspace) to service.  The application
> then picks up the CHALLENGE and requests a RESPONSE packet be generated,
> allowing the app to include app-specific data in it if appropriate.  There
> is, however, no actual limit on the capacity of the CHALLENGE queue, and
> this could be abused remotely - and also getting the OOB mechanism right
> has proven tricky.
> 
> Further, by analogy with other AFS codebases, it's not actually necessary
> to generate the application data in response to the CHALLENGE.  The reason
> I did this was to set the encryption on the app-data to be the same as that
> specified in the CHALLENGE as the server must be able to handle that.
> However, it's sufficient to use the encoding type set in the token that is
> going to be sent to the server; presumably the kerberos server knows that
> the fileserver can handle that type - otherwise why tell the client to use
> it?
> 
> With this in mind, make the following changes to the rxrpc server:
> 
>  (1) Remove the OOB queuing stuff and retire the related CMSG values and
>      sockopt.
> 
>  (2) Revert to making the connection event processor work item parse the
>      CHALLENGE and generate the RESPONSE directly.
> 
>  (3) Add another (optional) parameter that is passed in when an rxrpc
>      client call is created and ends up attached to the connection bundle
>      and is a user-type key containing the application data.
> 
>  (4) RESPONSE generation looks at the bundle and if the app-data is there,
>      it will include it (if the security class is YFS-RxGK; RxKAD ignores
>      it).
> 
>  (5) The AFS filesystem driver will create an app-data key when it probes a
>      fileserver and will attach it to the afs_server struct.  This is then
>      picked up when a call is made to that server and passed to rxrpc.
> 
>  (6) Direct userspace users of AF_RXRPC can partake by creating a user-type
>      key containing the app-data they want to use and passing its serial ID
>      in a CMSG of type RXRPC_RESPONSE_APPDATA in the initial sendmsg() of a
>      call.
> 
> To support this, I've made a change outside of afs and rxrc:
> 
>  (7) Add a refcount to the user-type key payload.  The problem is that the
>      RESPONSE packet generator needs to look at the length of the app-data,
>      allocate a buffer for the packet and then copy the app-data in - but
>      the RCU read lock cannot be held across the allocation and the key
>      might get updated.  To get around this, a ref is taken on the payload
>      before the allocation and then put afterwards.
> 
>  (8) Since the DNS resolver makes use of the user-defined type's code, but
>      allocates the payload itself, make it initialise the refcount.
> 
> Fixes: 5800b1cf3fd8 ("rxrpc: Allow CHALLENGEs to the passed to the app for a RESPONSE")
> Link: https://sashiko.dev/#/patchset/20260624163819.3017002-1-dhowells%40redhat.com
> Signed-off-by: David Howells <dhowells@xxxxxxxxxx>
> cc: Marc Dionne <marc.dionne@xxxxxxxxxxxx>
> cc: Jeffrey Altman <jaltman@xxxxxxxxxxxx>
> cc: Eric Dumazet <edumazet@xxxxxxxxxx>
> cc: "David S. Miller" <davem@xxxxxxxxxxxxx>
> cc: Jakub Kicinski <kuba@xxxxxxxxxx>
> cc: Paolo Abeni <pabeni@xxxxxxxxxx>
> cc: Simon Horman <horms@xxxxxxxxxx>
> cc: Jarkko Sakkinen <jarkko@xxxxxxxxxx>
> cc: linux-afs@xxxxxxxxxxxxxxxxxxx
> cc: keyrings@xxxxxxxxxxxxxxx
> cc: stable@xxxxxxxxxx
> ---
>  fs/afs/cm_security.c         | 151 ++++++--------
>  fs/afs/fs_probe.c            |   5 +
>  fs/afs/internal.h            |  37 ++--
>  fs/afs/main.c                |   1 -
>  fs/afs/rxrpc.c               |  39 ++--
>  fs/afs/server.c              |   2 +-
>  include/keys/user-type.h     |   2 +
>  include/net/af_rxrpc.h       |  20 +-
>  include/trace/events/afs.h   |   1 +
>  include/trace/events/rxrpc.h |   2 -
>  include/uapi/linux/rxrpc.h   |   6 +-
>  net/dns_resolver/dns_key.c   |   1 +
>  net/rxrpc/Makefile           |   1 -
>  net/rxrpc/af_rxrpc.c         |  49 +----
>  net/rxrpc/ar-internal.h      |  22 +-
>  net/rxrpc/call_object.c      |   4 +-
>  net/rxrpc/conn_client.c      |   2 +
>  net/rxrpc/conn_event.c       |  68 +-----
>  net/rxrpc/key.c              |  36 ++++
>  net/rxrpc/oob.c              | 387 -----------------------------------
>  net/rxrpc/recvmsg.c          |  84 +-------
>  net/rxrpc/rxgk.c             | 128 +++---------
>  net/rxrpc/rxkad.c            |  27 ---
>  net/rxrpc/sendmsg.c          |  15 ++
>  net/rxrpc/server_key.c       |  40 ----
>  security/keys/user_defined.c |  23 ++-
>  26 files changed, 238 insertions(+), 915 deletions(-)
>  delete mode 100644 net/rxrpc/oob.c

This is quite big and difficult to review, could you please someout
break it in smaller chunk?

/P