Re: [PATCH v36 28/31] ALSA: usb-audio: qcom: Introduce QC USB SND offloading support

[Wesley Cheng]

Hi Stephan,

On 3/26/2025 3:09 AM, Stephan Gerhold wrote:
> On Tue, Mar 25, 2025 at 06:32:14PM -0700, Wesley Cheng wrote:
>> On 3/25/2025 2:47 AM, Stephan Gerhold wrote:
>>> On Tue, Mar 18, 2025 at 05:51:38PM -0700, Wesley Cheng wrote:
>>>> Several Qualcomm SoCs have a dedicated audio DSP, which has the ability to
>>>> support USB sound devices.  This vendor driver will implement the required
>>>> handshaking with the DSP, in order to pass along required resources that
>>>> will be utilized by the DSP's USB SW.  The communication channel used for
>>>> this handshaking will be using the QMI protocol.  Required resources
>>>> include:
>>>> - Allocated secondary event ring address
>>>> - EP transfer ring address
>>>> - Interrupter number
>>>>
>>>> The above information will allow for the audio DSP to execute USB transfers
>>>> over the USB bus.  It will also be able to support devices that have an
>>>> implicit feedback and sync endpoint as well.  Offloading these data
>>>> transfers will allow the main/applications processor to enter lower CPU
>>>> power modes, and sustain a longer duration in those modes.
>>>>
>>>> Audio offloading is initiated with the following sequence:
>>>> 1. Userspace configures to route audio playback to USB backend and starts
>>>> playback on the platform soundcard.
>>>> 2. The Q6DSP AFE will communicate to the audio DSP to start the USB AFE
>>>> port.
>>>> 3. This results in a QMI packet with a STREAM enable command.
>>>> 4. The QC audio offload driver will fetch the required resources, and pass
>>>> this information as part of the QMI response to the STREAM enable command.
>>>> 5. Once the QMI response is received the audio DSP will start queuing data
>>>> on the USB bus.
>>>>
>>>> As part of step#2, the audio DSP is aware of the USB SND card and pcm
>>>> device index that is being selected, and is communicated as part of the QMI
>>>> request received by QC audio offload.  These indices will be used to handle
>>>> the stream enable QMI request.
>>>>
>>>> Signed-off-by: Wesley Cheng <quic_wcheng@xxxxxxxxxxx>
>>>> ---
>>>>  sound/usb/Kconfig                 |   14 +
>>>>  sound/usb/Makefile                |    2 +-
>>>>  sound/usb/qcom/Makefile           |    2 +
>>>>  sound/usb/qcom/qc_audio_offload.c | 1988 +++++++++++++++++++++++++++++
>>>>  4 files changed, 2005 insertions(+), 1 deletion(-)
>>>>  create mode 100644 sound/usb/qcom/Makefile
>>>>  create mode 100644 sound/usb/qcom/qc_audio_offload.c
>>>>
>>>> diff --git a/sound/usb/Kconfig b/sound/usb/Kconfig
>>>> index 4a9569a3a39a..6daa551738da 100644
>>>> --- a/sound/usb/Kconfig
>>>> +++ b/sound/usb/Kconfig
>>>> @@ -176,6 +176,20 @@ config SND_BCD2000
>>>>  	  To compile this driver as a module, choose M here: the module
>>>>  	  will be called snd-bcd2000.
>>>>  
>>>> +config SND_USB_AUDIO_QMI
>>>> +	tristate "Qualcomm Audio Offload driver"
>>>> +	depends on QCOM_QMI_HELPERS && SND_USB_AUDIO && USB_XHCI_SIDEBAND && SND_SOC_USB
>>>> +	help
>>>> +	  Say Y here to enable the Qualcomm USB audio offloading feature.
>>>> +
>>>> +	  This module sets up the required QMI stream enable/disable
>>>> +	  responses to requests generated by the audio DSP.  It passes the
>>>> +	  USB transfer resource references, so that the audio DSP can issue
>>>> +	  USB transfers to the host controller.
>>>> +
>>>> +	  To compile this driver as a module, choose M here: the module
>>>> +	  will be called snd-usb-audio-qmi.
>>>> [...]
>>>> diff --git a/sound/usb/qcom/qc_audio_offload.c b/sound/usb/qcom/qc_audio_offload.c
>>>> new file mode 100644
>>>> index 000000000000..3319363a0fd0
>>>> --- /dev/null
>>>> +++ b/sound/usb/qcom/qc_audio_offload.c
>>>> @@ -0,0 +1,1988 @@
>>>> [...]
>>>> +static int __init qc_usb_audio_offload_init(void)
>>>> +{
>>>> +	struct uaudio_qmi_svc *svc;
>>>> +	int ret;
>>>> +
>>>> +	svc = kzalloc(sizeof(*svc), GFP_KERNEL);
>>>> +	if (!svc)
>>>> +		return -ENOMEM;
>>>> +
>>>> +	svc->uaudio_svc_hdl = kzalloc(sizeof(*svc->uaudio_svc_hdl), GFP_KERNEL);
>>>> +	if (!svc->uaudio_svc_hdl) {
>>>> +		ret = -ENOMEM;
>>>> +		goto free_svc;
>>>> +	}
>>>> +
>>>> +	ret = qmi_handle_init(svc->uaudio_svc_hdl,
>>>> +			      QMI_UAUDIO_STREAM_REQ_MSG_V01_MAX_MSG_LEN,
>>>> +			      &uaudio_svc_ops_options,
>>>> +			      &uaudio_stream_req_handlers);
>>>> +	ret = qmi_add_server(svc->uaudio_svc_hdl, UAUDIO_STREAM_SERVICE_ID_V01,
>>>> +			     UAUDIO_STREAM_SERVICE_VERS_V01, 0);
>>>> +
>>>> +	uaudio_svc = svc;
>>>> +
>>>> +	ret = snd_usb_register_platform_ops(&offload_ops);
>>>> +	if (ret < 0)
>>>> +		goto release_qmi;
>>>> +
>>>> +	return 0;
>>>> +
>>>> +release_qmi:
>>>> +	qmi_handle_release(svc->uaudio_svc_hdl);
>>>> +free_svc:
>>>> +	kfree(svc);
>>>> +
>>>> +	return ret;
>>>> +}
>>>> +
>>>> +static void __exit qc_usb_audio_offload_exit(void)
>>>> +{
>>>> +	struct uaudio_qmi_svc *svc = uaudio_svc;
>>>> +	int idx;
>>>> +
>>>> +	/*
>>>> +	 * Remove all connected devices after unregistering ops, to ensure
>>>> +	 * that no further connect events will occur.  The disconnect routine
>>>> +	 * will issue the QMI disconnect indication, which results in the
>>>> +	 * external DSP to stop issuing transfers.
>>>> +	 */
>>>> +	snd_usb_unregister_platform_ops();
>>>> +	for (idx = 0; idx < SNDRV_CARDS; idx++)
>>>> +		qc_usb_audio_offload_disconnect(uadev[idx].chip);
>>>> +
>>>> +	qmi_handle_release(svc->uaudio_svc_hdl);
>>>> +	kfree(svc);
>>>> +	uaudio_svc = NULL;
>>>> +}
>>>> +
>>>> +module_init(qc_usb_audio_offload_init);
>>>> +module_exit(qc_usb_audio_offload_exit);
>>>> +
>>>> +MODULE_DESCRIPTION("QC USB Audio Offloading");
>>>> +MODULE_LICENSE("GPL");
>>>
>>> What will trigger loading this if this code is built as module?
>>>
>>> Testing suggests nothing does at the moment: If this is built as module,
>>> playback via USB_RX will fail until you manually modprobe
>>> snd-usb-audio-qmi.
>>>
>>
>> Yes, it would only get triggered on a modprobe.  I think the more important
>> part is when snd_usb_register_platform_ops() is called.  This is what would
>> register the vendor USB offload driver callbacks for USB connect/disconnect
>> events.
>>
>>> I think the easiest way to solve this would be to drop the
>>> module_init()/module_exit() and instead call into these init/exit
>>> functions from one of the other audio modules. This would also ensure
>>> that the QMI server is only registered if we actually need it (if the
>>> board sound card actually has a USB DAI link).
>>>
>>
>> It would be difficult from the perspective of USB SND, because if we got
>> rid of the vendor ops, it would be messy, since the USB offload vendor
>> driver will be specific for every SoC.
>>
> 
> What I meant is calling qc_usb_audio_offload_init() from any of the
> other drivers that are involved, e.g. q6usb. Or register an auxilliary
> device like in qcom_pd_mapper, so the modules don't need to link
> together directly. That shouldn't get too messy.
> 

Thanks for clarifying.  I took a look and implemented the auxiliary path,
and it works pretty well :).  Wasn't aware of the aux bus...very useful.

Thanks
Wesley Cheng