Re: [PATCH 2/2] media: v4l2-mem2mem: add a list for buf used by hw

[Nicolas Dufresne]

Hi Randy,

Le vendredi 04 août 2023 à 00:16 +0800, Randy Li a écrit :
> On 2023/7/29 00:19, Nicolas Dufresne wrote:
> > Le vendredi 28 juillet 2023 à 15:37 +0800, Hsia-Jun Li a écrit :
> > > > I think this is one reason to migrate to the stateless decoder design.
> > > > 
> > > I didn't know such plan here. I don't think the current stateless API
> > > could export the reconstruction buffers for encoder or post-processing
> > > buffer for decoder to us.
> > Someone suggested introduce auxiliary queues in our meeting in Lyon a while ago,
> > but I bet everyone got too busy with finalizing APIs, fixing fluster tests etc.
> > The suggestion felt like it would be possible to add it after the fact. This was
> > also being discussed in the context of supporting multi-scalers (standalone our
> > inline with the codec, like VC8000D+). It could also cover for primary and
> > secondary buffers, along with encoder primary, and reconstruction buffers, but
> > also auxiliary reference data. This would also be needed to properly support
> > Vulkan Video fwiw, and could also help with a transition to DMABuf Heaps and
> > memory accounting.
> > 
> > I've also had corridor discussion around having multi-instance media constroller
> > devices. It wasn't clear how to bind the media instance to the video node
> > instances, but assuming there is a way, it would be a tad more flexible (but
> > massively more complex).
> 
> I think we should answer to those questions before we decided what we want:
> 
> A. Should a queue only has the buffers for the same format and sizes?

I initially started answering these, but ended up concluding this is more some
sort of personal notes. I believe the discussion is diverging. Remember that in
an existing API, one cannot fix all theoretical issues in one go. In order to
move forward, you have to tackle very specific use case and find a way forward.
If you are to break compatibility as much as you suggest, you'd rather look into
writing a new API.

[...]