Re: [PATCH v3 3/4] staging: media: Add support for the Allwinner A31 ISP

From: Sakari Ailus
Date: Thu Apr 28 2022 - 08:15:10 EST

Next message: Rob Herring: "Re: [v4 16/18] dt-bindings: mediatek: mt8186: add mt8186-mt6366-da7219-max98357 document"
Previous message: Artur Bujdoso: "Re: [PATCH] staging: octeon-usb: remove unnecessary parentheses"
Next in thread: Paul Kocialkowski: "Re: [PATCH v3 3/4] staging: media: Add support for the Allwinner A31 ISP"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Hi Paul,

Thanks for the set.

A few comments below.

On Fri, Apr 15, 2022 at 05:37:07PM +0200, Paul Kocialkowski wrote:
> Some Allwinner platforms come with an Image Signal Processor, which
> supports various features in order to enhance and transform data
> received by image sensors into good-looking pictures. In most cases,
> the data is raw bayer, which gets internally converted to RGB and
> finally YUV, which is what the hardware produces.
>
> This driver supports ISPs that are similar to the A31 ISP, which was
> the first standalone ISP found in Allwinner platforms. Simpler ISP
> blocks were found in the A10 and A20, where they are tied to a CSI
> controller. Newer generations of Allwinner SoCs (starting with the
> H6, H616, etc) come with a new camera subsystem and revised ISP.
> Even though these previous and next-generation ISPs are somewhat
> similar to the A31 ISP, they have enough significant differences to
> be out of the scope of this driver.
>
> While the ISP supports many features, including 3A and many
> enhancement blocks, this implementation is limited to the following:
> - V3s (V3/S3) platform support;
> - Bayer media bus formats as input;
> - Semi-planar YUV (NV12/NV21) as output;
> - Debayering with per-component gain and offset configuration;
> - 2D noise filtering with configurable coefficients.
>
> Since many features are missing from the associated uAPI, the driver
> is aimed to integrate staging until all features are properly
> described.
>
> On the technical side, it uses the v4l2 and media controller APIs,
> with a video node for capture, a processor subdev and a video node
> for parameters submission. A specific uAPI structure and associated
> v4l2 meta format are used to configure parameters of the supported
> modules.
>
> One particular thing about the hardware is that configuration for
> module registers needs to be stored in a DMA buffer and gets copied
> to actual registers by the hardware at the next vsync, when instructed
> by a flag. This is handled by the "state" mechanism in the driver.
>
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@xxxxxxxxxxx>
> ---
> drivers/staging/media/sunxi/Kconfig | 1 +
> drivers/staging/media/sunxi/Makefile | 1 +
> drivers/staging/media/sunxi/sun6i-isp/Kconfig | 15 +
> .../staging/media/sunxi/sun6i-isp/Makefile | 4 +
> .../staging/media/sunxi/sun6i-isp/TODO.txt | 6 +

This should be called TODO (without .txt).

I understand this is an online ISP. How do you schedule the video buffer
queues? Say, what happens if it's time to set up buffers for a frame and
there's a buffer queued in the parameter queue but not in the image data
queue? Or the other way around?

> .../staging/media/sunxi/sun6i-isp/sun6i_isp.c | 556 +++++++++++++
> .../staging/media/sunxi/sun6i-isp/sun6i_isp.h | 85 ++
> .../media/sunxi/sun6i-isp/sun6i_isp_capture.c | 749 ++++++++++++++++++
> .../media/sunxi/sun6i-isp/sun6i_isp_capture.h | 78 ++
> .../media/sunxi/sun6i-isp/sun6i_isp_params.c | 573 ++++++++++++++
> .../media/sunxi/sun6i-isp/sun6i_isp_params.h | 52 ++
> .../media/sunxi/sun6i-isp/sun6i_isp_proc.c | 578 ++++++++++++++
> .../media/sunxi/sun6i-isp/sun6i_isp_proc.h | 66 ++
> .../media/sunxi/sun6i-isp/sun6i_isp_reg.h | 275 +++++++
> .../sunxi/sun6i-isp/uapi/sun6i-isp-config.h | 43 +
> 15 files changed, 3082 insertions(+)
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Kconfig
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Makefile
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/TODO.txt
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h
>
> diff --git a/drivers/staging/media/sunxi/Kconfig b/drivers/staging/media/sunxi/Kconfig
> index 4549a135741f..62a486aba88b 100644
> --- a/drivers/staging/media/sunxi/Kconfig
> +++ b/drivers/staging/media/sunxi/Kconfig
> @@ -12,5 +12,6 @@ config VIDEO_SUNXI
> if VIDEO_SUNXI
>
> source "drivers/staging/media/sunxi/cedrus/Kconfig"
> +source "drivers/staging/media/sunxi/sun6i-isp/Kconfig"
>
> endif
> diff --git a/drivers/staging/media/sunxi/Makefile b/drivers/staging/media/sunxi/Makefile
> index b87140b0e15f..3d20b2f0e644 100644
> --- a/drivers/staging/media/sunxi/Makefile
> +++ b/drivers/staging/media/sunxi/Makefile
> @@ -1,2 +1,3 @@
> # SPDX-License-Identifier: GPL-2.0
> obj-$(CONFIG_VIDEO_SUNXI_CEDRUS) += cedrus/
> +obj-$(CONFIG_VIDEO_SUN6I_ISP) += sun6i-isp/
> diff --git a/drivers/staging/media/sunxi/sun6i-isp/Kconfig b/drivers/staging/media/sunxi/sun6i-isp/Kconfig
> new file mode 100644
> index 000000000000..68dcae9cd7d7
> --- /dev/null
> +++ b/drivers/staging/media/sunxi/sun6i-isp/Kconfig
> @@ -0,0 +1,15 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +config VIDEO_SUN6I_ISP
> + tristate "Allwinner A31 Image Signal Processor (ISP) Driver"
> + depends on V4L_PLATFORM_DRIVERS && VIDEO_DEV
> + depends on ARCH_SUNXI || COMPILE_TEST
> + depends on PM && COMMON_CLK && HAS_DMA
> + select MEDIA_CONTROLLER
> + select VIDEO_V4L2_SUBDEV_API
> + select VIDEOBUF2_DMA_CONTIG
> + select VIDEOBUF2_VMALLOC
> + select V4L2_FWNODE
> + select REGMAP_MMIO
> + help
> + Support for the Allwinner A31 Image Signal Processor (ISP), also
> + found on other platforms such as the A80, A83T or V3/V3s.
> diff --git a/drivers/staging/media/sunxi/sun6i-isp/Makefile b/drivers/staging/media/sunxi/sun6i-isp/Makefile
> new file mode 100644
> index 000000000000..da1034785144
> --- /dev/null
> +++ b/drivers/staging/media/sunxi/sun6i-isp/Makefile
> @@ -0,0 +1,4 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +sun6i-isp-y += sun6i_isp.o sun6i_isp_proc.o sun6i_isp_capture.o sun6i_isp_params.o
> +
> +obj-$(CONFIG_VIDEO_SUN6I_ISP) += sun6i-isp.o
> diff --git a/drivers/staging/media/sunxi/sun6i-isp/TODO.txt b/drivers/staging/media/sunxi/sun6i-isp/TODO.txt
> new file mode 100644
> index 000000000000..1e3236edc1ab
> --- /dev/null
> +++ b/drivers/staging/media/sunxi/sun6i-isp/TODO.txt
> @@ -0,0 +1,6 @@
> +Unstaging requirements:
> +- Add uAPI support and documentation for the configuration of all the hardware
> + modules and description of the statistics data structures;
> +- Add support for statistics reporting;
> +- Add userspace support in libcamera which demonstrates the ability to receive
> + statistics and adapt hardware modules configuration accordingly;
> diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c
> new file mode 100644
> index 000000000000..118eee625eba
> --- /dev/null
> +++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c
> @@ -0,0 +1,556 @@
> +// SPDX-License-Identifier: GPL-2.0+
> +/*
> + * Copyright 2021-2022 Bootlin
> + * Author: Paul Kocialkowski <paul.kocialkowski@xxxxxxxxxxx>
> + */
> +
> +#include <linux/clk.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/err.h>
> +#include <linux/interrupt.h>
> +#include <linux/module.h>
> +#include <linux/of.h>
> +#include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/regmap.h>
> +#include <linux/reset.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-mc.h>
> +
> +#include "sun6i_isp.h"
> +#include "sun6i_isp_capture.h"
> +#include "sun6i_isp_params.h"
> +#include "sun6i_isp_proc.h"
> +#include "sun6i_isp_reg.h"
> +
> +/* Helpers */
> +
> +u32 sun6i_isp_load_read(struct sun6i_isp_device *isp_dev, u32 offset)
> +{
> + u32 *data = (u32 *)(isp_dev->tables.load.data + offset);
> +
> + return *data;
> +}
> +
> +void sun6i_isp_load_write(struct sun6i_isp_device *isp_dev, u32 offset,
> + u32 value)
> +{
> + u32 *data = (u32 *)(isp_dev->tables.load.data + offset);
> +
> + *data = value;
> +}
> +
> +/* State */
> +
> +/*
> + * The ISP works with a load buffer, which gets copied to the actual registers
> + * by the hardware before processing a frame when a specific flag is set.
> + * This is represented by tracking the ISP state in the different parts of
> + * the code with explicit sync points:
> + * - state update: to update the load buffer for the next frame if necessary;
> + * - state complete: to indicate that the state update was applied.
> + */
> +
> +static void sun6i_isp_state_ready(struct sun6i_isp_device *isp_dev)
> +{
> + struct regmap *regmap = isp_dev->regmap;
> + u32 value;
> +
> + regmap_read(regmap, SUN6I_ISP_FE_CTRL_REG, &value);
> + value |= SUN6I_ISP_FE_CTRL_PARA_READY;
> + regmap_write(regmap, SUN6I_ISP_FE_CTRL_REG, value);
> +}
> +
> +static void sun6i_isp_state_complete(struct sun6i_isp_device *isp_dev)
> +{
> + unsigned long flags;
> +
> + spin_lock_irqsave(&isp_dev->state_lock, flags);
> +
> + sun6i_isp_capture_state_complete(isp_dev);
> + sun6i_isp_params_state_complete(isp_dev);
> +
> + spin_unlock_irqrestore(&isp_dev->state_lock, flags);
> +}
> +
> +void sun6i_isp_state_update(struct sun6i_isp_device *isp_dev, bool ready_hold)
> +{
> + bool update = false;
> + unsigned long flags;
> +
> + spin_lock_irqsave(&isp_dev->state_lock, flags);
> +
> + sun6i_isp_capture_state_update(isp_dev, &update);
> + sun6i_isp_params_state_update(isp_dev, &update);
> +
> + if (update && !ready_hold)
> + sun6i_isp_state_ready(isp_dev);
> +
> + spin_unlock_irqrestore(&isp_dev->state_lock, flags);
> +}
> +
> +/* Tables */
> +
> +static int sun6i_isp_table_setup(struct sun6i_isp_device *isp_dev,
> + struct sun6i_isp_table *table)
> +{
> + table->data = dma_alloc_coherent(isp_dev->dev, table->size,
> + &table->address, GFP_KERNEL);
> + if (!table->data)
> + return -ENOMEM;
> +
> + return 0;
> +}
> +
> +static void sun6i_isp_table_cleanup(struct sun6i_isp_device *isp_dev,
> + struct sun6i_isp_table *table)
> +{
> + dma_free_coherent(isp_dev->dev, table->size, table->data,
> + table->address);
> +}
> +
> +void sun6i_isp_tables_configure(struct sun6i_isp_device *isp_dev)
> +{
> + struct regmap *regmap = isp_dev->regmap;
> +
> + regmap_write(regmap, SUN6I_ISP_REG_LOAD_ADDR_REG,
> + SUN6I_ISP_ADDR_VALUE(isp_dev->tables.load.address));
> +
> + regmap_write(regmap, SUN6I_ISP_REG_SAVE_ADDR_REG,
> + SUN6I_ISP_ADDR_VALUE(isp_dev->tables.save.address));
> +
> + regmap_write(regmap, SUN6I_ISP_LUT_TABLE_ADDR_REG,
> + SUN6I_ISP_ADDR_VALUE(isp_dev->tables.lut.address));
> +
> + regmap_write(regmap, SUN6I_ISP_DRC_TABLE_ADDR_REG,
> + SUN6I_ISP_ADDR_VALUE(isp_dev->tables.drc.address));
> +
> + regmap_write(regmap, SUN6I_ISP_STATS_ADDR_REG,
> + SUN6I_ISP_ADDR_VALUE(isp_dev->tables.stats.address));
> +}
> +
> +static int sun6i_isp_tables_setup(struct sun6i_isp_device *isp_dev)
> +{
> + struct sun6i_isp_tables *tables = &isp_dev->tables;
> + int ret;
> +
> + /* Sizes are hardcoded for now but actually depend on the platform. */

Would it be cleaner to have them defined in a platform-specific way, e.g.
in a struct you obtain using device_get_match_data()?

> +
> + tables->load.size = 0x1000;
> + ret = sun6i_isp_table_setup(isp_dev, &tables->load);
> + if (ret)
> + return ret;
> +
> + tables->save.size = 0x1000;
> + ret = sun6i_isp_table_setup(isp_dev, &tables->save);
> + if (ret)
> + return ret;
> +
> + tables->lut.size = 0xe00;
> + ret = sun6i_isp_table_setup(isp_dev, &tables->lut);
> + if (ret)
> + return ret;
> +
> + tables->drc.size = 0x600;
> + ret = sun6i_isp_table_setup(isp_dev, &tables->drc);
> + if (ret)
> + return ret;
> +
> + tables->stats.size = 0x2100;
> + ret = sun6i_isp_table_setup(isp_dev, &tables->stats);

You can return already here.

> + if (ret)
> + return ret;
> +
> + return 0;
> +}
> +
> +static void sun6i_isp_tables_cleanup(struct sun6i_isp_device *isp_dev)
> +{
> + struct sun6i_isp_tables *tables = &isp_dev->tables;
> +
> + sun6i_isp_table_cleanup(isp_dev, &tables->stats);
> + sun6i_isp_table_cleanup(isp_dev, &tables->drc);
> + sun6i_isp_table_cleanup(isp_dev, &tables->lut);
> + sun6i_isp_table_cleanup(isp_dev, &tables->save);
> + sun6i_isp_table_cleanup(isp_dev, &tables->load);
> +}
> +
> +/* Media */
> +
> +static const struct media_device_ops sun6i_isp_media_ops = {
> + .link_notify = v4l2_pipeline_link_notify,
> +};
> +
> +/* V4L2 */
> +
> +static int sun6i_isp_v4l2_setup(struct sun6i_isp_device *isp_dev)
> +{
> + struct sun6i_isp_v4l2 *v4l2 = &isp_dev->v4l2;
> + struct v4l2_device *v4l2_dev = &v4l2->v4l2_dev;
> + struct media_device *media_dev = &v4l2->media_dev;
> + struct device *dev = isp_dev->dev;
> + int ret;
> +
> + /* Media Device */
> +
> + strscpy(media_dev->model, SUN6I_ISP_DESCRIPTION,
> + sizeof(media_dev->model));
> + snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> + "platform:%s", dev_name(dev));

This is no longer needed, see commit b0e38610f40a0f89e34939d2c7420590d67d86a3
.

> + media_dev->ops = &sun6i_isp_media_ops;
> + media_dev->hw_revision = 0;
> + media_dev->dev = dev;
> +
> + media_device_init(media_dev);
> +
> + ret = media_device_register(media_dev);
> + if (ret) {
> + dev_err(dev, "failed to register media device\n");
> + return ret;
> + }
> +
> + /* V4L2 Control Handler */
> +
> + ret = v4l2_ctrl_handler_init(&v4l2->ctrl_handler, 0);

I suppose you intend to add controls later on?

> + if (ret) {
> + dev_err(dev, "failed to init v4l2 control handler\n");
> + goto error_media;
> + }
> +
> + /* V4L2 Device */
> +
> + v4l2_dev->mdev = media_dev;
> + v4l2_dev->ctrl_handler = &v4l2->ctrl_handler;
> +
> + ret = v4l2_device_register(dev, v4l2_dev);
> + if (ret) {
> + dev_err(dev, "failed to register v4l2 device\n");
> + goto error_v4l2_ctrl;
> + }
> +
> + return 0;
> +
> +error_v4l2_ctrl:
> + v4l2_ctrl_handler_free(&v4l2->ctrl_handler);
> +
> +error_media:
> + media_device_unregister(media_dev);
> + media_device_cleanup(media_dev);
> +
> + return ret;
> +}

> +static void sun6i_isp_v4l2_cleanup(struct sun6i_isp_device *isp_dev)
> +{
> + struct sun6i_isp_v4l2 *v4l2 = &isp_dev->v4l2;
> +
> + media_device_unregister(&v4l2->media_dev);
> + v4l2_device_unregister(&v4l2->v4l2_dev);
> + v4l2_ctrl_handler_free(&v4l2->ctrl_handler);
> + media_device_cleanup(&v4l2->media_dev);
> +}
> +
> +/* Platform */
> +
> +static irqreturn_t sun6i_isp_interrupt(int irq, void *private)
> +{
> + struct sun6i_isp_device *isp_dev = private;
> + struct regmap *regmap = isp_dev->regmap;
> + u32 status = 0, enable = 0;
> +
> + regmap_read(regmap, SUN6I_ISP_FE_INT_STA_REG, &status);
> + regmap_read(regmap, SUN6I_ISP_FE_INT_EN_REG, &enable);
> +
> + if (!status)
> + return IRQ_NONE;
> + else if (!(status & enable))
> + goto complete;
> +
> + /*
> + * The ISP working cycle starts with a params-load, which makes the
> + * state from the load buffer active. Then it starts processing the
> + * frame and gives a finish interrupt. Soon after that, the next state
> + * coming from the load buffer will be applied for the next frame,
> + * giving a params-load as well.
> + *
> + * Because both frame finish and params-load are received almost
> + * at the same time (one ISR call), handle them in chronology order.
> + */
> +
> + if (status & SUN6I_ISP_FE_INT_STA_FINISH)
> + sun6i_isp_capture_finish(isp_dev);
> +
> + if (status & SUN6I_ISP_FE_INT_STA_PARA_LOAD) {
> + sun6i_isp_state_complete(isp_dev);
> + sun6i_isp_state_update(isp_dev, false);
> + }
> +
> +complete:
> + regmap_write(regmap, SUN6I_ISP_FE_INT_STA_REG, status);
> +
> + return IRQ_HANDLED;
> +}
> +
> +static int sun6i_isp_suspend(struct device *dev)
> +{
> + struct sun6i_isp_device *isp_dev = dev_get_drvdata(dev);
> +
> + reset_control_assert(isp_dev->reset);
> + clk_disable_unprepare(isp_dev->clock_ram);
> + clk_disable_unprepare(isp_dev->clock_mod);
> +
> + return 0;
> +}
> +
> +static int sun6i_isp_resume(struct device *dev)
> +{
> + struct sun6i_isp_device *isp_dev = dev_get_drvdata(dev);
> + int ret;
> +
> + ret = reset_control_deassert(isp_dev->reset);
> + if (ret) {
> + dev_err(dev, "failed to deassert reset\n");
> + return ret;
> + }
> +
> + ret = clk_prepare_enable(isp_dev->clock_mod);
> + if (ret) {
> + dev_err(dev, "failed to enable module clock\n");
> + goto error_reset;
> + }
> +
> + ret = clk_prepare_enable(isp_dev->clock_ram);
> + if (ret) {
> + dev_err(dev, "failed to enable ram clock\n");
> + goto error_clock_mod;
> + }
> +
> + return 0;
> +
> +error_clock_mod:
> + clk_disable_unprepare(isp_dev->clock_mod);
> +
> +error_reset:
> + reset_control_assert(isp_dev->reset);
> +
> + return ret;
> +}
> +
> +static const struct dev_pm_ops sun6i_isp_pm_ops = {
> + .runtime_suspend = sun6i_isp_suspend,
> + .runtime_resume = sun6i_isp_resume,
> +};
> +
> +static const struct regmap_config sun6i_isp_regmap_config = {
> + .reg_bits = 32,
> + .reg_stride = 4,
> + .val_bits = 32,
> + .max_register = 0x400,
> +};
> +
> +static int sun6i_isp_resources_setup(struct sun6i_isp_device *isp_dev,
> + struct platform_device *platform_dev)
> +{
> + struct device *dev = isp_dev->dev;
> + void __iomem *io_base;
> + int irq;
> + int ret;
> +
> + /* Registers */
> +
> + io_base = devm_platform_ioremap_resource(platform_dev, 0);
> + if (IS_ERR(io_base))
> + return PTR_ERR(io_base);
> +
> + isp_dev->regmap = devm_regmap_init_mmio_clk(dev, "bus", io_base,
> + &sun6i_isp_regmap_config);
> + if (IS_ERR(isp_dev->regmap)) {
> + dev_err(dev, "failed to init register map\n");
> + return PTR_ERR(isp_dev->regmap);
> + }
> +
> + /* Clocks */
> +
> + isp_dev->clock_mod = devm_clk_get(dev, "mod");
> + if (IS_ERR(isp_dev->clock_mod)) {
> + dev_err(dev, "failed to acquire module clock\n");
> + return PTR_ERR(isp_dev->clock_mod);
> + }
> +
> + isp_dev->clock_ram = devm_clk_get(dev, "ram");
> + if (IS_ERR(isp_dev->clock_ram)) {
> + dev_err(dev, "failed to acquire ram clock\n");
> + return PTR_ERR(isp_dev->clock_ram);
> + }
> +
> + ret = clk_set_rate_exclusive(isp_dev->clock_mod, 297000000);

Is this also specific to the model?

> + if (ret) {
> + dev_err(dev, "failed to set mod clock rate\n");
> + return ret;
> + }
> +
> + /* Reset */
> +
> + isp_dev->reset = devm_reset_control_get_shared(dev, NULL);
> + if (IS_ERR(isp_dev->reset)) {
> + dev_err(dev, "failed to acquire reset\n");
> + ret = PTR_ERR(isp_dev->reset);
> + goto error_clock_rate_exclusive;
> + }
> +
> + /* Interrupt */
> +
> + irq = platform_get_irq(platform_dev, 0);
> + if (irq < 0) {
> + dev_err(dev, "failed to get interrupt\n");
> + ret = -ENXIO;
> + goto error_clock_rate_exclusive;
> + }
> +
> + ret = devm_request_irq(dev, irq, sun6i_isp_interrupt, IRQF_SHARED,
> + SUN6I_ISP_NAME, isp_dev);
> + if (ret) {
> + dev_err(dev, "failed to request interrupt\n");
> + goto error_clock_rate_exclusive;
> + }
> +
> + /* Runtime PM */
> +
> + pm_runtime_enable(dev);
> +
> + return 0;
> +
> +error_clock_rate_exclusive:
> + clk_rate_exclusive_put(isp_dev->clock_mod);
> +
> + return ret;
> +}
> +
> +static void sun6i_isp_resources_cleanup(struct sun6i_isp_device *isp_dev)
> +{
> + struct device *dev = isp_dev->dev;
> +
> + pm_runtime_disable(dev);
> + clk_rate_exclusive_put(isp_dev->clock_mod);
> +}
> +
> +static int sun6i_isp_probe(struct platform_device *platform_dev)
> +{
> + struct sun6i_isp_device *isp_dev;
> + struct device *dev = &platform_dev->dev;
> + int ret;
> +
> + isp_dev = devm_kzalloc(dev, sizeof(*isp_dev), GFP_KERNEL);
> + if (!isp_dev)
> + return -ENOMEM;
> +
> + isp_dev->dev = dev;
> + platform_set_drvdata(platform_dev, isp_dev);
> +
> + spin_lock_init(&isp_dev->state_lock);
> +
> + ret = sun6i_isp_resources_setup(isp_dev, platform_dev);
> + if (ret)
> + return ret;
> +
> + ret = sun6i_isp_tables_setup(isp_dev);
> + if (ret) {
> + dev_err(dev, "failed to setup tables\n");
> + goto error_resources;
> + }
> +
> + ret = sun6i_isp_v4l2_setup(isp_dev);
> + if (ret) {
> + dev_err(dev, "failed to setup v4l2\n");
> + goto error_tables;
> + }
> +
> + ret = sun6i_isp_proc_setup(isp_dev);
> + if (ret) {
> + dev_err(dev, "failed to setup proc\n");
> + goto error_v4l2;
> + }
> +
> + ret = sun6i_isp_capture_setup(isp_dev);
> + if (ret) {
> + dev_err(dev, "failed to setup capture\n");
> + goto error_proc;
> + }
> +
> + ret = sun6i_isp_params_setup(isp_dev);
> + if (ret) {
> + dev_err(dev, "failed to setup params\n");
> + goto error_capture;
> + }
> +
> + return 0;
> +
> +error_capture:
> + sun6i_isp_capture_cleanup(isp_dev);
> +
> +error_proc:
> + sun6i_isp_proc_cleanup(isp_dev);
> +
> +error_v4l2:
> + sun6i_isp_v4l2_cleanup(isp_dev);
> +
> +error_tables:
> + sun6i_isp_tables_cleanup(isp_dev);
> +
> +error_resources:
> + sun6i_isp_resources_cleanup(isp_dev);
> +
> + return ret;
> +}
> +
> +static int sun6i_isp_remove(struct platform_device *platform_dev)
> +{
> + struct sun6i_isp_device *isp_dev = platform_get_drvdata(platform_dev);
> +
> + sun6i_isp_params_cleanup(isp_dev);
> + sun6i_isp_capture_cleanup(isp_dev);
> + sun6i_isp_proc_cleanup(isp_dev);
> + sun6i_isp_v4l2_cleanup(isp_dev);
> + sun6i_isp_tables_cleanup(isp_dev);
> + sun6i_isp_resources_cleanup(isp_dev);
> +
> + return 0;
> +}
> +
> +/*
> + * History of sun6i-isp:
> + * - sun4i-a10-isp: initial ISP tied to the CSI0 controller,
> + * apparently unused in software implementations;
> + * - sun6i-a31-isp: separate ISP loosely based on sun4i-a10-isp,
> + * adding extra modules and features;
> + * - sun9i-a80-isp: based on sun6i-a31-isp with some register offset changes
> + * and new modules like saturation and cnr;
> + * - sun8i-a23-isp/sun8i-h3-isp: based on sun9i-a80-isp with most modules
> + * related to raw removed;
> + * - sun8i-a83t-isp: based on sun9i-a80-isp with some register offset changes
> + * - sun8i-v3s-isp: based on sun8i-a83t-isp with a new disc module;
> + */
> +
> +static const struct of_device_id sun6i_isp_of_match[] = {
> + { .compatible = "allwinner,sun8i-v3s-isp" },
> + {},
> +};
> +
> +MODULE_DEVICE_TABLE(of, sun6i_isp_of_match);
> +
> +static struct platform_driver sun6i_isp_platform_driver = {
> + .probe = sun6i_isp_probe,
> + .remove = sun6i_isp_remove,
> + .driver = {
> + .name = SUN6I_ISP_NAME,
> + .of_match_table = of_match_ptr(sun6i_isp_of_match),
> + .pm = &sun6i_isp_pm_ops,
> + },
> +};
> +
> +module_platform_driver(sun6i_isp_platform_driver);
> +
> +MODULE_DESCRIPTION("Allwinner A31 Image Signal Processor driver");
> +MODULE_AUTHOR("Paul Kocialkowski <paul.kocialkowski@xxxxxxxxxxx>");
> +MODULE_LICENSE("GPL");
> diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h
> new file mode 100644
> index 000000000000..60017606babe
> --- /dev/null
> +++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h
> @@ -0,0 +1,85 @@
> +/* SPDX-License-Identifier: GPL-2.0+ */
> +/*
> + * Copyright 2021-2022 Bootlin
> + * Author: Paul Kocialkowski <paul.kocialkowski@xxxxxxxxxxx>
> + */
> +
> +#ifndef _SUN6I_ISP_H_
> +#define _SUN6I_ISP_H_
> +
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-device.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#include "sun6i_isp_capture.h"
> +#include "sun6i_isp_params.h"
> +#include "sun6i_isp_proc.h"
> +
> +#define SUN6I_ISP_NAME "sun6i-isp"
> +#define SUN6I_ISP_DESCRIPTION "Allwinner A31 ISP Device"
> +
> +enum sun6i_isp_port {
> + SUN6I_ISP_PORT_CSI0 = 0,
> + SUN6I_ISP_PORT_CSI1 = 1,
> +};
> +
> +struct sun6i_isp_buffer {
> + struct vb2_v4l2_buffer v4l2_buffer;
> + struct list_head list;
> +};
> +
> +struct sun6i_isp_v4l2 {
> + struct v4l2_device v4l2_dev;
> + struct v4l2_ctrl_handler ctrl_handler;
> + struct media_device media_dev;
> +};
> +
> +struct sun6i_isp_table {
> + void *data;
> + dma_addr_t address;
> + unsigned int size;
> +};
> +
> +struct sun6i_isp_tables {
> + struct sun6i_isp_table load;
> + struct sun6i_isp_table save;
> +
> + struct sun6i_isp_table lut;
> + struct sun6i_isp_table drc;
> + struct sun6i_isp_table stats;
> +};
> +
> +struct sun6i_isp_device {
> + struct device *dev;
> +
> + struct sun6i_isp_tables tables;
> +
> + struct sun6i_isp_v4l2 v4l2;
> + struct sun6i_isp_proc proc;
> + struct sun6i_isp_capture capture;
> + struct sun6i_isp_params params;
> +
> + struct regmap *regmap;
> + struct clk *clock_mod;
> + struct clk *clock_ram;
> + struct reset_control *reset;
> +
> + spinlock_t state_lock; /* State helpers lock. */
> +};
> +
> +/* Helpers */
> +
> +u32 sun6i_isp_load_read(struct sun6i_isp_device *isp_dev, u32 offset);
> +void sun6i_isp_load_write(struct sun6i_isp_device *isp_dev, u32 offset,
> + u32 value);
> +u32 sun6i_isp_address_value(dma_addr_t address);
> +
> +/* State */
> +
> +void sun6i_isp_state_update(struct sun6i_isp_device *isp_dev, bool ready_hold);
> +
> +/* Tables */
> +
> +void sun6i_isp_tables_configure(struct sun6i_isp_device *isp_dev);
> +
> +#endif
> diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c
> new file mode 100644
> index 000000000000..5cd069891461
> --- /dev/null
> +++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c
> @@ -0,0 +1,749 @@
> +// SPDX-License-Identifier: GPL-2.0+
> +/*
> + * Copyright 2021-2022 Bootlin
> + * Author: Paul Kocialkowski <paul.kocialkowski@xxxxxxxxxxx>
> + */
> +
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/v4l2-mc.h>
> +#include <media/videobuf2-dma-contig.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#include "sun6i_isp.h"
> +#include "sun6i_isp_capture.h"
> +#include "sun6i_isp_proc.h"
> +#include "sun6i_isp_reg.h"
> +
> +/* Helpers */
> +
> +void sun6i_isp_capture_dimensions(struct sun6i_isp_device *isp_dev,
> + unsigned int *width, unsigned int *height)
> +{
> + if (width)
> + *width = isp_dev->capture.format.fmt.pix.width;
> + if (height)
> + *height = isp_dev->capture.format.fmt.pix.height;
> +}
> +
> +void sun6i_isp_capture_format(struct sun6i_isp_device *isp_dev,
> + u32 *pixelformat)
> +{
> + if (pixelformat)
> + *pixelformat = isp_dev->capture.format.fmt.pix.pixelformat;
> +}
> +
> +/* Format */
> +
> +static const struct sun6i_isp_capture_format sun6i_isp_capture_formats[] = {
> + {
> + .pixelformat = V4L2_PIX_FMT_NV12,
> + .output_format = SUN6I_ISP_OUTPUT_FMT_YUV420SP,
> + },
> + {
> + .pixelformat = V4L2_PIX_FMT_NV21,
> + .output_format = SUN6I_ISP_OUTPUT_FMT_YVU420SP,
> + },
> +};
> +
> +const struct sun6i_isp_capture_format *
> +sun6i_isp_capture_format_find(u32 pixelformat)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(sun6i_isp_capture_formats); i++)
> + if (sun6i_isp_capture_formats[i].pixelformat == pixelformat)
> + return &sun6i_isp_capture_formats[i];
> +
> + return NULL;
> +}
> +
> +/* Capture */
> +
> +static void
> +sun6i_isp_capture_buffer_configure(struct sun6i_isp_device *isp_dev,
> + struct sun6i_isp_buffer *isp_buffer)
> +{
> + const struct v4l2_format_info *info;
> + struct vb2_buffer *vb2_buffer;
> + unsigned int width, height;
> + unsigned int width_aligned;
> + dma_addr_t address;
> + u32 pixelformat;
> +
> + vb2_buffer = &isp_buffer->v4l2_buffer.vb2_buf;
> + address = vb2_dma_contig_plane_dma_addr(vb2_buffer, 0);
> +
> + sun6i_isp_load_write(isp_dev, SUN6I_ISP_MCH_Y_ADDR0_REG,
> + SUN6I_ISP_ADDR_VALUE(address));
> +
> + sun6i_isp_capture_dimensions(isp_dev, &width, &height);
> + sun6i_isp_capture_format(isp_dev, &pixelformat);
> +
> + info = v4l2_format_info(pixelformat);
> + if (WARN_ON(!info))
> + return;
> +
> + /* Stride needs to be aligned to 4. */
> + width_aligned = ALIGN(width, 2);
> +
> + if (info->comp_planes > 1) {
> + address += info->bpp[0] * width_aligned * height;
> +
> + sun6i_isp_load_write(isp_dev, SUN6I_ISP_MCH_U_ADDR0_REG,
> + SUN6I_ISP_ADDR_VALUE(address));
> + }
> +
> + if (info->comp_planes > 2) {
> + address += info->bpp[1] *
> + DIV_ROUND_UP(width_aligned, info->hdiv) *
> + DIV_ROUND_UP(height, info->vdiv);
> +
> + sun6i_isp_load_write(isp_dev, SUN6I_ISP_MCH_V_ADDR0_REG,
> + SUN6I_ISP_ADDR_VALUE(address));
> + }
> +}
> +
> +void sun6i_isp_capture_configure(struct sun6i_isp_device *isp_dev)
> +{
> + unsigned int width, height;
> + unsigned int stride_luma, stride_chroma = 0;
> + unsigned int stride_luma_div4, stride_chroma_div4;
> + const struct sun6i_isp_capture_format *format;
> + const struct v4l2_format_info *info;
> + u32 pixelformat;
> +
> + sun6i_isp_capture_dimensions(isp_dev, &width, &height);
> + sun6i_isp_capture_format(isp_dev, &pixelformat);
> +
> + format = sun6i_isp_capture_format_find(pixelformat);
> + if (WARN_ON(!format))
> + return;
> +
> + sun6i_isp_load_write(isp_dev, SUN6I_ISP_MCH_SIZE_CFG_REG,
> + SUN6I_ISP_MCH_SIZE_CFG_WIDTH(width) |
> + SUN6I_ISP_MCH_SIZE_CFG_HEIGHT(height));
> +
> + info = v4l2_format_info(pixelformat);
> + if (WARN_ON(!info))
> + return;
> +
> + stride_luma = width * info->bpp[0];
> + stride_luma_div4 = DIV_ROUND_UP(stride_luma, 4);
> +
> + if (info->comp_planes > 1) {
> + stride_chroma = width * info->bpp[1] / info->hdiv;
> + stride_chroma_div4 = DIV_ROUND_UP(stride_chroma, 4);
> + }
> +
> + sun6i_isp_load_write(isp_dev, SUN6I_ISP_MCH_CFG_REG,
> + SUN6I_ISP_MCH_CFG_EN |
> + SUN6I_ISP_MCH_CFG_OUTPUT_FMT(format->output_format) |
> + SUN6I_ISP_MCH_CFG_STRIDE_Y_DIV4(stride_luma_div4) |
> + SUN6I_ISP_MCH_CFG_STRIDE_UV_DIV4(stride_chroma_div4));
> +}
> +
> +/* State */
> +
> +static void sun6i_isp_capture_state_cleanup(struct sun6i_isp_device *isp_dev,
> + bool error)
> +{
> + struct sun6i_isp_capture_state *state = &isp_dev->capture.state;
> + struct sun6i_isp_buffer **isp_buffer_states[] = {
> + &state->pending, &state->current, &state->complete,
> + };
> + struct sun6i_isp_buffer *isp_buffer;
> + struct vb2_buffer *vb2_buffer;
> + unsigned long flags;
> + unsigned int i;
> +
> + spin_lock_irqsave(&state->lock, flags);
> +
> + for (i = 0; i < ARRAY_SIZE(isp_buffer_states); i++) {
> + isp_buffer = *isp_buffer_states[i];
> + if (!isp_buffer)
> + continue;
> +
> + vb2_buffer = &isp_buffer->v4l2_buffer.vb2_buf;
> + vb2_buffer_done(vb2_buffer, error ? VB2_BUF_STATE_ERROR :
> + VB2_BUF_STATE_QUEUED);
> +
> + *isp_buffer_states[i] = NULL;
> + }
> +
> + list_for_each_entry(isp_buffer, &state->queue, list) {
> + vb2_buffer = &isp_buffer->v4l2_buffer.vb2_buf;
> + vb2_buffer_done(vb2_buffer, error ? VB2_BUF_STATE_ERROR :
> + VB2_BUF_STATE_QUEUED);
> + }
> +
> + INIT_LIST_HEAD(&state->queue);
> +
> + spin_unlock_irqrestore(&state->lock, flags);
> +}
> +
> +void sun6i_isp_capture_state_update(struct sun6i_isp_device *isp_dev,
> + bool *update)
> +{
> + struct sun6i_isp_capture_state *state = &isp_dev->capture.state;
> + struct sun6i_isp_buffer *isp_buffer;
> + unsigned long flags;
> +
> + spin_lock_irqsave(&state->lock, flags);
> +
> + if (list_empty(&state->queue))
> + goto complete;
> +
> + if (state->pending)
> + goto complete;
> +
> + isp_buffer = list_first_entry(&state->queue, struct sun6i_isp_buffer,
> + list);
> +
> + sun6i_isp_capture_buffer_configure(isp_dev, isp_buffer);
> +
> + list_del(&isp_buffer->list);
> +
> + state->pending = isp_buffer;
> +
> + if (update)
> + *update = true;
> +
> +complete:
> + spin_unlock_irqrestore(&state->lock, flags);
> +}
> +
> +void sun6i_isp_capture_state_complete(struct sun6i_isp_device *isp_dev)
> +{
> + struct sun6i_isp_capture_state *state = &isp_dev->capture.state;
> + unsigned long flags;
> +
> + spin_lock_irqsave(&state->lock, flags);
> +
> + if (!state->pending)
> + goto complete;
> +
> + state->complete = state->current;
> + state->current = state->pending;
> + state->pending = NULL;
> +
> + if (state->complete) {
> + struct sun6i_isp_buffer *isp_buffer = state->complete;
> + struct vb2_buffer *vb2_buffer =
> + &isp_buffer->v4l2_buffer.vb2_buf;
> +
> + vb2_buffer->timestamp = ktime_get_ns();
> + isp_buffer->v4l2_buffer.sequence = state->sequence;
> +
> + vb2_buffer_done(vb2_buffer, VB2_BUF_STATE_DONE);
> +
> + state->complete = NULL;
> + }
> +
> +complete:
> + spin_unlock_irqrestore(&state->lock, flags);
> +}
> +
> +void sun6i_isp_capture_finish(struct sun6i_isp_device *isp_dev)
> +{
> + struct sun6i_isp_capture_state *state = &isp_dev->capture.state;
> + unsigned long flags;
> +
> + spin_lock_irqsave(&state->lock, flags);
> + state->sequence++;
> + spin_unlock_irqrestore(&state->lock, flags);
> +}
> +
> +/* Queue */
> +
> +static int sun6i_isp_capture_queue_setup(struct vb2_queue *queue,
> + unsigned int *buffers_count,
> + unsigned int *planes_count,
> + unsigned int sizes[],
> + struct device *alloc_devs[])
> +{
> + struct sun6i_isp_device *isp_dev = vb2_get_drv_priv(queue);
> + unsigned int size = isp_dev->capture.format.fmt.pix.sizeimage;
> +
> + if (*planes_count)
> + return sizes[0] < size ? -EINVAL : 0;
> +
> + *planes_count = 1;
> + sizes[0] = size;
> +
> + return 0;
> +}
> +
> +static int sun6i_isp_capture_buffer_prepare(struct vb2_buffer *vb2_buffer)
> +{
> + struct sun6i_isp_device *isp_dev =
> + vb2_get_drv_priv(vb2_buffer->vb2_queue);
> + struct v4l2_device *v4l2_dev = &isp_dev->v4l2.v4l2_dev;
> + unsigned int size = isp_dev->capture.format.fmt.pix.sizeimage;
> +
> + if (vb2_plane_size(vb2_buffer, 0) < size) {
> + v4l2_err(v4l2_dev, "buffer too small (%lu < %u)\n",
> + vb2_plane_size(vb2_buffer, 0), size);
> + return -EINVAL;
> + }
> +
> + vb2_set_plane_payload(vb2_buffer, 0, size);
> +
> + return 0;
> +}
> +
> +static void sun6i_isp_capture_buffer_queue(struct vb2_buffer *vb2_buffer)
> +{
> + struct sun6i_isp_device *isp_dev =
> + vb2_get_drv_priv(vb2_buffer->vb2_queue);
> + struct sun6i_isp_capture_state *state = &isp_dev->capture.state;
> + struct vb2_v4l2_buffer *v4l2_buffer = to_vb2_v4l2_buffer(vb2_buffer);
> + struct sun6i_isp_buffer *isp_buffer =
> + container_of(v4l2_buffer, struct sun6i_isp_buffer, v4l2_buffer);
> + unsigned long flags;
> +
> + spin_lock_irqsave(&state->lock, flags);
> + list_add_tail(&isp_buffer->list, &state->queue);
> + spin_unlock_irqrestore(&state->lock, flags);
> +
> + /* Update the state to schedule our buffer as soon as possible. */
> + if (state->streaming)
> + sun6i_isp_state_update(isp_dev, false);
> +}
> +
> +static int sun6i_isp_capture_start_streaming(struct vb2_queue *queue,
> + unsigned int count)
> +{
> + struct sun6i_isp_device *isp_dev = vb2_get_drv_priv(queue);
> + struct sun6i_isp_capture_state *state = &isp_dev->capture.state;
> + struct video_device *video_dev = &isp_dev->capture.video_dev;
> + struct v4l2_subdev *subdev = &isp_dev->proc.subdev;
> + int ret;
> +
> + state->sequence = 0;
> +
> + ret = media_pipeline_start(&video_dev->entity, &video_dev->pipe);
> + if (ret < 0)
> + goto error_state;
> +
> + state->streaming = true;
> +
> + ret = v4l2_subdev_call(subdev, video, s_stream, 1);
> + if (ret && ret != -ENOIOCTLCMD)
> + goto error_streaming;
> +
> + return 0;
> +
> +error_streaming:
> + state->streaming = false;
> +
> + media_pipeline_stop(&video_dev->entity);
> +
> +error_state:
> + sun6i_isp_capture_state_cleanup(isp_dev, false);
> +
> + return ret;
> +}
> +
> +static void sun6i_isp_capture_stop_streaming(struct vb2_queue *queue)
> +{
> + struct sun6i_isp_device *isp_dev = vb2_get_drv_priv(queue);
> + struct sun6i_isp_capture_state *state = &isp_dev->capture.state;
> + struct video_device *video_dev = &isp_dev->capture.video_dev;
> + struct v4l2_subdev *subdev = &isp_dev->proc.subdev;
> +
> + v4l2_subdev_call(subdev, video, s_stream, 0);
> +
> + state->streaming = false;
> +
> + media_pipeline_stop(&video_dev->entity);
> +
> + sun6i_isp_capture_state_cleanup(isp_dev, true);
> +}
> +
> +static const struct vb2_ops sun6i_isp_capture_queue_ops = {
> + .queue_setup = sun6i_isp_capture_queue_setup,
> + .buf_prepare = sun6i_isp_capture_buffer_prepare,
> + .buf_queue = sun6i_isp_capture_buffer_queue,
> + .start_streaming = sun6i_isp_capture_start_streaming,
> + .stop_streaming = sun6i_isp_capture_stop_streaming,
> + .wait_prepare = vb2_ops_wait_prepare,
> + .wait_finish = vb2_ops_wait_finish,
> +};
> +
> +/* Video Device */
> +
> +static void sun6i_isp_capture_format_prepare(struct v4l2_format *format)
> +{
> + struct v4l2_pix_format *pix_format = &format->fmt.pix;
> + const struct v4l2_format_info *info;
> + unsigned int width, height;
> + unsigned int width_aligned;
> + unsigned int i;
> +
> + v4l_bound_align_image(&pix_format->width, SUN6I_ISP_CAPTURE_WIDTH_MIN,
> + SUN6I_ISP_CAPTURE_WIDTH_MAX, 1,
> + &pix_format->height, SUN6I_ISP_CAPTURE_HEIGHT_MIN,
> + SUN6I_ISP_CAPTURE_HEIGHT_MAX, 1, 0);
> +
> + if (!sun6i_isp_capture_format_find(pix_format->pixelformat))
> + pix_format->pixelformat =
> + sun6i_isp_capture_formats[0].pixelformat;
> +
> + info = v4l2_format_info(pix_format->pixelformat);
> + if (WARN_ON(!info))
> + return;
> +
> + width = pix_format->width;
> + height = pix_format->height;
> +
> + /* Stride needs to be aligned to 4. */
> + width_aligned = ALIGN(width, 2);
> +
> + pix_format->bytesperline = width_aligned * info->bpp[0];
> + pix_format->sizeimage = 0;
> +
> + for (i = 0; i < info->comp_planes; i++) {
> + unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> + unsigned int vdiv = (i == 0) ? 1 : info->vdiv;
> +
> + pix_format->sizeimage += info->bpp[i] *
> + DIV_ROUND_UP(width_aligned, hdiv) *
> + DIV_ROUND_UP(height, vdiv);
> + }
> +
> + pix_format->field = V4L2_FIELD_NONE;
> +
> + pix_format->colorspace = V4L2_COLORSPACE_RAW;
> + pix_format->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> + pix_format->quantization = V4L2_QUANTIZATION_DEFAULT;
> + pix_format->xfer_func = V4L2_XFER_FUNC_DEFAULT;
> +}
> +
> +static int sun6i_isp_capture_querycap(struct file *file, void *private,
> + struct v4l2_capability *capability)
> +{
> + struct sun6i_isp_device *isp_dev = video_drvdata(file);
> + struct video_device *video_dev = &isp_dev->capture.video_dev;
> +
> + strscpy(capability->driver, SUN6I_ISP_NAME, sizeof(capability->driver));
> + strscpy(capability->card, video_dev->name, sizeof(capability->card));
> + snprintf(capability->bus_info, sizeof(capability->bus_info),
> + "platform:%s", dev_name(isp_dev->dev));
> +
> + return 0;
> +}
> +
> +static int sun6i_isp_capture_enum_fmt(struct file *file, void *private,
> + struct v4l2_fmtdesc *fmtdesc)
> +{
> + u32 index = fmtdesc->index;
> +
> + if (index >= ARRAY_SIZE(sun6i_isp_capture_formats))
> + return -EINVAL;
> +
> + fmtdesc->pixelformat = sun6i_isp_capture_formats[index].pixelformat;
> +
> + return 0;
> +}
> +
> +static int sun6i_isp_capture_g_fmt(struct file *file, void *private,
> + struct v4l2_format *format)
> +{
> + struct sun6i_isp_device *isp_dev = video_drvdata(file);
> +
> + *format = isp_dev->capture.format;
> +
> + return 0;
> +}
> +
> +static int sun6i_isp_capture_s_fmt(struct file *file, void *private,
> + struct v4l2_format *format)
> +{
> + struct sun6i_isp_device *isp_dev = video_drvdata(file);
> +
> + if (vb2_is_busy(&isp_dev->capture.queue))
> + return -EBUSY;
> +
> + sun6i_isp_capture_format_prepare(format);
> +
> + isp_dev->capture.format = *format;
> +
> + return 0;
> +}
> +
> +static int sun6i_isp_capture_try_fmt(struct file *file, void *private,
> + struct v4l2_format *format)
> +{
> + sun6i_isp_capture_format_prepare(format);
> +
> + return 0;
> +}
> +
> +static int sun6i_isp_capture_enum_input(struct file *file, void *private,
> + struct v4l2_input *input)
> +{
> + if (input->index != 0)
> + return -EINVAL;
> +
> + input->type = V4L2_INPUT_TYPE_CAMERA;
> + strscpy(input->name, "Camera", sizeof(input->name));
> +
> + return 0;
> +}
> +
> +static int sun6i_isp_capture_g_input(struct file *file, void *private,
> + unsigned int *index)
> +{
> + *index = 0;
> +
> + return 0;
> +}
> +
> +static int sun6i_isp_capture_s_input(struct file *file, void *private,
> + unsigned int index)
> +{
> + if (index != 0)
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> +static const struct v4l2_ioctl_ops sun6i_isp_capture_ioctl_ops = {
> + .vidioc_querycap = sun6i_isp_capture_querycap,
> +
> + .vidioc_enum_fmt_vid_cap = sun6i_isp_capture_enum_fmt,
> + .vidioc_g_fmt_vid_cap = sun6i_isp_capture_g_fmt,
> + .vidioc_s_fmt_vid_cap = sun6i_isp_capture_s_fmt,
> + .vidioc_try_fmt_vid_cap = sun6i_isp_capture_try_fmt,
> +
> + .vidioc_enum_input = sun6i_isp_capture_enum_input,
> + .vidioc_g_input = sun6i_isp_capture_g_input,
> + .vidioc_s_input = sun6i_isp_capture_s_input,
> +
> + .vidioc_create_bufs = vb2_ioctl_create_bufs,
> + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> + .vidioc_reqbufs = vb2_ioctl_reqbufs,
> + .vidioc_querybuf = vb2_ioctl_querybuf,
> + .vidioc_expbuf = vb2_ioctl_expbuf,
> + .vidioc_qbuf = vb2_ioctl_qbuf,
> + .vidioc_dqbuf = vb2_ioctl_dqbuf,
> + .vidioc_streamon = vb2_ioctl_streamon,
> + .vidioc_streamoff = vb2_ioctl_streamoff,
> +
> + .vidioc_log_status = v4l2_ctrl_log_status,
> + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> + .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> +};
> +
> +static int sun6i_isp_capture_open(struct file *file)
> +{
> + struct sun6i_isp_device *isp_dev = video_drvdata(file);
> + struct video_device *video_dev = &isp_dev->capture.video_dev;
> + struct mutex *lock = &isp_dev->capture.lock;
> + int ret;
> +
> + if (mutex_lock_interruptible(lock))
> + return -ERESTARTSYS;
> +
> + ret = v4l2_pipeline_pm_get(&video_dev->entity);

Do you need this?

Drivers should primarily depend on runtime PM, this is only needed for
compatibility reasons. Instead I'd like to see sensor drivers being moved
to runtime PM.

> + if (ret)
> + goto error_mutex;
> +
> + ret = v4l2_fh_open(file);
> + if (ret)
> + goto error_pipeline;
> +
> + mutex_unlock(lock);
> +
> + return 0;
> +
> +error_pipeline:
> + v4l2_pipeline_pm_put(&video_dev->entity);
> +
> +error_mutex:
> + mutex_unlock(lock);
> +
> + return ret;
> +}
> +
> +static int sun6i_isp_capture_release(struct file *file)
> +{
> + struct sun6i_isp_device *isp_dev = video_drvdata(file);
> + struct video_device *video_dev = &isp_dev->capture.video_dev;
> + struct mutex *lock = &isp_dev->capture.lock;
> +
> + mutex_lock(lock);
> +
> + _vb2_fop_release(file, NULL);
> + v4l2_pipeline_pm_put(&video_dev->entity);
> +
> + mutex_unlock(lock);
> +
> + return 0;
> +}
> +
> +static const struct v4l2_file_operations sun6i_isp_capture_fops = {
> + .owner = THIS_MODULE,
> + .open = sun6i_isp_capture_open,
> + .release = sun6i_isp_capture_release,
> + .unlocked_ioctl = video_ioctl2,
> + .poll = vb2_fop_poll,
> + .mmap = vb2_fop_mmap,
> +};
> +
> +/* Media Entity */
> +
> +static int sun6i_isp_capture_link_validate(struct media_link *link)
> +{
> + struct video_device *video_dev =
> + media_entity_to_video_device(link->sink->entity);
> + struct sun6i_isp_device *isp_dev = video_get_drvdata(video_dev);
> + struct v4l2_device *v4l2_dev = &isp_dev->v4l2.v4l2_dev;
> + unsigned int capture_width, capture_height;
> + unsigned int proc_width, proc_height;
> +
> + sun6i_isp_capture_dimensions(isp_dev, &capture_width, &capture_height);
> + sun6i_isp_proc_dimensions(isp_dev, &proc_width, &proc_height);
> +
> + /* No cropping/scaling is supported (yet). */
> + if (capture_width != proc_width || capture_height != proc_height) {
> + v4l2_err(v4l2_dev,
> + "invalid input/output dimensions: %ux%u/%ux%u\n",
> + proc_width, proc_height, capture_width,
> + capture_height);
> + return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
> +static const struct media_entity_operations sun6i_isp_capture_entity_ops = {
> + .link_validate = sun6i_isp_capture_link_validate,
> +};
> +
> +/* Capture */
> +
> +int sun6i_isp_capture_setup(struct sun6i_isp_device *isp_dev)
> +{
> + struct sun6i_isp_capture *capture = &isp_dev->capture;
> + struct sun6i_isp_capture_state *state = &capture->state;
> + struct v4l2_device *v4l2_dev = &isp_dev->v4l2.v4l2_dev;
> + struct v4l2_subdev *proc_subdev = &isp_dev->proc.subdev;
> + struct video_device *video_dev = &capture->video_dev;
> + struct vb2_queue *queue = &capture->queue;
> + struct media_pad *pad = &capture->pad;
> + struct v4l2_format *format = &capture->format;
> + struct v4l2_pix_format *pix_format = &format->fmt.pix;
> + int ret;
> +
> + /* State */
> +
> + INIT_LIST_HEAD(&state->queue);
> + spin_lock_init(&state->lock);
> +
> + /* Media Entity */
> +
> + video_dev->entity.ops = &sun6i_isp_capture_entity_ops;
> +
> + /* Media Pads */
> +
> + pad->flags = MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_MUST_CONNECT;
> +
> + ret = media_entity_pads_init(&video_dev->entity, 1, pad);
> + if (ret)
> + goto error_mutex;

return ret;

> +
> + /* Queue */
> +
> + mutex_init(&capture->lock);
> +
> + queue->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> + queue->io_modes = VB2_MMAP | VB2_DMABUF;
> + queue->buf_struct_size = sizeof(struct sun6i_isp_buffer);
> + queue->ops = &sun6i_isp_capture_queue_ops;
> + queue->mem_ops = &vb2_dma_contig_memops;
> + queue->min_buffers_needed = 2;
> + queue->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> + queue->lock = &capture->lock;
> + queue->dev = isp_dev->dev;
> + queue->drv_priv = isp_dev;
> +
> + ret = vb2_queue_init(queue);
> + if (ret) {
> + v4l2_err(v4l2_dev, "failed to initialize vb2 queue: %d\n", ret);
> + goto error_media_entity;
> + }
> +
> + /* V4L2 Format */
> +
> + format->type = queue->type;
> + pix_format->pixelformat = sun6i_isp_capture_formats[0].pixelformat;
> + pix_format->width = 1280;
> + pix_format->height = 720;
> +
> + sun6i_isp_capture_format_prepare(format);
> +
> + /* Video Device */
> +
> + strscpy(video_dev->name, SUN6I_ISP_CAPTURE_NAME,
> + sizeof(video_dev->name));
> + video_dev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
> + video_dev->vfl_dir = VFL_DIR_RX;
> + video_dev->release = video_device_release_empty;
> + video_dev->fops = &sun6i_isp_capture_fops;
> + video_dev->ioctl_ops = &sun6i_isp_capture_ioctl_ops;
> + video_dev->v4l2_dev = v4l2_dev;
> + video_dev->queue = queue;
> + video_dev->lock = &capture->lock;
> +
> + video_set_drvdata(video_dev, isp_dev);
> +
> + ret = video_register_device(video_dev, VFL_TYPE_VIDEO, -1);
> + if (ret) {
> + v4l2_err(v4l2_dev, "failed to register video device: %d\n",
> + ret);
> + goto error_media_entity;
> + }
> +
> + v4l2_info(v4l2_dev, "device %s registered as %s\n", video_dev->name,
> + video_device_node_name(video_dev));

This isn't really driver specific. I'd drop it.

> +
> + /* Media Pad Link */
> +
> + ret = media_create_pad_link(&proc_subdev->entity,
> + SUN6I_ISP_PROC_PAD_SOURCE,
> + &video_dev->entity, 0,
> + MEDIA_LNK_FL_ENABLED |
> + MEDIA_LNK_FL_IMMUTABLE);
> + if (ret < 0) {
> + v4l2_err(v4l2_dev, "failed to create %s:%u -> %s:%u link\n",
> + proc_subdev->entity.name, SUN6I_ISP_PROC_PAD_SOURCE,
> + video_dev->entity.name, 0);

This error message printing would be better to be added to
media_create_pad_link().

> + goto error_video_device;
> + }
> +
> + return 0;
> +
> +error_video_device:
> + vb2_video_unregister_device(video_dev);
> +
> +error_media_entity:
> + media_entity_cleanup(&video_dev->entity);
> +
> +error_mutex:
> + mutex_destroy(&capture->lock);
> +
> + return ret;
> +}
> +
> +void sun6i_isp_capture_cleanup(struct sun6i_isp_device *isp_dev)
> +{
> + struct sun6i_isp_capture *capture = &isp_dev->capture;
> + struct video_device *video_dev = &capture->video_dev;
> +
> + vb2_video_unregister_device(video_dev);
> + media_entity_cleanup(&video_dev->entity);
> + mutex_destroy(&capture->lock);
> +}
> diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h
> new file mode 100644
> index 000000000000..0e3e4fa7a0f4
> --- /dev/null
> +++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h
> @@ -0,0 +1,78 @@
> +/* SPDX-License-Identifier: GPL-2.0+ */
> +/*
> + * Copyright 2021-2022 Bootlin
> + * Author: Paul Kocialkowski <paul.kocialkowski@xxxxxxxxxxx>
> + */
> +
> +#ifndef _SUN6I_ISP_CAPTURE_H_
> +#define _SUN6I_ISP_CAPTURE_H_
> +
> +#include <media/v4l2-device.h>
> +
> +#define SUN6I_ISP_CAPTURE_NAME "sun6i-isp-capture"
> +
> +#define SUN6I_ISP_CAPTURE_WIDTH_MIN 16
> +#define SUN6I_ISP_CAPTURE_WIDTH_MAX 3264
> +#define SUN6I_ISP_CAPTURE_HEIGHT_MIN 16
> +#define SUN6I_ISP_CAPTURE_HEIGHT_MAX 2448
> +
> +struct sun6i_isp_device;
> +
> +struct sun6i_isp_capture_format {
> + u32 pixelformat;
> + u8 output_format;
> +};
> +
> +#undef current
> +struct sun6i_isp_capture_state {
> + struct list_head queue;
> + spinlock_t lock; /* Queue and buffers lock. */
> +
> + struct sun6i_isp_buffer *pending;
> + struct sun6i_isp_buffer *current;
> + struct sun6i_isp_buffer *complete;
> +
> + unsigned int sequence;
> + bool streaming;
> +};
> +
> +struct sun6i_isp_capture {
> + struct sun6i_isp_capture_state state;
> +
> + struct video_device video_dev;
> + struct vb2_queue queue;
> + struct mutex lock; /* Queue lock. */
> + struct media_pad pad;
> +
> + struct v4l2_format format;
> +};
> +
> +/* Helpers */
> +
> +void sun6i_isp_capture_dimensions(struct sun6i_isp_device *isp_dev,
> + unsigned int *width, unsigned int *height);
> +void sun6i_isp_capture_format(struct sun6i_isp_device *isp_dev,
> + u32 *pixelformat);
> +
> +/* Format */
> +
> +const struct sun6i_isp_capture_format *
> +sun6i_isp_capture_format_find(u32 pixelformat);
> +
> +/* Capture */
> +
> +void sun6i_isp_capture_configure(struct sun6i_isp_device *isp_dev);
> +
> +/* State */
> +
> +void sun6i_isp_capture_state_update(struct sun6i_isp_device *isp_dev,
> + bool *update);
> +void sun6i_isp_capture_state_complete(struct sun6i_isp_device *isp_dev);
> +void sun6i_isp_capture_finish(struct sun6i_isp_device *isp_dev);
> +
> +/* Capture */
> +
> +int sun6i_isp_capture_setup(struct sun6i_isp_device *isp_dev);
> +void sun6i_isp_capture_cleanup(struct sun6i_isp_device *isp_dev);
> +
> +#endif
> diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c
> new file mode 100644
> index 000000000000..8f6ee73b3bdc
> --- /dev/null
> +++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c
> @@ -0,0 +1,573 @@
> +// SPDX-License-Identifier: GPL-2.0+
> +/*
> + * Copyright 2021-2022 Bootlin
> + * Author: Paul Kocialkowski <paul.kocialkowski@xxxxxxxxxxx>
> + */
> +
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/v4l2-mc.h>
> +#include <media/videobuf2-vmalloc.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#include "sun6i_isp.h"
> +#include "sun6i_isp_params.h"
> +#include "sun6i_isp_reg.h"
> +#include "uapi/sun6i-isp-config.h"
> +
> +/* Params */
> +
> +static const struct sun6i_isp_params_config sun6i_isp_params_config_default = {
> + .modules_used = SUN6I_ISP_MODULE_BAYER,
> +
> + .bayer = {
> + .offset_r = 32,
> + .offset_gr = 32,
> + .offset_gb = 32,
> + .offset_b = 32,
> +
> + .gain_r = 256,
> + .gain_gr = 256,
> + .gain_gb = 256,
> + .gain_b = 256,
> +
> + },
> +
> + .bdnf = {
> + .in_dis_min = 8,
> + .in_dis_max = 16,
> +
> + .coefficients_g = { 15, 4, 1 },
> + .coefficients_rb = { 15, 4 },
> + },
> +};
> +
> +static void sun6i_isp_params_configure_ob(struct sun6i_isp_device *isp_dev)
> +{
> + unsigned int width, height;
> +
> + sun6i_isp_proc_dimensions(isp_dev, &width, &height);
> +
> + sun6i_isp_load_write(isp_dev, SUN6I_ISP_OB_SIZE_REG,
> + SUN6I_ISP_OB_SIZE_WIDTH(width) |
> + SUN6I_ISP_OB_SIZE_HEIGHT(height));
> +
> + sun6i_isp_load_write(isp_dev, SUN6I_ISP_OB_VALID_REG,
> + SUN6I_ISP_OB_VALID_WIDTH(width) |
> + SUN6I_ISP_OB_VALID_HEIGHT(height));
> +
> + sun6i_isp_load_write(isp_dev, SUN6I_ISP_OB_SRC0_VALID_START_REG,
> + SUN6I_ISP_OB_SRC0_VALID_START_HORZ(0) |
> + SUN6I_ISP_OB_SRC0_VALID_START_VERT(0));
> +}
> +
> +static void sun6i_isp_params_configure_ae(struct sun6i_isp_device *isp_dev)
> +{
> + /* These are default values that need to be set to get an output. */
> +
> + sun6i_isp_load_write(isp_dev, SUN6I_ISP_AE_CFG_REG,
> + SUN6I_ISP_AE_CFG_LOW_BRI_TH(0xff) |
> + SUN6I_ISP_AE_CFG_HORZ_NUM(8) |
> + SUN6I_ISP_AE_CFG_HIGH_BRI_TH(0xf00) |
> + SUN6I_ISP_AE_CFG_VERT_NUM(8));
> +}
> +
> +static void
> +sun6i_isp_params_configure_bayer(struct sun6i_isp_device *isp_dev,
> + const struct sun6i_isp_params_config *config)
> +{
> + const struct sun6i_isp_params_config_bayer *bayer = &config->bayer;
> +
> + sun6i_isp_load_write(isp_dev, SUN6I_ISP_BAYER_OFFSET0_REG,
> + SUN6I_ISP_BAYER_OFFSET0_R(bayer->offset_r) |
> + SUN6I_ISP_BAYER_OFFSET0_GR(bayer->offset_gr));
> +
> + sun6i_isp_load_write(isp_dev, SUN6I_ISP_BAYER_OFFSET1_REG,
> + SUN6I_ISP_BAYER_OFFSET1_GB(bayer->offset_gb) |
> + SUN6I_ISP_BAYER_OFFSET1_B(bayer->offset_b));
> +
> + sun6i_isp_load_write(isp_dev, SUN6I_ISP_BAYER_GAIN0_REG,
> + SUN6I_ISP_BAYER_GAIN0_R(bayer->gain_r) |
> + SUN6I_ISP_BAYER_GAIN0_GR(bayer->gain_gr));
> +
> + sun6i_isp_load_write(isp_dev, SUN6I_ISP_BAYER_GAIN1_REG,
> + SUN6I_ISP_BAYER_GAIN1_GB(bayer->gain_gb) |
> + SUN6I_ISP_BAYER_GAIN1_B(bayer->gain_b));
> +}
> +
> +static void sun6i_isp_params_configure_wb(struct sun6i_isp_device *isp_dev)
> +{
> + /* These are default values that need to be set to get an output. */
> +
> + sun6i_isp_load_write(isp_dev, SUN6I_ISP_WB_GAIN0_REG,
> + SUN6I_ISP_WB_GAIN0_R(256) |
> + SUN6I_ISP_WB_GAIN0_GR(256));
> +
> + sun6i_isp_load_write(isp_dev, SUN6I_ISP_WB_GAIN1_REG,
> + SUN6I_ISP_WB_GAIN1_GB(256) |
> + SUN6I_ISP_WB_GAIN1_B(256));
> +
> + sun6i_isp_load_write(isp_dev, SUN6I_ISP_WB_CFG_REG,
> + SUN6I_ISP_WB_CFG_CLIP(0xfff));
> +}
> +
> +static void sun6i_isp_params_configure_base(struct sun6i_isp_device *isp_dev)
> +{
> + sun6i_isp_params_configure_ae(isp_dev);
> + sun6i_isp_params_configure_ob(isp_dev);
> + sun6i_isp_params_configure_wb(isp_dev);
> +}
> +
> +static void
> +sun6i_isp_params_configure_bdnf(struct sun6i_isp_device *isp_dev,
> + const struct sun6i_isp_params_config *config)
> +{
> + const struct sun6i_isp_params_config_bdnf *bdnf = &config->bdnf;
> +
> + sun6i_isp_load_write(isp_dev, SUN6I_ISP_BDNF_CFG_REG,
> + SUN6I_ISP_BDNF_CFG_IN_DIS_MIN(bdnf->in_dis_min) |
> + SUN6I_ISP_BDNF_CFG_IN_DIS_MAX(bdnf->in_dis_max));
> +
> + sun6i_isp_load_write(isp_dev, SUN6I_ISP_BDNF_COEF_RB_REG,
> + SUN6I_ISP_BDNF_COEF_RB(0, bdnf->coefficients_rb[0]) |
> + SUN6I_ISP_BDNF_COEF_RB(1, bdnf->coefficients_rb[1]) |
> + SUN6I_ISP_BDNF_COEF_RB(2, bdnf->coefficients_rb[2]) |
> + SUN6I_ISP_BDNF_COEF_RB(3, bdnf->coefficients_rb[3]) |
> + SUN6I_ISP_BDNF_COEF_RB(4, bdnf->coefficients_rb[4]));
> +
> + sun6i_isp_load_write(isp_dev, SUN6I_ISP_BDNF_COEF_G_REG,
> + SUN6I_ISP_BDNF_COEF_G(0, bdnf->coefficients_g[0]) |
> + SUN6I_ISP_BDNF_COEF_G(1, bdnf->coefficients_g[1]) |
> + SUN6I_ISP_BDNF_COEF_G(2, bdnf->coefficients_g[2]) |
> + SUN6I_ISP_BDNF_COEF_G(3, bdnf->coefficients_g[3]) |
> + SUN6I_ISP_BDNF_COEF_G(4, bdnf->coefficients_g[4]) |
> + SUN6I_ISP_BDNF_COEF_G(5, bdnf->coefficients_g[5]) |
> + SUN6I_ISP_BDNF_COEF_G(6, bdnf->coefficients_g[6]));
> +}
> +
> +static void
> +sun6i_isp_params_configure_modules(struct sun6i_isp_device *isp_dev,
> + const struct sun6i_isp_params_config *config)
> +{
> + u32 value;
> +
> + if (config->modules_used & SUN6I_ISP_MODULE_BDNF)
> + sun6i_isp_params_configure_bdnf(isp_dev, config);
> +
> + if (config->modules_used & SUN6I_ISP_MODULE_BAYER)
> + sun6i_isp_params_configure_bayer(isp_dev, config);
> +
> + value = sun6i_isp_load_read(isp_dev, SUN6I_ISP_MODULE_EN_REG);
> + /* Clear all modules but keep input configuration. */
> + value &= SUN6I_ISP_MODULE_EN_SRC0 | SUN6I_ISP_MODULE_EN_SRC1;
> +
> + if (config->modules_used & SUN6I_ISP_MODULE_BDNF)
> + value |= SUN6I_ISP_MODULE_EN_BDNF;
> +
> + /* Bayer stage is always enabled. */
> +
> + sun6i_isp_load_write(isp_dev, SUN6I_ISP_MODULE_EN_REG, value);
> +}
> +
> +void sun6i_isp_params_configure(struct sun6i_isp_device *isp_dev)
> +{
> + struct sun6i_isp_params_state *state = &isp_dev->params.state;
> + unsigned long flags;
> +
> + spin_lock_irqsave(&state->lock, flags);
> +
> + sun6i_isp_params_configure_base(isp_dev);
> +
> + /* Default config is only applied at the very first stream start. */
> + if (state->configured)
> + goto complete;
> +
> + sun6i_isp_params_configure_modules(isp_dev,

Indentation. Doesn't checkpatch.pl complain?

> + &sun6i_isp_params_config_default);
> +
> + state->configured = true;
> +
> +complete:
> + spin_unlock_irqrestore(&state->lock, flags);
> +}
> +
> +/* State */
> +
> +static void sun6i_isp_params_state_cleanup(struct sun6i_isp_device *isp_dev,
> + bool error)
> +{
> + struct sun6i_isp_params_state *state = &isp_dev->params.state;
> + struct sun6i_isp_buffer *isp_buffer;
> + struct vb2_buffer *vb2_buffer;
> + unsigned long flags;
> +
> + spin_lock_irqsave(&state->lock, flags);
> +
> + if (state->pending) {
> + vb2_buffer = &state->pending->v4l2_buffer.vb2_buf;
> + vb2_buffer_done(vb2_buffer, error ? VB2_BUF_STATE_ERROR :
> + VB2_BUF_STATE_QUEUED);
> + }
> +
> + list_for_each_entry(isp_buffer, &state->queue, list) {
> + vb2_buffer = &isp_buffer->v4l2_buffer.vb2_buf;
> + vb2_buffer_done(vb2_buffer, error ? VB2_BUF_STATE_ERROR :
> + VB2_BUF_STATE_QUEUED);
> + }
> +
> + INIT_LIST_HEAD(&state->queue);
> +
> + spin_unlock_irqrestore(&state->lock, flags);
> +}
> +
> +void sun6i_isp_params_state_update(struct sun6i_isp_device *isp_dev,
> + bool *update)
> +{
> + struct sun6i_isp_params_state *state = &isp_dev->params.state;
> + struct sun6i_isp_buffer *isp_buffer;
> + struct vb2_buffer *vb2_buffer;
> + const struct sun6i_isp_params_config *config;
> + unsigned long flags;
> +
> + spin_lock_irqsave(&state->lock, flags);
> +
> + if (list_empty(&state->queue))
> + goto complete;
> +
> + if (state->pending)
> + goto complete;
> +
> + isp_buffer = list_first_entry(&state->queue, struct sun6i_isp_buffer,
> + list);
> +
> + vb2_buffer = &isp_buffer->v4l2_buffer.vb2_buf;
> + config = vb2_plane_vaddr(vb2_buffer, 0);
> +
> + sun6i_isp_params_configure_modules(isp_dev, config);
> +
> + list_del(&isp_buffer->list);
> +
> + state->pending = isp_buffer;
> +
> + if (update)
> + *update = true;
> +
> +complete:
> + spin_unlock_irqrestore(&state->lock, flags);
> +}
> +
> +void sun6i_isp_params_state_complete(struct sun6i_isp_device *isp_dev)
> +{
> + struct sun6i_isp_params_state *state = &isp_dev->params.state;
> + struct sun6i_isp_buffer *isp_buffer;
> + struct vb2_buffer *vb2_buffer;
> + unsigned long flags;
> +
> + spin_lock_irqsave(&state->lock, flags);
> +
> + if (!state->pending)
> + goto complete;
> +
> + isp_buffer = state->pending;
> + vb2_buffer = &isp_buffer->v4l2_buffer.vb2_buf;
> +
> + vb2_buffer->timestamp = ktime_get_ns();
> +
> + /* Parameters will be applied starting from the next frame. */
> + isp_buffer->v4l2_buffer.sequence = isp_dev->capture.state.sequence + 1;
> +
> + vb2_buffer_done(vb2_buffer, VB2_BUF_STATE_DONE);
> +
> + state->pending = NULL;
> +
> +complete:
> + spin_unlock_irqrestore(&state->lock, flags);
> +}
> +
> +/* Queue */
> +
> +static int sun6i_isp_params_queue_setup(struct vb2_queue *queue,
> + unsigned int *buffers_count,
> + unsigned int *planes_count,
> + unsigned int sizes[],
> + struct device *alloc_devs[])
> +{
> + struct sun6i_isp_device *isp_dev = vb2_get_drv_priv(queue);
> + unsigned int size = isp_dev->params.format.fmt.meta.buffersize;
> +
> + if (*planes_count)
> + return sizes[0] < size ? -EINVAL : 0;
> +
> + *planes_count = 1;
> + sizes[0] = size;
> +
> + return 0;
> +}
> +
> +static int sun6i_isp_params_buffer_prepare(struct vb2_buffer *vb2_buffer)
> +{
> + struct sun6i_isp_device *isp_dev =
> + vb2_get_drv_priv(vb2_buffer->vb2_queue);
> + struct v4l2_device *v4l2_dev = &isp_dev->v4l2.v4l2_dev;
> + unsigned int size = isp_dev->params.format.fmt.meta.buffersize;
> +
> + if (vb2_plane_size(vb2_buffer, 0) < size) {
> + v4l2_err(v4l2_dev, "buffer too small (%lu < %u)\n",
> + vb2_plane_size(vb2_buffer, 0), size);
> + return -EINVAL;
> + }
> +
> + vb2_set_plane_payload(vb2_buffer, 0, size);
> +
> + return 0;
> +}
> +
> +static void sun6i_isp_params_buffer_queue(struct vb2_buffer *vb2_buffer)
> +{
> + struct sun6i_isp_device *isp_dev =
> + vb2_get_drv_priv(vb2_buffer->vb2_queue);
> + struct sun6i_isp_params_state *state = &isp_dev->params.state;
> + struct vb2_v4l2_buffer *v4l2_buffer = to_vb2_v4l2_buffer(vb2_buffer);
> + struct sun6i_isp_buffer *isp_buffer =
> + container_of(v4l2_buffer, struct sun6i_isp_buffer, v4l2_buffer);
> + bool capture_streaming = isp_dev->capture.state.streaming;
> + unsigned long flags;
> +
> + spin_lock_irqsave(&state->lock, flags);
> + list_add_tail(&isp_buffer->list, &state->queue);
> + spin_unlock_irqrestore(&state->lock, flags);
> +
> + if (state->streaming && capture_streaming)
> + sun6i_isp_state_update(isp_dev, false);
> +}
> +
> +static int sun6i_isp_params_start_streaming(struct vb2_queue *queue,
> + unsigned int count)
> +{
> + struct sun6i_isp_device *isp_dev = vb2_get_drv_priv(queue);
> + struct sun6i_isp_params_state *state = &isp_dev->params.state;
> + bool capture_streaming = isp_dev->capture.state.streaming;
> +
> + state->streaming = true;
> +
> + /*
> + * Update the state as soon as possible if capture is streaming,
> + * otherwise it will be applied when capture starts streaming.
> + */
> +
> + if (capture_streaming)
> + sun6i_isp_state_update(isp_dev, false);
> +
> + return 0;
> +}
> +
> +static void sun6i_isp_params_stop_streaming(struct vb2_queue *queue)
> +{
> + struct sun6i_isp_device *isp_dev = vb2_get_drv_priv(queue);
> + struct sun6i_isp_params_state *state = &isp_dev->params.state;
> +
> + state->streaming = false;
> + sun6i_isp_params_state_cleanup(isp_dev, true);
> +}
> +
> +static const struct vb2_ops sun6i_isp_params_queue_ops = {
> + .queue_setup = sun6i_isp_params_queue_setup,
> + .buf_prepare = sun6i_isp_params_buffer_prepare,
> + .buf_queue = sun6i_isp_params_buffer_queue,
> + .start_streaming = sun6i_isp_params_start_streaming,
> + .stop_streaming = sun6i_isp_params_stop_streaming,
> + .wait_prepare = vb2_ops_wait_prepare,
> + .wait_finish = vb2_ops_wait_finish,
> +};
> +
> +/* Video Device */
> +
> +static int sun6i_isp_params_querycap(struct file *file, void *private,
> + struct v4l2_capability *capability)
> +{
> + struct sun6i_isp_device *isp_dev = video_drvdata(file);
> + struct video_device *video_dev = &isp_dev->params.video_dev;
> +
> + strscpy(capability->driver, SUN6I_ISP_NAME, sizeof(capability->driver));
> + strscpy(capability->card, video_dev->name, sizeof(capability->card));
> + snprintf(capability->bus_info, sizeof(capability->bus_info),
> + "platform:%s", dev_name(isp_dev->dev));

This is no longer needed with commit
2a792fd5cf669d379d82354a99998d9ae9ff7d14 .

> +
> + return 0;
> +}
> +
> +static int sun6i_isp_params_enum_fmt(struct file *file, void *private,
> + struct v4l2_fmtdesc *fmtdesc)
> +{
> + struct sun6i_isp_device *isp_dev = video_drvdata(file);
> + struct v4l2_meta_format *params_format =
> + &isp_dev->params.format.fmt.meta;
> +
> + if (fmtdesc->index > 0)
> + return -EINVAL;
> +
> + fmtdesc->pixelformat = params_format->dataformat;
> +
> + return 0;
> +}
> +
> +static int sun6i_isp_params_g_fmt(struct file *file, void *private,
> + struct v4l2_format *format)
> +{
> + struct sun6i_isp_device *isp_dev = video_drvdata(file);
> +
> + *format = isp_dev->params.format;
> +
> + return 0;
> +}
> +
> +static const struct v4l2_ioctl_ops sun6i_isp_params_ioctl_ops = {
> + .vidioc_querycap = sun6i_isp_params_querycap,
> +
> + .vidioc_enum_fmt_meta_out = sun6i_isp_params_enum_fmt,
> + .vidioc_g_fmt_meta_out = sun6i_isp_params_g_fmt,
> + .vidioc_s_fmt_meta_out = sun6i_isp_params_g_fmt,
> + .vidioc_try_fmt_meta_out = sun6i_isp_params_g_fmt,
> +
> + .vidioc_create_bufs = vb2_ioctl_create_bufs,
> + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> + .vidioc_reqbufs = vb2_ioctl_reqbufs,
> + .vidioc_querybuf = vb2_ioctl_querybuf,
> + .vidioc_expbuf = vb2_ioctl_expbuf,
> + .vidioc_qbuf = vb2_ioctl_qbuf,
> + .vidioc_dqbuf = vb2_ioctl_dqbuf,
> + .vidioc_streamon = vb2_ioctl_streamon,
> + .vidioc_streamoff = vb2_ioctl_streamoff,
> +
> + .vidioc_log_status = v4l2_ctrl_log_status,
> + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> + .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> +};
> +
> +static const struct v4l2_file_operations sun6i_isp_params_fops = {
> + .owner = THIS_MODULE,
> + .unlocked_ioctl = video_ioctl2,
> + .open = v4l2_fh_open,
> + .release = vb2_fop_release,
> + .mmap = vb2_fop_mmap,
> + .poll = vb2_fop_poll,
> +};
> +
> +/* Params */
> +
> +int sun6i_isp_params_setup(struct sun6i_isp_device *isp_dev)
> +{
> + struct sun6i_isp_params *params = &isp_dev->params;
> + struct sun6i_isp_params_state *state = &params->state;
> + struct v4l2_device *v4l2_dev = &isp_dev->v4l2.v4l2_dev;
> + struct v4l2_subdev *proc_subdev = &isp_dev->proc.subdev;
> + struct video_device *video_dev = &params->video_dev;
> + struct vb2_queue *queue = &isp_dev->params.queue;
> + struct media_pad *pad = &isp_dev->params.pad;
> + struct v4l2_format *format = &isp_dev->params.format;
> + struct v4l2_meta_format *params_format = &format->fmt.meta;
> + int ret;
> +
> + /* State */
> +
> + INIT_LIST_HEAD(&state->queue);
> + spin_lock_init(&state->lock);
> +
> + /* Media Pads */
> +
> + pad->flags = MEDIA_PAD_FL_SOURCE | MEDIA_PAD_FL_MUST_CONNECT;
> +
> + ret = media_entity_pads_init(&video_dev->entity, 1, pad);
> + if (ret)
> + goto error_mutex;
> +
> + /* Queue */
> +
> + mutex_init(&params->lock);
> +
> + queue->type = V4L2_BUF_TYPE_META_OUTPUT;
> + queue->io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF;
> + queue->buf_struct_size = sizeof(struct sun6i_isp_buffer);
> + queue->ops = &sun6i_isp_params_queue_ops;
> + queue->mem_ops = &vb2_vmalloc_memops;
> + queue->min_buffers_needed = 1;
> + queue->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> + queue->lock = &params->lock;
> + queue->dev = isp_dev->dev;
> + queue->drv_priv = isp_dev;
> +
> + ret = vb2_queue_init(queue);
> + if (ret) {
> + v4l2_err(v4l2_dev, "failed to initialize vb2 queue: %d\n", ret);
> + goto error_media_entity;
> + }
> +
> + /* V4L2 Format */
> +
> + format->type = queue->type;
> + params_format->dataformat = V4L2_META_FMT_SUN6I_ISP_PARAMS;
> + params_format->buffersize = sizeof(struct sun6i_isp_params_config);
> +
> + /* Video Device */
> +
> + strscpy(video_dev->name, SUN6I_ISP_PARAMS_NAME,
> + sizeof(video_dev->name));
> + video_dev->device_caps = V4L2_CAP_META_OUTPUT | V4L2_CAP_STREAMING;
> + video_dev->vfl_dir = VFL_DIR_TX;
> + video_dev->release = video_device_release_empty;
> + video_dev->fops = &sun6i_isp_params_fops;
> + video_dev->ioctl_ops = &sun6i_isp_params_ioctl_ops;
> + video_dev->v4l2_dev = v4l2_dev;
> + video_dev->queue = queue;
> + video_dev->lock = &params->lock;
> +
> + video_set_drvdata(video_dev, isp_dev);
> +
> + ret = video_register_device(video_dev, VFL_TYPE_VIDEO, -1);
> + if (ret) {
> + v4l2_err(v4l2_dev, "failed to register video device: %d\n",
> + ret);
> + goto error_media_entity;
> + }
> +
> + v4l2_info(v4l2_dev, "device %s registered as %s\n", video_dev->name,
> + video_device_node_name(video_dev));
> +
> + /* Media Pad Link */
> +
> + ret = media_create_pad_link(&video_dev->entity, 0,
> + &proc_subdev->entity,
> + SUN6I_ISP_PROC_PAD_SINK_PARAMS,
> + MEDIA_LNK_FL_ENABLED |
> + MEDIA_LNK_FL_IMMUTABLE);
> + if (ret < 0) {
> + v4l2_err(v4l2_dev, "failed to create %s:%u -> %s:%u link\n",
> + video_dev->entity.name, 0, proc_subdev->entity.name,
> + SUN6I_ISP_PROC_PAD_SINK_PARAMS);
> + goto error_video_device;
> + }
> +
> + return 0;
> +
> +error_video_device:
> + vb2_video_unregister_device(video_dev);
> +
> +error_media_entity:
> + media_entity_cleanup(&video_dev->entity);
> +
> +error_mutex:
> + mutex_destroy(&params->lock);
> +
> + return ret;
> +}
> +
> +void sun6i_isp_params_cleanup(struct sun6i_isp_device *isp_dev)
> +{
> + struct sun6i_isp_params *params = &isp_dev->params;
> + struct video_device *video_dev = &params->video_dev;
> +
> + vb2_video_unregister_device(video_dev);
> + media_entity_cleanup(&video_dev->entity);
> + mutex_destroy(&params->lock);
> +}
> diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h
> new file mode 100644
> index 000000000000..50f10f879c42
> --- /dev/null
> +++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h
> @@ -0,0 +1,52 @@
> +/* SPDX-License-Identifier: GPL-2.0+ */
> +/*
> + * Copyright 2021-2022 Bootlin
> + * Author: Paul Kocialkowski <paul.kocialkowski@xxxxxxxxxxx>
> + */
> +
> +#ifndef _SUN6I_ISP_PARAMS_H_
> +#define _SUN6I_ISP_PARAMS_H_
> +
> +#include <media/v4l2-device.h>
> +
> +#define SUN6I_ISP_PARAMS_NAME "sun6i-isp-params"
> +
> +struct sun6i_isp_device;
> +
> +struct sun6i_isp_params_state {
> + struct list_head queue; /* Queue and buffers lock. */
> + spinlock_t lock;
> +
> + struct sun6i_isp_buffer *pending;
> +
> + bool configured;
> + bool streaming;
> +};
> +
> +struct sun6i_isp_params {
> + struct sun6i_isp_params_state state;
> +
> + struct video_device video_dev;
> + struct vb2_queue queue;
> + struct mutex lock; /* Queue lock. */
> + struct media_pad pad;
> +
> + struct v4l2_format format;
> +};
> +
> +/* Params */
> +
> +void sun6i_isp_params_configure(struct sun6i_isp_device *isp_dev);
> +
> +/* State */
> +
> +void sun6i_isp_params_state_update(struct sun6i_isp_device *isp_dev,
> + bool *update);
> +void sun6i_isp_params_state_complete(struct sun6i_isp_device *isp_dev);
> +
> +/* Params */
> +
> +int sun6i_isp_params_setup(struct sun6i_isp_device *isp_dev);
> +void sun6i_isp_params_cleanup(struct sun6i_isp_device *isp_dev);
> +
> +#endif
> diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c
> new file mode 100644
> index 000000000000..edaf5ee8b61c
> --- /dev/null
> +++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c
> @@ -0,0 +1,578 @@
> +// SPDX-License-Identifier: GPL-2.0+
> +/*
> + * Copyright 2021-2022 Bootlin
> + * Author: Paul Kocialkowski <paul.kocialkowski@xxxxxxxxxxx>
> + */
> +
> +#include <linux/pm_runtime.h>
> +#include <linux/regmap.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-fwnode.h>
> +
> +#include "sun6i_isp.h"
> +#include "sun6i_isp_capture.h"
> +#include "sun6i_isp_params.h"
> +#include "sun6i_isp_proc.h"
> +#include "sun6i_isp_reg.h"
> +
> +/* Helpers */
> +
> +void sun6i_isp_proc_dimensions(struct sun6i_isp_device *isp_dev,
> + unsigned int *width, unsigned int *height)
> +{
> + if (width)
> + *width = isp_dev->proc.mbus_format.width;
> + if (height)
> + *height = isp_dev->proc.mbus_format.height;
> +}
> +
> +/* Format */
> +
> +static const struct sun6i_isp_proc_format sun6i_isp_proc_formats[] = {
> + {
> + .mbus_code = MEDIA_BUS_FMT_SBGGR8_1X8,
> + .input_format = SUN6I_ISP_INPUT_FMT_RAW_BGGR,
> + },
> + {
> + .mbus_code = MEDIA_BUS_FMT_SGBRG8_1X8,
> + .input_format = SUN6I_ISP_INPUT_FMT_RAW_GBRG,
> + },
> + {
> + .mbus_code = MEDIA_BUS_FMT_SGRBG8_1X8,
> + .input_format = SUN6I_ISP_INPUT_FMT_RAW_GRBG,
> + },
> + {
> + .mbus_code = MEDIA_BUS_FMT_SRGGB8_1X8,
> + .input_format = SUN6I_ISP_INPUT_FMT_RAW_RGGB,
> + },
> +
> + {
> + .mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10,
> + .input_format = SUN6I_ISP_INPUT_FMT_RAW_BGGR,
> + },
> + {
> + .mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10,
> + .input_format = SUN6I_ISP_INPUT_FMT_RAW_GBRG,
> + },
> + {
> + .mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10,
> + .input_format = SUN6I_ISP_INPUT_FMT_RAW_GRBG,
> + },
> + {
> + .mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10,
> + .input_format = SUN6I_ISP_INPUT_FMT_RAW_RGGB,
> + },
> +};
> +
> +const struct sun6i_isp_proc_format *sun6i_isp_proc_format_find(u32 mbus_code)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(sun6i_isp_proc_formats); i++)
> + if (sun6i_isp_proc_formats[i].mbus_code == mbus_code)
> + return &sun6i_isp_proc_formats[i];
> +
> + return NULL;
> +}
> +
> +/* Processor */
> +
> +static void sun6i_isp_proc_irq_enable(struct sun6i_isp_device *isp_dev)
> +{
> + struct regmap *regmap = isp_dev->regmap;
> +
> + regmap_write(regmap, SUN6I_ISP_FE_INT_EN_REG,
> + SUN6I_ISP_FE_INT_EN_FINISH |
> + SUN6I_ISP_FE_INT_EN_START |
> + SUN6I_ISP_FE_INT_EN_PARA_SAVE |
> + SUN6I_ISP_FE_INT_EN_PARA_LOAD |
> + SUN6I_ISP_FE_INT_EN_SRC0_FIFO |
> + SUN6I_ISP_FE_INT_EN_ROT_FINISH);
> +}
> +
> +static void sun6i_isp_proc_irq_disable(struct sun6i_isp_device *isp_dev)
> +{
> + struct regmap *regmap = isp_dev->regmap;
> +
> + regmap_write(regmap, SUN6I_ISP_FE_INT_EN_REG, 0);
> +}
> +
> +static void sun6i_isp_proc_irq_clear(struct sun6i_isp_device *isp_dev)
> +{
> + struct regmap *regmap = isp_dev->regmap;
> +
> + regmap_write(regmap, SUN6I_ISP_FE_INT_EN_REG, 0);
> + regmap_write(regmap, SUN6I_ISP_FE_INT_STA_REG,
> + SUN6I_ISP_FE_INT_STA_CLEAR);
> +}
> +
> +static void sun6i_isp_proc_enable(struct sun6i_isp_device *isp_dev,
> + struct sun6i_isp_proc_source *source)
> +{
> + struct sun6i_isp_proc *proc = &isp_dev->proc;
> + struct regmap *regmap = isp_dev->regmap;
> + u8 mode;
> +
> + /* Frontend */
> +
> + if (source == &proc->source_csi0)
> + mode = SUN6I_ISP_SRC_MODE_CSI(0);
> + else
> + mode = SUN6I_ISP_SRC_MODE_CSI(1);
> +
> + regmap_write(regmap, SUN6I_ISP_FE_CFG_REG,
> + SUN6I_ISP_FE_CFG_EN | SUN6I_ISP_FE_CFG_SRC0_MODE(mode));
> +
> + regmap_write(regmap, SUN6I_ISP_FE_CTRL_REG,
> + SUN6I_ISP_FE_CTRL_VCAP_EN | SUN6I_ISP_FE_CTRL_PARA_READY);
> +}
> +
> +static void sun6i_isp_proc_disable(struct sun6i_isp_device *isp_dev)
> +{
> + struct regmap *regmap = isp_dev->regmap;
> +
> + /* Frontend */
> +
> + regmap_write(regmap, SUN6I_ISP_FE_CTRL_REG, 0);
> + regmap_write(regmap, SUN6I_ISP_FE_CFG_REG, 0);
> +}
> +
> +static void sun6i_isp_proc_configure(struct sun6i_isp_device *isp_dev)
> +{
> + struct v4l2_mbus_framefmt *mbus_format = &isp_dev->proc.mbus_format;
> + const struct sun6i_isp_proc_format *format;
> + u32 value;
> +
> + /* Module */
> +
> + value = sun6i_isp_load_read(isp_dev, SUN6I_ISP_MODULE_EN_REG);
> + value |= SUN6I_ISP_MODULE_EN_SRC0;
> + sun6i_isp_load_write(isp_dev, SUN6I_ISP_MODULE_EN_REG, value);
> +
> + /* Input */
> +
> + format = sun6i_isp_proc_format_find(mbus_format->code);
> + if (WARN_ON(!format))
> + return;
> +
> + sun6i_isp_load_write(isp_dev, SUN6I_ISP_MODE_REG,
> + SUN6I_ISP_MODE_INPUT_FMT(format->input_format) |
> + SUN6I_ISP_MODE_INPUT_YUV_SEQ(format->input_yuv_seq) |
> + SUN6I_ISP_MODE_SHARP(1) |
> + SUN6I_ISP_MODE_HIST(2));
> +}
> +
> +/* V4L2 Subdev */
> +
> +static int sun6i_isp_proc_s_stream(struct v4l2_subdev *subdev, int on)
> +{
> + struct sun6i_isp_device *isp_dev = v4l2_get_subdevdata(subdev);
> + struct sun6i_isp_proc *proc = &isp_dev->proc;
> + struct media_entity *proc_entity = &proc->subdev.entity;
> + struct device *dev = isp_dev->dev;
> + struct sun6i_isp_proc_source *source;
> + struct v4l2_subdev *source_subdev;
> + struct media_link *link;
> + /* Initialize to 0 to use both in disable label (ret != 0) and off. */
> + int ret = 0;
> +
> + /* Source */
> +
> + link = media_entity_get_single_enabled_link(proc_entity,
> + SUN6I_ISP_PROC_PAD_SINK_CSI);
> + if (IS_ERR(link)) {
> + dev_err(dev,
> + "zero or more than a single source connected to the bridge\n");
> + return PTR_ERR(link);
> + }
> +
> + source_subdev = media_entity_to_v4l2_subdev(link->source->entity);
> +
> + if (source_subdev == proc->source_csi0.subdev)
> + source = &proc->source_csi0;
> + else
> + source = &proc->source_csi1;
> +
> + if (!on) {
> + sun6i_isp_proc_irq_disable(isp_dev);
> + v4l2_subdev_call(source_subdev, video, s_stream, 0);
> + goto disable;
> + }
> +
> + /* PM */
> +
> + ret = pm_runtime_resume_and_get(dev);
> + if (ret < 0)
> + return ret;
> +
> + /* Clear */
> +
> + sun6i_isp_proc_irq_clear(isp_dev);
> +
> + /* Configure */
> +
> + sun6i_isp_tables_configure(isp_dev);
> + sun6i_isp_params_configure(isp_dev);
> + sun6i_isp_proc_configure(isp_dev);
> + sun6i_isp_capture_configure(isp_dev);
> +
> + /* State Update */
> +
> + sun6i_isp_state_update(isp_dev, true);
> +
> + /* Enable */
> +
> + sun6i_isp_proc_irq_enable(isp_dev);
> + sun6i_isp_proc_enable(isp_dev, source);
> +
> + ret = v4l2_subdev_call(source_subdev, video, s_stream, 1);
> + if (ret && ret != -ENOIOCTLCMD) {
> + sun6i_isp_proc_irq_disable(isp_dev);
> + goto disable;
> + }
> +
> + return 0;
> +
> +disable:
> + sun6i_isp_proc_disable(isp_dev);
> +
> + pm_runtime_put(dev);
> +
> + return ret;
> +}
> +
> +static const struct v4l2_subdev_video_ops sun6i_isp_proc_video_ops = {
> + .s_stream = sun6i_isp_proc_s_stream,
> +};
> +
> +static void
> +sun6i_isp_proc_mbus_format_prepare(struct v4l2_mbus_framefmt *mbus_format)
> +{
> + if (!sun6i_isp_proc_format_find(mbus_format->code))
> + mbus_format->code = sun6i_isp_proc_formats[0].mbus_code;
> +
> + mbus_format->field = V4L2_FIELD_NONE;
> + mbus_format->colorspace = V4L2_COLORSPACE_RAW;
> + mbus_format->quantization = V4L2_QUANTIZATION_DEFAULT;
> + mbus_format->xfer_func = V4L2_XFER_FUNC_DEFAULT;
> +}
> +
> +static int sun6i_isp_proc_init_cfg(struct v4l2_subdev *subdev,
> + struct v4l2_subdev_state *state)
> +{
> + struct sun6i_isp_device *isp_dev = v4l2_get_subdevdata(subdev);
> + unsigned int pad = SUN6I_ISP_PROC_PAD_SINK_CSI;
> + struct v4l2_mbus_framefmt *mbus_format =
> + v4l2_subdev_get_try_format(subdev, state, pad);
> + struct mutex *lock = &isp_dev->proc.lock;
> +
> + mutex_lock(lock);
> +
> + mbus_format->code = sun6i_isp_proc_formats[0].mbus_code;
> + mbus_format->width = 1280;
> + mbus_format->height = 720;
> +
> + sun6i_isp_proc_mbus_format_prepare(mbus_format);
> +
> + mutex_unlock(lock);
> +
> + return 0;
> +}
> +
> +static int
> +sun6i_isp_proc_enum_mbus_code(struct v4l2_subdev *subdev,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_mbus_code_enum *code_enum)
> +{
> + if (code_enum->index >= ARRAY_SIZE(sun6i_isp_proc_formats))
> + return -EINVAL;
> +
> + code_enum->code = sun6i_isp_proc_formats[code_enum->index].mbus_code;
> +
> + return 0;
> +}
> +
> +static int sun6i_isp_proc_get_fmt(struct v4l2_subdev *subdev,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_format *format)
> +{
> + struct sun6i_isp_device *isp_dev = v4l2_get_subdevdata(subdev);
> + struct v4l2_mbus_framefmt *mbus_format = &format->format;
> + struct mutex *lock = &isp_dev->proc.lock;
> +
> + mutex_lock(lock);
> +
> + if (format->which == V4L2_SUBDEV_FORMAT_TRY)
> + *mbus_format = *v4l2_subdev_get_try_format(subdev, state,
> + format->pad);
> + else
> + *mbus_format = isp_dev->proc.mbus_format;
> +
> + mutex_unlock(lock);
> +
> + return 0;
> +}
> +
> +static int sun6i_isp_proc_set_fmt(struct v4l2_subdev *subdev,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_format *format)
> +{
> + struct sun6i_isp_device *isp_dev = v4l2_get_subdevdata(subdev);
> + struct v4l2_mbus_framefmt *mbus_format = &format->format;
> + struct mutex *lock = &isp_dev->proc.lock;
> +
> + mutex_lock(lock);
> +
> + sun6i_isp_proc_mbus_format_prepare(mbus_format);
> +
> + if (format->which == V4L2_SUBDEV_FORMAT_TRY)
> + *v4l2_subdev_get_try_format(subdev, state, format->pad) =
> + *mbus_format;
> + else
> + isp_dev->proc.mbus_format = *mbus_format;
> +
> + mutex_unlock(lock);
> +
> + return 0;
> +}
> +
> +static const struct v4l2_subdev_pad_ops sun6i_isp_proc_pad_ops = {
> + .init_cfg = sun6i_isp_proc_init_cfg,
> + .enum_mbus_code = sun6i_isp_proc_enum_mbus_code,
> + .get_fmt = sun6i_isp_proc_get_fmt,
> + .set_fmt = sun6i_isp_proc_set_fmt,
> +};
> +
> +const struct v4l2_subdev_ops sun6i_isp_proc_subdev_ops = {

This can be static, can't it?

> + .video = &sun6i_isp_proc_video_ops,
> + .pad = &sun6i_isp_proc_pad_ops,
> +};
> +
> +/* Media Entity */
> +
> +static const struct media_entity_operations sun6i_isp_proc_entity_ops = {
> + .link_validate = v4l2_subdev_link_validate,
> +};
> +
> +/* V4L2 Async */
> +
> +static int sun6i_isp_proc_link(struct sun6i_isp_device *isp_dev,
> + int sink_pad_index,
> + struct v4l2_subdev *remote_subdev, bool enabled)
> +{
> + struct device *dev = isp_dev->dev;
> + struct v4l2_subdev *subdev = &isp_dev->proc.subdev;
> + struct media_entity *sink_entity = &subdev->entity;
> + struct media_entity *source_entity = &remote_subdev->entity;
> + int source_pad_index;
> + int ret;
> +
> + /* Get the first remote source pad. */
> + ret = media_entity_get_fwnode_pad(source_entity, remote_subdev->fwnode,
> + MEDIA_PAD_FL_SOURCE);
> + if (ret < 0) {
> + dev_err(dev, "missing source pad in external entity %s\n",
> + source_entity->name);
> + return -EINVAL;
> + }
> +
> + source_pad_index = ret;
> +
> + dev_dbg(dev, "creating %s:%u -> %s:%u link\n", source_entity->name,
> + source_pad_index, sink_entity->name, sink_pad_index);
> +
> + ret = media_create_pad_link(source_entity, source_pad_index,
> + sink_entity, sink_pad_index,
> + enabled ? MEDIA_LNK_FL_ENABLED : 0);
> + if (ret < 0) {
> + dev_err(dev, "failed to create %s:%u -> %s:%u link\n",
> + source_entity->name, source_pad_index,
> + sink_entity->name, sink_pad_index);
> + return ret;
> + }
> +
> + return 0;
> +}
> +
> +static int sun6i_isp_proc_notifier_bound(struct v4l2_async_notifier *notifier,
> + struct v4l2_subdev *remote_subdev,
> + struct v4l2_async_subdev *async_subdev)
> +{
> + struct sun6i_isp_device *isp_dev =
> + container_of(notifier, struct sun6i_isp_device, proc.notifier);
> + struct sun6i_isp_proc_async_subdev *proc_async_subdev =
> + container_of(async_subdev, struct sun6i_isp_proc_async_subdev,
> + async_subdev);
> + struct sun6i_isp_proc *proc = &isp_dev->proc;
> + struct sun6i_isp_proc_source *source = proc_async_subdev->source;
> + bool enabled;
> +
> + switch (source->endpoint.base.port) {
> + case SUN6I_ISP_PORT_CSI0:
> + source = &proc->source_csi0;
> + enabled = true;
> + break;
> + case SUN6I_ISP_PORT_CSI1:
> + source = &proc->source_csi1;
> + enabled = !proc->source_csi0.expected;
> + break;
> + default:
> + break;
> + }
> +
> + source->subdev = remote_subdev;
> +
> + return sun6i_isp_proc_link(isp_dev, SUN6I_ISP_PROC_PAD_SINK_CSI,
> + remote_subdev, enabled);
> +}
> +
> +static int
> +sun6i_isp_proc_notifier_complete(struct v4l2_async_notifier *notifier)
> +{
> + struct sun6i_isp_device *isp_dev =
> + container_of(notifier, struct sun6i_isp_device, proc.notifier);
> + struct v4l2_device *v4l2_dev = &isp_dev->v4l2.v4l2_dev;
> + int ret;
> +
> + ret = v4l2_device_register_subdev_nodes(v4l2_dev);
> + if (ret)
> + return ret;
> +
> + return 0;
> +}
> +
> +static const struct v4l2_async_notifier_operations
> +sun6i_isp_proc_notifier_ops = {
> + .bound = sun6i_isp_proc_notifier_bound,
> + .complete = sun6i_isp_proc_notifier_complete,
> +};
> +
> +/* Processor */
> +
> +static int sun6i_isp_proc_source_setup(struct sun6i_isp_device *isp_dev,
> + struct sun6i_isp_proc_source *source,
> + u32 port)
> +{
> + struct device *dev = isp_dev->dev;
> + struct v4l2_async_notifier *notifier = &isp_dev->proc.notifier;
> + struct v4l2_fwnode_endpoint *endpoint = &source->endpoint;
> + struct sun6i_isp_proc_async_subdev *proc_async_subdev;
> + struct fwnode_handle *handle = NULL;
> + int ret;
> +
> + handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(dev), port, 0, 0);
> + if (!handle)
> + return -ENODEV;
> +
> + ret = v4l2_fwnode_endpoint_parse(handle, endpoint);
> + if (ret)
> + goto complete;
> +
> + proc_async_subdev =
> + v4l2_async_nf_add_fwnode_remote(notifier, handle,
> + struct
> + sun6i_isp_proc_async_subdev);
> + if (IS_ERR(proc_async_subdev)) {
> + ret = PTR_ERR(proc_async_subdev);
> + goto complete;
> + }
> +
> + proc_async_subdev->source = source;
> +
> + source->expected = true;
> +
> +complete:
> + fwnode_handle_put(handle);
> +
> + return ret;
> +}
> +
> +int sun6i_isp_proc_setup(struct sun6i_isp_device *isp_dev)
> +{
> + struct device *dev = isp_dev->dev;
> + struct sun6i_isp_proc *proc = &isp_dev->proc;
> + struct v4l2_device *v4l2_dev = &isp_dev->v4l2.v4l2_dev;
> + struct v4l2_async_notifier *notifier = &proc->notifier;
> + struct v4l2_subdev *subdev = &proc->subdev;
> + struct media_pad *pads = proc->pads;
> + int ret;
> +
> + mutex_init(&proc->lock);
> +
> + /* V4L2 Subdev */
> +
> + v4l2_subdev_init(subdev, &sun6i_isp_proc_subdev_ops);
> + strscpy(subdev->name, SUN6I_ISP_PROC_NAME, sizeof(subdev->name));
> + subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> + subdev->owner = THIS_MODULE;
> + subdev->dev = dev;
> +
> + v4l2_set_subdevdata(subdev, isp_dev);
> +
> + /* Media Entity */
> +
> + subdev->entity.function = MEDIA_ENT_F_PROC_VIDEO_ISP;
> + subdev->entity.ops = &sun6i_isp_proc_entity_ops;
> +
> + /* Media Pads */
> +
> + pads[SUN6I_ISP_PROC_PAD_SINK_CSI].flags = MEDIA_PAD_FL_SINK |
> + MEDIA_PAD_FL_MUST_CONNECT;
> + pads[SUN6I_ISP_PROC_PAD_SINK_PARAMS].flags = MEDIA_PAD_FL_SINK |
> + MEDIA_PAD_FL_MUST_CONNECT;
> + pads[SUN6I_ISP_PROC_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
> +
> + ret = media_entity_pads_init(&subdev->entity, SUN6I_ISP_PROC_PAD_COUNT,
> + pads);
> + if (ret)
> + return ret;
> +
> + /* V4L2 Subdev */
> +
> + ret = v4l2_device_register_subdev(v4l2_dev, subdev);
> + if (ret < 0) {
> + v4l2_err(v4l2_dev, "failed to register v4l2 subdev: %d\n", ret);
> + goto error_media_entity;
> + }
> +
> + /* V4L2 Async */
> +
> + v4l2_async_nf_init(notifier);
> + notifier->ops = &sun6i_isp_proc_notifier_ops;
> +
> + sun6i_isp_proc_source_setup(isp_dev, &proc->source_csi0,
> + SUN6I_ISP_PORT_CSI0);
> + sun6i_isp_proc_source_setup(isp_dev, &proc->source_csi1,
> + SUN6I_ISP_PORT_CSI1);
> +
> + ret = v4l2_async_nf_register(v4l2_dev, notifier);
> + if (ret) {
> + v4l2_err(v4l2_dev,
> + "failed to register v4l2 async notifier: %d\n", ret);
> + goto error_v4l2_async_notifier;
> + }
> +
> + return 0;
> +
> +error_v4l2_async_notifier:
> + v4l2_async_nf_cleanup(notifier);
> +
> + v4l2_device_unregister_subdev(subdev);
> +
> +error_media_entity:
> + media_entity_cleanup(&subdev->entity);
> +
> + return ret;
> +}
> +
> +void sun6i_isp_proc_cleanup(struct sun6i_isp_device *isp_dev)
> +{
> + struct v4l2_async_notifier *notifier = &isp_dev->proc.notifier;
> + struct v4l2_subdev *subdev = &isp_dev->proc.subdev;
> +
> + v4l2_async_nf_unregister(notifier);
> + v4l2_async_nf_cleanup(notifier);
> +
> + v4l2_device_unregister_subdev(subdev);
> + media_entity_cleanup(&subdev->entity);
> +}
> diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h
> new file mode 100644
> index 000000000000..c5c274e21ad5
> --- /dev/null
> +++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h
> @@ -0,0 +1,66 @@
> +/* SPDX-License-Identifier: GPL-2.0+ */
> +/*
> + * Copyright 2021-2022 Bootlin
> + * Author: Paul Kocialkowski <paul.kocialkowski@xxxxxxxxxxx>
> + */
> +
> +#ifndef _SUN6I_ISP_PROC_H_
> +#define _SUN6I_ISP_PROC_H_
> +
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-fwnode.h>
> +
> +#define SUN6I_ISP_PROC_NAME "sun6i-isp-proc"
> +
> +enum sun6i_isp_proc_pad {
> + SUN6I_ISP_PROC_PAD_SINK_CSI = 0,
> + SUN6I_ISP_PROC_PAD_SINK_PARAMS = 1,
> + SUN6I_ISP_PROC_PAD_SOURCE = 2,
> + SUN6I_ISP_PROC_PAD_COUNT = 3,
> +};
> +
> +struct sun6i_isp_device;
> +
> +struct sun6i_isp_proc_format {
> + u32 mbus_code;
> + u8 input_format;
> + u8 input_yuv_seq;
> +};
> +
> +struct sun6i_isp_proc_source {
> + struct v4l2_subdev *subdev;
> + struct v4l2_fwnode_endpoint endpoint;
> + bool expected;
> +};
> +
> +struct sun6i_isp_proc_async_subdev {
> + struct v4l2_async_subdev async_subdev;
> + struct sun6i_isp_proc_source *source;
> +};
> +
> +struct sun6i_isp_proc {
> + struct v4l2_subdev subdev;
> + struct media_pad pads[3];
> + struct v4l2_async_notifier notifier;
> + struct v4l2_mbus_framefmt mbus_format;
> + struct mutex lock; /* Mbus format lock. */
> +
> + struct sun6i_isp_proc_source source_csi0;
> + struct sun6i_isp_proc_source source_csi1;
> +};
> +
> +/* Helpers */
> +
> +void sun6i_isp_proc_dimensions(struct sun6i_isp_device *isp_dev,
> + unsigned int *width, unsigned int *height);
> +
> +/* Format */
> +
> +const struct sun6i_isp_proc_format *sun6i_isp_proc_format_find(u32 mbus_code);
> +
> +/* Proc */
> +
> +int sun6i_isp_proc_setup(struct sun6i_isp_device *isp_dev);
> +void sun6i_isp_proc_cleanup(struct sun6i_isp_device *isp_dev);
> +
> +#endif
> diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h
> new file mode 100644
> index 000000000000..83b9cdab2134
> --- /dev/null
> +++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h
> @@ -0,0 +1,275 @@
> +/* SPDX-License-Identifier: GPL-2.0+ */
> +/*
> + * Copyright 2021-2022 Bootlin
> + * Author: Paul Kocialkowski <paul.kocialkowski@xxxxxxxxxxx>
> + */
> +
> +#ifndef _SUN6I_ISP_REG_H_
> +#define _SUN6I_ISP_REG_H_
> +
> +#include <linux/kernel.h>
> +
> +#define SUN6I_ISP_ADDR_VALUE(a) ((a) >> 2)
> +
> +/* Frontend */
> +
> +#define SUN6I_ISP_SRC_MODE_DRAM 0
> +#define SUN6I_ISP_SRC_MODE_CSI(n) (1 + (n))
> +
> +#define SUN6I_ISP_FE_CFG_REG 0x0
> +#define SUN6I_ISP_FE_CFG_EN BIT(0)
> +#define SUN6I_ISP_FE_CFG_SRC0_MODE(v) (((v) << 8) & GENMASK(9, 8))
> +#define SUN6I_ISP_FE_CFG_SRC1_MODE(v) (((v) << 16) & GENMASK(17, 16))
> +
> +#define SUN6I_ISP_FE_CTRL_REG 0x4
> +#define SUN6I_ISP_FE_CTRL_SCAP_EN BIT(0)
> +#define SUN6I_ISP_FE_CTRL_VCAP_EN BIT(1)
> +#define SUN6I_ISP_FE_CTRL_PARA_READY BIT(2)
> +#define SUN6I_ISP_FE_CTRL_LUT_UPDATE BIT(3)
> +#define SUN6I_ISP_FE_CTRL_LENS_UPDATE BIT(4)
> +#define SUN6I_ISP_FE_CTRL_GAMMA_UPDATE BIT(5)
> +#define SUN6I_ISP_FE_CTRL_DRC_UPDATE BIT(6)
> +#define SUN6I_ISP_FE_CTRL_DISC_UPDATE BIT(7)
> +#define SUN6I_ISP_FE_CTRL_OUTPUT_SPEED_CTRL(v) (((v) << 16) & GENMASK(17, 16))
> +#define SUN6I_ISP_FE_CTRL_VCAP_READ_START BIT(31)
> +
> +#define SUN6I_ISP_FE_INT_EN_REG 0x8
> +#define SUN6I_ISP_FE_INT_EN_FINISH BIT(0)
> +#define SUN6I_ISP_FE_INT_EN_START BIT(1)
> +#define SUN6I_ISP_FE_INT_EN_PARA_SAVE BIT(2)
> +#define SUN6I_ISP_FE_INT_EN_PARA_LOAD BIT(3)
> +#define SUN6I_ISP_FE_INT_EN_SRC0_FIFO BIT(4)
> +#define SUN6I_ISP_FE_INT_EN_SRC1_FIFO BIT(5)
> +#define SUN6I_ISP_FE_INT_EN_ROT_FINISH BIT(6)
> +#define SUN6I_ISP_FE_INT_EN_LINE_NUM_START BIT(7)
> +
> +#define SUN6I_ISP_FE_INT_STA_REG 0xc
> +#define SUN6I_ISP_FE_INT_STA_CLEAR 0xff
> +#define SUN6I_ISP_FE_INT_STA_FINISH BIT(0)
> +#define SUN6I_ISP_FE_INT_STA_START BIT(1)
> +#define SUN6I_ISP_FE_INT_STA_PARA_SAVE BIT(2)
> +#define SUN6I_ISP_FE_INT_STA_PARA_LOAD BIT(3)
> +#define SUN6I_ISP_FE_INT_STA_SRC0_FIFO BIT(4)
> +#define SUN6I_ISP_FE_INT_STA_SRC1_FIFO BIT(5)
> +#define SUN6I_ISP_FE_INT_STA_ROT_FINISH BIT(6)
> +#define SUN6I_ISP_FE_INT_STA_LINE_NUM_START BIT(7)
> +
> +/* Only since sun9i-a80-isp. */
> +#define SUN6I_ISP_FE_INT_LINE_NUM_REG 0x18
> +#define SUN6I_ISP_FE_ROT_OF_CFG_REG 0x1c
> +
> +/* Buffers/tables */
> +
> +#define SUN6I_ISP_REG_LOAD_ADDR_REG 0x20
> +#define SUN6I_ISP_REG_SAVE_ADDR_REG 0x24
> +
> +#define SUN6I_ISP_LUT_TABLE_ADDR_REG 0x28
> +#define SUN6I_ISP_DRC_TABLE_ADDR_REG 0x2c
> +#define SUN6I_ISP_STATS_ADDR_REG 0x30
> +
> +/* SRAM */
> +
> +#define SUN6I_ISP_SRAM_RW_OFFSET_REG 0x38
> +#define SUN6I_ISP_SRAM_RW_DATA_REG 0x3c
> +
> +/* Global */
> +
> +#define SUN6I_ISP_MODULE_EN_REG 0x40
> +#define SUN6I_ISP_MODULE_EN_AE BIT(0)
> +#define SUN6I_ISP_MODULE_EN_OBC BIT(1)
> +#define SUN6I_ISP_MODULE_EN_DPC_LUT BIT(2)
> +#define SUN6I_ISP_MODULE_EN_DPC_OTF BIT(3)
> +#define SUN6I_ISP_MODULE_EN_BDNF BIT(4)
> +#define SUN6I_ISP_MODULE_EN_AWB BIT(6)
> +#define SUN6I_ISP_MODULE_EN_WB BIT(7)
> +#define SUN6I_ISP_MODULE_EN_LSC BIT(8)
> +#define SUN6I_ISP_MODULE_EN_BGC BIT(9)
> +#define SUN6I_ISP_MODULE_EN_SAP BIT(10)
> +#define SUN6I_ISP_MODULE_EN_AF BIT(11)
> +#define SUN6I_ISP_MODULE_EN_RGB2RGB BIT(12)
> +#define SUN6I_ISP_MODULE_EN_RGB_DRC BIT(13)
> +#define SUN6I_ISP_MODULE_EN_TDNF BIT(15)
> +#define SUN6I_ISP_MODULE_EN_AFS BIT(16)
> +#define SUN6I_ISP_MODULE_EN_HIST BIT(17)
> +#define SUN6I_ISP_MODULE_EN_YUV_GAIN_OFFSET BIT(18)
> +#define SUN6I_ISP_MODULE_EN_YUV_DRC BIT(19)
> +#define SUN6I_ISP_MODULE_EN_TG BIT(20)
> +#define SUN6I_ISP_MODULE_EN_ROT BIT(21)
> +#define SUN6I_ISP_MODULE_EN_CONTRAST BIT(22)
> +#define SUN6I_ISP_MODULE_EN_SATU BIT(24)
> +#define SUN6I_ISP_MODULE_EN_SRC1 BIT(30)
> +#define SUN6I_ISP_MODULE_EN_SRC0 BIT(31)
> +
> +#define SUN6I_ISP_MODE_REG 0x44
> +#define SUN6I_ISP_MODE_INPUT_FMT(v) ((v) & GENMASK(2, 0))
> +#define SUN6I_ISP_MODE_INPUT_YUV_SEQ(v) (((v) << 3) & GENMASK(4, 3))
> +#define SUN6I_ISP_MODE_OTF_DPC(v) (((v) << 16) & BIT(16))
> +#define SUN6I_ISP_MODE_SHARP(v) (((v) << 17) & BIT(17))
> +#define SUN6I_ISP_MODE_HIST(v) (((v) << 20) & GENMASK(21, 20))
> +
> +#define SUN6I_ISP_INPUT_FMT_YUV420 0
> +#define SUN6I_ISP_INPUT_FMT_YUV422 1
> +#define SUN6I_ISP_INPUT_FMT_RAW_BGGR 4
> +#define SUN6I_ISP_INPUT_FMT_RAW_RGGB 5
> +#define SUN6I_ISP_INPUT_FMT_RAW_GBRG 6
> +#define SUN6I_ISP_INPUT_FMT_RAW_GRBG 7
> +
> +#define SUN6I_ISP_INPUT_YUV_SEQ_YUYV 0
> +#define SUN6I_ISP_INPUT_YUV_SEQ_YVYU 1
> +#define SUN6I_ISP_INPUT_YUV_SEQ_UYVY 2
> +#define SUN6I_ISP_INPUT_YUV_SEQ_VYUY 3
> +
> +#define SUN6I_ISP_IN_CFG_REG 0x48
> +#define SUN6I_ISP_IN_CFG_STRIDE_DIV16(v) ((v) & GENMASK(10, 0))
> +
> +#define SUN6I_ISP_IN_LUMA_RGB_ADDR0_REG 0x4c
> +#define SUN6I_ISP_IN_CHROMA_ADDR0_REG 0x50
> +#define SUN6I_ISP_IN_LUMA_RGB_ADDR1_REG 0x54
> +#define SUN6I_ISP_IN_CHROMA_ADDR1_REG 0x58
> +
> +/* AE */
> +
> +#define SUN6I_ISP_AE_CFG_REG 0x60
> +#define SUN6I_ISP_AE_CFG_LOW_BRI_TH(v) ((v) & GENMASK(11, 0))
> +#define SUN6I_ISP_AE_CFG_HORZ_NUM(v) (((v) << 12) & GENMASK(15, 12))
> +#define SUN6I_ISP_AE_CFG_HIGH_BRI_TH(v) (((v) << 16) & GENMASK(27, 16))
> +#define SUN6I_ISP_AE_CFG_VERT_NUM(v) (((v) << 28) & GENMASK(31, 28))
> +
> +#define SUN6I_ISP_AE_SIZE_REG 0x64
> +#define SUN6I_ISP_AE_SIZE_WIDTH(v) ((v) & GENMASK(10, 0))
> +#define SUN6I_ISP_AE_SIZE_HEIGHT(v) (((v) << 16) & GENMASK(26, 16))
> +
> +#define SUN6I_ISP_AE_POS_REG 0x68
> +#define SUN6I_ISP_AE_POS_HORZ_START(v) ((v) & GENMASK(10, 0))
> +#define SUN6I_ISP_AE_POS_VERT_START(v) (((v) << 16) & GENMASK(26, 16))
> +
> +/* OB */
> +
> +#define SUN6I_ISP_OB_SIZE_REG 0x78
> +#define SUN6I_ISP_OB_SIZE_WIDTH(v) ((v) & GENMASK(13, 0))
> +#define SUN6I_ISP_OB_SIZE_HEIGHT(v) (((v) << 16) & GENMASK(29, 16))
> +
> +#define SUN6I_ISP_OB_VALID_REG 0x7c
> +#define SUN6I_ISP_OB_VALID_WIDTH(v) ((v) & GENMASK(12, 0))
> +#define SUN6I_ISP_OB_VALID_HEIGHT(v) (((v) << 16) & GENMASK(28, 16))
> +
> +#define SUN6I_ISP_OB_SRC0_VALID_START_REG 0x80
> +#define SUN6I_ISP_OB_SRC0_VALID_START_HORZ(v) ((v) & GENMASK(11, 0))
> +#define SUN6I_ISP_OB_SRC0_VALID_START_VERT(v) (((v) << 16) & GENMASK(27, 16))
> +
> +#define SUN6I_ISP_OB_SRC1_VALID_START_REG 0x84
> +#define SUN6I_ISP_OB_SRC1_VALID_START_HORZ(v) ((v) & GENMASK(11, 0))
> +#define SUN6I_ISP_OB_SRC1_VALID_START_VERT(v) (((v) << 16) & GENMASK(27, 16))
> +
> +#define SUN6I_ISP_OB_SPRITE_REG 0x88
> +#define SUN6I_ISP_OB_SPRITE_WIDTH(v) ((v) & GENMASK(12, 0))
> +#define SUN6I_ISP_OB_SPRITE_HEIGHT(v) (((v) << 16) & GENMASK(28, 16))
> +
> +#define SUN6I_ISP_OB_SPRITE_START_REG 0x8c
> +#define SUN6I_ISP_OB_SPRITE_START_HORZ(v) ((v) & GENMASK(11, 0))
> +#define SUN6I_ISP_OB_SPRITE_START_VERT(v) (((v) << 16) & GENMASK(27, 16))
> +
> +#define SUN6I_ISP_OB_CFG_REG 0x90
> +#define SUN6I_ISP_OB_HORZ_POS_REG 0x94
> +#define SUN6I_ISP_OB_VERT_PARA_REG 0x98
> +#define SUN6I_ISP_OB_OFFSET_FIXED_REG 0x9c
> +
> +/* BDNF */
> +
> +#define SUN6I_ISP_BDNF_CFG_REG 0xcc
> +#define SUN6I_ISP_BDNF_CFG_IN_DIS_MIN(v) ((v) & GENMASK(7, 0))
> +#define SUN6I_ISP_BDNF_CFG_IN_DIS_MAX(v) (((v) << 16) & GENMASK(23, 16))
> +
> +#define SUN6I_ISP_BDNF_COEF_RB_REG 0xd0
> +#define SUN6I_ISP_BDNF_COEF_RB(i, v) (((v) << (4 * (i))) & \
> + GENMASK(4 * (i) + 3, 4 * (i)))
> +
> +#define SUN6I_ISP_BDNF_COEF_G_REG 0xd4
> +#define SUN6I_ISP_BDNF_COEF_G(i, v) (((v) << (4 * (i))) & \
> + GENMASK(4 * (i) + 3, 4 * (i)))
> +
> +/* Bayer */
> +
> +#define SUN6I_ISP_BAYER_OFFSET0_REG 0xe0
> +#define SUN6I_ISP_BAYER_OFFSET0_R(v) ((v) & GENMASK(12, 0))
> +#define SUN6I_ISP_BAYER_OFFSET0_GR(v) (((v) << 16) & GENMASK(28, 16))
> +
> +#define SUN6I_ISP_BAYER_OFFSET1_REG 0xe4
> +#define SUN6I_ISP_BAYER_OFFSET1_GB(v) ((v) & GENMASK(12, 0))
> +#define SUN6I_ISP_BAYER_OFFSET1_B(v) (((v) << 16) & GENMASK(28, 16))
> +
> +#define SUN6I_ISP_BAYER_GAIN0_REG 0xe8
> +#define SUN6I_ISP_BAYER_GAIN0_R(v) ((v) & GENMASK(11, 0))
> +#define SUN6I_ISP_BAYER_GAIN0_GR(v) (((v) << 16) & GENMASK(27, 16))
> +
> +#define SUN6I_ISP_BAYER_GAIN1_REG 0xec
> +#define SUN6I_ISP_BAYER_GAIN1_GB(v) ((v) & GENMASK(11, 0))
> +#define SUN6I_ISP_BAYER_GAIN1_B(v) (((v) << 16) & GENMASK(27, 16))
> +
> +/* WB */
> +
> +#define SUN6I_ISP_WB_GAIN0_REG 0x140
> +#define SUN6I_ISP_WB_GAIN0_R(v) ((v) & GENMASK(11, 0))
> +#define SUN6I_ISP_WB_GAIN0_GR(v) (((v) << 16) & GENMASK(27, 16))
> +
> +#define SUN6I_ISP_WB_GAIN1_REG 0x144
> +#define SUN6I_ISP_WB_GAIN1_GB(v) ((v) & GENMASK(11, 0))
> +#define SUN6I_ISP_WB_GAIN1_B(v) (((v) << 16) & GENMASK(27, 16))
> +
> +#define SUN6I_ISP_WB_CFG_REG 0x148
> +#define SUN6I_ISP_WB_CFG_CLIP(v) ((v) & GENMASK(11, 0))
> +
> +/* Global */
> +
> +#define SUN6I_ISP_MCH_SIZE_CFG_REG 0x1e0
> +#define SUN6I_ISP_MCH_SIZE_CFG_WIDTH(v) ((v) & GENMASK(12, 0))
> +#define SUN6I_ISP_MCH_SIZE_CFG_HEIGHT(v) (((v) << 16) & GENMASK(28, 16))
> +
> +#define SUN6I_ISP_MCH_SCALE_CFG_REG 0x1e4
> +#define SUN6I_ISP_MCH_SCALE_CFG_X_RATIO(v) ((v) & GENMASK(11, 0))
> +#define SUN6I_ISP_MCH_SCALE_CFG_Y_RATIO(v) (((v) << 16) & GENMASK(27, 16))
> +#define SUN6I_ISP_MCH_SCALE_CFG_WEIGHT_SHIFT(v) (((v) << 28) & GENMASK(31, 28))
> +
> +#define SUN6I_ISP_SCH_SIZE_CFG_REG 0x1e8
> +#define SUN6I_ISP_SCH_SIZE_CFG_WIDTH(v) ((v) & GENMASK(12, 0))
> +#define SUN6I_ISP_SCH_SIZE_CFG_HEIGHT(v) (((v) << 16) & GENMASK(28, 16))
> +
> +#define SUN6I_ISP_SCH_SCALE_CFG_REG 0x1ec
> +#define SUN6I_ISP_SCH_SCALE_CFG_X_RATIO(v) ((v) & GENMASK(11, 0))
> +#define SUN6I_ISP_SCH_SCALE_CFG_Y_RATIO(v) (((v) << 16) & GENMASK(27, 16))
> +#define SUN6I_ISP_SCH_SCALE_CFG_WEIGHT_SHIFT(v) (((v) << 28) & GENMASK(31, 28))
> +
> +#define SUN6I_ISP_MCH_CFG_REG 0x1f0
> +#define SUN6I_ISP_MCH_CFG_EN BIT(0)
> +#define SUN6I_ISP_MCH_CFG_SCALE_EN BIT(1)
> +#define SUN6I_ISP_MCH_CFG_OUTPUT_FMT(v) (((v) << 2) & GENMASK(4, 2))
> +#define SUN6I_ISP_MCH_CFG_MIRROR_EN BIT(5)
> +#define SUN6I_ISP_MCH_CFG_FLIP_EN BIT(6)
> +#define SUN6I_ISP_MCH_CFG_STRIDE_Y_DIV4(v) (((v) << 8) & GENMASK(18, 8))
> +#define SUN6I_ISP_MCH_CFG_STRIDE_UV_DIV4(v) (((v) << 20) & GENMASK(30, 20))
> +
> +#define SUN6I_ISP_OUTPUT_FMT_YUV420SP 0
> +#define SUN6I_ISP_OUTPUT_FMT_YUV422SP 1
> +#define SUN6I_ISP_OUTPUT_FMT_YVU420SP 2
> +#define SUN6I_ISP_OUTPUT_FMT_YVU422SP 3
> +#define SUN6I_ISP_OUTPUT_FMT_YUV420P 4
> +#define SUN6I_ISP_OUTPUT_FMT_YUV422P 5
> +#define SUN6I_ISP_OUTPUT_FMT_YVU420P 6
> +#define SUN6I_ISP_OUTPUT_FMT_YVU422P 7
> +
> +#define SUN6I_ISP_SCH_CFG_REG 0x1f4
> +
> +#define SUN6I_ISP_MCH_Y_ADDR0_REG 0x1f8
> +#define SUN6I_ISP_MCH_U_ADDR0_REG 0x1fc
> +#define SUN6I_ISP_MCH_V_ADDR0_REG 0x200
> +#define SUN6I_ISP_MCH_Y_ADDR1_REG 0x204
> +#define SUN6I_ISP_MCH_U_ADDR1_REG 0x208
> +#define SUN6I_ISP_MCH_V_ADDR1_REG 0x20c
> +#define SUN6I_ISP_SCH_Y_ADDR0_REG 0x210
> +#define SUN6I_ISP_SCH_U_ADDR0_REG 0x214
> +#define SUN6I_ISP_SCH_V_ADDR0_REG 0x218
> +#define SUN6I_ISP_SCH_Y_ADDR1_REG 0x21c
> +#define SUN6I_ISP_SCH_U_ADDR1_REG 0x220
> +#define SUN6I_ISP_SCH_V_ADDR1_REG 0x224
> +
> +#endif
> diff --git a/drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h b/drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h
> new file mode 100644
> index 000000000000..fd2a0820aa98
> --- /dev/null
> +++ b/drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h
> @@ -0,0 +1,43 @@
> +/* SPDX-License-Identifier: ((GPL-2.0+ WITH Linux-syscall-note) OR MIT) */
> +/*
> + * Allwinner A31 ISP Configuration
> + */
> +
> +#ifndef _UAPI_SUN6I_ISP_CONFIG_H
> +#define _UAPI_SUN6I_ISP_CONFIG_H
> +
> +#include <linux/types.h>
> +
> +#define V4L2_META_FMT_SUN6I_ISP_PARAMS v4l2_fourcc('S', '6', 'I', 'P') /* Allwinner A31 ISP Parameters */
> +
> +#define SUN6I_ISP_MODULE_BAYER (1U << 0)
> +#define SUN6I_ISP_MODULE_BDNF (1U << 1)
> +
> +struct sun6i_isp_params_config_bayer {
> + __u16 offset_r;
> + __u16 offset_gr;
> + __u16 offset_gb;
> + __u16 offset_b;
> +
> + __u16 gain_r;
> + __u16 gain_gr;
> + __u16 gain_gb;
> + __u16 gain_b;
> +};
> +
> +struct sun6i_isp_params_config_bdnf {
> + __u8 in_dis_min; // 8
> + __u8 in_dis_max; // 10

Are these default values or something else? Better documentation was in the
TODO.txt file already.

> +
> + __u8 coefficients_g[7];
> + __u8 coefficients_rb[5];
> +};
> +
> +struct sun6i_isp_params_config {
> + __u32 modules_used;
> +
> + struct sun6i_isp_params_config_bayer bayer;
> + struct sun6i_isp_params_config_bdnf bdnf;
> +};
> +
> +#endif /* _UAPI_SUN6I_ISP_CONFIG_H */

--
Kind regards,

Sakari Ailus

Next message: Rob Herring: "Re: [v4 16/18] dt-bindings: mediatek: mt8186: add mt8186-mt6366-da7219-max98357 document"
Previous message: Artur Bujdoso: "Re: [PATCH] staging: octeon-usb: remove unnecessary parentheses"
Next in thread: Paul Kocialkowski: "Re: [PATCH v3 3/4] staging: media: Add support for the Allwinner A31 ISP"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]