RE: [PATCH v11 07/10] mtd: spi-nor: Add stacked memories support in spi-nor

From: Mahapatra, Amit Kumar
Date: Fri Jul 26 2024 - 08:36:15 EST


> -----Original Message-----
> From: Mahapatra, Amit Kumar <amit.kumar-mahapatra@xxxxxxx>
> Sent: Wednesday, March 13, 2024 9:34 PM
> To: Tudor Ambarus <tudor.ambarus@xxxxxxxxxx>; broonie@xxxxxxxxxx;
> pratyush@xxxxxxxxxx; miquel.raynal@xxxxxxxxxxx; richard@xxxxxx;
> vigneshr@xxxxxx; sbinding@xxxxxxxxxxxxxxxxxxxxx; lee@xxxxxxxxxx;
> james.schulman@xxxxxxxxxx; david.rhodes@xxxxxxxxxx;
> rf@xxxxxxxxxxxxxxxxxxxxx; perex@xxxxxxxx; tiwai@xxxxxxxx
> Cc: linux-spi@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx;
> michael@xxxxxxxx; linux-mtd@xxxxxxxxxxxxxxxxxxx;
> nicolas.ferre@xxxxxxxxxxxxx; alexandre.belloni@xxxxxxxxxxx;
> claudiu.beznea@xxxxxxxxx; Simek, Michal <michal.simek@xxxxxxx>; linux-
> arm-kernel@xxxxxxxxxxxxxxxxxxx; alsa-devel@xxxxxxxxxxxxxxxx;
> patches@xxxxxxxxxxxxxxxxxxxxx; linux-sound@xxxxxxxxxxxxxxx; git (AMD-
> Xilinx) <git@xxxxxxx>; amitrkcian2002@xxxxxxxxx; Conor Dooley
> <conor.dooley@xxxxxxxxxxxxx>; beanhuo@xxxxxxxxxx
> Subject: RE: [PATCH v11 07/10] mtd: spi-nor: Add stacked memories support
> in spi-nor
>
>
>
> > -----Original Message-----
> > From: Tudor Ambarus <tudor.ambarus@xxxxxxxxxx>
> > Sent: Friday, February 9, 2024 9:44 PM
> > To: Mahapatra, Amit Kumar <amit.kumar-mahapatra@xxxxxxx>;
> > broonie@xxxxxxxxxx; pratyush@xxxxxxxxxx; miquel.raynal@xxxxxxxxxxx;
> > richard@xxxxxx; vigneshr@xxxxxx; sbinding@xxxxxxxxxxxxxxxxxxxxx;
> > lee@xxxxxxxxxx; james.schulman@xxxxxxxxxx; david.rhodes@xxxxxxxxxx;
> > rf@xxxxxxxxxxxxxxxxxxxxx; perex@xxxxxxxx; tiwai@xxxxxxxx
> > Cc: linux-spi@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx;
> > michael@xxxxxxxx; linux-mtd@xxxxxxxxxxxxxxxxxxx;
> > nicolas.ferre@xxxxxxxxxxxxx; alexandre.belloni@xxxxxxxxxxx;
> > claudiu.beznea@xxxxxxxxx; Simek, Michal <michal.simek@xxxxxxx>;
> linux-
> > arm-kernel@xxxxxxxxxxxxxxxxxxx; alsa-devel@xxxxxxxxxxxxxxxx;
> > patches@xxxxxxxxxxxxxxxxxxxxx; linux-sound@xxxxxxxxxxxxxxx; git (AMD-
> > Xilinx) <git@xxxxxxx>; amitrkcian2002@xxxxxxxxx; Conor Dooley
> > <conor.dooley@xxxxxxxxxxxxx>; beanhuo@xxxxxxxxxx
> > Subject: Re: [PATCH v11 07/10] mtd: spi-nor: Add stacked memories
> > support in spi-nor

Hello Everyone,

I would like to propose another approach for handling stacked and
parallel connection modes and would appreciate your thoughts on it.
But before that, here is some background on what we are trying to achieve.

The AMD QSPI controller supports two advanced connection modes(Stacked and
Dual Parallel) which allow the controller to treat two different flashes
as one storage.

Stacked:
Flashes share the same SPI bus, but different CS line, controller asserts
the CS of the flash to which it needs to communicate.

Dual Parallel:
Both the flashes have their separate SPI bus CS of both the flashes will
be asserted/de-asserted at the same time. In this mode data will be split
across both the flashes by enabling the STRIPE setting in the controller.
If STRIPE is not enabled, then same data will be sent to both the devices.

For more information on the modes please feel free to go through the
controller flash interface below
https://docs.amd.com/r/en-US/am011-versal-acap-trm/QSPI-Flash-Device-Interface

Mirochip QSPI controller also supports "Dual Parallel 8-bit IO mode", but
they call it "Twin Quad Mode".
https://ww1.microchip.com/downloads/aemDocuments/documents/MPU32/ProductDocuments/DataSheets/SAMA7G5-Series-Data-Sheet-DS60001765.pdf

DT binding changes were added through the following commits:
https://github.com/torvalds/linux/commit/f89504300e94524d5d5846ff8b728592ac72cec4
https://github.com/torvalds/linux/commit/eba5368503b4291db7819512600fa014ea17c5a8
https://github.com/torvalds/linux/commit/e2edd1b64f1c79e8abda365149ed62a2a9a494b4

SPI core changes were adds through the following commit:
https://github.com/torvalds/linux/commit/4d8ff6b0991d5e86b17b235fc46ec62e9195cb9b

Based on the inputs/suggestions from Tudor, i am planning to add a new
layer between the SPI-NOR and MTD layers to support stacked and parallel
configurations. This new layer will be part of the spi-nor and located in
mtd/spi-nor/

This layer would perform the following tasks:
- During probing, store information from all the connected flashes,
whether in stacked or parallel mode, and present it as a single device
to the MTD layer.
- Register callbacks through this new layer instead of spi-nor/core.c and
handle MTD device registration.
- In stacked mode, select the appropriate spi-nor flash based on the
address provided by the MTD layer during flash operations.
- Manage flash crossover operations in stacked mode.
- Ensure both connected flashes are identical in parallel mode.
- Handle odd byte count requests from the MTD layer during flash
operations in parallel mode.

For implementing this the current DT binding need to be updated as
follows.

stacked-memories DT changes:
- Flash size information can be retrieved directly from the flash, so it
has been removed from the DT binding.
- Each stacked flash will have its own flash node. This approach allows
flashes of different makes and sizes to be stacked together, as each
flash will be probed individually.
- The stacked-memories DT bindings will contain the phandles of the flash
nodes connected in stacked mode.

spi@0 {

flash@0 {
compatible = "jedec,spi-nor"
reg = <0x00>;
stacked-memories = <&flash@0 &flash@1>;
spi-max-frequency = <50000000>;
...
partition@0 {
label = "qspi-0";
reg = <0x0 0xf00000>;
};


}

flash@1 {
compatible = "jedec,spi-nor"
reg = <0x01>;
spi-max-frequency = <50000000>;
...
partition@0 {
label = "qspi-1";
reg = <0x0 0x800000>;
};
}
}

parallel-memories DT changes:
- Flash size information can be retrieved directly from the flash, so it
has been removed from the DT binding.
- Each flash connected in parallel mode will have its own flash node.
This allows us to verify that both flashes connected in parallel are
identical, as each flash node will be probed separately.
- The parallel-memories DT bindings will contain the phandles of the
flash nodes connected in parallel.

spi@0 {

flash@0 {
compatible = "jedec,spi-nor"
reg = <0x00>;
parallel-memories = <&flash@0 &flash@1>;
spi-max-frequency = <50000000>;
...
partition@0 {
label = "qspi-0";
reg = <0x0 0xf00000>;
};


}

flash@1 {
compatible = "jedec,spi-nor"
reg = <0x01>;
spi-max-frequency = <50000000>;
...
partition@0 {
label = "qspi-1";
reg = <0x0 0x800000>;
};
}
}

Regards,
Amit

> >
> >
> >
> > On 2/9/24 11:06, Tudor Ambarus wrote:
> > >
> > >
> > > On 12/21/23 06:54, Mahapatra, Amit Kumar wrote:
> > >>> Something else to consider: I see that Micron has a twin quad mode:
> > >>> https://media-www.micron.com/-
> > >>> /media/client/global/documents/products/data-sheet/nor-flash/seria
> > >>> l-
> > >>> nor/mt25t/generation-
> > >>>
> > b/mt25t_qljs_l_512_xba_0.pdf?rev=de70b770c5dc4da8b8ead06b57c03500
> > >>>
> > >>> The micron's "Separate Chip-Select and Clock Signals" resembles
> > >>> the AMD's dual parallel 8-bit.
> > >> Yes, I agree.
> > >>
> > >>> Micron's "Shared Chip-Select and Clock Signals" differs from the
> > >>> AMD's stacked mode, as Micron uses DQ[3:0] and DQ[7:4], whereas
> > >>> AMD considers both as DQ[3:0].
> > >> Yes, correct.
> > >
> > > Amit, please help me to assess this. I assume Micron and Microchip
> > > is using the same concepts as AMD uses for the "Dual Parallel 8-bit
> > > IO mode", but they call it "Twin Quad Mode".
> > >
> > > I was wrong, the AMD datasheet [1] was misleading [2], it described
> > > the IOs for both flashes as IO[3:0], but later on in the "Table QSPI
> > > Interface Signals" the second flash is described with IO[7:4].
> > >
> > > The AMD's 8-bit Dual Flash Parallel Interface is using dedicated CS#
> > > and CLK# lines for each flash. As Micron does, isn't it?
> > >
> > > Micron says [3] that:
> > > "The device contains two quad I/O die, each able to operate
> > > independently for a total of eight I/Os. The memory map applies to
> > > each die. Each die has internal registers for status, configuration,
> > > and device protection that can be set and read independently from
> > > one
> > other.
> > > Micron recommends that internal configuration settings for the two
> > > die be set identically."
> >
>
> Hello Tudor,
>
> > Amit,
> >
> > I forgot to say my first conclusion about the quote from above. Even
> > if the dies are in the same physical package, micron asks users to
> > configure each die as it is a self-standing entity, IOW to configure
> > each die as it is a flash on its own. To me it looks like 2
> > concatenated flashes at first look. Thus identical to how AMD controller
> works.
> > Please clarify this.
>
> That’s correct, the Micron flash that you referred can communicate with the
> AMD QSPI controller in both parallel and stacked mode.
> >
> > >
> > > it also says that:
> > > "When using quad commands in XIO-SPI or when using QIO-SPI,
> > > DQ[3:0]/DQ[7:4] are I/O."
> >
> > and this would be a parallel concatenation of two flashes.
>
> That's correct.
>
> Regards,
> Amit
> >
> > Then it would be good if you let us now the similarities and
> > differences between how amd and mchp controller work, I scrawled few
> ideas below.
> >
> > thanks,
> > ta
> > >
> > > So I guess the upper layers just ask for a chunk of memory to be
> > > written and the controller handles the cs# lines automatically. How
> > > is the AMD controller working, do you have to drive the cs# lines
> > > manually, or you just set the parallel mode and the controller takes
> > > care of
> > everything?
> > >
> > > I assume this is how mchp is handling things, they seem to just set
> > > a bit the protocol into the QSPI_IFR.PROTTYP register field and
> > > that's all [4]. They even seem to write the registers of both
> > > flashes at the same
> > time.
> > >
> > > In what regards the AMD's "dual stack interface", AMD is sharing the
> > > clock and IO lines and uses dedicated CS# lines for the flashes,
> > > whereas Micron shares the CS# and CLK# lines with different IO lines.
> > >
> > > Amit, please study the architectures used by mchp, micron and amd
> > > and let us know if they are the same or they differ, and if they
> > > differ what are the differences.
> > >
> > > I added Conor from mchp in cc, I see Nicolas is already there, and
> > > Bean from micron.
> > >
> > > Thanks,
> > > ta
> > >
> > > [1]
> > > https://docs.xilinx.com/r/en-US/am011-versal-acap-trm/QSPI-Flash-Int
> > > er
> > > face-Signals
> > > [2]
> > >
> >
> https://docs.xilinx.com/viewer/attachment/dwmjhDJGICdJqD4swyVzcQ/fD8nv
> > > 4ry78xM0_EF5kv4mA
> > > [3]
> > > https://media-www.micron.com/-
> > /media/client/global/documents/products/
> > > data-sheet/nor-flash/serial-nor/mt25t/generation-b/mt25t_qljs_l_512_
> > > xb
> > > a_0.pdf?rev=de70b770c5dc4da8b8ead06b57c03500
> > > [4]
> > >
> >
> https://ww1.microchip.com/downloads/aemDocuments/documents/MPU32/
> > Produ
> > > ctDocuments/DataSheets/SAMA7G5-Series-Data-Sheet-DS60001765.pdf