Re: [REGRESSION] LVM-on-LVM: error while submitting device barriers

From: Patrick Plenefisch
Date: Thu Feb 29 2024 - 15:23:25 EST

On Thu, Feb 29, 2024 at 2:56 PM Goffredo Baroncelli <kreijack@xxxxxxxxx> wrote:
>
> > Your understanding is correct. The only thing that comes to my mind to
> > cause the problem is asymmetry of the SATA devices. I have one 8TB
> > device, plus a 1.5TB, 3TB, and 3TB drives. Doing math on the actual
> > extents, lowerVG/single spans (3TB+3TB), and
> > lowerVG/lvmPool/lvm/brokenDisk spans (3TB+1.5TB). Both obviously have
> > the other leg of raid1 on the 8TB drive, but my thought was that the
> > jump across the 1.5+3TB drive gap was at least "interesting"
>
>
> what about lowerVG/works ?
>

That one is only on two disks, it doesn't span any gaps

> However yes, I agree that the pair of disks involved may be the answer
> of the problem.
>
> Could you show us the output of
>
> $ sudo pvdisplay -m
>
>

I trimmed it, but kept the relevant bits (Free PE is thus not correct):


--- Physical volume ---
PV Name /dev/lowerVG/lvmPool
VG Name lvm
PV Size <3.00 TiB / not usable 3.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 786431
Free PE 82943
Allocated PE 703488
PV UUID 7p3LSU-EAHd-xUg0-r9vT-Gzkf-tYFV-mvlU1M

--- Physical Segments ---
Physical extent 0 to 159999:
Logical volume /dev/lvm/brokenDisk
Logical extents 0 to 159999
Physical extent 160000 to 339199:
Logical volume /dev/lvm/a
Logical extents 0 to 179199
Physical extent 339200 to 349439:
Logical volume /dev/lvm/brokenDisk
Logical extents 160000 to 170239
Physical extent 349440 to 351999:
FREE
Physical extent 352000 to 460026:
Logical volume /dev/lvm/brokenDisk
Logical extents 416261 to 524287
Physical extent 460027 to 540409:
FREE
Physical extent 540410 to 786430:
Logical volume /dev/lvm/brokenDisk
Logical extents 170240 to 416260


--- Physical volume ---
PV Name /dev/sda3
VG Name lowerVG
PV Size <2.70 TiB / not usable 3.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 707154
Free PE 909
Allocated PE 706245
PV UUID W8gJ0P-JuMs-1y3g-b5cO-4RuA-MoFs-3zgKBn

--- Physical Segments ---
Physical extent 0 to 52223:
Logical volume /dev/lowerVG/single_corig_rimage_0_iorig
Logical extents 629330 to 681553
Physical extent 52224 to 628940:
Logical volume /dev/lowerVG/single_corig_rimage_0_iorig
Logical extents 0 to 576716
Physical extent 628941 to 628941:
Logical volume /dev/lowerVG/single_corig_rmeta_0
Logical extents 0 to 0
Physical extent 628942 to 628962:
Logical volume /dev/lowerVG/single_corig_rimage_0_iorig
Logical extents 681554 to 681574
Physical extent 628963 to 634431:
Logical volume /dev/lowerVG/single_corig_rimage_0_imeta
Logical extents 0 to 5468
Physical extent 634432 to 654540:
FREE
Physical extent 654541 to 707153:
Logical volume /dev/lowerVG/single_corig_rimage_0_iorig
Logical extents 576717 to 629329

--- Physical volume ---
PV Name /dev/sdf2
VG Name lowerVG
PV Size <7.28 TiB / not usable 4.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 1907645
Free PE 414967
Allocated PE 1492678
PV UUID my0zQM-832Z-HYPD-sNfW-68ms-nddg-lMyWJM

--- Physical Segments ---
Physical extent 0 to 0:
Logical volume /dev/lowerVG/single_corig_rmeta_1
Logical extents 0 to 0
Physical extent 1 to 681575:
Logical volume /dev/lowerVG/single_corig_rimage_1_iorig
Logical extents 0 to 681574
Physical extent 681576 to 687044:
Logical volume /dev/lowerVG/single_corig_rimage_1_imeta
Logical extents 0 to 5468
Physical extent 687045 to 687045:
Logical volume /dev/lowerVG/lvmPool_rmeta_0
Logical extents 0 to 0
Physical extent 687046 to 1049242:
Logical volume /dev/lowerVG/lvmPool_rimage_0
Logical extents 0 to 362196
Physical extent 1049243 to 1056551:
FREE
Physical extent 1056552 to 1473477:
Logical volume /dev/lowerVG/lvmPool_rimage_0
Logical extents 369506 to 786431
Physical extent 1473478 to 1480786:
Logical volume /dev/lowerVG/lvmPool_rimage_0
Logical extents 362197 to 369505
Physical extent 1480787 to 1907644:
FREE

--- Physical volume ---
PV Name /dev/sdb3
VG Name lowerVG
PV Size 1.33 TiB / not usable 3.00 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 349398
Free PE 0
Allocated PE 349398
PV UUID Ncmgdw-ZOXS-qTYL-1jAz-w7zt-38V2-f53EpI

--- Physical Segments ---
Physical extent 0 to 0:
Logical volume /dev/lowerVG/lvmPool_rmeta_1
Logical extents 0 to 0
Physical extent 1 to 349397:
Logical volume /dev/lowerVG/lvmPool_rimage_1
Logical extents 0 to 349396


--- Physical volume ---
PV Name /dev/sde2
VG Name lowerVG
PV Size 2.71 TiB / not usable 3.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 711346
Free PE 255111
Allocated PE 456235
PV UUID xUG8TG-wvp0-roBo-GPo7-sbvn-aE7I-NAHU07

--- Physical Segments ---
Physical extent 0 to 416925:
Logical volume /dev/lowerVG/lvmPool_rimage_1
Logical extents 369506 to 786431
Physical extent 416926 to 437034:
Logical volume /dev/lowerVG/lvmPool_rimage_1
Logical extents 349397 to 369505
Physical extent 437035 to 711345:
FREE


Finally, I am not sure if it's relevant, but I did struggle to expand
the raid1 volumes across gaps when creating this setup. I did file a
bug about that, though I am not sure if it's relevant, as I removed
integrity and cache for brokenDisk & lvmPool:
https://gitlab.com/lvmteam/lvm2/-/issues/6

Patrick