Re: [PATCH 00/13] mvneta Buffer Management and enhancements
From: Gregory CLEMENT
Date: Tue Dec 01 2015 - 08:13:51 EST
Hi Marcin,
On lun., nov. 30 2015, Marcin Wojtas <mw@xxxxxxxxxxxx> wrote:
[...]
>>> 5. Enable BM on Armada XP and 38X development boards - those ones and
>>> A370 I could check on my own. In all cases they survived night-long
>>> linerate iperf. Also tests were performed with A388 SoC working as a
>>> network bridge between two packet generators. They showed increase of
>>> maximum processed 64B packets by ~20k (~555k packets with BM enabled
>>> vs ~535 packets without BM). Also when pushing 1500B-packets with a
>>> line rate achieved, CPU load decreased from around 25% without BM vs
>>> 18-20% with BM.
>>
>> I was trying to test the BM part of tour series on the Armada XP GP
>> board. However it failed very quickly during the pool allocation. After
>> a first debug I found that the size of the cs used in the
>> mvebu_mbus_dram_info struct was 0. I have applied your series on a
>> v4.4-rc1 kernel. At this stage I don't know if it is a regression in the
>> mbus driver, a misconfiguration on my side or something else.
>>
>> Does it ring a bell for you?
>
> Frankly, I'm a bit surprised, I've never seen such problems on any of
> the boards (AXP-GP/DB, A38X-DB/GP/AP). Did mvebu_mbus_dram_win_info
> function exit with an error? Can you please apply below diff:
> http://pastebin.com/2ws1txWk
Yes it exited with errors and I added the same kind traces. It was how I
knew that the size was 0!
I've just rebuild a fresh kernel using mvebu_v7_defconfig and adding
your patch, I got the same issue (see the log at the end of the email.)
But the good news is that on the same kernel on Armada 388 GP the pool
allocation does not fail. I really suspect an issue with my u-boot.
> And send me a full log beginning from u-boot?
>
>>
>> How do you test test it exactly?
>> Especially on which kernel and with which U-Boot?
>>
>
> I've just re-built the patchset I sent, which is on top of 4.4-rc1.
>
> I use AXP-GP, 78460 @ 1600MHz, 2GB DRAM, and everything works fine. My
> u-boot version: v2011.12 2014_T2.0_eng_dropv2.
My config is AXP-GP, 78460 @ 1300MHz, 8GB DRAM (only 4GB are used
because I didn't activated LPAE), but the main difference is the U-Boot
version: v2011.12 2014_T2.eng_dropv1.ATAG-test02.
Thanks,
Gregory
[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Linux version 4.4.0-rc1-00013-g76f111f9bdf8-dirty (gclement@FE-laptop) (gcc version 5.2.1 20151010 (Ubuntu 5.2.1-22ubuntu1) ) #1024 SMP Tue Dec 1 14:02:52 CET 2015
[ 0.000000] CPU: ARMv7 Processor [562f5842] revision 2 (ARMv7), cr=10c5387d
[ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
[ 0.000000] Machine model: Marvell Armada XP Development Board DB-MV784MP-GP
[ 0.000000] Memory policy: Data cache writealloc
[ 0.000000] PERCPU: Embedded 12 pages/cpu @ee1ac000 s18752 r8192 d22208 u49152
[ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 981504
[ 0.000000] Kernel command line: console=ttyS0,115200 earlyprintk mvneta.rxq_def=2
[ 0.000000] log_buf_len individual max cpu contribution: 4096 bytes
[ 0.000000] log_buf_len total cpu_extra contributions: 12288 bytes
[ 0.000000] log_buf_len min size: 16384 bytes
[ 0.000000] log_buf_len: 32768 bytes
[ 0.000000] early log buf free: 14924(91%)
[ 0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes)
[ 0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
[ 0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
[ 0.000000] Memory: 3888204K/3932160K available (5576K kernel code, 251K rwdata, 1544K rodata, 4460K init, 207K bss, 43956K reserved, 0K cma-reserved, 3145728K highmem)
[ 0.000000] Virtual kernel memory layout:
[ 0.000000] vector : 0xffff0000 - 0xffff1000 ( 4 kB)
[ 0.000000] fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
[ 0.000000] vmalloc : 0xf0800000 - 0xff800000 ( 240 MB)
[ 0.000000] lowmem : 0xc0000000 - 0xf0000000 ( 768 MB)
[ 0.000000] pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB)
[ 0.000000] modules : 0xbf000000 - 0xbfe00000 ( 14 MB)
[ 0.000000] .text : 0xc0008000 - 0xc06fc374 (7121 kB)
[ 0.000000] .init : 0xc06fd000 - 0xc0b58000 (4460 kB)
[ 0.000000] .data : 0xc0b58000 - 0xc0b96d00 ( 252 kB)
[ 0.000000] .bss : 0xc0b96d00 - 0xc0bcaa58 ( 208 kB)
[ 0.000000] Hierarchical RCU implementation.
[ 0.000000] Build-time adjustment of leaf fanout to 32.
[ 0.000000] NR_IRQS:16 nr_irqs:16 16
[ 0.000000] L2C: DT/platform modifies aux control register: 0x1a69ef10 -> 0x1a69ef12
[ 0.000000] Aurora cache controller enabled, 32 ways, 2048 kB
[ 0.000000] Aurora: CACHE_ID 0x00000100, AUX_CTRL 0x1a69ef12
[ 0.000000] Switching to timer-based delay loop, resolution 40ns
[ 0.000004] sched_clock: 32 bits at 25MHz, resolution 40ns, wraps every 85899345900ns
[ 0.000015] clocksource: armada_370_xp_clocksource: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 76450417870 ns
[ 0.000305] Console: colour dummy device 80x30
[ 0.000322] Calibrating delay loop (skipped), value calculated using timer frequency.. 50.00 BogoMIPS (lpj=250000)
[ 0.000332] pid_max: default: 32768 minimum: 301
[ 0.000417] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes)
[ 0.000425] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes)
[ 0.000737] CPU: Testing write buffer coherency: ok
[ 0.000880] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
[ 0.001006] Setting up static identity map for 0x8280 - 0x82d8
[ 0.001224] mvebu-soc-id: MVEBU SoC ID=0x7846, Rev=0x2
[ 0.001321] mvebu-pmsu: Initializing Power Management Service Unit
[ 0.002062] Booting CPU 1
[ 0.004402] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001
[ 0.004645] Booting CPU 2
[ 0.006984] CPU2: thread -1, cpu 2, socket 0, mpidr 80000002
[ 0.007219] Booting CPU 3
[ 0.009558] CPU3: thread -1, cpu 3, socket 0, mpidr 80000003
[ 0.009605] Brought up 4 CPUs
[ 0.009615] SMP: Total of 4 processors activated (200.00 BogoMIPS).
[ 0.009620] CPU: All CPU(s) started in SVC mode.
[ 0.009957] devtmpfs: initialized
[ 0.012369] VFP support v0.3: implementor 56 architecture 2 part 20 variant 9 rev 6
[ 0.012525] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[ 0.012657] pinctrl core: initialized pinctrl subsystem
[ 0.013188] NET: Registered protocol family 16
[ 0.013368] DMA: preallocated 256 KiB pool for atomic coherent allocations
[ 0.040064] cpuidle: using governor ladder
[ 0.070061] cpuidle: using governor menu
[ 0.075168] hw-breakpoint: Debug register access (0xee003e17) caused undefined instruction on CPU 3
[ 0.075172] hw-breakpoint: Debug register access (0xee003e17) caused undefined instruction on CPU 1
[ 0.075176] hw-breakpoint: Debug register access (0xee003e17) caused undefined instruction on CPU 2
[ 0.075179] hw-breakpoint: CPU 1 failed to disable vector catch
[ 0.075196] hw-breakpoint: Debug register access (0xee003e17) caused undefined instruction on CPU 0
[ 0.110815] vgaarb: loaded
[ 0.110981] SCSI subsystem initialized
[ 0.111306] usbcore: registered new interface driver usbfs
[ 0.111349] usbcore: registered new interface driver hub
[ 0.111398] usbcore: registered new device driver usb
[ 0.111640] Advanced Linux Sound Architecture Driver Initialized.
[ 0.111991] Bluetooth: Core ver 2.21
[ 0.112024] NET: Registered protocol family 31
[ 0.112029] Bluetooth: HCI device and connection manager initialized
[ 0.112040] Bluetooth: HCI socket layer initialized
[ 0.112048] Bluetooth: L2CAP socket layer initialized
[ 0.112067] Bluetooth: SCO socket layer initialized
[ 0.112462] clocksource: Switched to clocksource armada_370_xp_clocksource
[ 0.119267] NET: Registered protocol family 2
[ 0.119629] TCP established hash table entries: 8192 (order: 3, 32768 bytes)
[ 0.119690] TCP bind hash table entries: 8192 (order: 4, 65536 bytes)
[ 0.119749] TCP: Hash tables configured (established 8192 bind 8192)
[ 0.119791] UDP hash table entries: 512 (order: 2, 16384 bytes)
[ 0.119814] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
[ 0.119945] NET: Registered protocol family 1
[ 0.120143] RPC: Registered named UNIX socket transport module.
[ 0.120150] RPC: Registered udp transport module.
[ 0.120155] RPC: Registered tcp transport module.
[ 0.120160] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 0.290263] hw perfevents: enabled with armv7_cortex_a9 PMU driver, 7 counters available
[ 0.291305] futex hash table entries: 1024 (order: 4, 65536 bytes)
[ 0.292696] bounce: pool size: 64 pages
[ 0.292746] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
[ 0.292758] io scheduler noop registered
[ 0.292766] io scheduler deadline registered
[ 0.292779] io scheduler cfq registered (default)
[ 0.293754] armada-xp-pinctrl f1018000.pin-ctrl: registered pinctrl driver
[ 0.294186] irq: Cannot allocate irq_descs @ IRQ48, assuming pre-allocated
[ 0.294451] irq: Cannot allocate irq_descs @ IRQ80, assuming pre-allocated
[ 0.294612] irq: Cannot allocate irq_descs @ IRQ112, assuming pre-allocated
[ 0.295033] mv_xor f1060900.xor: Marvell shared XOR driver
[ 0.332873] mv_xor f1060900.xor: Marvell XOR (Registers Mode): ( xor cpy intr )
[ 0.372848] mv_xor f1060900.xor: Marvell XOR (Registers Mode): ( xor cpy intr )
[ 0.372958] mv_xor f10f0900.xor: Marvell shared XOR driver
[ 0.412850] mv_xor f10f0900.xor: Marvell XOR (Registers Mode): ( xor cpy intr )
[ 0.452845] mv_xor f10f0900.xor: Marvell XOR (Registers Mode): ( xor cpy intr )
[ 0.492634] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[ 0.493897] console [ttyS0] disabled
[ 0.514007] f1012000.serial: ttyS0 at MMIO 0xf1012000 (irq = 20, base_baud = 15625000) is a 16550A
[ 1.200947] console [ttyS0] enabled
[ 1.225041] f1012100.serial: ttyS1 at MMIO 0xf1012100 (irq = 21, base_baud = 15625000) is a 16550A
[ 1.254648] f1012200.serial: ttyS2 at MMIO 0xf1012200 (irq = 33, base_baud = 15625000) is a 16550A
[ 1.284237] f1012300.serial: ttyS3 at MMIO 0xf1012300 (irq = 34, base_baud = 15625000) is a 16550A
[ 1.294303] sata_mv f10a0000.sata: slots 32 ports 2
[ 1.300098] scsi host0: sata_mv
[ 1.303511] scsi host1: sata_mv
[ 1.306805] ata1: SATA max UDMA/133 irq 31
[ 1.310913] ata2: SATA max UDMA/133 irq 31
[ 1.315482] pxa3xx-nand f10d0000.nand: This platform can't do DMA on this device
[ 1.323280] nand: device found, Manufacturer ID: 0x2c, Chip ID: 0x38
[ 1.329650] nand: Micron MT29F8G08ABABAWP
[ 1.333684] nand: 1024 MiB, SLC, erase size: 512 KiB, page size: 4096, OOB size: 224
[ 1.341450] pxa3xx-nand f10d0000.nand: ECC strength 16, ECC step size 2048
[ 1.349163] Bad block table found at page 261888, version 0x01
[ 1.355385] Bad block table found at page 262016, version 0x01
[ 1.364466] pxa3xx_nand-0: 'partitions' subnode not found on /soc/internal-regs/nand@d0000. Trying to parse direct subnodes as partitions.
[ 1.378215] m25p80 spi0.0: n25q128a13 (16384 Kbytes)
[ 1.385098] spi0.0: 'partitions' subnode not found on /soc/internal-regs/spi@10600/spi-flash@0. Trying to parse direct subnodes as partitions.
[ 1.399136] libphy: Fixed MDIO Bus: probed
[ 1.403396] libphy: orion_mdio_bus: probed
[ 1.415856] mvneta_bm f10c0000.bm: Buffer Manager for network controller enabled
[ 1.424232] size = 0x0, target = 0x0, attr = 0x1e
[ 1.428949] mvebu_mbus: invalid dram address 0x2d194000
[ 1.434204] mvneta_bm f10c0000.bm: fail to create pool 0
[ 1.439532] mvneta f1070000.ethernet eth0: fail to obtain long pool for port
[ 1.446607] mvneta f1070000.ethernet: use SW buffer management
[ 1.452696] mvneta f1070000.ethernet eth0: Using hardware mac address 00:50:43:68:1c:28
[ 1.461529] size = 0x0, target = 0x0, attr = 0x1e
[ 1.466258] mvebu_mbus: invalid dram address 0x2d198000
[ 1.471501] mvneta_bm f10c0000.bm: fail to create pool 1
[ 1.476836] mvneta f1074000.ethernet eth1: fail to obtain long pool for port
[ 1.483910] mvneta f1074000.ethernet: use SW buffer management
[ 1.489994] mvneta f1074000.ethernet eth1: Using random mac address be:14:de:e2:be:cf
[ 1.498661] size = 0x0, target = 0x0, attr = 0x1e
[ 1.503391] mvebu_mbus: invalid dram address 0x2d10c000
[ 1.508633] mvneta_bm f10c0000.bm: fail to create pool 2
[ 1.513968] mvneta f1030000.ethernet eth2: fail to obtain long pool for port
[ 1.521035] mvneta f1030000.ethernet: use SW buffer management
[ 1.527121] mvneta f1030000.ethernet eth2: Using random mac address 86:3c:f8:02:31:cf
[ 1.535810] size = 0x0, target = 0x0, attr = 0x1e
[ 1.540527] mvebu_mbus: invalid dram address 0x2d752000
[ 1.545781] mvneta_bm f10c0000.bm: fail to create pool 3
[ 1.551108] mvneta f1034000.ethernet eth3: fail to obtain long pool for port
[ 1.558182] mvneta f1034000.ethernet: use SW buffer management
[ 1.564269] mvneta f1034000.ethernet eth3: Using random mac address 7e:3e:74:8c:93:50
[ 1.572297] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 1.578863] ehci-pci: EHCI PCI platform driver
[ 1.583371] ehci-orion: EHCI orion driver
[ 1.587476] orion-ehci f1050000.usb: EHCI Host Controller
[ 1.592916] orion-ehci f1050000.usb: new USB bus registered, assigned bus number 1
[ 1.600567] orion-ehci f1050000.usb: irq 27, io mem 0xf1050000
[ 1.622494] orion-ehci f1050000.usb: USB 2.0 started, EHCI 1.00
[ 1.628967] hub 1-0:1.0: USB hub found
[ 1.632806] hub 1-0:1.0: 1 port detected
[ 1.637018] orion-ehci f1051000.usb: EHCI Host Controller
[ 1.642468] orion-ehci f1051000.usb: new USB bus registered, assigned bus number 2
[ 1.650111] orion-ehci f1051000.usb: irq 28, io mem 0xf1051000
[ 1.672467] orion-ehci f1051000.usb: USB 2.0 started, EHCI 1.00
[ 1.678908] hub 2-0:1.0: USB hub found
[ 1.682736] hub 2-0:1.0: 1 port detected
[ 1.687203] usbcore: registered new interface driver usb-storage
[ 1.693454] mousedev: PS/2 mouse device common for all mice
[ 1.699593] rtc-mv f1010300.rtc: rtc core: registered f1010300.rtc as rtc0
[ 1.706641] i2c /dev entries driver
[ 1.711074] orion_wdt: Initial timeout 171 sec
[ 1.715770] cpu: dev_pm_opp_of_cpumask_add_table: couldn't find opp table for cpu:0, -19
[ 1.723985] cpu: dev_pm_opp_of_cpumask_add_table: couldn't find opp table for cpu:1, -19
[ 1.732174] cpu: dev_pm_opp_of_cpumask_add_table: couldn't find opp table for cpu:2, -19
[ 1.740372] cpu: dev_pm_opp_of_cpumask_add_table: couldn't find opp table for cpu:3, -19
[ 1.748968] sdhci: Secure Digital Host Controller Interface driver
[ 1.755176] sdhci: Copyright(c) Pierre Ossman
[ 1.759723] sdhci-pltfm: SDHCI platform and OF driver helper
[ 1.765805] usbcore: registered new interface driver usbhid
[ 1.771394] usbhid: USB HID core driver
[ 1.775730] f0000000.nor: Found 1 x16 devices at 0x0 in 16-bit bank. Manufacturer ID 0x000089 Chip ID 0x000018
[ 1.785775] Intel/Sharp Extended Query Table at 0x0031
[ 1.790936] Intel/Sharp Extended Query Table at 0x0031
[ 1.796106] Using buffer write method
[ 1.799778] cfi_cmdset_0001: Erase suspend on write enabled
[ 1.809269] f0000000.nor: 'partitions' subnode not found on /soc/devbus-bootcs/nor@0. Trying to parse direct subnodes as partitions.
[ 1.822513] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl F300)
[ 1.829037] NET: Registered protocol family 10
[ 1.834180] sit: IPv6 over IPv4 tunneling driver
[ 1.839209] NET: Registered protocol family 17
[ 1.842687] ata1.00: ATA-9: TS8GHSD310, 20111220, max UDMA/133
[ 1.842692] ata1.00: 15621984 sectors, multi 1: LBA48 NCQ (depth 31/32)
[ 1.856343] ThumbEE CPU extension supported.
[ 1.860632] Registering SWP/SWPB emulation handler
[ 1.862685] ata1.00: configured for UDMA/133
[ 1.862993] scsi 0:0:0:0: Direct-Access ATA TS8GHSD310 1220 PQ: 0 ANSI: 5
[ 1.863646] sd 0:0:0:0: [sda] 15621984 512-byte logical blocks: (7.99 GB/7.44 GiB)
[ 1.863838] sd 0:0:0:0: [sda] Write Protect is off
[ 1.863907] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 1.873801] sda: sda1 sda2 sda3 < sda5 > sda4
[ 1.875389] sd 0:0:0:0: [sda] Attached SCSI disk
[ 1.909334] rtc-mv f1010300.rtc: setting system clock to 2001-01-20 07:57:42 UTC (979977462)
[ 1.918901] ALSA device list:
[ 1.921878] No soundcards found.
[ 2.222505] ata2: SATA link down (SStatus 0 SControl F300)
[ 2.230484] Freeing unused kernel memory: 4460K (c06fd000 - c0b58000)
--
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/