RE: Weird SysRq's during reboot

From: Alan
Date: Thu Jul 05 2018 - 10:02:55 EST


Panic over. It seems that my OnApp control server, when it loses contact
with a compute node, sends an snmptrap command to force a reboot of that
node. The trap script on the server uses sysrq-trigger, first "s", then "u",
then "b".

Please accept my apologies if you wasted any time on this.

Regards, Alan Griffiths

Bytehouse

-----Original Message-----
From: Alan [mailto:alan@xxxxxxxxxxxxxxx]
Sent: 05 July 2018 11:19
To: 'linux-kernel@xxxxxxxxxxxxxxx' <linux-kernel@xxxxxxxxxxxxxxx>
Subject: Weird SysRq's during reboot

Hello everyone, this is my first time posting to this list so please be
gentle.

I have a server, hardware spec:

HP DL360p G8
2 x Intel Xeon E5-2620 Hex-Core 2GHz processors
150GB Ram
Embedded HP Ethernet 1Gb 4-port 331FLR Adapter
Embedded Smart Array P420i Controller
PCI-E Slot 1 HP Ethernet 10Gb 2-port 530SFP+ Adapter

which I am trying to configure as an OnApp compute engine. Recently I have
seen the following weird behaviour:

1. Login on the server as root
2. Enter a 'shutdown -r now' command
3. Watch the reboot on the iLO console
4. System reboots except that right at the end, after the login
prompt appears on the console, I see:

SysRq: Emergency Sync
Emergency Sync complete
SysRq : Emergency Remount R/O

And then the system reboots again.
5. The second reboot always works without the SysRq.

Looking in /var/log/messages I see, just prior to the second reboot (*):

Jul 5 10:30:17 hv08 kernel: Ebtables v2.0 registered
Jul 5 10:30:17 hv08 kernel: Bridge firewalling registered
Jul 5 10:30:17 hv08 kernel: arp_tables: (C) 2002 David S. Miller
Jul 5 10:30:18 hv08 kernel: lo: Disabled Privacy Extensions
* Jul 5 10:30:27 hv08 kernel: SysRq : Emergency Sync
* Jul 5 10:30:27 hv08 kernel: Emergency Sync complete
* Jul 5 10:30:28 hv08 kernel: SysRq : Emergency Remount R/O
Jul 5 10:33:43 hv08 kernel: imklog 5.8.10, log source = /proc/kmsg
started.
Jul 5 10:33:43 hv08 rsyslogd: [origin software="rsyslogd"
swVersion="5.8.10" x-pid="2491" x-info="http://www.rsyslog.com";] start
Jul 5 10:33:43 hv08 kernel: Initializing cgroup subsys cpuset
Jul 5 10:33:43 hv08 kernel: Initializing cgroup subsys cpu
Jul 5 10:33:43 hv08 kernel: Linux version
2.6.32-696.30.1.el6.x86_64 (mockbuild@xxxxxxxxxxxxxxxxxxxx) (gcc version
4.4.7 20120313 (Red Hat 4.4.7-18) (GCC) ) #1 SMP Tue May 22 03:28:18 UTC
2018
Jul 5 10:33:43 hv08 kernel: Command line: ro
root=UUID=eafc533f-c5cb-4c89-bc0d-33125ab7f9d9 nomodeset rd_NO_LUKS
KEYBOARDTYPE=pc KEYTABLE=uk LANG=en_US.UTF-8 rd_NO_MD
SYSFONT=latarcyrheb-sun16 crash kernel=auto rd_NO_LVM rd_NO_DM rhgb quiet

I have tried booting the 2.6.32-696.28.1.el6.x86_64 and the
2.6.32-696.el6.x86_64 kernels with the same result. The problem does not
occur if I boot from the Centos 6.9 liveDVD iso which also uses kernel
version 2.6.32-696.el6.x86_64.

So, I am hoping someone here can answer any of these questions for me:

A. What have I done to make this happen?
B. Is there a way I can fix it?
C. If not is there some other way to stop it happening?

Thank you for taking the time to read this email. I shall look forward to
hearing from you. Please CC me on any replies.

Regards, Alan Griffiths

Bytehouse