Re: [RFC PATCH] watchdog: Add hook for kicking in kdump path

[Guenter Roeck]

On Tue, Apr 09, 2013 at 11:14:23AM -0400, Don Zickus wrote:
> On Tue, Apr 09, 2013 at 07:52:28AM -0700, Guenter Roeck wrote:
> > On Tue, Apr 09, 2013 at 10:44:31AM -0400, Don Zickus wrote:
> > > On Mon, Apr 08, 2013 at 08:15:09AM -0700, Guenter Roeck wrote:
> > > > On Mon, Apr 08, 2013 at 08:48:58AM -0400, Don Zickus wrote:
> > > > > On Mon, Apr 08, 2013 at 01:46:58PM +0800, Dave Young wrote:
> > > > > > On 04/06/2013 04:16 AM, Don Zickus wrote:
> > > > > > > A common problem with kdump is that during the boot up of the
> > > > > > > second kernel, the hardware watchdog times out and reboots the
> > > > > > > machine before a vmcore can be captured.
> > > > > > > 
> > > > > > > Instead of tellling customers to disable their hardware watchdog
> > > > > > > timers, I hacked up a hook to put in the kdump path that provides
> > > > > > > one last kick before jumping into the second kernel.
> > > > > > > 
> > > > > > > The assumption is the watchdog timeout is at least 10-30 seconds
> > > > > > > long, enough to get the second kernel to userspace to kick the watchdog
> > > > > > > again, if needed.
> > > > > > 
> > > > > > For kdump kernel some devices need to reset, this might increase the
> > > > > > boot time, it's not so reliable for the 10-30s for us to kicking the
> > > > > > watchdog.
> > > > > > 
> > > > > > Could we have another option to disable/stop the watchdog while panic
> > > > > > happens? Ie. add a kernel cmdline panic_stop_wd=<0|1> for 1st kernel, if
> > > > > > it's set to 1, then just stop the watchdog or we can kick the watchdog
> > > > > > like what you do in this patch. Of course stopping watchdog should be
> > > > > > lockless as well..
> > > > > 
> > > > > Hmm, I can look into that.  But I am not sure all watchdogs have the
> > > > > ability to stop once started.  I was also worried about the case where
> > > > 
> > > > Correct.
> > > > 
> > > > > kdump hangs for some reason.  Having the watchdog there to 'reboot' would
> > > > > be a nice safety net.
> > > > > 
> > > > Absolutely agree. After all, the reason for the kdump is most likely that
> > > > something went really wrong, meaning there is some likelyhood for the hang
> > > > to occur. Turning off the watchdog in this condition does not seem to be
> > > > a good idea.
> > > > 
> > > > > Perhaps adjusting the watchdog 'timeout' to something like 3 minutes would
> > > > > be easier?
> > > > > 
> > > > Not all watchdogs support such large timeouts, unfortunately. Maybe it would
> > > > make sense to implement infrastructure support for a softdog on top of the
> > > > hardware watchdog. Several drivers implement that outside the infrastructure
> > > > already.
> > > 
> > > Hi Guenter,
> > > 
> > > I am not familar with a softdog.  Can you give me an example of how it
> > > works?
> > > 
> > Just look for the use of mod_timer in the watchdog directory.
> 
> So looking at the mod_timer logic in various drivers, it seems regardless
> if the /dev/watchdog device is opened or not, if it is running, it will
> automagically kick the watchdog.
> 
yes

> This seems that we can avoid pulling in userspace pieces for this.  Just
> load the driver and the hardware starts getting kicked.
> 
Only if it is already running. Also, you don't want to rely on it, because you
lose protection against user space issues.

A second use is if the hw watchdog needs to be pinged more often than user
space can provide. Some of the HW watchdogs need a ping in one-second intervals
or even faster.

> Is that true?  And if so, do all drivers detect if the hardware is already
> running during their init?  Or is it based on the first device open?
> 
It is usually done in the probe function.

Guenter
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/