Re: interest in performance instrumentation

Chuck Lever (cel@monkey.org)
Thu, 11 Feb 1999 21:24:49 -0500 (EST)


On Fri, 12 Feb 1999, Richard Gooch wrote:
> > is there a list or group working on standardizing Linux performance
> > instrumentation and what tools are included with common Linux
> > distributions? for instance, is anyone working on "sar" or "iostat"
> > for Linux?
>
> What do these tools do?

sar comes from system V and is found in most Unix distributions. sar is a
tool that logs system performance and can extract certain parameters over
particular times. in other words, there's a log daemon part, and a user
part with which you can access the logs. stuff that sar might log
includes:

network packet rates
disk I/O parms like avg. request time, blocks in, blocks out
CPU user, system, idle
paging and swapping behavior

and so on. it's a sophisticated form of vmstat with logging built in.
it's very good for maintaining a picture of system performance trends.
you can also use the data for correlation of performance problems.

iostat is similar to vmstat, but it prints statistics about the I/O
subsystem rather than VM. sct has a skeleton C++ version of iostat, but i
don't think he's worked on it in a while. it comes with a kernel patch
because the data iostat is looking for isn't gathered in the stock Linux
kernel. so, to make tools like sar and iostat work, some effort has to be
focused on deciding what additional kernel instrumentation is required,
and which tools we like best.

when i was running a large production Unix shop on Solaris, my job
depended on having this kind of performance monitoring. we also developed
a Perl/web-based analysis tool that grabbed sar data on a regular basis,
graphed it, then compressed it and archived it. that kind of stuff is
great to show off to management, as well as being actually useful ;)

> Are you referring to measuring CPU performance
> (cache misses, TLB refills and so on) or something else? My MSR patch
> includes a thin driver for performance monitoring:
> http://www.atnf.csiro.au/~rgooch/linux/
>
> and is supported by a userspace library to yield easy access. The idea
> is that you can then build glitzy tools on top of the library.

i have your patch... it's very cool. i'd like some form of Intel MSR
support to be included in the stock kernel.

however, it's probably not useful for someone who's trying to tune their
web server, for example, since it focuses on very specific parts of the
performance picture. sar and vm/iostat can provide a good overall picture
of a system's performance, and sar can maintain performance history, which
is valuable when suddenly things stop working "the way they did
yesterday."

and besides, what about the alpha and sparc owners out there? :)

- Chuck Lever

--
corporate:	<chuckl@netscape.com>
personal:	<chucklever@netscape.net> or <cel@monkey.org>

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/