Re: PROBLEM: high system usage / poor SMP network performance

From: Andrew Morton (akpm@zip.com.au)
Date: Sun Jan 27 2002 - 17:42:49 EST


Vincent Sweeney wrote:
>
> Naturally I would really like to know where / what is using up all this
> system cpu so I would like to try profiling the kernel as I'm sure this is a
> pure kernel network layer performance issue but I have no idea where to
> start so does anyone have some advice / tips on where I should start?

Yes, profiling the kernel is clearly the next step. And it's
really easy.

1: If possible, rebuild your kernel with as few modules as possible.
   Current profiler doesn't cope with code which is in modules.

2: If you're on uniprocessor, enable the "Local APIC support on
   Uniprocessors" option. This allows higher-resolution profiling.

3: Arrange for the kernel to be provided the `profile=1' boot
   option. I use

        append="profile=1"

   in /etc/lilo.conf

   After a reboot, the kernel is profiling itself. The overhead is
   really low.

4: Bring the server online and wait until it starts to play up.

Now we can profile it. I use this script:

mnm:/home/akpm> cat $(which akpm-prof)
#!/bin/sh
TIMEFILE=/tmp/$(basename $1).time
sudo readprofile -r
sudo readprofile -M10
time "$@"
readprofile -v -m /boot/System.map | sort -n +2 | tail -40 | tee $TIMEFILE
echo created $TIMEFILE

Let's go through it:

        readprofile -r

                This clears out the kernel's current profiling info

        readprofile -M10

                This attempts to set the profiling interval to 10*HZ
                (1000 Hz). This requires a local APIC, and a recent
                util-linux package. Not very important if this fails.
                This command also cleans out the kernel's current
                profiling info (it's a superset of -r).

        time "$@"

                Runs the command which we wish to profile

        readprofile ...

                Emits the profile info, sorted in useful order.
                You must make sure that /boot/System.map is the
                correct one for the currently-running kernel!

So in your situation, the command which you want to profile isn't
important - you want to profile kernel activity arising from
*existing* processes. So you can use:

        akpm-prof sleep 30

Please send the results!

-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Thu Jan 31 2002 - 21:00:46 EST