See my recent message where I quote numbers showing 30% variance in
the lat_ctx numbers. It appears your code has more variance than you
expected.
In case you haven't noticed, I *have* been "debugging" my
benchmark. Why don't you stop arguing and be constructive instead?
> : Maybe I have a "complex" because your first (private) message to me
> : started off with "What the fuck are you talking about?". And you've
> : been pretty agressive about the whole thing and have used emotive
> : words.
>
> If I've hurt your feelings, my apologies (as I said immediately in
> private mail as well, nice of you to leave that out).
Yes, you did apologise, but a few messages later you were back to
being impolite: "Your test is broken", "if you think about it for a
millisecond". It's clear your first response is to come out all guns
blazing with maximum agression. You exhibit a pattern of behaviour
that is unproductive and offensive.
Every suggestion you've made as to why my test is broken (ranging from
"look at top and see the low-priority processes run instead" to "read
/proc/stat and count the unexpected context switches" (paraphrased))
has not identified the source of the variance in my tests.
Now I'm finding that you code also yields greater-than-expected
variance. So are you still convinced that my test is broken (even
though neither you, Linus or anyone else has shown *what* is broken)?
> : Even if the variances in my measurement turn out to be due to a broken
> : test or sensitivity to some "uninteresting" effect, there is no
> : denying that an increased run queue length slows down context switch
> : times. With my test I get a cost of 0.2 us per process and with yours
> : I get 0.19 us (PPro 180).
>
> I couldn't agree more. So maybe rather than trying to fix an
> operating system that isn't broken, you should focus on getting rid
> of all those extra processes. A lot of my frustration with this
> argument is that I've been through this with customer after customer
> at Sun and SGI. They all want you to change the operating system to
> suit their application when what they should do is write their
> application correctly in the first place. Too many processes are
> going to screw you up no matter what queue they are on and no matter
> what scheduler you use.
There are at least two separate discussions going on here. One is
about the cost of extra processes on the run queue. I've demonstrated
similar numbers for this with lmbench and my code. It see we have
provisional agreement on this cost. The discussion now is whether the
cost is significant.
The other discussion regards the variance I have measured. Here you
persist in saying my test is flawed, and yet so far not a single one
of your suggestions as to why it may be flawed has stood up to the
test. Irrespective of whether my test turns out to be flawed, my point
about run queue length costs remains.
Now, I agree in principle that you should fix broken applications
rather than fix the OS. But it doesn't always work this way. There may
be legitimate reasons for doing things a particular way. I have
demonstrated a real-world case where there are up to 10 normal
processes on the run queue. The "obvious" solution is to place the RT
application on a separate machine and avoid the whole problem. This is
not necessarily the right solution, as I have already indicated.
One issue is the cost of transferring data between machines. The other
issue is cost of hardware: buying a second machine is not always an
option.
The change I'm proposing (two run queues) is very simple, it is robust
and it in fact will *simplfy* some code. The current scheduler has to
special-case RT processes, complicating the code and making it more
prone to bugs. In fact, I found a bug this morning: RT processes which
yield can end up giving control to lower priority processes. This is
contrary to the POSIX.4 spec. and contrary to what I'd expect.
I've fixed it and sent a patch to Linus. Not surprisingly, the patch
complicates the code path, because we need yet another special case.
> This discussion reminds me of the C++ people that keep asking for
> bigger hardware caches so their code will run fast. They are
> constantly amazed when you show the less lines of C code, that
> solves the problem they are trying to solve, using less cache and
> running faster. You tell us - what's the right thing to do - fix
> the hardware or fix the C++?
Don't get me started on C++. It's the most powerful language in use
today. It is now an order of magnitude easier to write bloatware in
10% of the time. Fantastic.
However, sometimes we don't get to make the choices. We're stuck with
the situation we're placed in and we have to make the best of it.
If there is a cheap solution that allows the application to run
faster, it should be considered seriously.
We should not allow general design principles to become mantra. That
way lies inflexibility.
> : If you're not interested in being constructive (i.e. "I think your
> : variance may be due to XYZ effect"), then why not get off my back and
> : let me get on with tracking down the problem?
> : If I propose a mechanism which I think may explain the variance, and
> : you don't agree, all you need to do is say so and why. Repeating the
> : mantra "your test is broken" is pointless.
>
> You keep asking me to debug it.
No, I'm not asking you to debug it. All I ask is that you don't feel
like being constructive, at least don't be destructive. Constructive
criticism is welcome. Finger pointing, abuse and agression are not.
> Please remember that I'm not at some cushy job, I'm spending my
> savings working tools for Linux kernel developers. It's not like I
> have tons of idle cycles to waste on your code. I've already put
> out the effort, for free, to make an as portable and accurate
> benchmark as I possibly can - a benchmark, by the way, that measures
> a lot more than just toy context switches. It's been a tremendous
> amount of work over quite a few years for which I've not received a
> dime.
Ra, ra, ra. Do you want a medal? Do you have any idea of how much time
I've contributed to the Linux community? Is it relevant? No.
You seem to have enough time argue about how my test is fundamentally
flawed. You could have saved yourself time by just pointing out that
you haven't measured this variance with lmbench and left it at that.
Instead, you feel the need to constantly point out how my test is
broken. An assertion you haven't been able to substantiate, BTW.
Unexpected results don't automatically invalidate the results.
> It's bad enough making sure that my code is not full of bugs -
> explain to me what possible reason I could have for debugging your
> code. I think it's enough that I've proved that a properly written
> test won't show that variance.
I don't demand you debug my code (again: it's not clear that my code
actually has a bug).
> I think that the onus is on you to debug your own code. And until
> you can show that you have it debugged, maybe it would be better if
> you deferred to the code which has been show to work.
You carry on as though I'm not investigating the cause of the variance
in my measurements.
I'm not going to "defer" to code that (it now transpires) also shows
variance. That will just hide the problem. I want to expose the
problem and track it down.
BTW: your comment about "deferring" betrays an underlying arrogance.
So, if you're not willing to be constructive, or willing to admit that
your code may be flawed because it's *insensitive* to the effect(s),
just go away and let me get on with tracking down the problem.
Regards,
Richard....
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/