Paul> Jes Sorensen <Jes.Sorensen@cern.ch> wrote:
>> My point is that the current implementation expects no protection
>> against reordering and many current device drivers already deal
>> with this correctly (I checked the aic7xxx this this morning and I
>> noticed that it deals with this explicitly, even on the PPC).
Paul> Richard Henderson mentioned four drivers which he uses which
Paul> deal with this: ncr, aic7xxx, tulip, epic100. How many others
Paul> would there be in this category? It sounds to me like the vast
Paul> majority of drivers which use readl/writel would be assuming
Paul> that they prevent reordering.
I grepped through a bunch of drivers in 2.3.13 and I think we are
about 50/50 on this one. The Cyclades driver deals with it as well and
I know my AceNIC Gigabit Ethernet and the Road Runner HIPPI drivers
deal with it as well. However a couple of 100 Mbit Ethernet drivers
were not handling this.
That aside, the number of drivers actually using PCI shared memory
access at the moment is very limited and it wouldn't be much work
fixing the existing drivers if we decide on writel not guaranteeing
write ordering.
Paul> My position is that readl/writel should prevent reordering (of
Paul> the accesses generated by read*/write*), and that we should have
Paul> variants, maybe called readl_fast etc., for the cases where
Paul> performance is crucial and the driver writer is willing to put
Paul> in explicit barriers where necessary.
The things I do not like about this is that we will then get four
versions of writel out of this, slow/fast and little/big endian and it
will make current drivers, which are already written to handle this,
less efficient. IMHO there are already enough interesting issues for
driver authors to deal with that hiding write ordering issues is not
going to help much. I'd personally go for the performance and then
spend more effort on teaching driver authors to do things the right
way.
I'd be happy to write a 'how to deal with write ordering issue
correctly on PCI shared memory' document for Documentation/ if we
decide on a writel implementation working the way the current Alpha
implementation does.
Paul> IMHO the time spent doing readl/writel is absolutely
Paul> insignificant in the vast majority of cases. If you have to
Paul> feed a device so fast that the time spent doing readl/writel is
Paul> significant, then you should either be using DMA (or for a
Paul> framebuffer, a graphics engine) or you should be feeding it from
Paul> user level (as in the case of a 3D graphics card).
For frame buffers I think it is significant enough for the console
code. However for Gigabit Ethernet at high speeds 60MB/sec+ TCP I have
already experienced ``interesting'' effects from having cache line
ping-pongs in the wrong places caused by simple pointer writes by
multiple processors on SMP systems - I can see write ordering writel's
having impacts here as well.
Paul> Interestingly, the 4 drivers mentioned by Richard all do DMA
Paul> (AFAIK). Which leads me to wonder why the speed of readl/writel
Paul> is so significant that it is necessary to have the barriers
Paul> explicit in the driver instead of including them in
Paul> readl/writel. Surely those drivers can't be doing more than a
Paul> couple of readl/writel per packet or block?
On Gigabit Ethernet we are easily talking 80000 pkts/sec ;-)
Jes
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/