On Thu, Mar 16, 2000 at 12:12:11PM -0500, Richard B. Johnson wrote:
> On Thu, 16 Mar 2000, John Wilson wrote:
>
> > On Thu, Mar 16, 2000 at 08:22:38AM -0500, Richard B. Johnson wrote:
> > > No. Not a "user-buffer". It may be possible to memory-map something that's
> > > DMA..able to user space, but even that's not guaranteed. To have something
> > > DMA..able in the ix86 machines, it needs to be:
> > >
> > > (1) Physical, below 16 Megabytes
> > > (2) DATA alignment must be such that a 64k block is not crossed
> > > during a DMA transfer. The ix86 machines use a page register
> > > to access 64k pages.
> > > (3) You need access to the DMA controller which the kernel isn't
> > > going to give you, without a fight iopl(3), etc., from user-
> > > space.
> >
> > No no, these requirements are for ISA DMA, not DMA in general. I'm using
> > a PCI device, so there are no address restrictions (on x86 anyway, I dunno
> > how PCI works on CPUs with >32 addr bits), and it has its own DMA paging
> > registers so it doesn't require a contiguous buffer either. The buffer
> > just has to be locked in place, which sounds like mlock() would be my friend.
> >
> > Also, my driver is a real driver, it doesn't run in user space. I just want
> > it to be able to set the device up so that the DMA buffer (*only*) is in
> > user space, and let 'er rip.
> >
>
> If your device is doing it's own DMA, basically you are telling it to
> write its stuff to some physical address that you give it. If this
> is correct, then your driver needs to lock user pages in place if
> you are not going to use kernel memory.
>
We are dealing with the same sort of issue at LLNL for a high speed
interconnect card that has its own MMU. We wrote a generic
co-processor interface into the kernel to handle the issues involved.
What happens is the card's MMU has tables on it to map pages of memory
on the host cpu to a contiguous block of memory that it can use to
DMA. In essence, it duplicates the memory tables for a particular
process in the card's memory. When a page in that process is swapped
out or is no longer valid, the hooks we added allows a device driver
to register a callback that gets called when this happens. It seems
to work really well. This makes it so no pages need to be locked
down, which makes GANG scheduling of large jobs MUCH easier to deal
with.
We are in the process of getting permission to release all of the
kernel modifications we've been making. As you can imagine, sometimes
it is hard to get things out of a DOE controlled lab. :)
BAPper
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Thu Mar 23 2000 - 21:00:23 EST