Re: Linux/IA-64 byte order

Richard Gooch (rgooch@atnf.csiro.au)
Tue, 9 Mar 1999 15:59:51 +1100

Oliver Xymoron writes:
> On Tue, 9 Mar 1999, Richard Gooch wrote:
>
> > I write visualisation software for astronomy. This software is used
> > all over the world, and often has to deal with very large
> > datasets. It's not uncommon to "load" a dataset (a cube) but only view
> > a small portion of it (a single plane (channel) of the cube). On
> > big-endian machines I can avoid loading data and instead use memory
> > mapping, because all the portable binary data formats are big-endian
> > (FITS, Miriad and my own).
> >
> > Being able to memory map a gigabyte dataset makes "loading" extremely
> > fast. It also means you don't need stacks of swap space.
>
> Make an empty map for little-endian direct access and make a backing
> map with the actual file. Then install a signal handler to catch
> reads in the empty map, copy a page (or more) with byteswapping, and
> you're done. "Loading" is still extremely fast.

Yes, "loading" a subsection of the data is still (fairly) fast. But
that doesn't solve the other problem: huge swap space consumption when
you want to access the entire dataset. It's often the case that you
have much more data disc than you have swap disc.

> As David Miller suggests, this might be a complete non-issue. We can
> for instance speculate that the processor has a byteswap micro-op
> that can be added to an instruction pipeline with little or no
> cost. Or other such magic. Remember, we're talking about a fairly
> novel architecture.

Special BE load instructions are going to be messy to use, because you
have to keep track of "where" the data came from (mmap() or malloc())
all the way along.

The possibility of tagging regions for BE access via the page tables
looks good, though. Hopefully IA-64 will support it. And hopefully
Linus accepts such an approach.

Regards,

Richard....

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/