Re: [PATCH 03/18] f2fs crypto: declare some definitions for f2fs encryption feature

[Tom Marshall]

On Fri, May 15, 2015 at 06:14:24PM -0700, Jaegeuk Kim wrote:
> On Thu, May 14, 2015 at 09:50:44AM -0700, Tom Marshall wrote:
> > Please keep in mind that I'm also working on transparent
> > compression.  I'm watching this thread closely so that I can
> > implement a compression library alongside the crypto library.  If
> > there is any interest or benefit, I would be glad to work together
> > so that the two can be done cooperatively at the same time.
> 
> I can't imagine quickly how compression code can be shared with crypto.
> The basic approach for compression would be that X pages can be compressed into
> small number of pages, Y, which can be a X to Y mapping.
> But, this per-file encryption supports only 1 to 1 4KB mapping, so that it could
> be quite a simple implementation.

No, I don't intend to share actual code with crypto -- at least not much. 
I'm more interested in looking at how the crypto layer is implemented to
give me clues about how to implement a compression layer.

> Could you elaborate on your approach or design? Or, codes?
> Whatever, IMO, it needs to implement it by any filesystem first.

I don't really have any working code yet.  I will probably get to that in
the coming few weeks.  Right now I'm still working with the ugly VFS
stacking implementation that I posted initially.

The thing that I have done is dismissed the standard compression framing
formats.

zlib (and gzip) are designed for streaming and it is quite difficult to
implement random access on it.  See the example code in the zlib source,
zran.c.  It's not really tenable because 32kb of prior data is required to
be kept as priming information.  Even doing fully encapsulated blocks with
Z_FINISH, there is still no way to skip over data without decompressing it
first to build an index.

lz4 is somewhat better in that blocks are self contained.  But block lengths
must be read sequentially.  This means that reading an arbitrary position in
a file requires a proportional number of reads to find the desired block.

So, I am working with a simple framing format that I threw together.  The
header has a compression method (zlib or lz4), block size, original input
size, and a block map.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/