> You have to make changes in both the main copy and the dependency
> structures; think about it. Future references to the meta data *either*
> need to get the up-to-date information in the main copy, *OR* the
> code which references the metatdata would have to first search the
> "outstanding modifications" structure, and then search the main copy.
> This is both (a) complex and (b) slow.
oops, you are right, i misunderstood the paper.
> The paper quite explicitly states that the changes in the main copy are
> *undone*, written to disk, and then *redone*. This way, in-memory
> references to the filesystem get all of the latest changes, but what is
> written to disk may noto have the latest changes, in order to preserve
> metadata consistency.
ok, but this really brings in that contention problem you were talking
about.
> Alternatively, you can double buffer things by copying the block to
> scratch space and then doing the undo operations, but then you have to
> pay the cost of the copy, and the buffer layer needs to know that it
> should be writing the block from the scratch space instead of the main
> memory copy.
What about the following modification to your scratch copy technique.
Instead of copying to scratch buffer before write, what about keeping two
metadata blocks around, modifying the 'kernel copy', and correctly
propagating changes to the 'disk copy'. This propagation does not advance
when there is a write operation going on to that block. The propagation
stuff has no blocking effect on modifications done to the kernel copy.
(full asynchronity)
this way we do not pay the full cost of copying a full block, only
double-modification cost, double memory for metadata, plus the overhead of
the dependency technique, which is hopefully constant and smallish.
> Finally, it can't be filesystem-independent because the code to do the
> undo and redo operations inherently must be filesysten-dependent,
> because it depends on the metadata structure.
yes, but i mean the 'main bulk' can be filesystem-independent. The
filesystem itself has to represent it's metadata operations properly, and
this is distinct from the current 'modify buffer directly and mark it
dirty plus be smart about the order things'.
but if a filesystem is compiled with 'no-soft-updates', then it would be
the same as currently, on the assembly level. I think this can be done
with GCC, but it's not trivial.
it is a largescale change. [this is what you said too]
-- mingo