Re: [PATCH v3] exfat: remove EXFAT_SB_DIRTY flag

[Tetsuhiro Kohada]

> > When should VOL_DIRTY be cleared?
> >
> > The current behavior is ...
> >
> > Case of  mkdir, rmdir, rename:
> >     - set VOL_DIRTY before operation
> >     - set VOL_CLEAN after operating.
> > In async mode, it is actually written to the media after 30 seconds.
> >
> > Case of  cp, touch:
> >     - set VOL_DIRTY before operation
> >     - however, VOL_CLEAN is not called in this context.
> > VOL_CLEAN will call by sync_fs or unmount.
> >
> > I added VOL_CLEAN in last of __exfat_write_inode() and exfat_map_cluster().
> > As a result, VOL_DIRTY is cleared with cp and touch.
> > However, when copying a many files ...
> >    - Async mode: VOL_DIRTY is written to the media twice every 30 seconds.
> >    - Sync mode: Of course,  VOL_DIRTY and VOL_CLEAN to the media for each
> > file.
> >
> > Frequent writing VOL_DIRTY and VOL_CLEAN  increases the risk of boot-
> > sector curruption.
> > If the boot-sector corrupted, it causes the following serious problems  on
> > some OSs.
> >    - misjudge as unformatted
> >    - can't judge as exfat
> >    - can't repair
> >
> > I want to minimize boot sector writes, to reduce these risk.
> >
> > I looked vfat/udf implementation, which manages similar dirty information
> > on linux, and found that they ware mark-dirty at mount and cleared at
> > unmount.
> >
> > Here are some ways to clear VOL_DIRTY.
> >
> > (A) VOL_CLEAN after every write operation.
> >     :-) Ejectable at any time after a write operation.
> >     :-( Many times write to Boot-sector.
> >
> > (B) dirty at mount, clear at unmount (same as vfat/udf)
> >     :-) Write to boot-sector twice.
> >     :-( It remains dirty unless unmounted.
> >     :-( Write to boot-sector even if there is no write operation.
> >
> > (C) dirty on first write operation, clear on unmount
> >     :-) Writing to boot-sector is minimal.
> >     :-) Will not write to the boot-sector if there is no write operation.
> >     :-( It remains dirty unless unmounted.
> >
> > (D) dirty on first write operation,  clear on sync-fs/unmount
> >    :-) Writing to boot-sector can be reduced.
> >    :-) Will not write to the boot-sector if there is no write operation.
> >    :-) sync-fs makes it clean and ejectable immidiately.
> >    :-( It remains dirty unless sync-fs or unmount.
> >    :-( Frequent sync-fs will  increases writes to boot-sector.
> >
> > I think it should be (C) or(D).
> > What do you think?
>
> First of all, I'm sorry for the late reply.
> And thank you for the suggestion.

Thanks for thinking on this complicated issue.


> Most of the NAND flash devices and HDDs have wear leveling and bad sector replacement algorithms applied.
> So I think that the life of the boot sector will not be exhausted first.

I'm not too worried about the life of the boot-sector.
I'm worried about write failures caused by external factors.
(power failure/system down/vibration/etc. during writing)
They rarely occur on SD cards, but occur on many HDDs, some SSDs and USB storages by 0.1% or more.
Especially with AFT-HDD, not only boot-sector but also the following multiple sectors become unreadable.
It is not possible to completely solve this problem, as long as writing to the boot-sector.
(I think it's a exFAT's specification defect)
The only effective way to reduce this problem is to reduce writes to the boot-sector.


> Currently the volume dirty/clean policy of exfat-fs is not perfect,

Thank you for sharing the problem with you.


> but I think it behaves similarly to the policy of MS Windows.

On Windows10, the dirty flag is cleared after more than 15 seconds after all write operations are completed.
(dirty-flag is never updated during the write operation continues)


> Therefore,
> I think code improvements should be made to reduce volume flag records while maintaining the current policy.

Current policy is inconsistent.
As I wrote last mail, the problem with the current implementation is that the dirty-flag may not be cleared
after the write operation.(even if sync is enabled or disabled)
Because, some write operations clear the dirty-flag but some don't clear.
Unmount or sync command is the only way to ensure that the dirty-flag is cleared.
This has no effect on clearing the dirty-flag after a write operations, it only increases risk of destroying
the boot-sector.
 - Clear the dirty-flag after every write operation.
 - Never clear the dirty-flag after every write operation.
Unless unified to either one,  I think that sync policy cannot be consistent.

How do you think?


BR
---
etsuhiro Kohada <kohada.t2@xxxxxxxxx>