Re: [f2fs-dev] [PATCH 1/2] f2fs: pass down write hints to block layer for bufferd write

[Jaegeuk Kim]

On 12/23, Chao Yu wrote:
> On 2017/12/15 10:06, Jaegeuk Kim wrote:
> > On 12/14, Hyunchul Lee wrote:
> >> Hi Jaegeuk,
> >>
> >> I need your comment about the fs_iohint mount option.
> >>
> >> a) w/o fs_iohint, propagate user hints to low layer.
> >> b) w/ fs_iohint, ignore user hints, and use hints which is generated
> >> with F2FS.
> >>
> >> Chao suggests this option. because user hints are more accurate than
> >> file system.
> >>
> >> This is resonable, But I have some concerns about this option. 
> >> The first thing is that blocks of a segments have different hints. This
> >> could make GC less effective. 
> >> The second is that the separation between LIFE_MEDIUM and LIFE_LONG is 
> >> really needed. I think that difference between them is a little ambigous 
> >> for users, and LIFE_SHORT and LIFE_EXTREME is converted to different 
> >> hints by F2FS.
> > 
> > I think what we really can do would assign many user hints to our 3 DATA
> > logs likewise rw_hint_to_seg_type(), since it's just hints for user data.
> > Then, we can decide how to keep that as much as possible, since we have
> > another filesystem metadata such as meta and nodes. In addition, I don't
> > think we have to keep the original user-hints which makes F2FS logs be
> > messed up.
> > 
> > With that mind, I can think of the below cases. Especially, if user wants
> > to keep their io_hints, we'd better recommend to use direct_io w/o fs_iohints.
> 
> 
> 
> > In order to keep this policy, I think fs_iohints would be better to be a
> > feature set by mkfs.f2fs and detected by sysfs entries for users.
> > 
> > 1) w/ fs_iohints
> > 
> > User                        F2FS               Block
> > -------------------------------------------------------------------
> >                             Meta               WRITE_LIFE_MEDIUM
> >                             HOT_NODE           WRITE_LIFE_NOTSET
> >                             WARM_NODE          -'
> >                             COLD_NODE          WRITE_LIFE_NONE
> > ioctl(cold)                 COLD_DATA          WRITE_LIFE_EXTREME
> > extention list              -'                 -'
> > WRITE_LIFE_EXTREME          -'                 -'
> > WRITE_LIFE_SHORT            HOT_DATA           WRITE_LIFE_SHORT
> > 
> > -- buffered_io
> > WRITE_LIFE_NOT_SET          WARM_DATA          WRITE_LIFE_LONG
> > WRITE_LIFE_NONE             -'                 -'
> > WRITE_LIFE_MEDIUM           -'                 -'
> > WRITE_LIFE_LONG             -'                 -'
> > 
> > -- direct_io (Not recommendable)
> > WRITE_LIFE_NOT_SET          WARM_DATA          WRITE_LIFE_NOT_SET
> > WRITE_LIFE_NONE             -'                 WRITE_LIFE_NONE
> > WRITE_LIFE_MEDIUM           -'                 WRITE_LIFE_MEDIUM
> > WRITE_LIFE_LONG             -'                 WRITE_LIFE_LONG
> 
> Agreed with above IO hint mapping rule.
> 
> > 
> > 2) w/o fs_iohints
> > 
> > User                        F2FS               Block
> > -------------------------------------------------------------------
> >                             Meta               -
> >                             HOT_NODE           -
> >                             WARM_NODE          -
> >                             COLD_NODE          -
> > ioctl(cold)                 COLD_DATA          -
> > extention list              -'                 -
> > 
> > -- buffered_io
> > WRITE_LIFE_EXTREME          COLD_DATA          -
> > WRITE_LIFE_SHORT            HOT_DATA           -
> > WRITE_LIFE_NOT_SET          WARM_DATA          -
> > WRITE_LIFE_NONE             -'                 -
> > WRITE_LIFE_MEDIUM           -'                 -
> > WRITE_LIFE_LONG             -'                 -
> 
> Now we recommend direct_io if user wants to give IO hint for storage, I suspect
> that user would suffer performance regression issue w/o buffered IO.
> 
> Another problem is that, now, in Android, it will be very hard to prompt
> application to migrate their IO pattern from buffered IO to direct IO, one
> possible way is distinguishing user data lifetime from FWK, e.g. set
> WRITE_LIFE_SHORT for cache file or tmp file, set WRITE_LIFE_EXTREME for media file.
> 
> In order to support buffered_io, would it be better to change mapping as below?
> 
> -- buffered_io
> WRITE_LIFE_EXTREME          COLD_DATA          WRITE_LIFE_EXTREME
> WRITE_LIFE_SHORT            HOT_DATA           WRITE_LIFE_SHORT
> WRITE_LIFE_NOT_SET          WARM_DATA          WRITE_LIFE_NOT_SET
> WRITE_LIFE_NONE             -'                 -'
> WRITE_LIFE_MEDIUM           -'                 -'
> WRITE_LIFE_LONG             -'                 -'

Agreed, and it makes more sense that we'd better keep the write hints on
userdata given by applications.

BTW, since we couldn't get any performance numbers with these, how about
adding a mount option like "-o iohints=MODE" where MODE may be one of
"fs-based", "user-based", and "off"?

Thanks,

> 
> Thanks,
> 
> > 
> > -- direct_io
> > WRITE_LIFE_EXTREME          COLD_DATA          WRITE_LIFE_EXTREME
> > WRITE_LIFE_SHORT            HOT_DATA           WRITE_LIFE_SHORT
> > WRITE_LIFE_NOT_SET          WARM_DATA          WRITE_LIFE_NOT_SET
> > WRITE_LIFE_NONE             -'                 WRITE_LIFE_NONE
> > WRITE_LIFE_MEDIUM           -'                 WRITE_LIFE_MEDIUM
> > WRITE_LIFE_LONG             -'                 WRITE_LIFE_LONG
> > 
> > 
> > Note that, I don't much care about how to manipulate streamid in nvme driver
> > in terms of LIFE_NONE or LIFE_NOTSET, since other drivers can handle them
> > in different ways. Taking a look at the definition, at least, we don't need
> > to assume that those are same at all. For example, if we can expolit this in
> > UFS driver, we can pass all the stream ids to the device as context ids.
> > 
> > Thanks,
> > 
> >>
> >> Thanks.
> >>
> >> On 12/12/2017 11:45 AM, Chao Yu wrote:
> >>> Hi Hyunchul,
> >>>
> >>> On 2017/12/12 10:15, Hyunchul Lee wrote:
> >>>> Hi Chao,
> >>>>
> >>>> On 12/11/2017 10:15 PM, Chao Yu wrote:
> >>>>> Hi Hyunchul,
> >>>>>
> >>>>> On 2017/12/1 16:28, Hyunchul Lee wrote:
> >>>>>> Hi Chao,
> >>>>>>
> >>>>>> On 11/30/2017 04:06 PM, Chao Yu wrote:
> >>>>>>> Hi Hyunchul,
> >>>>>>>
> >>>>>>> On 2017/11/28 8:23, Hyunchul Lee wrote:
> >>>>>>>> From: Hyunchul Lee <cheol.lee@xxxxxxx>
> >>>>>>>>
> >>>>>>>> This implements which hint is passed down to block layer
> >>>>>>>> for datas from the specific segment type.
> >>>>>>>>
> >>>>>>>> segment type                     hints
> >>>>>>>> ------------                     -----
> >>>>>>>> COLD_NODE & COLD_DATA            WRITE_LIFE_EXTREME
> >>>>>>>> WARM_DATA                        WRITE_LIFE_NONE
> >>>>>>>> HOT_NODE & WARM_NODE             WRITE_LIFE_LONG
> >>>>>>>> HOT_DATA                         WRITE_LIFE_MEDIUM
> >>>>>>>> META_DATA                        WRITE_LIFE_SHORT
> >>>>>>>
> >>>>>>> Just noticed, if our user do not give the hint via ioctl, f2fs can
> >>>>>>> provider hint to lower layer according to hot/cold separation ability,
> >>>>>>> it will be okay. But once user give his hint which may be more accurate
> >>>>>>> than filesystem, hint converted by f2fs may be wrong.
> >>>>>>>
> >>>>>>> So what do you think of adding an option to control whether filesystem
> >>>>>>> can convert hint user given?
> >>>>>>>
> >>>>>>
> >>>>>> I think it is okay for LIFE_SHORT and LIFE_EXTREME. because they are 
> >>>>>> converted to different hints.
> >>>>>
> >>>>> What I mean is introducing a mount option, e.g. fs_iohint,
> >>>>> a) w/o fs_iohint, propagate file/inode io_hint to low layer.
> >>>>> b) w/ fs_iohint, ignore file/inode io_hint, use io_hint which is generated
> >>>>> with filesystem's private rule.
> >>>>>
> >>>>
> >>>> Okay, I will implement this option and send this patch again.
> >>>
> >>> Let's wait for Jaegeuk's comments first?
> >>>
> >>>>
> >>>> Without fs_iohint, Even if data blocks are moved due to GC, 
> >>>> we should keep user hints. And if user hints are not given, 
> >>>> any hints are not passed down to block layer, right?
> >>>
> >>> Hmm.. that will be a problem, IMO, we can store last user's io_hint into inode
> >>> layout, so later when we trigger GC, we can use the last io_hint in inode rather
> >>> than giving no hint or fs' hint.
> >>>
> >>> I think it needs to discuss with original author of IO hint, what is the IO hint
> >>> policy when filesystem move block by itself after inode has been released in system.
> >>>
> >>> Thanks,
> >>>
> >>>>
> >>>> Thank you for comments.
> >>>>
> >>>>> Thanks,
> >>>>>
> >>>>>>
> >>>>>> file hint      segment type        io hint
> >>>>>> ---------      ------------        -------
> >>>>>> LIFE_SHORT     HOT_DATA            LIFE_MEDIUM
> >>>>>> LIFE_MEDIUM    WARM_DATA           LIFE_NONE
> >>>>>> LIFE_LONG      WARM_DATA           LIFE_NONE
> >>>>>> LIFE_EXTREME   COLD_DATA           LIFE_EXTREME
> >>>>>>
> >>>>>> the problem is that LIFE_MEDIUM and LIFE_LONG are converted to 
> >>>>>> the same hint, LIFE_NONE. I am not sure that the seperation between 
> >>>>>> LIFE_MEDIUM and LIFE_LONG is really needed. Because I guess that the 
> >>>>>> difference between them is a little ambigous for users, and if WARM_DATA 
> >>>>>> segment has two different hints, it can makes GC non-efficient.
> >>>>>>
> >>>>>> I wonder your thought about this.
> >>>>>>
> >>>>>> Thanks.
> >>>>>>
> >>>>>>> Thanks,
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>> ------------------------------------------------------------------------------
> >>>>>> Check out the vibrant tech community on one of the world's most
> >>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> >>>>>> _______________________________________________
> >>>>>> Linux-f2fs-devel mailing list
> >>>>>> Linux-f2fs-devel@xxxxxxxxxxxxxxxxxxxxx
> >>>>>> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
> >>>>>>
> >>>>>
> >>>>
> >>>> .
> >>>>
> >>>
> >>>
> >>> ------------------------------------------------------------------------------
> >>> Check out the vibrant tech community on one of the world's most
> >>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> >>> _______________________________________________
> >>> Linux-f2fs-devel mailing list
> >>> Linux-f2fs-devel@xxxxxxxxxxxxxxxxxxxxx
> >>> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
> >>>
> > 
> > .
> >