Re: page migration patchset

[Ray Bryant]

Andi Kleen wrote:
> Ray Bryant <raybry@xxxxxxx> writes:
>
>
> > http://sr71.net/patches/2.6.10/2.6.10-mm1-mhp-test7/
> >
> > A number of us are interested in using the page migration patchset by itself:
> >
> > (1)  Myself, for a manual page migration project I am working on.  (This
> >      is for migrating jobs from one set of nodes to another under batch
> >      scheduler control).
> > (2)  Marcello, for his memory defragmentation work.
> > (3)  Of course, the memory hotplug project itself.
> >
> > (there are probably other "users" that I have not enumerated here).
>
>
> Could you coordinate that with Steve Longerbeam (cc'ed) ? 
>
> He has a NUMA API extension ready to be merged into -mm* that also
> does kind of page migration when changing the policies of files.
>
> -Andi
Yes, Steve's patch tries to move page cache pages that are found to be 
allocated in the "wrong" place.  (See remove_invalid_filemap_page() in his
patch of 11/02/2004 on lkml).  But if the page is found to be busy, the code
gives up, as near as I can tell.

If the page migration patch were merged, Steve could call 
migrate_onepage(page,node) to move the page to the correct node. even if it
is busy [hopefully his code can "wait" at that point, I haven't looked into it 
further to see if that is the case.]

[This is really the page migration patch plus a small patch of
mine that addss the node argument to migrate_onepage(), and that I hope will
get merged into the page migration patch shortly]

Other than that, I don't see a big intersection between the two patches.
Steve, do you see anything else where we need to coordinate?

On the other hand, there is some work to be done wrt memory policies
and page migration.  For the project I am working on, we need to be able
to move all of the pages used by a process on one set of nodes to another
set of nodes.  At some point during this process we will need to update
the memory policy for that process.  For Steve's patch, we will
similarly need to update the policy associated with files associated with
the process, I would think, elsewise new pages will get allocated on the
old set of nodes, which is something we don't want.  Sounds like some
new interfaces will have to be developed here.  Does that make sense
to you, Andi and Steve?

My personal preference would be to keep as much of this as possible
under user space control; that is, rather than having a big autonomous
system call that migrates pages and then updates policy information,
I'd prefer to split the work into several smaller system calls that
are issued by a user space program responsible for coordinating the
process migration as a series of steps, e. g.:

(1)  suspend the process via SIGSTOP
(2)  update the mempolicy information
(3)  migrate the process's pages
(4)  migrate the process to the new cpu via set_schedaffinity()
(5)  resume the process via SIGCONT

that way the user program actually implements the process memory
migration fuctionality rather than having it all done by the kernel.
Thid also lets the user (or sys admin) modify or add new steps to
the page migration to satisfy local requirements without having to
modify the kernel.
--
Best Regards,
Ray
-----------------------------------------------
                  Ray Bryant
512-453-9679 (work)         512-507-7807 (cell)
raybry@xxxxxxx             raybry@xxxxxxxxxxxxx
The box said: "Requires Windows 98 or better",
           so I installed Linux.
-----------------------------------------------
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/