Re: [Xen-devel] xen: Can't insert balloon page into VM userspace (WAS Re: [linux-linus bisection] complete test-arm64-arm64-xl-xsm)

[David Hildenbrand]

On 12.03.19 19:23, David Hildenbrand wrote:
> On 12.03.19 19:02, Boris Ostrovsky wrote:
>> On 3/12/19 1:24 PM, Andrew Cooper wrote:
>>> On 12/03/2019 17:18, David Hildenbrand wrote:
>>>> On 12.03.19 18:14, Matthew Wilcox wrote:
>>>>> On Tue, Mar 12, 2019 at 05:05:39PM +0000, Julien Grall wrote:
>>>>>> On 3/12/19 3:59 PM, Julien Grall wrote:
>>>>>>> It looks like all the arm test for linus [1] and next [2] tree
>>>>>>> are now failing. x86 seems to be mostly ok.
>>>>>>>
>>>>>>> The bisector fingered the following commit:
>>>>>>>
>>>>>>> commit 0ee930e6cafa048c1925893d0ca89918b2814f2c
>>>>>>> Author: Matthew Wilcox <willy@xxxxxxxxxxxxx>
>>>>>>> Date:   Tue Mar 5 15:46:06 2019 -0800
>>>>>>>
>>>>>>>      mm/memory.c: prevent mapping typed pages to userspace
>>>>>>>      Pages which use page_type must never be mapped to userspace as it would
>>>>>>>      destroy their page type.  Add an explicit check for this instead of
>>>>>>>      assuming that kernel drivers always get this right.
>>>>> Oh good, it found a real problem.
>>>>>
>>>>>> It turns out the problem is because the balloon driver will call
>>>>>> __SetPageOffline() on allocated page. Therefore the page has a type and
>>>>>> vm_insert_pages will deny the insertion.
>>>>>>
>>>>>> My knowledge is quite limited in this area. So I am not sure how we can
>>>>>> solve the problem.
>>>>>>
>>>>>> I would appreciate if someone could provide input of to fix the mapping.
>>>>> I don't know the balloon driver, so I don't know why it was doing this,
>>>>> but what it was doing was Wrong and has been since 2014 with:
>>>>>
>>>>> commit d6d86c0a7f8ddc5b38cf089222cb1d9540762dc2
>>>>> Author: Konstantin Khlebnikov <k.khlebnikov@xxxxxxxxxxx>
>>>>> Date:   Thu Oct 9 15:29:27 2014 -0700
>>>>>
>>>>>     mm/balloon_compaction: redesign ballooned pages management
>>>>>
>>>>> If ballooned pages are supposed to be mapped into userspace, you can't mark
>>>>> them as ballooned pages using the mapcount field.
>>>>>
>>>> Asking myself why anybody would want to map balloon inflated pages into
>>>> user space (this just sounds plain wrong but my understanding to what
>>>> XEN balloon driver does might be limited), but I assume the easy fix
>>>> would be to revert
>>> I suspect the bug here is that the balloon driver is (ab)used for a
>>> second purpose
>>
>> Yes. And its name is alloc_xenballooned_pages().
>>
> 
> Haven't had a look at the code yet, but would another temporary fix be
> to clear/set PG_offline when allocating/freeing a ballooned page?
> (assuming here that only such pages will be mapped to user space)
> 

I guess something like this could do the trick if I understood it correctly:

diff --git a/drivers/xen/balloon.c b/drivers/xen/balloon.c
index 39b229f9e256..d37dd5bb7a8f 100644
--- a/drivers/xen/balloon.c
+++ b/drivers/xen/balloon.c
@@ -604,6 +604,7 @@ int alloc_xenballooned_pages(int nr_pages, struct
page **pages)
        while (pgno < nr_pages) {
                page = balloon_retrieve(true);
                if (page) {
+                       __ClearPageOffline(page);
                        pages[pgno++] = page;
 #ifdef CONFIG_XEN_HAVE_PVMMU
                        /*
@@ -645,8 +646,10 @@ void free_xenballooned_pages(int nr_pages, struct
page **pages)
        mutex_lock(&balloon_mutex);

        for (i = 0; i < nr_pages; i++) {
-               if (pages[i])
+               if (pages[i]) {
+                       __SetPageOffline(pages[i]);
                        balloon_append(pages[i]);
+               }
        }

        balloon_stats.target_unpopulated -= nr_pages;


At least this way, the pages allocated (and thus eventually mapped to
user space) would not be marked, but the other ones would remain marked
and could be excluded by makedumptool.

-- 

Thanks,

David / dhildenb