mm, thp: avoid excessive compaction latency during fault fix

mm-thp-avoid-excessive-compaction-latency-during-fault.patch excludes sync
compaction for all high order allocations other than thp.  What we really
want to do is suppress sync compaction for thp, but only during the page
fault path.

Orders greater than PAGE_ALLOC_COSTLY_ORDER aren't necessarily going to
loop again so this is the only way to exhaust our capabilities before
declaring that we can't allocate.

Signed-off-by: David Rientjes <rientjes@google.com>
Reported-by: Hugh Dickins <hughd@google.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Greg Thelen <gthelen@google.com>
Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 1f8100c747b3636ec466dfd870c26e62ddc94d86..dddf43f2f545e344c5c7458458a1dea6bcb94d10 100644 (file)
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -2576,16 +2576,13 @@ rebalance:
        if (page)
                goto got_pg;
 
-       if (gfp_mask & __GFP_NO_KSWAPD) {
-               /*
-                * Khugepaged is allowed to try MIGRATE_SYNC_LIGHT, the latency
-                * of this allocation isn't critical.  Everything else, however,
-                * should only be allowed to do MIGRATE_ASYNC to avoid excessive
-                * stalls during fault.
-                */
-               if ((current->flags & (PF_KTHREAD | PF_KSWAPD)) == PF_KTHREAD)
-                       migration_mode = MIGRATE_SYNC_LIGHT;
-       }
+       /*
+        * It can become very expensive to allocate transparent hugepages at
+        * fault, so use asynchronous memory compaction for THP unless it is
+        * khugepaged trying to collapse.
+        */
+       if (!(gfp_mask & __GFP_NO_KSWAPD) || (current->flags & PF_KTHREAD))
+               migration_mode = MIGRATE_SYNC_LIGHT;
 
        /*
         * If compaction is deferred for high-order allocations, it is because