memcg, mm: introduce lowlimit reclaim

memcg, mm: introduce lowlimit reclaim

Previous discussions have shown that soft limits cannot be reformed
(http://lwn.net/Articles/555249/).  This series introduces an alternative
approach for protecting memory allocated to processes executing within a
memory cgroup controller.  It is based on a new tunable that was discussed
with Johannes and Tejun held during the kernel summit 2013 and at LSF
2014.

This patchset introduces such low limit that is functionally similar to a
minimum guarantee.  Memcgs which are under their lowlimit are not
considered eligible for the reclaim (both global and hardlimit) unless all
groups under the reclaimed hierarchy are below the low limit when all of
them are considered eligible.

The previous version of the patchset posted as a RFC
(http://marc.info/?l=linux-mm&m=138677140628677&w=2) suggested a hard
guarantee without any fallback.  More discussions led me to reconsidering
the default behavior and come up a more relaxed one.  The hard requirement
can be added later based on a use case which really requires.  It would be
controlled by memory.reclaim_flags knob which would specify whether to OOM
or fallback (default) when all groups are bellow low limit.

The default value of the limit is 0 so all groups are eligible by default
and an interested party has to explicitly set the limit.

The primary use case is to protect an amount of memory allocated to a
workload without it being reclaimed by an unrelated activity.  In some
cases this requirement can be fulfilled by mlock but it is not suitable
for many loads and generally requires application awareness.  Such
application awareness can be complex.  It effectively forbids the use of
memory overcommit as the application must explicitly manage memory
residency.

With the low limit, such workloads can be placed in a memcg with a low
limit that protects the estimated working set.

The hierarchical behavior of the lowlimit is described in the first patch.
 The second patch allows setting the lowlimit.  The last 2 patches clarify
documentation about the memcg reclaim in gereneral (3rd patch) and low
limit (4th patch).

This patch (of 4)

This patch introduces low limit reclaim.  The low_limit acts as a reclaim
protection because groups which are under their low_limit are considered
ineligible for reclaim.  While hardlimit protects from using more memory
than allowed lowlimit protects from getting below memory assigned to the
group due to external memory pressure.

More precisely a group is considered eligible for the reclaim under a
specific hierarchy represented by its root only if the group is above its
low limit and the same applies to all parents up the hierarchy to the
root.  Nevertheless the limit still might be ignored if all groups under
the reclaimed hierarchy are under their low limits.  This will prevent
from OOM rather than protecting the memory.

Consider the following hierarchy with memory pressure coming from the
group A (hard limit reclaim - l-low_limit_in_bytes, u-usage_in_bytes,
h-limit_in_bytes):

		root_mem_cgroup
			.
		  _____/
		 /
		A (l = 80 u=90 h=90)
	       /
	      / \_________
	     /            \
	    B (l=0 u=50)   C (l=50 u=40)
	                    \
			     D (l=0 u=30)

A and B are reclaimable but C and D are not (D is protected by C).

The low_limit is 0 by default so every group is eligible.  This patch
doesn't provide a way to set the limit yet although the core
infrastructure is there already.

Signed-off-by: Michal Hocko <mhocko@suse.cz>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Greg Thelen <gthelen@google.com>
Cc: Michel Lespinasse <walken@google.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Hugh Dickins <hughd@google.com>
Cc: Roman Gushchin <klamm@yandex-team.ru>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>