This patch fixes a regression in 2.6.14 against 2.6.13 that causes an
imbalance in memory allocation during bootup.
The slab allocator in 2.6.13 is not numa aware and simply calls
alloc_pages(). This means that memory policies may control the behavior of
alloc_pages(). During bootup the memory policy is set to MPOL_INTERLEAVE
resulting in the spreading out of allocations during bootup over all
available nodes. The slab allocator in 2.6.13 has only a single list of
slab pages. As a result the per cpu slab cache and the spinlock controlled
page lists may contain slab entries from off node memory. The slab
allocator in 2.6.13 makes no effort to discern the locality of an entry on
its lists.
The NUMA aware slab allocator in 2.6.14 controls locality of the slab pages
explicitly by calling alloc_pages_node(). The NUMA slab allocator manages
slab entries by having lists of available slab pages for each node. The
per cpu slab cache can only contain slab entries associated with the node
local to the processor. This guarantees that the default allocation mode
of the slab allocator always assigns local memory if available.
Setting MPOL_INTERLEAVE as a default policy during bootup has no effect
anymore. In 2.6.14 all node unspecific slab allocations are performed on
the boot processor. This means that most of key data structures are
allocated on one node. Most processors will have to refer to these
structures making the boot node a potential bottleneck. This may reduce
performance and cause unnecessary memory pressure on the boot node.
This patch implements NUMA policies in the slab layer. There is the need
of explicit application of NUMA memory policies by the slab allcator itself
since the NUMA slab allocator does no longer let the page_allocator control
locality.
The check for policies is made directly at the beginning of __cache_alloc
using current->mempolicy. The memory policy is already frequently checked
by the page allocator (alloc_page_vma() and alloc_page_current()). So it
is highly likely that the cacheline is present. For MPOL_INTERLEAVE
kmalloc() will spread out each request to one node after another so that an
equal distribution of allocations can be obtained during bootup.
It is not possible to push the policy check to lower layers of the NUMA
slab allocator since the per cpu caches are now only containing slab
entries from the current node. If the policy says that the local node is
not to be preferred or forbidden then there is no point in checking the
slab cache or local list of slab pages. The allocation better be directed
immediately to the lists containing slab entries for the allowed set of
nodes.
This way of applying policy also fixes another strange behavior in 2.6.13.
alloc_pages() is controlled by the memory allocation policy of the current
process. It could therefore be that one process is running with
MPOL_INTERLEAVE and would f.e. obtain a new page following that policy
since no slab entries are in the lists anymore. A page can typically be
used for multiple slab entries but lets say that the current process is
only using one. The other entries are then added to the slab lists. These
are now non local entries in the slab lists despite of the possible
availability of local pages that would provide faster access and increase
the performance of the application.
Another process without MPOL_INTERLEAVE may now run and expect a local slab
entry from kmalloc(). However, there are still these free slab entries
from the off node page obtained from the other process via MPOL_INTERLEAVE
in the cache. The process will then get an off node slab entry although
other slab entries may be available that are local to that process. This
means that the policy if one process may contaminate the locality of the
slab caches for other processes.
This patch in effect insures that a per process policy is followed for the
allocation of slab entries and that there cannot be a memory policy
influence from one process to another. A process with default policy will
always get a local slab entry if one is available. And the process using
memory policies will get its memory arranged as requested. Off-node slab
allocation will require the use of spinlocks and will make the use of per
cpu caches not possible. A process using memory policies to redirect
allocations offnode will have to cope with additional lock overhead in
addition to the latency added by the need to access a remote slab entry.
Changes V1->V2
- Remove #ifdef CONFIG_NUMA by moving forward declaration into
prior #ifdef CONFIG_NUMA section.
- Give the function determining the node number to use a saner
name.
Signed-off-by: Christoph Lameter <clameter@sgi.com> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This patch creates a proc entry /proc/sys/vm/zone_reclaim_mode that may be
used to override the automatic determination of the zone reclaim made on
bootup.
Signed-off-by: Christoph Lameter <clameter@sgi.com> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Some bits for zone reclaim exists in 2.6.15 but they are not usable. This
patch fixes them up, removes unused code and makes zone reclaim usable.
Zone reclaim allows the reclaiming of pages from a zone if the number of
free pages falls below the watermarks even if other zones still have enough
pages available. Zone reclaim is of particular importance for NUMA
machines. It can be more beneficial to reclaim a page than taking the
performance penalties that come with allocating a page on a remote zone.
Zone reclaim is enabled if the maximum distance to another node is higher
than RECLAIM_DISTANCE, which may be defined by an arch. By default
RECLAIM_DISTANCE is 20. 20 is the distance to another node in the same
component (enclosure or motherboard) on IA64. The meaning of the NUMA
distance information seems to vary by arch.
If zone reclaim is not successful then no further reclaim attempts will
occur for a certain time period (ZONE_RECLAIM_INTERVAL).
Zone reclaim has a huge impact on NUMA performance (f.e. our maximum
throughput with XFS is raised from 4GB to 6GB/sec / page cache contamination
of numa nodes destroys locality if one just does a large copy operation which
results in performance dropping for good until reboot).
This patch:
Resurrect may_swap in struct scan_control
Signed-off-by: Christoph Lameter <clameter@sgi.com> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Nick Piggin [Thu, 19 Jan 2006 01:42:27 +0000 (17:42 -0800)]
[PATCH] mm: migration page refcounting fix
Migration code currently does not take a reference to target page
properly, so between unlocking the pte and trying to take a new
reference to the page with isolate_lru_page, anything could happen to
it.
Fix this by holding the pte lock until we get a chance to elevate the
refcount.
Other small cleanups while we're here.
Signed-off-by: Nick Piggin <npiggin@suse.de> Signed-off-by: Christoph Lameter <clameter@sgi.com> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Andrew Morton [Thu, 19 Jan 2006 01:42:26 +0000 (17:42 -0800)]
[PATCH] mm: dirty_exceeded speedup
Ravikiran reports that this variable is bouncing all around nodes on NUMA
machines, causing measurable performance problems. Fix that up by only
writing to it when it actually changed.
And put it in a new cacheline to prevent it sharing with other things (this
happened).
Jon Mason [Thu, 19 Jan 2006 01:42:25 +0000 (17:42 -0800)]
[PATCH] Prevent trident driver from grabbing pcnet32 hardware
Some pcnet32 hardware erroneously has the Vendor ID for Trident. The
pcnet32 driver looks for the PCI ethernet class before grabbing the
hardware, but the current trident driver does not check against the PCI
audio class. This allows the trident driver to claim the pcnet32 hardware.
This patch prevents that.
This revised version of the OSS Trident patch includes PCI_DEVICE Macro
usage.
Signed-off-by: Jon Mason <jdmason@us.ibm.com> Signed-off-by: Muli Ben-Yehuda <mulix@mulix.org> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Andrew Morton [Thu, 19 Jan 2006 01:42:22 +0000 (17:42 -0800)]
[PATCH] scsi_transport_spi build fix
On alpha:
In file included from drivers/scsi/sym53c8xx_2/sym_glue.h:59,
from drivers/scsi/sym53c8xx_2/sym_fw.c:40:
include/scsi/scsi_transport_spi.h:57: error: field `dv_mutex' has incomplete type
Cc: James Bottomley <James.Bottomley@steeleye.com> Cc: Ingo Molnar <mingo@elte.hu> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Jan Kara [Thu, 19 Jan 2006 01:42:19 +0000 (17:42 -0800)]
[PATCH] jbd: log_do_checkpoint fix
While checkpointing we have to check that our transaction still is in the
checkpoint list *and* (not or) that it's not just a different transaction
with the same address.
Signed-off-by: Jan Kara <jack@suse.cz> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Eddie C. Dost [Wed, 18 Jan 2006 22:54:31 +0000 (14:54 -0800)]
[SPARC64]: Serial Console for E250 Patch
From: Eddie C. Dost <ecd@brainaid.de>
I have the following patch for serial console over the RSC
(remote system controller) on my E250 machine. It basically adds
support for input-device=rsc and output-device=rsc from OBP, and
allows 115200,8,n,1,- serial mode setting.
Signed-off-by: David S. Miller <davem@davemloft.net>
David L Stevens [Wed, 18 Jan 2006 22:20:56 +0000 (14:20 -0800)]
[IPV4]: Fix multiple bugs in IGMPv3
1) fix "mld_marksources()" to
a) send nothing when all queried sources are excluded
b) send full exclude report when source queried sources are
not excluded
c) don't schedule a timer when there's nothing to report
2) fix "add_grec()" to send empty-source records when it should
The original check doesn't account for a non-empty source
list with all sources inactive; the new code keeps that
short-circuit case, and also generates the group header
with an empty list if needed.
3) fix mca_crcount decrement to be after add_grec(), which needs
its original value
4) add/remove delete records and prevent current advertisements
when an exclude-mode filter moves from "active" to "inactive"
or vice versa based on new filter additions.
Items 1-3 are just IPv4 versions of the IPv6 bugs found
by Yan Zheng and fixed earlier. Item #4 is a related bug that
affects exclude-mode change records only (but not queries) and
also occurs in IPv6 (IPv6 version coming soon).
Signed-off-by: David L Stevens <dlstevens@us.ibm.com> Signed-off-by: David S. Miller <davem@davemloft.net>
Jesse Brandeburg [Wed, 18 Jan 2006 21:01:41 +0000 (13:01 -0800)]
[PATCH] e1000: Added driver comments
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: John Ronciak <john.ronciak@intel.com> Signed-off-by: Jeff Garzik <jgarzik@pobox.com>
Jesse Brandeburg [Wed, 18 Jan 2006 21:01:39 +0000 (13:01 -0800)]
[PATCH] e1000: Fix whitespace
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: John Ronciak <john.ronciak@intel.com> Signed-off-by: Jeff Garzik <jgarzik@pobox.com>
These are not static functions, that is why we have them declared in the header.
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: John Ronciak <john.ronciak@intel.com> Signed-off-by: Jeff Garzik <jgarzik@pobox.com>
Jesse Brandeburg [Wed, 18 Jan 2006 21:01:34 +0000 (13:01 -0800)]
[PATCH] e1000: Added functions to save and restore config
These functions help restore the driver to active configuration when coming out of resume for power management.
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: John Ronciak <john.ronciak@intel.com> Signed-off-by: Jeff Garzik <jgarzik@pobox.com>
Jesse Brandeburg [Wed, 18 Jan 2006 21:01:32 +0000 (13:01 -0800)]
[PATCH] e1000: Added RX buffer enhancements
Align the prefetches to a dword to help speed them up.
Recycle skb's and early replenish.
Force memory writes to complete before fetching more descriptors.
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: John Ronciak <john.ronciak@intel.com> Signed-off-by: Jeff Garzik <jgarzik@pobox.com>
Adds the ability to disability packet split at compile time and use the legacy receive path on PCI express hardware. Made this a CONFIG option and modified the Kconfig, to reflect the new option.
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: John Ronciak <john.ronciak@intel.com> Signed-off-by: Jeff Garzik <jgarzik@pobox.com>
Commit 7493a314cb83797ce612a577475aacaedc553fed changed the ordering
of the registration of the platform device driver vs the 8250 drivers
internal initialisation. This led to the probe function being called
before the driver had finished its internal initialisation, causing
mayhem. Revert the ordering change.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Bryan O'Sullivan [Wed, 18 Jan 2006 01:00:05 +0000 (17:00 -0800)]
[PATCH] Fix sparse parse error in lppaca.h
sparse can't parse a struct definition in include/asm-powerpc/lppaca.h,
even though gcc can accept it. The form looks like this:
struct __attribute__((whatever)) foo { };
An equivalent that both gcc and sparse can handle is
struct foo { } __attribute__((whatever));
This is the only definition of this type in the tree, and fixing it is
easier than fixing sparse.
Signed-off-by: Bryan O'Sullivan <bos@serpentine.com>
[ Side note: fixing sparse wouldn't be hard, but the "attribute at the
end" version is the canonical one, and the one that makes sense. So
let's just fix the kernel instead. Luc Van Oostenryck already sent
out a sparse patch to the sparse mailing list in case anybody cares.
-- Linus ] Signed-off-by: Linus Torvalds <torvalds@osdl.org>
There are a couple of problems in the DMA setup code for skge.
* In the 64 bit case, it doesn't set the consistent mask.
* In the 32 bit case, the error check is backwards!
It likely will only be visible as a bug on 64 bit platforms.
Signed-off-by: Stephen Hemminger <shemminger@osdl.org> Signed-off-by: Jeff Garzik <jgarzik@pobox.com>
Alan Cox [Tue, 17 Jan 2006 20:53:50 +0000 (20:53 +0000)]
[PATCH] libata: Code for the IRQ mask flag
Signed-off-by: Alan Cox <alan@redhat.com>
Somewhat cleaner in the resync as someone cleaned up the pio xfer users Signed-off-by: Jeff Garzik <jgarzik@pobox.com>
Fix problems with Yukon FE rev 2 chipset. Don't cut and paste bugs in from
sk98lin driver. Change how the ram buffer is divided up, and make the math
clearer. Also, set the thresholds where rx takes precedence. The threshold
values are just guesses at this point, it might be worth tuning them later.
Signed-off-by: Stephen Hemminger <shemminger@osdl.org> Signed-off-by: Jeff Garzik <jgarzik@pobox.com>
Per Liden [Mon, 16 Jan 2006 15:39:13 +0000 (16:39 +0100)]
[TIPC] Group protocols with sub-options in Kconfig
This is just a cosmetic change that moves the TIPC configuration
entry next to the other protocols that also have sub-options.
Makes the the networking options menu look a bit better.
Per Liden [Fri, 13 Jan 2006 21:22:22 +0000 (22:22 +0100)]
[TIPC] Updated link priority macros
Added macros for min/default/max link priority in tipc_config.h.
Also renamed TIPC_NUM_LINK_PRI to TIPC_MEDIA_LINK_PRI since that
is a more accurate description of what it is used for.
Ensure a consistent value is read from the STICK register by ensuring
that both high and low are read without high changing due to a roll
over of the low register.
Various Debian/SPARC users (myself include) have noticed problems with
Hummingbird based systems. The symptoms are that the system time is
seen to jump forward 3 days, 6 hours, 11 minutes give or take a few
seconds. In many cases the system then hangs some time afterwards.
I've spotted a race condition in the code to read the STICK register.
I could not work out why 3d, 6h, 11m is important but guess that it is
due to the 2^32 jump of STICK (forwards on one read and then the next
read will seem to be backwards) during a timer interrupt. I'm guessing
that a change of -2^32 will get converted to a large unsigned
increment after the arithmetic manipulation between STICK,
nanoseconds, jiffies etc.
I did a test where I modified __hbird_read_stick to artificially
inject rollover faults forcefully every few seconds. With this I saw
the clock jump over 6 times in 12 hours compared to once every month
or so.
Signed-off-by: Richard Mortimer <richm@oldelvet.org.uk> Signed-off-by: David S. Miller <davem@davemloft.net>
Alan Cox [Mon, 9 Jan 2006 17:14:40 +0000 (17:14 +0000)]
[PATCH] libata: Pre UDMA EIDE PIO mode selection
I misread the spec when doing the original. I've tested the corrected
version with pre UDMA drives and it now picks the right modes. This is a
specific bug fix rather than an update or new feature item.
Signed-off-by: Alan Cox <alan@redhat.com> Signed-off-by: Jeff Garzik <jgarzik@pobox.com>
projects / mv-sheeva.git / log