Jim Ramsay [Fri, 28 Jun 2013 00:53:27 +0000 (10:53 +1000)]
dm-switch is a new target that maps IO to underlying block devices
efficiently when there are a large number of fixed-sized address regions
but there is no simple pattern to allow for a compact mapping
representation such as dm-stripe.
Motivation
----------
Dell EqualLogic and some other iSCSI storage arrays use a distributed
frameless architecture. In this architecture, the storage group
consists of a number of distinct storage arrays ("members"), each having
independent controllers, disk storage and network adapters. When a LUN
is created it is spread across multiple members. The details of the
spreading are hidden from initiators connected to this storage system.
The storage group exposes a single target discovery portal, no matter
how many members are being used. When iSCSI sessions are created, each
session is connected to an eth port on a single member. Data to a LUN
can be sent on any iSCSI session, and if the blocks being accessed are
stored on another member the IO will be forwarded as required. This
forwarding is invisible to the initiator. The storage layout is also
dynamic, and the blocks stored on disk may be moved from member to
member as needed to balance the load.
This architecture simplifies the management and configuration of both
the storage group and initiators. In a multipathing configuration, it
is possible to set up multiple iSCSI sessions to use multiple network
interfaces on both the host and target to take advantage of the
increased network bandwidth. An initiator could use a simple round robin
algorithm to send IO across all paths and let the storage array members
forward it as necessary, but there is a performance advantage to
sending data directly to the correct member.
The Device Mapper table architecture already supports designating different
address regions with different targets, but in our architecture the LUN
is spread with an address region size on the order of 10s of MBs, which
means the resulting DM table could have more than a million entries
and consume far too much memory.
Solution
--------
Based on earlier discussion with the dm-devel contributors, we have
solved this problem by using Device Mapper to build a two-layer device
hierarchy:
Upper Tier – Determine which array member the IO should be sent to.
Lower Tier – Load balance amongst paths to a particular member.
The lower tier consists of a single dm multipath device for each member.
Each of these multipath devices contains the set of paths directly to
the array member in one priority group, and leverages existing path
selectors to load balance amongst these paths. We also build a
non-preferred priority group containing paths to other array members for
failover reasons.
The upper tier consists of a single dm switch device, using the new DM
target module. This device uses a bitmap to look up the location of the
IO and choose the appropriate lower tier device to route the IO. By
using a bitmap we are able to use 4 bits for each address range in a 16
member group (which is very large for us). This is a much denser
representation than the DM table B-tree can achieve.
Though we have developed this target for a specific storage device, we
have made an effort to keep it as general purpose as possible in the hope
that others may benefit.
Originally developed by Jim Ramsay. Simplified by Mikulas Patocka.
Signed-off-by: Jim Ramsay <jim_ramsay@dell.com> Signed-off-by: Mikulas Patocka <mpatocka@redhat.com> Signed-off-by: Alasdair G Kergon <agk@redhat.com>
Bart Van Assche [Wed, 12 Jun 2013 13:24:25 +0000 (15:24 +0200)]
IB/srp: Skip host settle delay
The SRP initiator implements host reset by reconnecting to the SRP
target. That means that communication with the target is possible as
soon as host reset finished. Hence skip the host settle delay.
Signed-off-by: Bart Van Assche <bvanassche@acm.org> Reviewed-by: Sebastian Riemer <sebastian.riemer@profitbricks.com> Reviewed-by: Christoph Hellwig <hch@infradead.org> Acked-by: David Dillow <dillowda@ornl.gov> Signed-off-by: Roland Dreier <roland@purestorage.com>
projects / karo-tx-linux.git / log