Yehuda Sadeh [Tue, 9 Feb 2010 19:14:41 +0000 (11:14 -0800)]
ceph: sync read/write considers page cache
In the cases where we either do a sync read or a write, we
need to make sure that everything in the page cache is flushed.
In the case of a sync write we invalidate the relevant pages,
so that subsequent read/write reflects the new data written.
Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net> Signed-off-by: Sage Weil <sage@newdream.net>
Yehuda Sadeh [Tue, 9 Feb 2010 19:08:40 +0000 (11:08 -0800)]
ceph: fix truncation when not holding caps
A truncation should occur when either we have the
specified caps for the file, or (in cases where we are
not the only ones referencing the file) when it is mapped
or when it is opened. The latter two cases were not
handled.
Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net> Signed-off-by: Sage Weil <sage@newdream.net>
Originally ceph_page_mkwrite called ceph_write_begin, hoping that
the returned locked page would be the page that it was requested
to mkwrite. Factored out relevant part of ceph_page_mkwrite and
we lock the right page anyway.
Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net> Signed-off-by: Sage Weil <sage@newdream.net>
Yehuda Sadeh [Thu, 4 Feb 2010 21:41:41 +0000 (13:41 -0800)]
ceph: fix short synchronous reads
Zeroing of holes was not done correctly: page_off was miscalculated and
zeroing the tail didn't not adjust the 'read' value to include the zeroed
portion.
Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net> Signed-off-by: Sage Weil <sage@newdream.net>
Yehuda Sadeh [Wed, 3 Feb 2010 19:00:26 +0000 (11:00 -0800)]
ceph: put unused osd connections on lru
Instead of removing osd connection immediately when the
requests list is empty, put the osd connection on an lru.
Only if that osd has not been used for more than a specified
time, will it be removed.
Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net> Signed-off-by: Sage Weil <sage@newdream.net>
Sage Weil [Wed, 3 Feb 2010 00:25:35 +0000 (16:25 -0800)]
ceph: add support for auth_x authentication protocol
The auth_x protocol implements support for a kerberos-like mutual
authentication infrastructure used by Ceph. We do not simply use vanilla
kerberos because of scalability and performance issues when dealing with
a large cluster of nodes providing a single logical service.
Auth_x provides mutual authentication of client and server and protects
against replay and man in the middle attacks. It does not encrypt
the full session over the wire, however, so data payload may still be
snooped.
Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net> Signed-off-by: Sage Weil <sage@newdream.net>
Sage Weil [Wed, 3 Feb 2010 00:21:06 +0000 (16:21 -0800)]
ceph: allow renewal of auth credentials
Add infrastructure to allow the mon_client to periodically renew its auth
credentials. Also add a messenger callback that will force such a renewal
if a peer rejects our authenticator.
Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net> Signed-off-by: Sage Weil <sage@newdream.net>
Sage Weil [Wed, 3 Feb 2010 00:34:04 +0000 (16:34 -0800)]
ceph: release all pages after successful osd write response
We release all the pages, even if the osd response was
different than the number of pages written. This could only
happen due to truncation that arrives the osd in
different order, for which we want the pages released anyway.
Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net> Signed-off-by: Sage Weil <sage@newdream.net>
Yehuda Sadeh [Mon, 11 Jan 2010 22:47:13 +0000 (14:47 -0800)]
ceph: alloc message data pages and check if tid exists
Now doing it in the same callback that is also responsible for
allocating the 'front' part of the message. If we get a message
that we haven't got a corresponding tid for, mark it for skipping.
Moving the mutex unlock/lock from the osd alloc_msg callback
to the calling function in the messenger.
Sage Weil [Mon, 25 Jan 2010 19:33:08 +0000 (11:33 -0800)]
ceph: properly handle aborted mds requests
Previously, if the MDS request was interrupted, we would unregister the
request and ignore any reply. This could cause the caps or other cache
state to become out of sync. (For instance, aborting dbench and doing
rm -r on clients would complain about a non-empty directory because the
client didn't realize it's aborted file create request completed.)
Even we don't unregister, we still can't process the reply normally because
we are no longer holding the caller's locks (like the dir i_mutex).
So, mark aborted operations with r_aborted, and in the reply handler, be
sure to process all the caps. Do not process the namespace changes,
though, since we no longer will hold the dir i_mutex. The dentry lease
state can also be ignored as it's more forgiving.
Yehuda Sadeh [Thu, 7 Jan 2010 23:36:32 +0000 (15:36 -0800)]
ceph: change dentry offset and position after splice_dentry
This fixes a bug, where we had the parent list have dentries with
offsets that are not monotonically increasing, which caused the ceph
dcache_readdir to skip entries.
Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net> Signed-off-by: Sage Weil <sage@newdream.net>
Sage Weil [Wed, 23 Dec 2009 20:21:51 +0000 (12:21 -0800)]
ceph: use ceph_pagelist for mds reconnect message; change encoding (protocol change)
Use the ceph_pagelist to encode the MDS reconnect message. We change the
message encoding (protocol change!) at the same time to make our life
easier (we don't know how many snaprealms we have when we start encoding).
An empty message implies the session is closed/does not exist.
Sage Weil [Wed, 23 Dec 2009 20:12:31 +0000 (12:12 -0800)]
ceph: support ceph_pagelist for message payload
The ceph_pagelist is a simple list of whole pages, strung together via
their lru list_head. It facilitates encoding to a "buffer" of unknown
size. Allow its use in place of the ceph_msg page vector.
This will be used to fix the huge buffer preallocation woes of MDS
reconnection.
Sage Weil [Wed, 23 Dec 2009 17:30:21 +0000 (09:30 -0800)]
ceph: add feature bits to connection handshake (protocol change)
Define supported and required feature set. Fail connection if the server
requires features we do not support (TAG_FEATURES), or if the server does
not support features we require.
Sage Weil [Tue, 22 Dec 2009 19:24:33 +0000 (11:24 -0800)]
ceph: include transaction id in ceph_msg_header (protocol change)
Many (most?) message types include a transaction id. By including it in
the fixed size header, we always have it available even when we are unable
to allocate memory for the (larger, variable sized) message body. This
will allow us to error out the appropriate request instead of (silently)
dropping the reply.
Sage Weil [Tue, 22 Dec 2009 18:45:45 +0000 (10:45 -0800)]
ceph: control access to page vector for incoming data
When we issue an OSD read, we specify a vector of pages that the data is to
be read into. The request may be sent multiple times, to multiple OSDs, if
the osdmap changes, which means we can get more than one reply.
Only read data into the page vector if the reply is coming from the
OSD we last sent the request to. Keep track of which connection is using
the vector by taking a reference. If another connection was already
using the vector before and a new reply comes in on the right connection,
revoke the pages from the other connection.
Sage Weil [Tue, 22 Dec 2009 18:43:42 +0000 (10:43 -0800)]
ceph: use connection mutex to protect read and write stages
Use a single mutex (previously out_mutex) to protect both read and write
activity from concurrent ceph_con_* calls. Drop the mutex when doing
callbacks to avoid nested locking (the callback may need to call something
like ceph_con_close).
Sage Weil [Tue, 22 Dec 2009 18:29:39 +0000 (10:29 -0800)]
ceph: unregister canceled/timed out osd requests
Canceled or timed out osd requests were getting left in the request list
and never deallocated (until umount). Unregister if they are canceled
(control-c) or time out.
This works around a bug in vfs_rename_dir() that rehashes the target
dentry. Ensure such dentries always fail revalidation by timing out the
dentry lease and kicking it out of the current directory lease gen.
Sage Weil [Fri, 20 Nov 2009 23:14:15 +0000 (15:14 -0800)]
ceph: reset msgr backoff during open, not after successful handshake
Reset the backoff delay when we reopen the connection, so that the delays
for any initial connection problems are reasonable. We were resetting only
after a successful handshake, which was of limited utility.
Sage Weil [Fri, 20 Nov 2009 21:43:45 +0000 (13:43 -0800)]
ceph: reset requested max_size after mds reconnect
The max_size increase request to the MDS can get lost during an MDS
restart and reconnect. Reset our requested value after the MDS recovers,
so that any blocked writes will re-request a larger max_size upon waking.
Also, explicit wake session caps after the reconnect. Normally the cap
renewal catches this, but not in the cases where the caps didn't go stale
in the first place, which would leave writers waiting on max_size asleep.
Sage Weil [Thu, 19 Nov 2009 00:50:41 +0000 (16:50 -0800)]
ceph: fix debugfs entry, simplify fsid checks
We may first learn our fsid from any of the mon, osd, or mds maps
(whichever the monitor sends first). Consolidate checks in a single
helper. Initialize the client debugfs entry then, since we need the
fsid (and global_id) for the directory name.
When we open a monitor session, we send an initial AUTH message listing
the auth protocols we support, our entity name, and (possibly) a previously
assigned global_id. The monitor chooses a protocol and responds with an
initial message.
Initially implement AUTH_NONE, a dummy protocol that provides no security,
but works within the new framework. It generates 'authorizers' that are
used when connecting to (mds, osd) services that simply state our entity
name and global_id.
Sage Weil [Wed, 18 Nov 2009 19:22:36 +0000 (11:22 -0800)]
ceph: remove bad calls to ceph_con_shutdown
We want to ceph_con_close when we're done with the connection, before
the ref count reaches 0. Once it does, do not call ceph_con_shutdown,
as that takes the con mutex and may sleep, and besides that is
unnecessary.
Sage Weil [Thu, 12 Nov 2009 23:07:24 +0000 (15:07 -0800)]
ceph: fix page invalidation deadlock
We occasionally want to make a best-effort attempt to invalidate cache
pages without fear of blocking. If this fails, we fall back to an async
invalidate in another thread.
Use invalidate_mapping_pages instead of invalidate_inode_page2, as that
will skip locked pages, and not deadlock.
Sage Weil [Wed, 11 Nov 2009 00:02:23 +0000 (16:02 -0800)]
ceph: remove recon_gen logic
We don't get an explicit affirmative confirmation that our caps reconnect,
nor do we necessarily want to pay that cost. So, take all this code out
for now.
Sage Weil [Tue, 10 Nov 2009 22:34:36 +0000 (14:34 -0800)]
ceph: separate banner and connect during handshake into distinct stages
We need to make sure we only swab the address during the banner once. So
break process_banner out of process_connect, and clean up the surrounding
code so that these are distinct phases of the handshake.
Sage Weil [Mon, 9 Nov 2009 20:05:48 +0000 (12:05 -0800)]
ceph: do not confuse stale and dead (unreconnected) caps
We were using the cap_gen to track both stale caps (caps that timed out
due to temporarily losing touch with the mds) and dead caps that did not
reconnect after an MDS failure. Introduce a recon_gen counter to track
reconnections to restarted MDSs and kill dead caps based on that instead.
Rename gen to cap_gen while we're at it to make it more clear which is
which.
Sage Weil [Sun, 8 Nov 2009 04:18:22 +0000 (20:18 -0800)]
ceph: make CRUSH hash function a bucket property
Make the integer hash function a property of the bucket it is used on. This
allows us to gracefully add support for new hash functions without starting
from scatch.
Sage Weil [Sat, 7 Nov 2009 05:55:25 +0000 (21:55 -0800)]
ceph: make object hash a pg_pool property
The object will be hashed to a placement seed (ps) based on the pg_pool's
hash function. This allows new hashes to be introduced into an existing
object store, or selection of a hash appropriate to the objects that
will be stored in a particular pool.
Sage Weil [Sat, 7 Nov 2009 00:44:05 +0000 (16:44 -0800)]
ceph: use strong hash function for mapping objects to pgs
We were using the (weak) dcache hash function, but it was leaving lower
bits consecutive for consecutive (inode) objects. We really want to make
the object to pg mapping random and uniform, so use a proper hash function
here.
This is Robert Jenkin's public domain hash function (with some minor
cleanup):
http://burtleburtle.net/bob/hash/evahash.html
Sage Weil [Wed, 4 Nov 2009 19:39:12 +0000 (11:39 -0800)]
ceph: fix endian conversions for ceph_pg
The endian conversions don't quite work with the old union ceph_pg. Just
make it a regular struct, and make each field __le. This is simpler and it
has the added bonus of actually working.
Sage Weil [Tue, 3 Nov 2009 23:17:56 +0000 (15:17 -0800)]
ceph: use fixed endian encoding for ceph_entity_addr
We exchange struct ceph_entity_addr over the wire and store it on disk.
The sockaddr_storage.ss_family field, however, is host endianness. So,
fix ss_family endianness to big endian when sending/receiving over the
wire.
projects / linux-beck.git / log