]> git.karo-electronics.de Git - linux-beck.git/log
linux-beck.git
14 years agoceph: fix flush_dirty_caps race with caps migration
Sage Weil [Mon, 1 Mar 2010 23:16:56 +0000 (15:16 -0800)]
ceph: fix flush_dirty_caps race with caps migration

The flush_dirty_caps() used to loop over the first entry of the cap_dirty
dirty list on the assumption that after calling ceph_check_caps() it would
be removed from the list.  This isn't true for caps that are being
migrated between MDSs, where we've received the EXPORT but not the IMPORT.

Instead, do a safe list iteration, and pin the next inode on the list via
the CEPH_I_NOFLUSH flag.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: include migrating caps in issued set
Sage Weil [Mon, 1 Mar 2010 23:17:34 +0000 (15:17 -0800)]
ceph: include migrating caps in issued set

We should include caps that are mid-migration (we've received the EXPORT,
but not the IMPORT) in the issued caps set.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix osdmap decoding when pools include (removed) snaps
Sage Weil [Mon, 1 Mar 2010 22:50:05 +0000 (14:50 -0800)]
ceph: fix osdmap decoding when pools include (removed) snaps

Add missing pointer dereference (p is a void **).

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: return EBADF if waiting for caps on closed file
Sage Weil [Mon, 1 Mar 2010 17:57:54 +0000 (09:57 -0800)]
ceph: return EBADF if waiting for caps on closed file

Verify the file is actually open for the given caps when we are
waiting for caps.  This ensures we will wake up and return EBADF
if another thread closes the file out from under us.

Note that EBADF is also the correct return code from write(2)
when called on a file handle opened for reading (although the
vfs should catch that).

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: set osd request message front length correctly
Sage Weil [Mon, 1 Mar 2010 23:26:41 +0000 (15:26 -0800)]
ceph: set osd request message front length correctly

We didn't set the front length correctly.  When messages used
the message pool we ended up with the conservative max (4 KB), and
the rest of the time the slightly less conservative estimate.  Even
though the OSD ignores the extra data, set it to the right value to avoid
sending extra data over the network.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: reset front len on return to msgpool; BUG on mismatched front iov
Sage Weil [Mon, 1 Mar 2010 23:25:00 +0000 (15:25 -0800)]
ceph: reset front len on return to msgpool; BUG on mismatched front iov

Reset msg front len when a message is returned to the pool: the caller
may have changed it.

BUG if we try to send a message with a hdr.front_len that doesn't match
the front iov.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix snaptrace decoding on cap migration between mds
Sage Weil [Mon, 1 Mar 2010 21:20:50 +0000 (13:20 -0800)]
ceph: fix snaptrace decoding on cap migration between mds

This was simply broken.  Apparently at some point we thought about putting
the snaptrace in the middle section, but didn't.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: use single osd op reply msg
Sage Weil [Mon, 1 Mar 2010 21:02:00 +0000 (13:02 -0800)]
ceph: use single osd op reply msg

Use a single ceph_msg for the osd reply, even when we are getting multiple
replies.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: reset bits on connection close
Sage Weil [Fri, 26 Feb 2010 21:55:51 +0000 (13:55 -0800)]
ceph: reset bits on connection close

Clear LOSSYTX bit, so that if/when we reconnect, said reconnect
will retry on failure.

Clear _PENDING bits too, to avoid polluting subsequent
connection state.

Drop unused REGISTERED bit.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: remove bogus mds forward warning
Sage Weil [Fri, 26 Feb 2010 00:40:07 +0000 (16:40 -0800)]
ceph: remove bogus mds forward warning

The must_resend flag is always true, not false.  In any case, we can
just ignore it anyway.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: remove fragile __map_osds optimization
Sage Weil [Fri, 26 Feb 2010 17:37:33 +0000 (09:37 -0800)]
ceph: remove fragile __map_osds optimization

We used to try to avoid freeing and then reallocating the osd
struct.  This is a bit fragile due to potential interactions with
other references (beyond o_requests), and may be the cause of
this crash:

[120633.442358] BUG: unable to handle kernel NULL pointer dereference at (null)
[120633.443292] IP: [<ffffffff812549b6>] rb_erase+0x11d/0x277
[120633.443292] PGD f7ff3067 PUD f7f53067 PMD 0
[120633.443292] Oops: 0000 [#1] PREEMPT SMP
[120633.443292] last sysfs file: /sys/kernel/uevent_seqnum
[120633.443292] CPU 1
[120633.443292] Modules linked in: ceph fan ac battery psmouse ehci_hcd ide_pci_generic ohci_hcd thermal processor button
[120633.443292] Pid: 3023, comm: ceph-msgr/1 Not tainted 2.6.32-rc2 #12 H8SSL
[120633.443292] RIP: 0010:[<ffffffff812549b6>]  [<ffffffff812549b6>] rb_erase+0x11d/0x277
[120633.443292] RSP: 0018:ffff8800f7b13a50  EFLAGS: 00010246
[120633.443292] RAX: ffff880022907819 RBX: ffff880022907818 RCX: 0000000000000000
[120633.443292] RDX: ffff8800f7b13a80 RSI: ffff8800f587eb48 RDI: 0000000000000000
[120633.443292] RBP: ffff8800f7b13a60 R08: 0000000000000000 R09: 0000000000000004
[120633.443292] R10: 0000000000000000 R11: ffff8800c4441000 R12: ffff8800f587eb48
[120633.443292] R13: ffff8800f58eaa00 R14: ffff8800f413c000 R15: 0000000000000001
[120633.443292] FS:  00007fbef6e226e0(0000) GS:ffff880009200000(0000) knlGS:0000000000000000
[120633.443292] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
[120633.443292] CR2: 0000000000000000 CR3: 00000000f7c53000 CR4: 00000000000006e0
[120633.443292] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[120633.443292] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[120633.443292] Process ceph-msgr/1 (pid: 3023, threadinfo ffff8800f7b12000, task ffff8800f5858b40)
[120633.443292] Stack:
[120633.443292]  ffff8800f413c000 ffff8800f587e9c0 ffff8800f7b13a80 ffffffffa0098a86
[120633.443292] <0> 00000000000006f1 0000000000000000 ffff8800f7b13af0 ffffffffa009959b
[120633.443292] <0> ffff8800f413c000 ffff880022a68400 ffff880022a68400 ffff8800f587e9c0
[120633.443292] Call Trace:
[120633.443292]  [<ffffffffa0098a86>] __remove_osd+0x4d/0xbc [ceph]
[120633.443292]  [<ffffffffa009959b>] __map_osds+0x199/0x4fa [ceph]
[120633.443292]  [<ffffffffa00999f4>] ? __send_request+0xf8/0x186 [ceph]
[120633.443292]  [<ffffffffa0099beb>] kick_requests+0x169/0x3cb [ceph]
[120633.443292]  [<ffffffffa009a8c1>] ceph_osdc_handle_map+0x370/0x522 [ceph]

Since we're probably screwed anyway if a small kmalloc is
failing, don't bother with trying to be clever here.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix connection fault STANDBY check
Sage Weil [Thu, 25 Feb 2010 20:40:45 +0000 (12:40 -0800)]
ceph: fix connection fault STANDBY check

Move any out_sent messages to out_queue _before_ checking if
out_queue is empty and going to STANDBY, or else we may drop
something that was never acked.

And clean up the code a bit (less goto).

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: invalidate_authorizer without con->mutex held
Sage Weil [Thu, 25 Feb 2010 20:38:57 +0000 (12:38 -0800)]
ceph: invalidate_authorizer without con->mutex held

This fixes lock ABBA inversion, as the ->invalidate_authorizer()
op may need to take a lock (or even call back into the
messenger).

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: don't clobber write return value when using O_SYNC
Yehuda Sadeh [Tue, 23 Feb 2010 18:16:23 +0000 (18:16 +0000)]
ceph: don't clobber write return value when using O_SYNC

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix client_request_forward decoding
Sage Weil [Tue, 23 Feb 2010 22:02:44 +0000 (14:02 -0800)]
ceph: fix client_request_forward decoding

The tid is in the message header, not body.  Broken since 6df058c0.

No need to look at next mds session; just mark the request and be done.
(The old error path was broken too, but now it's gone.)

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: drop messages on unregistered mds sessions; cleanup
Sage Weil [Mon, 22 Feb 2010 23:12:16 +0000 (15:12 -0800)]
ceph: drop messages on unregistered mds sessions; cleanup

Verify the mds session is currently registered before handling
incoming messages.  Clean up message handlers to pull mds out
of session->s_mds instead of less trustworthy src field.

Clean up con_{get,put} debug output.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix comments, locking in destroy_inode
Sage Weil [Mon, 22 Feb 2010 21:59:00 +0000 (13:59 -0800)]
ceph: fix comments, locking in destroy_inode

The destroy_inode path needs no inode locks since there are no
inode references.  Update __ceph_remove_cap comment to reflect
that it is called without cap->session->s_mutex in this case.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: move dereference after NULL test
Alexander Beregalov [Mon, 22 Feb 2010 14:17:44 +0000 (17:17 +0300)]
ceph: move dereference after NULL test

Signed-off-by: Alexander Beregalov <a.beregalov@gmail.com>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix up unexpected message handling
Sage Weil [Sat, 20 Feb 2010 05:43:23 +0000 (21:43 -0800)]
ceph: fix up unexpected message handling

Fix skipping of unexpected message types from osd, mon.

Clean up pr_info and debug output.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: cleanup redundant code in handle_cap_grant
Yehuda Sadeh [Fri, 19 Feb 2010 00:12:21 +0000 (00:12 +0000)]
ceph: cleanup redundant code in handle_cap_grant

There is no state in local vars that requires us to loop after temporarily
dropping i_lock.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: don't truncate dirty pages in invalidate work thread
Yehuda Sadeh [Fri, 19 Feb 2010 00:10:11 +0000 (00:10 +0000)]
ceph: don't truncate dirty pages in invalidate work thread

Instead of truncating the whole range of pages, we skip those
pages that are dirty or in the middle of writeback. Those pages
will be cleared later when the writeback completes.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: remove page upon writeback completion if lost cache cap
Yehuda Sadeh [Fri, 19 Feb 2010 00:07:01 +0000 (00:07 +0000)]
ceph: remove page upon writeback completion if lost cache cap

This page should have been removed earlier when the cache cap was
revoked, but a writeback was in flight, so it was skipped. We truncate
it here just as the writeback finishes, while it's still locked.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix check for invalidate_mapping_pages success
Sage Weil [Wed, 17 Feb 2010 18:43:37 +0000 (10:43 -0800)]
ceph: fix check for invalidate_mapping_pages success

We need to know whether there was any page left behind, and not the
return value (the total number of pages invalidated).  Look at the mapping
to see if we were successful or not.

Move it all into a helper to simplify the two callers.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix typo in ceph_queue_writeback debug output
Sage Weil [Wed, 17 Feb 2010 23:45:51 +0000 (15:45 -0800)]
ceph: fix typo in ceph_queue_writeback debug output

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: v0.19 release
Sage Weil [Wed, 17 Feb 2010 21:56:07 +0000 (13:56 -0800)]
ceph: v0.19 release

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: use rbtree for pg pools; decode new osdmap format
Sage Weil [Tue, 16 Feb 2010 23:55:03 +0000 (15:55 -0800)]
ceph: use rbtree for pg pools; decode new osdmap format

Since we can now create and destroy pg pools, the pool ids will be sparse,
and an array no longer makes sense for looking up by pool id.  Use an
rbtree instead.

The OSDMap encoding also no longer has a max pool count (previously used to
allocate the array).  There is a new pool_max, that is the largest pool id
we've ever used, although we don't actually need it in the client.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix memory leak when destroying osdmap with pg_temp mappings
Sage Weil [Tue, 16 Feb 2010 23:53:32 +0000 (15:53 -0800)]
ceph: fix memory leak when destroying osdmap with pg_temp mappings

Also move _lookup_pg_mapping into a helper.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix iterate_caps removal race
Sage Weil [Tue, 16 Feb 2010 19:39:45 +0000 (11:39 -0800)]
ceph: fix iterate_caps removal race

We need to be able to iterate over all caps on a session with a
possibly slow callback on each cap.  To allow this, we used to
prevent cap reordering while we were iterating.  However, we were
not safe from races with removal: removing the 'next' cap would
make the next pointer from list_for_each_entry_safe be invalid,
and cause a lock up or similar badness.

Instead, we keep an iterator pointer in the session pointing to
the current cap.  As before, we avoid reordering.  For removal,
if the cap isn't the current cap we are iterating over, we are
fine.  If it is, we clear cap->ci (to mark the cap as pending
removal) but leave it in the session list.  In iterate_caps, we
can safely finish removal and get the next cap pointer.

While we're at it, clean up put_cap to not take a cap reservation
context, as it was never used.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: clean up readdir caps reservation
Sage Weil [Wed, 17 Feb 2010 18:02:43 +0000 (10:02 -0800)]
ceph: clean up readdir caps reservation

Use a global counter for the minimum number of allocated caps instead of
hard coding a check against readdir_max.  This takes into account multiple
client instances, and avoids examining the superblock mount options when a
cap is dropped.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix authentication races, auth_none oops
Sage Weil [Tue, 16 Feb 2010 00:22:28 +0000 (16:22 -0800)]
ceph: fix authentication races, auth_none oops

Call __validate_auth() under monc->mutex, and use helper for
initial hello so that the pending_auth flag is set.  This fixes
possible races in which we have an authentication request (hello
or otherwise) pending and send another one.  In particular, with
auth_none, we _never_ want to call ceph_build_auth() from
__validate_auth(), since the ->build_request() method is NULL.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: use rbtree for mon statfs requests
Sage Weil [Mon, 15 Feb 2010 22:47:28 +0000 (14:47 -0800)]
ceph: use rbtree for mon statfs requests

An rbtree is lighter weight, particularly given we will generally have
very few in-flight statfs requests.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: use rbtree for snap_realms
Sage Weil [Mon, 15 Feb 2010 22:37:55 +0000 (14:37 -0800)]
ceph: use rbtree for snap_realms

Switch from radix tree to rbtree for snap realms.  This is much more
appropriate given that realm keys are few and far between.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: use rbtree for mds requests
Sage Weil [Mon, 15 Feb 2010 20:08:46 +0000 (12:08 -0800)]
ceph: use rbtree for mds requests

The rbtree is a more appropriate data structure than a radix_tree.  It
avoids extra memory usage and simplifies the code.

It also fixes a bug where the debugfs 'mdsc' file wasn't including the
most recent mds request.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: cancel delayed work when closing connection
Sage Weil [Mon, 15 Feb 2010 20:05:09 +0000 (12:05 -0800)]
ceph: cancel delayed work when closing connection

This ensures that if/when we reopen the connection, we can requeue work on
the connection immediately, without waiting for an old timer to expire.
Queue new delayed work inside con->mutex to avoid any race.

This fixes problems with clients failing to reconnect to the MDS due to
the client_reconnect message arriving too late (due to waiting for an old
delayed work timeout to expire).

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: allow connection to be reopened by fault callback
Sage Weil [Wed, 17 Feb 2010 06:01:03 +0000 (22:01 -0800)]
ceph: allow connection to be reopened by fault callback

Fix the messenger to allow a ceph_con_open() during the fault callback.
Previously the work wasn't getting queued on the connection because the
fault path avoids requeued work (normally spurious).  Loop on reopening by
checking for the OPENING state bit.

This fixes OSD reconnects when a TCP connection drops.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: reset osd connections after fault
Sage Weil [Mon, 15 Feb 2010 20:11:51 +0000 (12:11 -0800)]
ceph: reset osd connections after fault

A single osd connection fault (e.g. tcp disconnect) wasn't
reopening the connection, which causes all current and future
requests for that osd to hang.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix msgr to keep sent messages until acked
Sage Weil [Sun, 14 Feb 2010 04:29:31 +0000 (20:29 -0800)]
ceph: fix msgr to keep sent messages until acked

The test was backwards from commit b3d1dbbd: keep the message if the
connection _isn't_ lossy.  This allows the client to continue when the
TCP connection drops for some reason (network glitch) but both ends
survive.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: remove bogus invalidate_mapping_pages
Sage Weil [Wed, 10 Feb 2010 00:43:11 +0000 (16:43 -0800)]
ceph: remove bogus invalidate_mapping_pages

We were invalidating mapping pages when dropping FILE_CACHE in
__send_cap().  But ceph_check_caps attempts to invalidate already, and
also checks for success, so we should never get to this point.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: invalidate pages even if truncate is pending
Sage Weil [Tue, 9 Feb 2010 23:44:16 +0000 (15:44 -0800)]
ceph: invalidate pages even if truncate is pending

There is no reason not to invalidate pages when a truncate is pending.
Both throw out page cache pages.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: cleanup async writeback, truncation, invalidate helpers
Sage Weil [Tue, 9 Feb 2010 23:24:44 +0000 (15:24 -0800)]
ceph: cleanup async writeback, truncation, invalidate helpers

Grab inode ref in helper.  Make work functions static, with consistent
naming.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix sync read eof check deadlock
Sage Weil [Tue, 9 Feb 2010 22:04:02 +0000 (14:04 -0800)]
ceph: fix sync read eof check deadlock

If a sync read gets a short result from the OSD, it may need to do a
getattr to see if it is short due to reaching end-of-file.  The getattr
was being done while holding a reference to FILE_RD, which can lead to
a deadlock if the MDS is revoking that capability bit and can't process
the getattr until it does.

We fix this by setting a flag if EOF size validation is needed, and doing
the getattr in ceph_aio_read, after the RD cap ref is dropped.  If the
read needs to be continued, we loop and continue traversing the file.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: do not retain caps that are being revoked
Sage Weil [Tue, 9 Feb 2010 21:41:47 +0000 (13:41 -0800)]
ceph: do not retain caps that are being revoked

Never retain caps in __send_cap() that are being revoked.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: cap revocation fixes
Sage Weil [Tue, 9 Feb 2010 21:41:18 +0000 (13:41 -0800)]
ceph: cap revocation fixes

Try to invalidate pages in ceph_check_caps() if FILE_CACHE is being
revoked.  If we fail, queue an immediate async invalidate if FILE_CACHE
is being revoked.  (If it's not being revoked, we just queue the caps
for later evaluation later, as per the old behavior.)

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: sync read/write considers page cache
Yehuda Sadeh [Tue, 9 Feb 2010 19:14:41 +0000 (11:14 -0800)]
ceph: sync read/write considers page cache

In the cases where we either do a sync read or a write, we
need to make sure that everything in the page cache is flushed.
In the case of a sync write we invalidate the relevant pages,
so that subsequent read/write reflects the new data written.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix truncation when not holding caps
Yehuda Sadeh [Tue, 9 Feb 2010 19:08:40 +0000 (11:08 -0800)]
ceph: fix truncation when not holding caps

A truncation should occur when either we have the
specified caps for the file, or (in cases where we are
not the only ones referencing the file) when it is mapped
or when it is opened. The latter two cases were not
handled.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: refactor ceph_write_begin, fix ceph_page_mkwrite
Yehuda Sadeh [Tue, 9 Feb 2010 19:02:51 +0000 (11:02 -0800)]
ceph: refactor ceph_write_begin, fix ceph_page_mkwrite

Originally ceph_page_mkwrite called ceph_write_begin, hoping that
the returned locked page would be the page that it was requested
to mkwrite. Factored out relevant part of ceph_page_mkwrite and
we lock the right page anyway.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix short synchronous reads
Yehuda Sadeh [Thu, 4 Feb 2010 21:41:41 +0000 (13:41 -0800)]
ceph: fix short synchronous reads

Zeroing of holes was not done correctly: page_off was miscalculated and
zeroing the tail didn't not adjust the 'read' value to include the zeroed
portion.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: add uid field to ceph_pg_pool
Sage Weil [Fri, 5 Feb 2010 00:18:10 +0000 (16:18 -0800)]
ceph: add uid field to ceph_pg_pool

Also verify encoding version as we go.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: put unused osd connections on lru
Yehuda Sadeh [Wed, 3 Feb 2010 19:00:26 +0000 (11:00 -0800)]
ceph: put unused osd connections on lru

Instead of removing osd connection immediately when the
requests list is empty, put the osd connection on an lru.
Only if that osd has not been used for more than a specified
time, will it be removed.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: remove unused variable
Yehuda Sadeh [Wed, 3 Feb 2010 18:47:48 +0000 (10:47 -0800)]
ceph: remove unused variable

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: add support for auth_x authentication protocol
Sage Weil [Wed, 3 Feb 2010 00:25:35 +0000 (16:25 -0800)]
ceph: add support for auth_x authentication protocol

The auth_x protocol implements support for a kerberos-like mutual
authentication infrastructure used by Ceph.  We do not simply use vanilla
kerberos because of scalability and performance issues when dealing with
a large cluster of nodes providing a single logical service.

Auth_x provides mutual authentication of client and server and protects
against replay and man in the middle attacks.  It does not encrypt
the full session over the wire, however, so data payload may still be
snooped.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: add struct version to auth encoding
Sage Weil [Thu, 4 Feb 2010 17:42:20 +0000 (09:42 -0800)]
ceph: add struct version to auth encoding

Inlucde struct version in encoding. This will streamline future protocol
changes.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: allow renewal of auth credentials
Sage Weil [Wed, 3 Feb 2010 00:21:06 +0000 (16:21 -0800)]
ceph: allow renewal of auth credentials

Add infrastructure to allow the mon_client to periodically renew its auth
credentials.  Also add a messenger callback that will force such a renewal
if a peer rejects our authenticator.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: aes crypto and base64 encode/decode helpers
Sage Weil [Wed, 3 Feb 2010 00:07:07 +0000 (16:07 -0800)]
ceph: aes crypto and base64 encode/decode helpers

Helpers to encrypt/decrypt AES and base64.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: buffer decoding helpers
Sage Weil [Wed, 3 Feb 2010 00:11:19 +0000 (16:11 -0800)]
ceph: buffer decoding helpers

Helper for decoding into a ceph_buffer, and other misc decoding helpers
we will need.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: release all pages after successful osd write response
Sage Weil [Wed, 3 Feb 2010 00:34:04 +0000 (16:34 -0800)]
ceph: release all pages after successful osd write response

We release all the pages, even if the osd response was
different than the number of pages written. This could only
happen due to truncation that arrives the osd in
different order, for which we want the pages released anyway.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: always send truncation info with read and write osd ops
Yehuda Sadeh [Tue, 2 Feb 2010 00:10:45 +0000 (16:10 -0800)]
ceph: always send truncation info with read and write osd ops

This fixes a bug where the read/write ops arrive the osd after
a following truncation request.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: remove unreachable code
Yehuda Sadeh [Fri, 29 Jan 2010 19:01:11 +0000 (11:01 -0800)]
ceph: remove unreachable code

We never truncate to a smaller size without contacting the MDS.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: include type in ceph_entity_addr, filepath
Sage Weil [Wed, 27 Jan 2010 22:28:10 +0000 (14:28 -0800)]
ceph: include type in ceph_entity_addr, filepath

Include a type/version in ceph_entity_addr and filepath.  Include extra
byte in filepath encoding as necessary.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: precede encoded ceph_pg_pool struct with version
Sage Weil [Tue, 26 Jan 2010 00:03:02 +0000 (16:03 -0800)]
ceph: precede encoded ceph_pg_pool struct with version

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: keep reserved replies on the request structure
Yehuda Sadeh [Thu, 14 Jan 2010 01:03:23 +0000 (17:03 -0800)]
ceph: keep reserved replies on the request structure

This includes treating all the data preallocation and revokation
at the same place, not having to have a special case for
the reserved pages.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
14 years agoceph: alloc message data pages and check if tid exists
Yehuda Sadeh [Mon, 11 Jan 2010 22:47:13 +0000 (14:47 -0800)]
ceph: alloc message data pages and check if tid exists

Now doing it in the same callback that is also responsible for
allocating the 'front' part of the message. If we get a message
that we haven't got a corresponding tid for, mark it for skipping.

Moving the mutex unlock/lock from the osd alloc_msg callback
to the calling function in the messenger.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
14 years agoceph: refactor messages data section allocation
Yehuda Sadeh [Mon, 11 Jan 2010 18:32:02 +0000 (10:32 -0800)]
ceph: refactor messages data section allocation

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
14 years agoceph: allocate middle of message before stating to read
Yehuda Sadeh [Fri, 8 Jan 2010 21:58:34 +0000 (13:58 -0800)]
ceph: allocate middle of message before stating to read

Both front and middle parts of the message are now being
allocated at the ceph_alloc_msg().

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
14 years agoceph: properly handle aborted mds requests
Sage Weil [Mon, 25 Jan 2010 19:33:08 +0000 (11:33 -0800)]
ceph: properly handle aborted mds requests

Previously, if the MDS request was interrupted, we would unregister the
request and ignore any reply.  This could cause the caps or other cache
state to become out of sync.  (For instance, aborting dbench and doing
rm -r on clients would complain about a non-empty directory because the
client didn't realize it's aborted file create request completed.)

Even we don't unregister, we still can't process the reply normally because
we are no longer holding the caller's locks (like the dir i_mutex).

So, mark aborted operations with r_aborted, and in the reply handler, be
sure to process all the caps.  Do not process the namespace changes,
though, since we no longer will hold the dir i_mutex.  The dentry lease
state can also be ignored as it's more forgiving.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: mark MDS CREATE as a write op
Sage Weil [Mon, 25 Jan 2010 19:18:17 +0000 (11:18 -0800)]
ceph: mark MDS CREATE as a write op

CEPH_MDS_OP_CREATE was not correctly marked as a write operation.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: remove duplicate variable initialization
Julia Lawall [Wed, 20 Jan 2010 23:16:41 +0000 (15:16 -0800)]
ceph: remove duplicate variable initialization

The variable client is initialized twice to the same (side effect-free)
expression.  Drop one initialization.

A simplified version of the semantic match that finds this problem is:
(http://coccinelle.lip6.fr/)

// <smpl>
@forall@
idexpression *x;
identifier f!=ERR_PTR;
@@

x = f(...)
... when != x
(
x = f(...,<+...x...+>,...)
|
* x = f(...)
)
// </smpl>

Signed-off-by: Julia Lawall <julia@diku.dk>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: display pgid in debugfs osd request dump
Sage Weil [Fri, 8 Jan 2010 23:58:25 +0000 (15:58 -0800)]
ceph: display pgid in debugfs osd request dump

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: remove unused erank field
Sage Weil [Fri, 8 Jan 2010 00:12:36 +0000 (16:12 -0800)]
ceph: remove unused erank field

The ceph_entity_addr erank field is obsolete; remove it.  Get rid of
trivial addr comparison helpers while we're at it.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: change dentry offset and position after splice_dentry
Yehuda Sadeh [Thu, 7 Jan 2010 23:36:32 +0000 (15:36 -0800)]
ceph: change dentry offset and position after splice_dentry

This fixes a bug, where we had the parent list have dentries with
offsets that are not monotonically increasing, which caused the ceph
dcache_readdir to skip entries.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix copy_user_to_page_vector()
Yehuda Sadeh [Thu, 31 Dec 2009 20:04:58 +0000 (12:04 -0800)]
ceph: fix copy_user_to_page_vector()

The function was broken in the case where there was more than one page
involved, broke the ceph sync_write case.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: use ceph_pagelist for mds reconnect message; change encoding (protocol change)
Sage Weil [Wed, 23 Dec 2009 20:21:51 +0000 (12:21 -0800)]
ceph: use ceph_pagelist for mds reconnect message; change encoding (protocol change)

Use the ceph_pagelist to encode the MDS reconnect message.  We change the
message encoding (protocol change!) at the same time to make our life
easier (we don't know how many snaprealms we have when we start encoding).

An empty message implies the session is closed/does not exist.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: support ceph_pagelist for message payload
Sage Weil [Wed, 23 Dec 2009 20:12:31 +0000 (12:12 -0800)]
ceph: support ceph_pagelist for message payload

The ceph_pagelist is a simple list of whole pages, strung together via
their lru list_head.  It facilitates encoding to a "buffer" of unknown
size.  Allow its use in place of the ceph_msg page vector.

This will be used to fix the huge buffer preallocation woes of MDS
reconnection.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: add feature bits to connection handshake (protocol change)
Sage Weil [Wed, 23 Dec 2009 17:30:21 +0000 (09:30 -0800)]
ceph: add feature bits to connection handshake (protocol change)

Define supported and required feature set.  Fail connection if the server
requires features we do not support (TAG_FEATURES), or if the server does
not support features we require.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: include transaction id in ceph_msg_header (protocol change)
Sage Weil [Tue, 22 Dec 2009 19:24:33 +0000 (11:24 -0800)]
ceph: include transaction id in ceph_msg_header (protocol change)

Many (most?) message types include a transaction id.  By including it in
the fixed size header, we always have it available even when we are unable
to allocate memory for the (larger, variable sized) message body.  This
will allow us to error out the appropriate request instead of (silently)
dropping the reply.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: more informative msgpool errors
Sage Weil [Tue, 22 Dec 2009 18:45:18 +0000 (10:45 -0800)]
ceph: more informative msgpool errors

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: control access to page vector for incoming data
Sage Weil [Tue, 22 Dec 2009 18:45:45 +0000 (10:45 -0800)]
ceph: control access to page vector for incoming data

When we issue an OSD read, we specify a vector of pages that the data is to
be read into.  The request may be sent multiple times, to multiple OSDs, if
the osdmap changes, which means we can get more than one reply.

Only read data into the page vector if the reply is coming from the
OSD we last sent the request to.  Keep track of which connection is using
the vector by taking a reference.  If another connection was already
using the vector before and a new reply comes in on the right connection,
revoke the pages from the other connection.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: use connection mutex to protect read and write stages
Sage Weil [Tue, 22 Dec 2009 18:43:42 +0000 (10:43 -0800)]
ceph: use connection mutex to protect read and write stages

Use a single mutex (previously out_mutex) to protect both read and write
activity from concurrent ceph_con_* calls.  Drop the mutex when doing
callbacks to avoid nested locking (the callback may need to call something
like ceph_con_close).

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: unregister canceled/timed out osd requests
Sage Weil [Tue, 22 Dec 2009 18:29:39 +0000 (10:29 -0800)]
ceph: unregister canceled/timed out osd requests

Canceled or timed out osd requests were getting left in the request list
and never deallocated (until umount).  Unregister if they are canceled
(control-c) or time out.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: only unregister registered bdi
Sage Weil [Tue, 22 Dec 2009 05:04:26 +0000 (21:04 -0800)]
ceph: only unregister registered bdi

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: do not touch_caps while iterating over caps list
Sage Weil [Tue, 22 Dec 2009 04:40:34 +0000 (20:40 -0800)]
ceph: do not touch_caps while iterating over caps list

Avoid confusing iterate_session_caps(), flag the session while we are
iterating so that __touch_cap does not rearrange items on the list.

All other modifiers of session->s_caps do so under the protection of
s_mutex.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix incremental osdmap pg_temp decoding bug
Sage Weil [Tue, 22 Dec 2009 00:02:37 +0000 (16:02 -0800)]
ceph: fix incremental osdmap pg_temp decoding bug

An incremental pg_temp wasn't being decoded properly (wrong bound on
for loop).

Also remove unused local variable, while we're at it.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix error paths for corrupt osdmap messages
Sage Weil [Mon, 21 Dec 2009 22:49:37 +0000 (14:49 -0800)]
ceph: fix error paths for corrupt osdmap messages

Both osdmap_decode() and osdmap_apply_incremental() should never return
NULL.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: do not drop lease during revalidate
Sage Weil [Mon, 21 Dec 2009 19:48:44 +0000 (11:48 -0800)]
ceph: do not drop lease during revalidate

We need to hold session s_mutex for __ceph_mdsc_drop_dentry_lease(), which
we don't, so skip it.  It was purely an optimization.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: ensure rename target dentry fails revalidation
Sage Weil [Mon, 21 Dec 2009 19:42:18 +0000 (11:42 -0800)]
ceph: ensure rename target dentry fails revalidation

This works around a bug in vfs_rename_dir() that rehashes the target
dentry.  Ensure such dentries always fail revalidation by timing out the
dentry lease and kicking it out of the current directory lease gen.

This can be reverted when the vfs bug is fixed.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: writeback congestion control
Yehuda Sadeh [Fri, 18 Dec 2009 21:51:57 +0000 (13:51 -0800)]
ceph: writeback congestion control

Set bdi congestion bit when amount of write data in flight exceeds adjustable
threshold.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: writepage grabs and releases inode
Yehuda Sadeh [Wed, 16 Dec 2009 22:51:06 +0000 (14:51 -0800)]
ceph: writepage grabs and releases inode

Fixes a deadlock that is triggered due to kswapd,
while the page was locked and the iput couldn't tear
down the address space.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
14 years agoceph: remove unaccessible code
Yehuda Sadeh [Wed, 16 Dec 2009 22:22:17 +0000 (14:22 -0800)]
ceph: remove unaccessible code

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
14 years agoceph: include link to bdi in debugfs
Sage Weil [Tue, 15 Dec 2009 22:44:32 +0000 (14:44 -0800)]
ceph: include link to bdi in debugfs

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: make mds ops interruptible
Sage Weil [Tue, 15 Dec 2009 18:27:48 +0000 (10:27 -0800)]
ceph: make mds ops interruptible

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: plug leak of incoming message during connection fault/close
Sage Weil [Fri, 11 Dec 2009 17:48:05 +0000 (09:48 -0800)]
ceph: plug leak of incoming message during connection fault/close

If we explicitly close a connection, or there is a socket error, we need
to drop any partially received message.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: hex dump corrupt server data to KERN_DEBUG
Sage Weil [Mon, 14 Dec 2009 23:13:47 +0000 (15:13 -0800)]
ceph: hex dump corrupt server data to KERN_DEBUG

Also, print fsid using standard format, NOT hex dump.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix msgpool reservation leak
Yehuda Sadeh [Tue, 15 Dec 2009 17:50:36 +0000 (09:50 -0800)]
ceph: fix msgpool reservation leak

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
14 years agoceph: don't save sent messages on lossy connections
Sage Weil [Mon, 14 Dec 2009 22:58:11 +0000 (14:58 -0800)]
ceph: don't save sent messages on lossy connections

For lossy connections we drop all state on socket errors, so there is no
reason to keep sent ceph_msg's around.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: detect lossy state of connection
Sage Weil [Mon, 14 Dec 2009 22:56:56 +0000 (14:56 -0800)]
ceph: detect lossy state of connection

The server indicates whether a connection is lossy; set our LOSSYTX bit
appropriately.  Do not set lossy bit on outgoing connections.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: plug msg leak in con_fault
Sage Weil [Mon, 14 Dec 2009 22:30:34 +0000 (14:30 -0800)]
ceph: plug msg leak in con_fault

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: carry explicit msg reference for currently sending message
Sage Weil [Mon, 14 Dec 2009 22:04:30 +0000 (14:04 -0800)]
ceph: carry explicit msg reference for currently sending message

Carry a ceph_msg reference for connection->out_msg.  This will allow us to
make out_sent optional.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix leak of monc mutex
Sage Weil [Fri, 11 Dec 2009 16:55:23 +0000 (08:55 -0800)]
ceph: fix leak of monc mutex

Fix leak of monc mutex on ENOMEM or bad fsid when receiving new mon map.
Audited all other users.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: do not feed bad device ids to crush
Sage Weil [Wed, 9 Dec 2009 20:34:01 +0000 (12:34 -0800)]
ceph: do not feed bad device ids to crush

Do not feed bad (large) device ids to CRUSH.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: use kref for ceph_msg
Sage Weil [Mon, 7 Dec 2009 23:55:05 +0000 (15:55 -0800)]
ceph: use kref for ceph_msg

Signed-off-by: Sage Weil <sage@newdream.net>