Daniel Dressler [Fri, 21 Nov 2014 08:15:07 +0000 (17:15 +0900)]
Btrfs: disk-io: replace root args iff only fs_info used
This is the 3rd independent patch of a larger project to cleanup btrfs's
internal usage of btrfs_root. Many functions take btrfs_root only to
grab the fs_info struct.
By requiring a root these functions cause programmer overhead. That
these functions can accept any valid root is not obvious until
inspection.
This patch reduces the specificity of such functions to accept the
fs_info directly.
These patches can be applied independently and thus are not being
submitted as a patch series. There should be about 26 patches by the
project's completion. Each patch will cleanup between 1 and 34 functions
apiece. Each patch covers a single file's functions.
This patch affects the following function(s):
1) csum_tree_block
2) csum_dirty_buffer
3) check_tree_block_fsid
4) btrfs_find_tree_block
5) clean_tree_block
Signed-off-by: Daniel Dressler <danieru.dressler@gmail.com> Signed-off-by: David Sterba <dsterba@suse.cz>
Daniel Dressler [Mon, 17 Nov 2014 13:05:02 +0000 (22:05 +0900)]
Btrfs: delayed-inode: replace root args iff only fs_info used
This is the second independent patch of a larger project to cleanup
btrfs's internal usage of btrfs_root. Many functions take btrfs_root
only to grab the fs_info struct.
By requiring a root these functions cause programmer overhead. That
these functions can accept any valid root is not obvious until
inspection.
This patch reduces the specificity of such functions to accept the
fs_info directly.
These patches can be applied independently and thus are not being
submitted as a patch series. There should be about 26 patches by the
project's completion. Each patch will cleanup between 1 and 34 functions
apiece. Each patch covers a single file's functions.
This patch affects the following function(s):
1) btrfs_wq_run_delayed_node
Signed-off-by: Daniel Dressler <danieru.dressler@gmail.com> Signed-off-by: David Sterba <dsterba@suse.cz>
Daniel Dressler [Wed, 12 Nov 2014 04:43:09 +0000 (13:43 +0900)]
Btrfs: ctree: reduce args where only fs_info used
This patch is part of a larger project to cleanup btrfs's internal usage
of struct btrfs_root. Many functions take btrfs_root only to grab a
pointer to fs_info.
This causes programmers to ponder which root can be passed. Since only
the fs_info is read affected functions can accept any root, except this
is only obvious upon inspection.
This patch reduces the specificty of such functions to accept the
fs_info directly.
This patch does not address the two functions in ctree.c (insert_ptr,
and split_item) which only use root for BUG_ONs in ctree.c
This patch affects the following functions:
1) fixup_low_keys
2) btrfs_set_item_key_safe
Signed-off-by: Daniel Dressler <danieru.dressler@gmail.com> Signed-off-by: David Sterba <dsterba@suse.cz>
Filipe Manana [Fri, 13 Feb 2015 16:56:14 +0000 (16:56 +0000)]
Btrfs: don't remove extents and xattrs when logging new names
If we are recording in the tree log that an inode has new names (new hard
links were added), we would drop items, belonging to the inode, that we
shouldn't:
1) When the flag BTRFS_INODE_COPY_EVERYTHING is set in the inode's runtime
flags, we ended up dropping all the extent and xattr items that were
previously logged. This was done only in memory, since logging a new
name doesn't imply syncing the log;
2) When the flag BTRFS_INODE_COPY_EVERYTHING is set in the inode's runtime
flags, we ended up dropping all the xattr items that were previously
logged. Like the case before, this was done only in memory because
logging a new name doesn't imply syncing the log.
This led to some surprises in scenarios such as the following:
1) write some extents to an inode;
2) fsync the inode;
3) truncate the inode or delete/modify some of its xattrs
4) add a new hard link for that inode
5) fsync some other file, to force the log tree to be durably persisted
6) power failure happens
The next time the fs is mounted, the fsync log replay code is executed,
and the resulting file doesn't have the content it had when the last fsync
against it was performed, instead if has a content matching what it had
when the last transaction commit happened.
So change the behaviour such that when a new name is logged, only the inode
item and reference items are processed.
This is easy to reproduce with the test I just made for xfstests, whose
main body is:
# Create our test file with some data.
$XFS_IO_PROG -f -c "pwrite -S 0xaa -b 8K 0 8K" \
$SCRATCH_MNT/foo | _filter_xfs_io
# Make sure the file is durably persisted.
sync
# Append some data to our file, to increase its size.
$XFS_IO_PROG -f -c "pwrite -S 0xcc -b 4K 8K 4K" \
$SCRATCH_MNT/foo | _filter_xfs_io
# Fsync the file, so from this point on if a crash/power failure happens, our
# new data is guaranteed to be there next time the fs is mounted.
$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo
# Now shrink our file to 5000 bytes.
$XFS_IO_PROG -c "truncate 5000" $SCRATCH_MNT/foo
# Now do an expanding truncate to a size larger than what we had when we last
# fsync'ed our file. This is just to verify that after power failure and
# replaying the fsync log, our file matches what it was when we last fsync'ed
# it - 12Kb size, first 8Kb of data had a value of 0xaa and the last 4Kb of
# data had a value of 0xcc.
$XFS_IO_PROG -c "truncate 32K" $SCRATCH_MNT/foo
# Add one hard link to our file. This made btrfs drop all of our file's
# metadata from the fsync log, including the metadata relative to the
# extent we just wrote and fsync'ed. This change was made only to the fsync
# log in memory, so adding the hard link alone doesn't change the persisted
# fsync log. This happened because the previous truncates set the runtime
# flag BTRFS_INODE_NEEDS_FULL_SYNC in the btrfs inode structure.
ln $SCRATCH_MNT/foo $SCRATCH_MNT/foo_link
# Now make sure the in memory fsync log is durably persisted.
# Creating and fsync'ing another file will do it.
# After this our persisted fsync log will no longer have metadata for our file
# foo that points to the extent we wrote and fsync'ed before.
touch $SCRATCH_MNT/bar
$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/bar
# As expected, before the crash/power failure, we should be able to see a file
# with a size of 32Kb, with its first 5000 bytes having the value 0xaa and all
# the remaining bytes with value 0x00.
echo "File content before:"
od -t x1 $SCRATCH_MNT/foo
# Simulate a crash/power loss.
_load_flakey_table $FLAKEY_DROP_WRITES
_unmount_flakey
# After mounting the fs again, the fsync log was replayed.
# The expected result is to see a file with a size of 12Kb, with its first 8Kb
# of data having the value 0xaa and its last 4Kb of data having a value of 0xcc.
# The btrfs bug used to leave the file as it used te be as of the last
# transaction commit - that is, with a size of 8Kb with all bytes having a
# value of 0xaa.
echo "File content after:"
od -t x1 $SCRATCH_MNT/foo
The test case for xfstests follows soon.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Chris Mason <clm@fb.com>
Filipe Manana [Fri, 13 Feb 2015 12:30:56 +0000 (12:30 +0000)]
Btrfs: fix fsync data loss after adding hard link to inode
We have a scenario where after the fsync log replay we can lose file data
that had been previously fsync'ed if we added an hard link for our inode
and after that we sync'ed the fsync log (for example by fsync'ing some
other file or directory).
This is because when adding an hard link we updated the inode item in the
log tree with an i_size value of 0. At that point the new inode item was
in memory only and a subsequent fsync log replay would not make us lose
the file data. However if after adding the hard link we sync the log tree
to disk, by fsync'ing some other file or directory for example, we ended
up losing the file data after log replay, because the inode item in the
persisted log tree had an an i_size of zero.
This is easy to reproduce, and the following excerpt from my test for
xfstests shows this:
# Create one file with data and fsync it.
# This made the btrfs fsync log persist the data and the inode metadata with
# a correct inode->i_size (4096 bytes).
$XFS_IO_PROG -f -c "pwrite -S 0xaa -b 4K 0 4K" -c "fsync" \
$SCRATCH_MNT/foo | _filter_xfs_io
# Now add one hard link to our file. This made the btrfs code update the fsync
# log, in memory only, with an inode metadata having a size of 0.
ln $SCRATCH_MNT/foo $SCRATCH_MNT/foo_link
# Now force persistence of the fsync log to disk, for example, by fsyncing some
# other file.
touch $SCRATCH_MNT/bar
$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/bar
# Before a power loss or crash, we could read the 4Kb of data from our file as
# expected.
echo "File content before:"
od -t x1 $SCRATCH_MNT/foo
# Simulate a crash/power loss.
_load_flakey_table $FLAKEY_DROP_WRITES
_unmount_flakey
# After the fsync log replay, because the fsync log had a value of 0 for our
# inode's i_size, we couldn't read anymore the 4Kb of data that we previously
# wrote and fsync'ed. The size of the file became 0 after the fsync log replay.
echo "File content after:"
od -t x1 $SCRATCH_MNT/foo
Another alternative test, that doesn't need to fsync an inode in the same
transaction it was created, is:
# Create our test file with some data.
$XFS_IO_PROG -f -c "pwrite -S 0xaa -b 8K 0 8K" \
$SCRATCH_MNT/foo | _filter_xfs_io
# Make sure the file is durably persisted.
sync
# Append some data to our file, to increase its size.
$XFS_IO_PROG -f -c "pwrite -S 0xcc -b 4K 8K 4K" \
$SCRATCH_MNT/foo | _filter_xfs_io
# Fsync the file, so from this point on if a crash/power failure happens, our
# new data is guaranteed to be there next time the fs is mounted.
$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo
# Add one hard link to our file. This made btrfs write into the in memory fsync
# log a special inode with generation 0 and an i_size of 0 too. Note that this
# didn't update the inode in the fsync log on disk.
ln $SCRATCH_MNT/foo $SCRATCH_MNT/foo_link
# Now make sure the in memory fsync log is durably persisted.
# Creating and fsync'ing another file will do it.
touch $SCRATCH_MNT/bar
$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/bar
# As expected, before the crash/power failure, we should be able to read the
# 12Kb of file data.
echo "File content before:"
od -t x1 $SCRATCH_MNT/foo
# Simulate a crash/power loss.
_load_flakey_table $FLAKEY_DROP_WRITES
_unmount_flakey
# After mounting the fs again, the fsync log was replayed.
# The btrfs fsync log replay code didn't update the i_size of the persisted
# inode because the inode item in the log had a special generation with a
# value of 0 (and it couldn't know the correct i_size, since that inode item
# had a 0 i_size too). This made the last 4Kb of file data inaccessible and
# effectively lost.
echo "File content after:"
od -t x1 $SCRATCH_MNT/foo
This isn't a new issue/regression. This problem has been around since the
log tree code was added in 2008:
Forrest Liu [Wed, 11 Feb 2015 06:24:12 +0000 (14:24 +0800)]
Btrfs: fix BUG_ON in btrfs_orphan_add() when delete unused block group
Removing large amount of block group in a transaction may encounters
BUG_ON() in btrfs_orphan_add(). That is because btrfs_orphan_reserve_metadata()
will grab metadata reservation from transaction handle, and
btrfs_delete_unused_bgs() didn't reserve metadata for trnasaction handle when
delete unused block group.
Josef Bacik [Wed, 11 Feb 2015 20:08:59 +0000 (15:08 -0500)]
Btrfs: account for large extents with enospc
On our gluster boxes we stream large tar balls of backups onto our fses. With
160gb of ram this means we get really large contiguous ranges of dirty data, but
the way our ENOSPC stuff works is that as long as it's contiguous we only hold
metadata reservation for one extent. The problem is we limit our extents to
128mb, so we'll end up with at least 800 extents so our enospc accounting is
quite a bit lower than what we need. To keep track of this make sure we
increase outstanding_extents for every multiple of the max extent size so we can
be sure to have enough reserved metadata space. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com> Signed-off-by: Chris Mason <clm@fb.com>
Josef Bacik [Wed, 11 Feb 2015 20:08:58 +0000 (15:08 -0500)]
Btrfs: don't set and clear delalloc for O_DIRECT writes
We do this to get the space accounting, but this is just needless churn on the
io_tree, so just drop setting/clearing delalloc and just drop the reserved data
space when we have a successfull allocation. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com> Reviewed-by: Liu Bo <bo.li.liu@oracle.com> Signed-off-by: Chris Mason <clm@fb.com>
Josef Bacik [Wed, 11 Feb 2015 20:08:57 +0000 (15:08 -0500)]
Btrfs: only adjust outstanding_extents when we do a short write
We have this weird dance where we always inc outstanding_extents when we do a
O_DIRECT write, even if we allocate the entire range. To get around this we
also drop the metadata space if we successfully write. This is an unnecessary
dance, we only need to jack up outstanding_extents if we don't satisfy the
entire range request in get_blocks_direct, otherwise we are good using our
original reservation. So drop the unconditional inc and the drop of the
metadata space that we have for the unconditional inc. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com> Reviewed-by: Liu Bo <bo.li.liu@oracle.com> Signed-off-by: Chris Mason <clm@fb.com>
Zhao Lei [Thu, 12 Feb 2015 06:18:17 +0000 (14:18 +0800)]
btrfs: Fix out-of-space bug
Btrfs will report NO_SPACE when we create and remove files for several times,
and we can't write to filesystem until mount it again.
Steps to reproduce:
1: Create a single-dev btrfs fs with default option
2: Write a file into it to take up most fs space
3: Delete above file
4: Wait about 100s to let chunk removed
5: goto 2
Script is like following:
#!/bin/bash
# Recommend 1.2G space, too large disk will make test slow
DEV="/dev/sda16"
MNT="/mnt/tmp"
for ((i = 0; i < 10; i++)); do
df "$MNT" | tail -1
sleep 10
done
done
Reason:
It is triggered by commit: 47ab2a6c689913db23ccae38349714edf8365e0a
which is used to remove empty block groups automatically, but the
reason is not in that patch. Code before works well because btrfs
don't need to create and delete chunks so many times with high
complexity.
Above bug is caused by many reason, any of them can trigger it.
Reason1:
When we remove some continuous chunks but leave other chunks after,
these disk space should be used by chunk-recreating, but in current
code, only first create will successed.
Fixed by Forrest Liu <forrestl@synology.com> in:
Btrfs: fix find_free_dev_extent() malfunction in case device tree has hole
Reason2:
contains_pending_extent() return wrong value in calculation.
Fixed by Forrest Liu <forrestl@synology.com> in:
Btrfs: fix find_free_dev_extent() malfunction in case device tree has hole
Reason3:
btrfs_check_data_free_space() try to commit transaction and retry
allocating chunk when the first allocating failed, but space_info->full
is set in first allocating, and prevent second allocating in retry.
Fixed in this patch by clear space_info->full in commit transaction.
Tested for severial times by above script.
Changelog v3->v4:
use light weight int instead of atomic_t to record have_remove_bgs in
transaction, suggested by:
Josef Bacik <jbacik@fb.com>
Changelog v2->v3:
v2 fixed the bug by adding more commit-transaction, but we
only need to reclaim space when we are really have no space for
new chunk, noticed by:
Filipe David Manana <fdmanana@gmail.com>
Actually, our code already have this type of commit-and-retry,
we only need to make it working with removed-bgs.
v3 fixed the bug with above way.
Changelog v1->v2:
v1 will introduce a new bug when delete and create chunk in same disk
space in same transaction, noticed by:
Filipe David Manana <fdmanana@gmail.com>
V2 fix this bug by commit transaction after remove block grops.
Reported-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com> Suggested-by: Filipe David Manana <fdmanana@gmail.com> Suggested-by: Josef Bacik <jbacik@fb.com> Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com> Signed-off-by: Chris Mason <clm@fb.com>
Filipe Manana [Mon, 9 Feb 2015 21:14:24 +0000 (21:14 +0000)]
Btrfs: scrub, fix sleep in atomic context
My previous patch "Btrfs: fix scrub race leading to use-after-free"
introduced the possibility to sleep in an atomic context, which happens
when the scrub_lock mutex is held at the time scrub_pending_bio_dec()
is called - this function can be called under an atomic context.
Chris ran into this in a debug kernel which gave the following trace:
Shaohua Li [Fri, 9 Jan 2015 18:40:15 +0000 (10:40 -0800)]
btrfs: delete chunk allocation attemp when setting block group ro
Below test will fail currently:
mkfs.ext4 -F /dev/sda
btrfs-convert /dev/sda
mount /dev/sda /mnt
btrfs device add -f /dev/sdb /mnt
btrfs balance start -v -dconvert=raid1 -mconvert=raid1 /mnt
The reason is there are some block groups with usage 0, but the whole
disk hasn't free space to allocate new chunk, so we even can't set such
block group readonly. This patch deletes the chunk allocation when
setting block group ro. For META, we already have reserve. But for
SYSTEM, we don't have, so the check_system_chunk is still required.
Signed-off-by: Shaohua Li <shli@fb.com> Signed-off-by: Chris Mason <clm@fb.com>
Naohiro Aota [Mon, 5 Jan 2015 16:01:03 +0000 (01:01 +0900)]
btrfs: clear bio reference after submit_one_bio()
After submit_one_bio(), `bio' can go away. However submit_extent_page()
leave `bio' referable if submit_one_bio() failed (e.g. -ENOMEM on OOM).
It will cause invalid paging request when submit_extent_page() is called
next time.
I reproduced ENOMEM case with the following script (need
CONFIG_FAIL_PAGE_ALLOC, and CONFIG_FAULT_INJECTION_DEBUG_FS).
Filipe Manana [Fri, 23 Jan 2015 18:27:00 +0000 (18:27 +0000)]
Btrfs: add missing cleanup on sysfs init failure
If we failed during initialization of sysfs, we weren't unregistering the
top level btrfs sysfs entry nor the debugfs stuff.
Not unregistering the top level sysfs entry makes future attempts to reload
the btrfs module impossible and the following is reported in dmesg:
Filipe Manana [Thu, 29 Jan 2015 19:18:25 +0000 (19:18 +0000)]
Btrfs: fix race between transaction commit and empty block group removal
Committing a transaction can race with automatic removal of empty block
groups (cleaner kthread), leading to a BUG_ON() in the transaction
commit code while running btrfs_finish_extent_commit(). The following
sequence diagram shows how it can happen:
CPU 1 CPU 2
btrfs_commit_transaction()
fs_info->running_transaction = NULL
btrfs_finish_extent_commit()
find_first_extent_bit()
-> found range for block group X
in fs_info->freed_extents[]
btrfs_delete_unused_bgs()
-> found block group X
Removed block group X's range
from fs_info->freed_extents[]
Fix this by ensuring that unpining the block group's range in
btrfs_finish_extent_commit() is done in a synchronized fashion
with removing the block group's range from freed_extents[]
in btrfs_delete_unused_bgs()
David Sterba [Fri, 31 Oct 2014 16:18:08 +0000 (17:18 +0100)]
btrfs: more superblock checks, lower bounds on devices and sectorsize/nodesize
I received a few crafted images from Jiri, all got through the recently
added superblock checks. The lower bounds checks for num_devices and
sector/node -sizes were missing and caused a crash during mount.
Tools for symbolic code execution were used to prepare the images
contents.
Reported-by: Jiri Slaby <jslaby@suse.cz> Signed-off-by: David Sterba <dsterba@suse.cz> Signed-off-by: Chris Mason <clm@fb.com>
Yang Dongsheng [Tue, 6 Jan 2015 12:54:42 +0000 (20:54 +0800)]
btrfs: qgroup: move WARN_ON() to the correct location.
In function qgroup_excl_accounting(), we need to WARN when
qg->excl is less than what we want to free, same to child
and parents. But currently, for parent qgroup, the WARN_ON()
is located after freeing qg->excl. It will WARN out even we
free it normally.
This patch move this WARN_ON() before freeing qg->excl.
Signed-off-by: Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Reviewed-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com> Signed-off-by: Chris Mason <clm@fb.com>
Zhao Lei [Tue, 20 Jan 2015 07:11:44 +0000 (15:11 +0800)]
Btrfs: Introduce BTRFS_BLOCK_GROUP_RAID56_MASK to check raid56 simply
So we can check raid56 with:
(map->type & BTRFS_BLOCK_GROUP_RAID56_MASK)
instead of long:
(map->type & (BTRFS_BLOCK_GROUP_RAID5 | BTRFS_BLOCK_GROUP_RAID6))
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com> Signed-off-by: Miao Xie <miaox@cn.fujitsu.com> Signed-off-by: Chris Mason <clm@fb.com>
scrub_setup_recheck_block() have many arguments but most of them
can be get from one of them, we can remove them to make code clean.
Some other cleanup for that function also included in this patch.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com> Signed-off-by: Miao Xie <miaox@cn.fujitsu.com> Signed-off-by: Chris Mason <clm@fb.com>
Zhao Lei [Tue, 20 Jan 2015 07:11:41 +0000 (15:11 +0800)]
Btrfs: Combine per-page recover in dev-replace and scrub
The code are similar, combine them to make code clean and easy to maintenance.
Some lost condition are also completed with benefit of this combination.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com> Signed-off-by: Miao Xie <miaox@cn.fujitsu.com> Signed-off-by: Chris Mason <clm@fb.com>
Zhao Lei [Tue, 20 Jan 2015 07:11:40 +0000 (15:11 +0800)]
Btrfs: Separate finding-right-mirror and writing-to-target's process in scrub_handle_errored_block()
In corrent code, code of finding-right-mirror and writing-to-target
are mixed in logic, if we find a right mirror but failed in writing
to target, it will treat as "hadn't found right block", and fill the
target with sblock_bad.
Actually, "failed in writing to target" does not mean "source
block is wrong", this patch separate above two condition in logic,
and do some cleanup to make code clean.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com> Signed-off-by: Miao Xie <miaox@cn.fujitsu.com> Signed-off-by: Chris Mason <clm@fb.com>
Zhao Lei [Tue, 20 Jan 2015 07:11:34 +0000 (15:11 +0800)]
Btrfs: add ref_count and free function for btrfs_bio
1: ref_count is simple than current RBIO_HOLD_BBIO_MAP_BIT flag
to keep btrfs_bio's memory in raid56 recovery implement.
2: free function for bbio will make code clean and flexible, plus
forced data type checking in compile.
Changelog v1->v2:
Rename following by David Sterba's suggestion:
put_btrfs_bio() -> btrfs_put_bio()
get_btrfs_bio() -> btrfs_get_bio()
bbio->ref_count -> bbio->refs
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com> Signed-off-by: Miao Xie <miaox@cn.fujitsu.com> Signed-off-by: Chris Mason <clm@fb.com>
Zhao Lei [Tue, 20 Jan 2015 07:11:31 +0000 (15:11 +0800)]
Btrfs: fix a out-of-bound access of raid_map
We add the number of stripes on target devices into bbio->num_stripes
if we are under device replacement, and we just sort the raid_map of
those stripes that not on the target devices, so if when we need
real raid_map, we need skip the stripes on the target devices.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com> Signed-off-by: Miao Xie <miaox@cn.fujitsu.com> Signed-off-by: Chris Mason <clm@fb.com>
Filipe Manana [Wed, 14 Jan 2015 01:52:25 +0000 (01:52 +0000)]
Btrfs: fix fsync log replay for inodes with a mix of regular refs and extrefs
If we have an inode with a large number of hard links, some of which may
be extrefs, turn a regular ref into an extref, fsync the inode and then
replay the fsync log (after a crash/reboot), we can endup with an fsync
log that makes the replay code always fail with -EOVERFLOW when processing
the inode's references.
This is easy to reproduce with the test case I made for xfstests. Its steps
are the following:
# Create a test file with 3001 hard links. This number is large enough to
# make btrfs start using extrefs at some point even if the fs has the maximum
# possible leaf/node size (64Kb).
echo "hello world" > $SCRATCH_MNT/foo
for i in `seq 1 3000`; do
ln $SCRATCH_MNT/foo $SCRATCH_MNT/foo_link_`printf "%04d" $i`
done
# Make sure all metadata and data are durably persisted.
sync
# Now remove one link, add a new one with a new name, add another new one with
# the same name as the one we just removed and fsync the inode.
rm -f $SCRATCH_MNT/foo_link_0001
ln $SCRATCH_MNT/foo $SCRATCH_MNT/foo_link_3001
ln $SCRATCH_MNT/foo $SCRATCH_MNT/foo_link_0001
rm -f $SCRATCH_MNT/foo_link_0002
ln $SCRATCH_MNT/foo $SCRATCH_MNT/foo_link_3002
ln $SCRATCH_MNT/foo $SCRATCH_MNT/foo_link_3003
$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo
# Simulate a crash/power loss. This makes sure the next mount
# will see an fsync log and will replay that log.
# Check that the number of hard links is correct, we are able to remove all
# the hard links and read the file's data. This is just to verify we don't
# get stale file handle errors (due to dangling directory index entries that
# point to inodes that no longer exist).
echo "Link count: $(stat --format=%h $SCRATCH_MNT/foo)"
[ -f $SCRATCH_MNT/foo ] || echo "Link foo is missing"
for ((i = 1; i <= 3003; i++)); do
name=foo_link_`printf "%04d" $i`
if [ $i -eq 2 ]; then
[ -f $SCRATCH_MNT/$name ] && echo "Link $name found"
else
[ -f $SCRATCH_MNT/$name ] || echo "Link $name is missing"
fi
done
rm -f $SCRATCH_MNT/foo_link_*
cat $SCRATCH_MNT/foo
rm -f $SCRATCH_MNT/foo
status=0
exit
The fix is simply to correct the overflow condition when overwriting a
reference item because it was wrong, trying to increase the item in the
fs/subvol tree by an impossible amount. Also ensure that we don't insert
one normal ref and one ext ref for the same dentry - this happened because
processing a dir index entry from the parent in the log happened when
the normal ref item was full, which made the logic insert an extref and
later when the normal ref had enough room, it would be inserted again
when processing the ref item from the child inode in the log.
This issue has been present since the introduction of the extrefs feature
(2012).
A test case for xfstests follows soon. This test only passes if the previous
patch titled "Btrfs: fix fsync when extend references are added to an inode"
is applied too.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Chris Mason <clm@fb.com>
Filipe Manana [Tue, 13 Jan 2015 16:40:04 +0000 (16:40 +0000)]
Btrfs: fix fsync when extend references are added to an inode
If we added an extended reference to an inode and fsync'ed it, the log
replay code would make our inode have an incorrect link count, which
was lower then the expected/correct count.
This resulted in stale directory index entries after deleting some of
the hard links, and any access to the dangling directory entries resulted
in -ESTALE errors because the entries pointed to inode items that don't
exist anymore.
This is easy to reproduce with the test case I made for xfstests, and
the bulk of that test is:
# Create a test file with 3001 hard links. This number is large enough to
# make btrfs start using extrefs at some point even if the fs has the maximum
# possible leaf/node size (64Kb).
echo "hello world" > $SCRATCH_MNT/foo
for i in `seq 1 3000`; do
ln $SCRATCH_MNT/foo $SCRATCH_MNT/foo_link_`printf "%04d" $i`
done
# Make sure all metadata and data are durably persisted.
sync
# Add one more link to the inode that ends up being a btrfs extref and fsync
# the inode.
ln $SCRATCH_MNT/foo $SCRATCH_MNT/foo_link_3001
$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo
# Simulate a crash/power loss. This makes sure the next mount
# will see an fsync log and will replay that log.
# Now after the fsync log replay btrfs left our inode with a wrong link count N,
# which was smaller than the correct link count M (N < M).
# So after removing N hard links, the remaining M - N directory entries were
# still visible to user space but it was impossible to do anything with them
# because they pointed to an inode that didn't exist anymore. This resulted in
# stale file handle errors (-ESTALE) when accessing those dentries for example.
#
# So remove all hard links except the first one and then attempt to read the
# file, to verify we don't get an -ESTALE error when accessing the inodel
#
# The btrfs fsck tool also detected the incorrect inode link count and it
# reported an error message like the following:
#
# root 5 inode 257 errors 2001, no inode item, link count wrong
# unresolved ref dir 256 index 2978 namelen 13 name foo_link_2976 filetype 1 errors 4, no inode ref
#
# The fstests framework automatically calls fsck after a test is run, so we
# don't need to call fsck explicitly here.
So make sure an fsync always flushes the delayed inode item, so that the
fsync log contains it (needed in order to trigger the link count fixup
code) and fix the extref counting function, which always return -ENOENT
to its caller (and made it assume there were always 0 extrefs).
This issue has been present since the introduction of the extrefs feature
(2012).
A test case for xfstests follows soon.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Chris Mason <clm@fb.com>
Filipe Manana [Sat, 10 Jan 2015 10:56:48 +0000 (10:56 +0000)]
Btrfs: fix directory inconsistency after fsync log replay
If we have an inode (file) with a link count greater than 1, remove
one of its hard links, fsync the inode, power fail/crash and then
replay the fsync log on the next mount, we end up getting the parent
directory's metadata inconsistent - its i_size still reflects the
deleted hard link and has dangling index entries (with no matching
inode reference entries). This prevents the directory from ever being
deletable, as its i_size can never decrease to BTRFS_EMPTY_DIR_SIZE
even if all of its children inodes are deleted, and the dangling index
entries can never be removed (as they point to an inode that does not
exist anymore).
This is easy to reproduce with the following excerpt from the test case
for xfstests that I just made:
_scratch_mkfs >> $seqres.full 2>&1
_init_flakey
_mount_flakey
# Create a test file with 2 hard links in the same directory.
mkdir -p $SCRATCH_MNT/a/b
echo "hello world" > $SCRATCH_MNT/a/b/foo
ln $SCRATCH_MNT/a/b/foo $SCRATCH_MNT/a/b/bar
# Make sure all metadata and data are durably persisted.
sync
# Now remove one of the hard links and fsync the inode.
rm -f $SCRATCH_MNT/a/b/bar
$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/a/b/foo
# Simulate a crash/power loss. This makes sure the next mount
# will see an fsync log and will replay that log.
# Remove the last hard link of the file and attempt to remove its parent
# directory - this failed in btrfs because the fsync log and replay code
# didn't decrement the parent directory's i_size and left dangling directory
# index entries - this made the btrfs rmdir implementation always fail with
# the error -ENOTEMPTY.
#
# The dangling directory index entries were visible to user space, but it was
# impossible to do anything on them (unlink, open, read, write, stat, etc)
# because the inode they pointed to did not exist anymore.
#
# The parent directory's metadata inconsistency (stale index entries) was
# also detected by btrfs' fsck tool, which is run automatically by the fstests
# framework when the test finishes. The error message reported by fsck was:
#
# root 5 inode 259 errors 2001, no inode item, link count wrong
# unresolved ref dir 258 index 3 namelen 3 name bar filetype 1 errors 4, no inode ref
#
rm -f $SCRATCH_MNT/a/b/*
rmdir $SCRATCH_MNT/a/b
rmdir $SCRATCH_MNT/a
To fix this just make sure that after an unlink, if the inode is fsync'ed,
he parent inode is fully logged in the fsync log.
A test case for xfstests follows soon.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Chris Mason <clm@fb.com>
Filipe Manana [Tue, 6 Jan 2015 20:18:45 +0000 (20:18 +0000)]
Btrfs: lookup for block group only if needed when freeing a tree block
Very often our extent buffer's header generation doesn't match the current
transaction's id or it is also referenced by other trees (snapshots), so
we don't need the corresponding block group cache object. Therefore only
search for it if we are going to use it, so we avoid an unnecessary search
in the block groups rbtree (and acquiring and releasing its spinlock).
Freeing a tree block is performed when COWing or deleting a node/leaf,
which implies we are holding the node/leaf's parent node lock, therefore
reducing the amount of time spent when freeing a tree block helps reducing
the amount of time we are holding the parent node's lock.
For example, for a run of xfstests/generic/083, the block group cache
object was needed only 682 times for a total of 226691 calls to free
a tree block.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Chris Mason <clm@fb.com>
David Sterba [Wed, 14 Jan 2015 18:52:13 +0000 (19:52 +0100)]
btrfs: switch extent_state state to unsigned
Currently there's a 4B hole in the structure between refs and state and there
are only 16 bits used so we can make it unsigned. This will get a better
packing and may save some stack space for local variables.
The size of extent_state gets reduced by 8B and there are usually a lot
of slab objects.
Filipe Manana [Tue, 20 Jan 2015 12:40:53 +0000 (12:40 +0000)]
Btrfs: fix setup_leaf_for_split() to avoid leaf corruption
We were incorrectly detecting when the target key didn't exist anymore
after releasing the path and re-searching the tree. This could make
us split or duplicate (btrfs_split_item() and btrfs_duplicate_item()
are its only callers at the moment) an item when we should not.
For the case of duplicating an item, we currently only duplicate
checksum items (csum tree) and file extent items (fs/subvol trees).
For the checksum items we end up overriding the item completely,
but for file extent items we update only some of their fields in
the copy (done in __btrfs_drop_extents), which means we can end up
having a logical corruption for some values.
Also for the case where we duplicate a file extent item it will make
us produce a leaf with a wrong key order, as btrfs_duplicate_item()
advances us to the next slot and then its caller sets a smaller key
on the new item at that slot (like in __btrfs_drop_extents() e.g.).
Alternatively if the tree search in setup_leaf_for_split() leaves
with path->slots[0] == btrfs_header_nritems(path->nodes[0]), we end
up accessing beyond the leaf's end (when we check if the item's size
has changed) and make our caller insert an item at the invalid slot
btrfs_header_nritems(path->nodes[0]) + 1, causing an invalid memory
access if the leaf is full or nearly full.
This issue has been present since the introduction of this function
in 2009:
Josef Bacik [Mon, 17 Nov 2014 20:45:48 +0000 (15:45 -0500)]
Btrfs: track dirty block groups on their own list
Currently any time we try to update the block groups on disk we will walk _all_
block groups and check for the ->dirty flag to see if it is set. This function
can get called several times during a commit. So if you have several terabytes
of data you will be a very sad panda as we will loop through _all_ of the block
groups several times, which makes the commit take a while which slows down the
rest of the file system operations.
This patch introduces a dirty list for the block groups that we get added to
when we dirty the block group for the first time. Then we simply update any
block groups that have been dirtied since the last time we called
btrfs_write_dirty_block_groups. This allows us to clean up how we write the
free space cache out so it is much cleaner. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com> Signed-off-by: Chris Mason <clm@fb.com>
Josef Bacik [Tue, 16 Dec 2014 16:54:43 +0000 (08:54 -0800)]
Btrfs: change how we track dirty roots
I've been overloading root->dirty_list to keep track of dirty roots and which
roots need to have their commit roots switched at transaction commit time. This
could cause us to lose an update to the root which could corrupt the file
system. To fix this use a state bit to know if the root is dirty, and if it
isn't set we go ahead and move the root to the dirty list. This way if we
re-dirty the root after adding it to the switch_commit list we make sure to
update it. This also makes it so that the extent root is always the last root
on the dirty list to try and keep the amount of churn down at this point in the
commit. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com> Signed-off-by: Chris Mason <clm@fb.com>
Linus Torvalds [Sun, 18 Jan 2015 06:00:40 +0000 (18:00 +1200)]
Merge tag 'armsoc-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc
Pull ARM SoC fixes from Olof Johansson:
"We've been sitting on our fixes branch for a while, so this batch is
unfortunately on the large side.
A lot of these are tweaks and fixes to device trees, fixing various
bugs around clocks, reg ranges, etc. There's also a few defconfig
updates (which are on the late side, no more of those).
All in all the diffstat is bigger than ideal at this time, but nothing
in here seems particularly risky"
* tag 'armsoc-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc: (31 commits)
reset: sunxi: fix spinlock initialization
ARM: dts: disable CCI on exynos5420 based arndale-octa
drivers: bus: check cci device tree node status
ARM: rockchip: disable jtag/sdmmc autoswitching on rk3288
ARM: nomadik: fix up leftover device tree pins
ARM: at91: board-dt-sama5: add phy_fixup to override NAND_Tree
ARM: at91/dt: sam9263: Add missing clocks to lcdc node
ARM: at91: sama5d3: dt: correct the sound route
ARM: at91/dt: sama5d4: fix the timer reg length
ARM: exynos_defconfig: Enable LM90 driver
ARM: exynos_defconfig: Enable options for display panel support
arm: dts: Use pmu_system_controller phandle for dp phy
ARM: shmobile: sh73a0 legacy: Set .control_parent for all irqpin instances
ARM: dts: berlin: correct BG2Q's SM GPIO location.
ARM: dts: berlin: add broken-cd and set bus width for eMMC in Marvell DMP DT
ARM: dts: berlin: fix io clk and add missing core clk for BG2Q sdhci2 host
ARM: dts: Revert disabling of smc91x for n900
ARM: dts: imx51-babbage: Fix ULPI PHY reset modelling
ARM: dts: dra7-evm: fix qspi device tree partition size
ARM: omap2plus_defconfig: use CONFIG_CPUFREQ_DT
...
Linus Torvalds [Sun, 18 Jan 2015 03:29:11 +0000 (15:29 +1200)]
Merge tag 'clk-fixes-for-linus' of git://git.linaro.org/people/mike.turquette/linux
Pull clock driver fixes from Mike Turquette:
"Small number of fixes for clock drivers and a single null pointer
dereference fix in the framework core code.
The driver fixes vary from fixing section mismatch warnings to
preventing machines from hanging (and preventing developers from
crying)"
* tag 'clk-fixes-for-linus' of git://git.linaro.org/people/mike.turquette/linux:
clk: fix possible null pointer dereference
Revert "clk: ppc-corenet: Fix Section mismatch warning"
clk: rockchip: fix deadlock possibility in cpuclk
clk: berlin: bg2q: remove non-exist "smemc" gate clock
clk: at91: keep slow clk enabled to prevent system hang
clk: rockchip: fix rk3288 cpuclk core dividers
clk: rockchip: fix rk3066 pll lock bit location
clk: rockchip: Fix clock gate for rk3188 hclk_emem_peri
clk: rockchip: add CLK_IGNORE_UNUSED flag to fix rk3066/rk3188 USB Host
Linus Torvalds [Sun, 18 Jan 2015 03:26:52 +0000 (15:26 +1200)]
Merge tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi
Pull SCSI fixes from James Bottomley:
"This is one fix for a Multiqueue sleeping in invalid context problem
and a MAINTAINER file update for Qlogic"
* tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi:
scsi: ->queue_rq can't sleep
MAINTAINERS: Update maintainer list for qla4xxx
The commit 646cafc6 (clk: Change clk_ops->determine_rate to
return a clk_hw as the best parent) opens a possibility for
null pointer dereference, fix this.
Signed-off-by: Stanimir Varbanov <svarbanov@mm-sol.com> Reviewed-by: Stephen Boyd <sboyd@codeaurora.org> Signed-off-by: Michael Turquette <mturquette@linaro.org>
That commit tried to fix the section mismatch warning by moving the
ppc_corenet_clk_driver struct to init section. This is definitely wrong
because the kernel would free the memories occupied by this struct
after boot while this driver is still registered in the driver core.
The kernel would panic when accessing this driver struct.
Cc: stable@vger.kernel.org # 3.17 Signed-off-by: Kevin Hao <haokexin@gmail.com> Acked-by: Scott Wood <scottwood@freescale.com> Signed-off-by: Michael Turquette <mturquette@linaro.org>
Linus Torvalds [Sat, 17 Jan 2015 18:26:24 +0000 (06:26 +1200)]
Merge branch 'fixes' of git://git.infradead.org/users/vkoul/slave-dma
Pull dmaengine fixes from Vinod Koul:
"Two patches, the first by Andy to fix dw dmac runtime pm and second
one by me to fix the dmaengine headers in MAINTAINERS"
* 'fixes' of git://git.infradead.org/users/vkoul/slave-dma:
dmaengine: dw: balance PM runtime calls
MAINTAINERS: dmaengine: fix the header file for dmaengine
Linus Torvalds [Sat, 17 Jan 2015 18:24:30 +0000 (06:24 +1200)]
Merge branch 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull perf fixes from Ingo Molnar:
"Mostly tooling fixes, but also two PMU driver fixes"
* 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
perf tools powerpc: Use dwfl_report_elf() instead of offline.
perf tools: Fix segfault for symbol annotation on TUI
perf test: Fix dwarf unwind using libunwind.
perf tools: Avoid build splat for syscall numbers with uclibc
perf tools: Elide strlcpy warning with uclibc
perf tools: Fix statfs.f_type data type mismatch build error with uclibc
tools: Remove bitops/hweight usage of bits in tools/perf
perf machine: Fix __machine__findnew_thread() error path
perf tools: Fix building error in x86_64 when dwarf unwind is on
perf probe: Propagate error code when write(2) failed
perf/x86/intel: Fix bug for "cycles:p" and "cycles:pp" on SLM
perf/rapl: Fix sysfs_show() initialization for RAPL PMU
Olof Johansson [Thu, 15 Jan 2015 23:45:18 +0000 (15:45 -0800)]
Merge tag 'samsung-fixes-3.19' of git://git.kernel.org/pub/scm/linux/kernel/git/kgene/linux-samsung into fixes
Merge "Samsung fixes for v3.19" from Kukjin Kim:
Samsung fixes for v3.19
- exynos_defconfig: enable LM90 driver and display panel support
- HWMON
- SENSORS_LM90
- Direct Rendering Manager (DRM)
- DRM bridge registration and lookup framework
- Parade ps8622/ps8625 eDP/LVDS bridge
- NXP ptn3460 eDP/LVDS bridge
- Exynos Fully Interactive Mobile Display controller (FIMD)
- Panel registration and lookup framework
- Simple panels
- Backlight & LCD device support
- use pmu_system_controller phandle for dp phy
: DP PHY requires pmu_system_controller to handle PMU reg. now
* tag 'samsung-fixes-3.19' of git://git.kernel.org/pub/scm/linux/kernel/git/kgene/linux-samsung:
ARM: exynos_defconfig: Enable LM90 driver
ARM: exynos_defconfig: Enable options for display panel support
arm: dts: Use pmu_system_controller phandle for dp phy
Tyler Baker [Mon, 12 Jan 2015 15:54:46 +0000 (07:54 -0800)]
reset: sunxi: fix spinlock initialization
Call spin_lock_init() before the spinlocks are used, both in early init
and probe functions preventing a lockdep splat.
I have been observing lockdep complaining [1] during boot on my a80 optimus [2]
when CONFIG_PROVE_LOCKING has been enabled. This patch resolves the splat,
and has been tested on a few other sunxi platforms without issue.
Olof Johansson [Tue, 13 Jan 2015 23:32:43 +0000 (15:32 -0800)]
Merge tag 'renesas-soc-fixes-for-v3.19' of git://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas into fixes
Merge "Renesas ARM Based SoC Fixes for v3.19" from Simon Horman:
Renesas ARM Based SoC Fixes for v3.19
This pull request is based on the last round of SoC updates for v3.19,
Fourth Round of Renesas ARM Based SoC Updates for v3.19, tagged as
renesas-soc3-for-v3.19, merged into your next/soc branch and included in
v3.19-rc1.
- ARM: shmobile: r8a7740: Instantiate GIC from C board code in legacy builds
Set .control_parent for all irqpin instances for sh73a0 SoC when booting
using legacy C.
- ARM: shmobile: r8a7740: Instantiate GIC from C board code in legacy builds
This fixes a long standing problem which has been present since
the sh73a0 SoC started using the INTC External IRQ pin driver.
The patch that introduced the problem is 341eb5465f67437a ("ARM:
shmobile: INTC External IRQ pin driver on sh73a0") which was included
in v3.10.
* tag 'renesas-soc-fixes-for-v3.19' of git://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas:
ARM: shmobile: sh73a0 legacy: Set .control_parent for all irqpin instances
ARM: shmobile: r8a7740: Instantiate GIC from C board code in legacy builds
Abhilash Kesavan [Sat, 10 Jan 2015 03:11:36 +0000 (08:41 +0530)]
ARM: dts: disable CCI on exynos5420 based arndale-octa
The arndale-octa board was giving "imprecise external aborts" during
boot-up with MCPM enabled. CCI enablement of the boot cluster was found
to be the cause of these aborts (possibly because the secure f/w was not
allowing it). Hence, disable CCI for the arndale-octa board.
Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com> Tested-by: Krzysztof Kozlowski <k.kozlowski@samsung.com> Tested-by: Kevin Hilman <khilman@linaro.org> Tested-by: Tyler Baker <tyler.baker@linaro.org> Signed-off-by: Olof Johansson <olof@lixom.net>
Abhilash Kesavan [Sat, 10 Jan 2015 03:11:35 +0000 (08:41 +0530)]
drivers: bus: check cci device tree node status
The arm-cci driver completes the probe sequence even if the cci node is
marked as disabled. Add a check in the driver to honour the cci status
in the device tree.
Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com> Acked-by: Sudeep Holla <sudeep.holla@arm.com> Acked-by: Nicolas Pitre <nico@linaro.org> Tested-by: Sudeep Holla <sudeep.holla@arm.com> Tested-by: Kevin Hilman <khilman@linaro.org> Signed-off-by: Olof Johansson <olof@lixom.net>
Heiko Stübner [Sun, 11 Jan 2015 22:09:23 +0000 (23:09 +0100)]
ARM: rockchip: disable jtag/sdmmc autoswitching on rk3288
rk3288 SoCs have a function to automatically switch between jtag/sdmmc pinmux
settings depending on the card state. This collides with a lot of assumptions.
It only works when using the internal card-detect mechanism and breaks
horribly when using either the normal card-detect via the slot-gpio function
or via any other pin. Also there is of course no link between the mmc and jtag
on the software-side, so the jtag clocks may very well be disabled when the
card is ejected and the soc switches back to the jtag pinmux.
Leaving the switching function enabled did result in mmc timeouts and rcu
stalls thus hanging the system on 3.19-rc1. Therefore disable it in all cases,
as we expect the devicetree to explicitly select either mmc or jtag pinmuxes
anyway.
Signed-off-by: Heiko Stuebner <heiko@sntech.de> Signed-off-by: Olof Johansson <olof@lixom.net>
Olof Johansson [Sun, 11 Jan 2015 21:07:05 +0000 (13:07 -0800)]
Merge tag 'berlin-fixes-for-3.19-1' of git://git.infradead.org/users/hesselba/linux-berlin into fixes
Merge "ARM: berlin: Fixes for v3.19 (round 1)" from Sebastian Hesselbarth:
Marvell Berlin fixes for v3.19 round 1:
- SDHCI DT fixes for BG2Q and BG2Q reference board
- BG2Q SM GPIO DT node relocation
* tag 'berlin-fixes-for-3.19-1' of git://git.infradead.org/users/hesselba/linux-berlin:
ARM: dts: berlin: correct BG2Q's SM GPIO location.
ARM: dts: berlin: add broken-cd and set bus width for eMMC in Marvell DMP DT
ARM: dts: berlin: fix io clk and add missing core clk for BG2Q sdhci2 host
Linus Walleij [Fri, 9 Jan 2015 19:11:20 +0000 (20:11 +0100)]
ARM: nomadik: fix up leftover device tree pins
We altered the device tree bindings for the Nomadik family of
pin controllers to be standard, this file was merged out-of-order
so we missed fixing this. Fix it up.
Signed-off-by: Linus Walleij <linus.walleij@linaro.org> Signed-off-by: Olof Johansson <olof@lixom.net>
Olof Johansson [Thu, 8 Jan 2015 22:43:00 +0000 (14:43 -0800)]
Merge tag 'omap-for-v3.19/fixes-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tmlind/linux-omap into fixes
Merge "omap fixes against v3.19-rc1" from Tony Lindgren:
Fixes for omaps mostly to deal with dra7 timer issues
and hypervisor mode. The other fixes are minor fixes for
various boards. The summary of the fixes is:
- Fix real-time counter rate typos for some frequencies
- Fix counter frequency drift for am572x
- Fix booting of secondary CPU in HYP mode
- Fix n900 board name for legacy user space
- Fix cpufreq in omap2plus_defconfig after Kconfig change
- Fix dra7 qspi partitions
And also, let's re-enable smc91x on some n900 boards that
we have sitting in a few test boot systems after the boot
loader dependencies got fixed.
* tag 'omap-for-v3.19/fixes-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tmlind/linux-omap:
ARM: dts: Revert disabling of smc91x for n900
ARM: dts: dra7-evm: fix qspi device tree partition size
ARM: omap2plus_defconfig: use CONFIG_CPUFREQ_DT
ARM: OMAP2+: Fix n900 board name for legacy user space
ARM: omap5/dra7xx: Enable booting secondary CPU in HYP mode
ARM: dra7xx: Fix counter frequency drift for AM572x errata i856
ARM: omap5/dra7xx: Fix frequency typos
Olof Johansson [Thu, 8 Jan 2015 22:42:11 +0000 (14:42 -0800)]
Merge tag 'imx-fixes-3.19' of git://git.kernel.org/pub/scm/linux/kernel/git/shawnguo/linux into fixes
Merge "ARM: imx: fixes for 3.19" from Shawn Guo:
The i.MX fixes for 3.19:
- One fix for incorrect i.MX25 SPI1 clock assignment in device tree,
which causes system hang when accessing SPI1.
- Correct i.MX6SX QSPI parent clock configuration to fix a kernel Oops.
- Fix ULPI PHY reset modelling on imx51-babbage board to remove the
dependency on bootloader for USB3317 ULPI PHY reset.
- Correct video divider setting on i.MX6Q rev T0 1.0 to fix the issue
that HDMI is not working at high resolution on T0 1.0.
- One incremental fix for CODA960 VPU enabling in device tree to
correct interrupt order.
- LS1021A SCFG block works in BE mode, add device tree property
big-endian to make it right.
* tag 'imx-fixes-3.19' of git://git.kernel.org/pub/scm/linux/kernel/git/shawnguo/linux:
ARM: dts: imx51-babbage: Fix ULPI PHY reset modelling
ARM: imx6sx: Set PLL2 as parent of QSPI clocks
ARM: dts: imx25: Fix the SPI1 clocks
ARM: clk-imx6q: fix video divider for rev T0 1.0
ARM: dts: imx6qdl: Fix CODA960 interrupt order
ARM: ls1021a: dtsi: add 'big-endian' property for scfg node
Olof Johansson [Thu, 8 Jan 2015 22:38:36 +0000 (14:38 -0800)]
Merge tag 'v3.19-rockchip-dtsfixes1' of git://git.kernel.org/pub/scm/linux/kernel/git/mmind/linux-rockchip into fixes
Merge "ARM: rockchip: dts fix for 3.19" from Heiko Stübner:
Increase drive-strength to sdmmc pins on rk3288-evb to fix
an issue with the fixed highspeed card detection.
* tag 'v3.19-rockchip-dtsfixes1' of git://git.kernel.org/pub/scm/linux/kernel/git/mmind/linux-rockchip:
ARM: dts: rockchip: bump sd card pin drive strength up on rk3288-evb
Louis Langholtz [Fri, 16 Jan 2015 05:04:46 +0000 (22:04 -0700)]
kernel: avoid overflow in cmp_range
Avoid overflow possibility.
[ The overflow is purely theoretical, since this is used for memory
ranges that aren't even close to using the full 64 bits, but this is
the right thing to do regardless. - Linus ]
Signed-off-by: Louis Langholtz <lou_langholtz@me.com> Cc: Yinghai Lu <yinghai@kernel.org> Cc: Peter Anvin <hpa@linux.intel.com> Cc: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Namhyung Kim [Wed, 14 Jan 2015 11:18:05 +0000 (20:18 +0900)]
perf tools: Fix segfault for symbol annotation on TUI
Currently the symbol structure is allocated with symbol_conf.priv_size
to carry sideband information like annotation, map browser on TUI and
sort-by-name tree node. So retrieving these information from symbol
needs to care about the details of such placement.
However the annotation code just assumes that the symbol is placed after
the struct annotation. But actually there's other info between them.
So accessing those struct will lead to an undefined behavior (usually a
crash) after they write their info to the same location.
To reproduce the problem, please follow the steps below:
1. run perf report (TUI of course) with -v option
2. open map browser (by pressing right arrow key for any entry)
3. search any function (by pressing '/' key and input whatever..)
4. return to the hist browser (by pressing 'q' or left arrow key)
5. open annotation window for the same entry (by pressing 'a' key)
Signed-off-by: Namhyung Kim <namhyung@kernel.org> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1421234288-22758-1-git-send-email-namhyung@kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Wang Nan [Wed, 14 Jan 2015 02:36:47 +0000 (10:36 +0800)]
perf test: Fix dwarf unwind using libunwind.
Perf tool fails to unwind user stack if the event raises in a shared
object. This patch improves tests/dwarf-unwind.c to demonstrate the
problem by utilizing commonly used glibc function "bsearch". If perf is
not statically linked, the testcase will try to unwind a mixed call
trace.
By debugging libunwind I found that there is a bug in unwind-libunwind:
it always passes 0 as segbase to libunwind, cause libunwind unable to
locate debug_frame entry fir first level ip address (I add some more
debugging output into libunwind to make things clear):
This patch passes map->start as segbase to dwarf_find_debug_frame(), so
di will be initialized correctly.
In addition, dso and executable are different when setting segbase. This
patch first check whether the elf is executable, and pass segbase only
for shared object.
Signed-off-by: Wang Nan <wangnan0@huawei.com> Acked-by: Jiri Olsa <jolsa@kernel.org> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Li Zefan <lizefan@huawei.com> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1421203007-75799-1-git-send-email-wangnan0@huawei.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Vineet Gupta [Tue, 13 Jan 2015 13:43:22 +0000 (19:13 +0530)]
perf tools: Avoid build splat for syscall numbers with uclibc
This is due to duplicated unistd inclusion (via uClibc headers + kernel headers)
Also seen on ARM uClibc based tools
------- ARC build ---------->8-------------
CC util/evlist.o
In file included from
~/arc/k.org/arch/arc/include/uapi/asm/unistd.h:25:0,
from util/../perf-sys.h:10,
from util/../perf.h:15,
from util/event.h:7,
from util/event.c:3:
~/arc/k.org/include/uapi/asm-generic/unistd.h:906:0:
warning: "__NR_fcntl64" redefined [enabled by default]
#define __NR_fcntl64 __NR3264_fcntl
^
In file included from
~/arc/gnu/INSTALL_1412-arc-2014.12-rc1/arc-snps-linux-uclibc/sysroot/usr/include/sys/syscall.h:24:0,
from util/../perf-sys.h:6,
----------------->8-------------------
------- ARM build ---------->8-------------
CC FPIC plugin_scsi.o
In file included from util/../perf-sys.h:9:0,
from util/../perf.h:15,
from util/cache.h:7,
from perf.c:12:
~/arc/k.org/arch/arm/include/uapi/asm/unistd.h:28:0:
warning: "__NR_restart_syscall" redefined [enabled by default]
In file included from
~/buildroot/host/usr/arm-buildroot-linux-uclibcgnueabi/sysroot/usr/include/sys/syscall.h:25:0,
from util/../perf-sys.h:6,
from util/../perf.h:15,
from util/cache.h:7,
from perf.c:12:
~/buildroot/host/usr/arm-buildroot-linux-uclibcgnueabi/sysroot/usr/include/bits/sysnum.h:17:0:
note: this is the location of the previous definition
----------------->8-------------------
Signed-off-by: Vineet Gupta <vgupta@synopsys.com> Cc: Alexey Brodkin <Alexey.Brodkin@synopsys.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1421156604-30603-4-git-send-email-vgupta@synopsys.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Vineet Gupta [Tue, 13 Jan 2015 13:43:21 +0000 (19:13 +0530)]
perf tools: Elide strlcpy warning with uclibc
----------------->8------------------
CC bench/sched-pipe.o
In file included from builtin-annotate.c:13:0:
util/cache.h:76:15: warning: redundant redeclaration of 'strlcpy'
[-Wredundant-decls]
extern size_t strlcpy(char *dest, const char *src, size_t size);
^
In file included from util/util.h:55:0,
from builtin.h:4,
from builtin-annotate.c:8:
~/vineetg/arc/gnu/INSTALL_1412-arc-2014.12-rc1/arc-snps-linux-uclibc/sysroot/usr/include/string.h:396:15:
note: previous declaration of 'strlcpy' was here
extern size_t strlcpy(char *__restrict dst, const char *__restrict src,
----------------->8------------------
Signed-off-by: Vineet Gupta <vgupta@synopsys.com> Cc: Alexey Brodkin <Alexey.Brodkin@synopsys.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1421156604-30603-3-git-send-email-vgupta@synopsys.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
----------->8---------------
CC fs/fs.o
fs/fs.c: In function 'fs__valid_mount':
fs/fs.c:82:24: error: comparison between signed and unsigned integer
expressions [-Werror=sign-compare]
else if (st_fs.f_type != magic)
^
cc1: all warnings being treated as errors
----------->8---------------
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com> Acked-by: Jiri Olsa <jolsa@kernel.org> Cc: Borislav Petkov <bp@suse.de> Cc: Cody P Schafer <dev@codyps.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Vineet Gupta <Vineet.Gupta1@synopsys.com> Link: http://lkml.kernel.org/r/1420888254-17504-2-git-send-email-vgupta@synopsys.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
tools: Remove bitops/hweight usage of bits in tools/perf
We need to use lib/hweight.c for that, just like we do for lib/rbtree.c,
so tools need to link hweight.o. For now do it directly, but we need to
have a tools/lib/lk.a or .so that collects these goodies...
Reported-by: Jan Beulich <JBeulich@suse.com> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-a1e91dx3apzqw5kbdt7ut21s@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
When thread__init_map_groups() fails, a new thread should be removed
from the rbtree since it's gonna be freed. Also update last match cache
only if the function succeeded.
Reported-by: David Ahern <dsahern@gmail.com> Signed-off-by: Namhyung Kim <namhyung@kernel.org> Acked-by: Jiri Olsa <jolsa@kernel.org> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1420763892-15535-1-git-send-email-namhyung@kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Namhyung Kim [Mon, 12 Jan 2015 02:20:55 +0000 (10:20 +0800)]
perf tools: Fix building error in x86_64 when dwarf unwind is on
When build with 'make ARCH=x86' and dwarf unwind is on, there is a
compiling error:
CC /home/wn/perf/arch/x86/util/unwind-libdw.o
CC /home/wn/perf/arch/x86/tests/regs_load.o
arch/x86/tests/regs_load.S: Assembler messages:
arch/x86/tests/regs_load.S:65: Error: operand type mismatch for `push'
arch/x86/tests/regs_load.S:72: Error: operand type mismatch for `pop'
make[1]: *** [/home/wn/perf/arch/x86/tests/regs_load.o] Error 1
make[1]: INTERNAL: Exiting with 25 jobserver tokens available; should be 24!
make: *** [all] Error 2
...
Which is caused by incorrectly undefine macro HAVE_ARCH_X86_64_SUPPORT.
'config/Makefile.arch' tests __x86_64__ only when 'ARCH=x86_64'.
However, when building x86_64 kernel, ARCH=x86 is valid and commonly
used. Build systems, such as yocto, uses x86_64 compiler with 'ARCH=x86'
to build x86_64 perf, which causes mismatching.
As __LP64__ is defined for x86_64 as well, we can consolidate the
__x86_64__ check to the __LP64__ check and get rid of the IS_X86_64
IMHO.
(This patch is made by Namhyung Kim when replying my v1 patch:
https://lkml.org/lkml/2015/1/7/17
I modified the code to remove dependency on RAW_ARCH:
https://lkml.org/lkml/2015/1/7/865
Namhyung Kim didn't provide his SOB in his original email. I add
mine only for my modification.)
Signed-off-by: Namhyung Kim <namhyung@kernel.org> Signed-off-by: Wang Nan <wangnan0@huawei.com> Acked-by: Jiri Olsa <jolsa@kernel.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Li Zefan <lizefan@huawei.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Link: http://lkml.kernel.org/r/1421029255-23039-1-git-send-email-wangnan0@huawei.com
[ Namhyung provided his S-o-B on a followup to this patch thread on lkml ] Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Namhyung Kim [Sat, 10 Jan 2015 10:33:48 +0000 (19:33 +0900)]
perf probe: Propagate error code when write(2) failed
When it failed to write probe commands to the probe_event file in
debugfs, it needs to propagate the error code properly. Current code
blindly uses the return value of the write(2) so it always uses
-1 (-EPERM) and it might confuse users.
Signed-off-by: Namhyung Kim <namhyung@kernel.org> Acked-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1420886028-15135-4-git-send-email-namhyung@kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Linus Torvalds [Fri, 16 Jan 2015 19:18:08 +0000 (08:18 +1300)]
Merge tag 'char-misc-3.19-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc
Pull char/misc driver fixes from Greg KH:
"Here are three small driver fixes for reported issues for 3.19-rc5.
All of these have been in linux-next for a while with no reported
problems"
* tag 'char-misc-3.19-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc:
mcb: mcb-pci: Only remap the 1st 0x200 bytes of BAR 0
mei: add ABI documentation for fw_status exported through sysfs
mei: clean reset bit before reset
Linus Torvalds [Fri, 16 Jan 2015 19:15:49 +0000 (08:15 +1300)]
Merge tag 'tty-3.19-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty
Pull tty/serial driver fixes from Greg KH:
"Here are some tty and serial driver fixes for 3.19-rc5 that resolve
some reported issues, and add a new device id to the 8250 serial port
driver.
All have been in linux-next with no reported problems"
* tag 'tty-3.19-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty:
serial: samsung: Add the support for Exynos5433 SoC
Revert "tty: Fix pty master poll() after slave closes v2"
tty: Prevent hw state corruption in exclusive mode reopen
tty: Add support for the WCH384 4S multi-IO card
serial: fix parisc boot hang
Linus Torvalds [Fri, 16 Jan 2015 19:02:44 +0000 (08:02 +1300)]
Merge tag 'usb-3.19-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb
Pull USB fixes from Greg KH:
"Here is a bunch of USB fixes for 3.19-rc5.
Most of these are gadget driver fixes, along with the xhci driver fix
that we both reported having problems with, as well as some new device
ids and other tiny fixes.
All have been in linux-next with no problems"
* tag 'usb-3.19-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb: (43 commits)
usb: dwc3: gadget: Stop TRB preparation after limit is reached
usb: dwc3: gadget: Fix TRB preparation during SG
usb: phy: mv-usb: fix usb_phy build errors
usb: serial: handle -ENODEV quietly in generic_submit_read_urb
usb: serial: silence all non-critical read errors
USB: console: fix potential use after free
USB: console: fix uninitialised ldisc semaphore
usb: gadget: udc: atmel: fix possible oops when unloading module
usb: gadget: gadgetfs: fix an oops in ep_write()
usb: phy: Fix deferred probing
OHCI: add a quirk for ULi M5237 blocking on reset
uas: Add US_FL_NO_ATA_1X for 2 more Seagate disk enclosures
uas: Do not blacklist ASM1153 disk enclosures
usb: gadget: udc: avoid dereference before NULL check in ep_queue
usb: host: ehci-tegra: request deferred probe when failing to get phy
uas: disable UAS on Apricorn SATA dongles
uas: Add US_FL_NO_REPORT_OPCODES for JMicron JMS566 with usb-id 0bc2:a013
uas: Add US_FL_NO_ATA_1X for Seagate devices with usb-id 0bc2:a013
xhci: Add broken-streams quirk for Fresco Logic FL1000G xhci controllers
USB: EHCI: adjust error return code
...
Linus Torvalds [Fri, 16 Jan 2015 19:01:21 +0000 (08:01 +1300)]
Merge tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux
Pull arm64 fixes from Will Deacon:
- Wire up compat_sys_execveat for compat (AArch32) tasks
- Revert 421520ba9829, as this breaks our side of the boot protocol
* tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux:
arm64: partially revert "ARM: 8167/1: extend the reserved memory for initrd to be page aligned"
arm64: compat: wire up compat_sys_execveat
Linus Torvalds [Fri, 16 Jan 2015 18:59:06 +0000 (07:59 +1300)]
Merge tag 'nfs-for-3.19-2' of git://git.linux-nfs.org/projects/trondmy/linux-nfs
Pull NFS client bugfixes from Trond Myklebust:
"Highlights include:
- Stable fix for a NFSv3/lockd race
- Fixes for several NFSv4.1 client id trunking bugs
- Remove an incorrect test when checking for delegated opens"
* tag 'nfs-for-3.19-2' of git://git.linux-nfs.org/projects/trondmy/linux-nfs:
NFSv4: Remove incorrect check in can_open_delegated()
NFS: Ignore transport protocol when detecting server trunking
NFSv4/v4.1: Verify the client owner id during trunking detection
NFSv4: Cache the NFSv4/v4.1 client owner_id in the struct nfs_client
NFSv4.1: Fix client id trunking on Linux
LOCKD: Fix a race when initialising nlmsvc_timeout
projects / karo-tx-linux.git / log