lib/bitmap.c: speed up bitmap_find_free_region

In bitmap_find_free_region(), if we skip the all-ones words and find bits
in a not-all-ones word, we can improve performance of it.

For example, If bitmap_find_free_region() is called with order=0, First,
It scans bitmap array by the increment of long type, then find 1 free bit
within 1 long type value.  In 32 bits system and 1024 bits size, in the
worst case, We need 1024 for-loops to find 1 free bit.  But, If this is
applied, it takes 64 for-loops.  Instead, It will be needed additional
if-comparison of every word and It can take time slightly as 'Test case
3'.  But, In many cases, It will speed up significantly.

Test cases bellows show the time taken to execute bitmap_find_free_region()
before and after patch.

Test condition : ARMv7 1GHZ 32 bits system, 1024 bits size, gcc 4.6.2

Test case 1: order is 0. all bits are one except that last one bit is zero.
 before patch: 29727 ns
 after patch: 2349 ns

Test case 2: order is 1. all bits are one except that last 2 contiguous bits
are zero.
 before patch: 15475 ns
 after patch: 2225 ns

Test case 3: order is 1. all words are not-all-ones and don't have 2 contiguous
bits except that last 2 contiguous are zero.
 before patch: 15475 ns
 after patch: 16131 ns

Signed-off-by: Chanho Min <chanho.min@lge.com>
Cc: Nadia Yvette Chambers <nyc@holomorphy.com>
Cc: Jiri Kosina <jkosina@suse.cz>
Cc: Joe Perches <joe@perches.com>
Cc: anish singh <anish198519851985@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

diff --git a/lib/bitmap.c b/lib/bitmap.c
index 06f7e4fe8d2de4046a3139106058b9c831c9b789..95e8efc18f6834161cfc95c2554395f28e6785fd 100644 (file)
--- a/lib/bitmap.c
+++ b/lib/bitmap.c
@@ -1114,14 +1114,21 @@ done:
  */
 int bitmap_find_free_region(unsigned long *bitmap, int bits, int order)
 {
-       int pos, end;           /* scans bitmap by regions of size order */
-
-       for (pos = 0 ; (end = pos + (1 << order)) <= bits; pos = end) {
-               if (!__reg_op(bitmap, pos, order, REG_OP_ISFREE))
-                       continue;
-               __reg_op(bitmap, pos, order, REG_OP_ALLOC);
-               return pos;
-       }
+       int pos, end, nbit, i;  /* scans bitmap by regions of size order */
+       int nlongs = BITS_TO_LONGS(bits);
+
+       for (i = 0; i < nlongs; i++)
+               if (bitmap[i] != ~0UL) {
+                       pos = i * BITS_PER_LONG;
+                       nbit = min(bits, pos + BITS_PER_LONG);
+                       for (; (end = pos + (1 << order)) <= nbit; pos = end) {
+                               if (!__reg_op(bitmap, pos, order,
+                                       REG_OP_ISFREE))
+                                       continue;
+                               __reg_op(bitmap, pos, order, REG_OP_ALLOC);
+                               return pos;
+                       }
+               }
        return -ENOMEM;
 }
 EXPORT_SYMBOL(bitmap_find_free_region);