kvm tools: Process virito blk requests in separate thread

kvm tools: Process virito blk requests in separate thread

All blk requests are processed in notify_vq() which is in the context of
ioeventfd thread: ioeventfd__thread(). The processing in notify_vq() may
take a long time to complete and all devices share the single ioeventfd
thead, so this might block other device's notify_vq() being called and
starve other devices.

This patch makes virtio blk's notify_vq() just notify the blk thread
instead of doing the real hard read/write work. Tests show that the
overhead of the notification operations is small.

The reasons for using dedicated thead instead of using thead pool
follow:

1) In thread pool model, each job handling operation:
thread_pool__do_job() takes about 6 or 7 mutex_{lock,unlock} ops. Most
of the mutex are global (job_mutex) which are contented by the threads
in the pool. It's fine for the non performance critical virtio devices,
such as console, rng, etc. But it's not optimal for net and blk devices.

2) Using dedicated threads to handle blk requests opens the door for
user to set different IO priority for the blk threads.

3) It also reduces the contentions between net and blk devices if they
do not share the thead pool.

Signed-off-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>