We tracked down the first-0.5s-hdmi-audio-samples-lost problem to the
AC_VERB_SET_CHANNEL_STREAMID command. It is suspected that many HDMI
sinks need some time to adapt to the new state.
The workaround is to avoid changing stream id/format whenever possible.
Proposed by David.