Since commit 0c375501 (serial: imx: enable the clocks for console), the
imx_startup() function calls clk_prepare_enable conditionally, so we
need to call clk_disable_unprepare inside imx_shutdown() under the same
condition to avoid unbalanced clock calls.