The same serial hardware is present on LS2080A which is arm64, and
LS1021A which is arm32, so don't limit the workaround to PPC.
Unlike PPC which uses arch/powerpc/kernel/legacy_serial.c, the ARM
targets use drivers/tty/serial/of_serial.c, so add the handle_irq
override check there as well.