This is the final step to get rid of the one of the structures. A
further cleanup will follow. And I struct serial_state deserves cease
to exist after a switch to tty_port too.
While changing the lines, it removes also pointless tty->driver_data
casts.