serial: Remove unnecessary amba_set_drvdata()