pinctrl: qcom: restrict drivers per ARM/ARM64