core: normalize error handling a bit, in setup_pam()

core: normalize error handling a bit, in setup_pam()

Assign errno-style errors to a variable called "r" when they happen, the same way we do this in most other calls. It's
bad enough that the error handling part of the function deals with two different error variables (pam_code and r) now,
but before this fix it was even three!