GArray: Avoid unncessary atomic accesses to refcounts