Fix out-of-memory handling in GC_toggleref_add

Fix out-of-memory handling in GC_toggleref_add

* finalize.c (ensure_toggleref_capacity): Change return type from void
to GC_bool; return FALSE on allocation failure (or desired capacity
value overflow).
* finalize.c (GC_toggleref_add): Change return type from void to int.
* include/gc.h (GC_toggleref_add): Likewise.
* finalize.c (GC_toggleref_add): Return GC_NO_MEMORY if
ensure_toggleref_capacity failed, GC_SUCCESS otherwise (including the
case of no callback).