tree.c (build): Don't look at TREE_SIDE_EFFECTS or TREE_RAISES for non-trees.