Previously he function was misbehaving because of a missing check for
whether an 8-bit operand was a register operand; it would therefore
emit unnecessary REX prefixes, incorrectly assert on 32-bit targets, and
could potentially emit wrong code in rare cases (like a memory to register
operation involving AH.)
Also, some cleanup while I was in the area to make the function easier to
read.