Optimize PPC CR emulation by using magic 64 bit values

PowerPC has a 32 bit CR register, which is used to store flags for results of
computations. Most instructions have an optional bit that tells the CPU whether
the flags should be updated. This 32 bit register actually contains 8 sets of 4
flags: Summary Overflow (SO), Equals (EQ), Greater Than (GT), Less Than (LT).
These 8 sets are usually called CR0-CR7 and accessed independently. In the most
common operations, the flags are computed from the result of the operation in
the following fashion:
  * EQ is set iff result == 0
  * LT is set iff result < 0
  * GT is set iff result > 0
  * (Dolphin does not emulate SO)

While X86 architectures have a similar concept of flags, it is very difficult
to access the FLAGS register directly to translate its value to an equivalent
PowerPC value. With the current Dolphin implementation, updating a PPC CR
register requires CPU branching, which has a few performance issues: it uses
space in the BTB, and in the worst case (!GT, !LT, EQ) requires 2 branches not
taken.

After some brainstorming on IRC about how this could be improved, calc84maniac
figured out a neat trick that makes common CR operations way more efficient to
JIT on 64 bit X86 architectures. It relies on emulating each CRn bitfield with
a 64 bit register internally, whose value is the result of the operation from
which flags are updated, sign extended to 64 bits. Then, checking if a CR bit
is set can be done in the following way:
  * EQ is set iff LOWER_32_BITS(cr_64b_val) == 0
  * GT is set iff (s64)cr_64b_val > 0
  * LT is set iff bit 62 of cr_64b_val is set

To take a few examples, if the result of an operation is:
  * -1 (0xFFFFFFFFFFFFFFFF) -> lower 32 bits not 0       => !EQ
                            -> (s64)val (-1) is not > 0  => !GT
                            -> bit 62 is set             =>  LT
            !EQ, !GT, LT

  *  0 (0x0000000000000000) -> lower 32 bits are 0       =>  EQ
                            -> (s64)val (0) is not > 0   => !GT
                            -> bit 62 is not set         => !LT
            EQ, !GT, !LT

  *  1 (0x0000000000000001) -> lower 32 bits not 0       => !EQ
                            -> (s64)val (1) is > 0       =>  GT
                            -> bit 62 is not set         => !LT
            !EQ, GT, !LT

Sometimes we need to convert PPC CR values to these 64 bit values. The
following convention is used in this case:
  * Bit 0 (LSB) is set iff !EQ
  * Bit 62 is set iff LT
  * Bit 63 is set iff !GT
  * Bit 32 always set to disambiguize between EQ and GT

Some more examples:
  * !EQ, GT, LT -> 0x4000000100000001 (!B63, B62, B32, B0)
                -> lower 32 bits not 0          => !EQ
                -> (s64)val is > 0              =>  GT
                -> bit 62 is set                =>  LT
  * EQ, GT, !LT -> 0x0000000100000000
                -> lower 32 bits are 0          =>  EQ
                -> (s64)val is > 0 (note: B32)  =>  GT
                -> bit 62 is not set            => !LT

Source/Core/Core/PowerPC/PowerPC.h

@@ -38,7 +38,20 @@ struct GC_ALIGNED64(PowerPCState)
 	u32 pc;     // program counter
 	u32 npc;
 
-	u8 cr_fast[8];     // Possibly reorder to 0, 2, 4, 8, 1, 3, 5, 7 so that we can make Compact and Expand super fast?
+	// Optimized CR implementation. Instead of storing CR in its PowerPC format
+	// (4 bit value, SO/EQ/LT/GT), we store instead a 64 bit value for each of
+	// the 8 CR register parts. This 64 bit value follows this format:
+	//   - SO iff. bit 61 is set
+	//   - EQ iff. lower 32 bits == 0
+	//   - GT iff. (s64)cr_val > 0
+	//   - LT iff. bit 62 is set
+	//
+	// This has the interesting property that sign-extending the result of an
+	// operation from 32 to 64 bits results in a 64 bit value that works as a
+	// CR value. Checking each part of CR is also fast, as it is equivalent to
+	// testing one bit or the low 32 bit part of a register. And CR can still
+	// be manipulated bit by bit fairly easily.
+	u64 cr_val[8];
 
 	u32 msr;    // machine specific register
 	u32 fpscr;  // floating point flags/status bits
@@ -149,27 +162,54 @@ void UpdatePerformanceMonitor(u32 cycles, u32 num_load_stores, u32 num_fp_inst);
 
 }  // namespace
 
-// Fast CR system - store them in single bytes instead of nibbles to not have to
-// mask/shift them out.
+// Convert between PPC and internal representation of CR.
+inline u64 PPCCRToInternal(u8 value)
+{
+	u64 cr_val = 0x100000000;
+	// SO
+	cr_val |= (u64)!!(value & 1) << 61;
+	// EQ
+	cr_val |= (u64)!(value & 2);
+	// GT
+	cr_val |= (u64)!(value & 4) << 63;
+	// LT
+	cr_val |= (u64)!!(value & 8) << 62;
 
-// These are intended to stay fast, probably become faster, and are not likely to slow down much if at all.
+	return cr_val;
+}
+
+// Warning: these CR operations are fairly slow since they need to convert from
+// PowerPC format (4 bit) to our internal 64 bit format. See the definition of
+// ppcState.cr_val for more explanations.
 inline void SetCRField(int cr_field, int value) {
-	PowerPC::ppcState.cr_fast[cr_field] = value;
+	PowerPC::ppcState.cr_val[cr_field] = PPCCRToInternal(value);
 }
 
 inline u32 GetCRField(int cr_field) {
-	return PowerPC::ppcState.cr_fast[cr_field];
+	u64 cr_val = PowerPC::ppcState.cr_val[cr_field];
+	u32 ppc_cr = 0;
+
+	// SO
+	ppc_cr |= !!(cr_val & (1ull << 61));
+	// EQ
+	ppc_cr |= ((cr_val & 0xFFFFFFFF) == 0) << 1;
+	// GT
+	ppc_cr |= ((s64)cr_val > 0) << 2;
+	// LT
+	ppc_cr |= !!(cr_val & (1ull << 62)) << 3;
+
+	return ppc_cr;
 }
 
 inline u32 GetCRBit(int bit) {
-	return (PowerPC::ppcState.cr_fast[bit >> 2] >> (3 - (bit & 3))) & 1;
+	return (GetCRField(bit >> 2) >> (3 - (bit & 3))) & 1;
 }
 
 inline void SetCRBit(int bit, int value) {
 	if (value & 1)
-		PowerPC::ppcState.cr_fast[bit >> 2] |= 0x8 >> (bit & 3);
+		SetCRField(bit >> 2, GetCRField(bit >> 2) | (0x8 >> (bit & 3)));
 	else
-		PowerPC::ppcState.cr_fast[bit >> 2] &= ~(0x8 >> (bit & 3));
+		SetCRField(bit >> 2, GetCRField(bit >> 2) & ~(0x8 >> (bit & 3)));
 }
 
 // SetCR and GetCR are fairly slow. Should be avoided if possible.