Even though the number of He3 is small than D, it is likely that the probability of fusion with Deuterium is hugely enhanced by 2 factors:
1. It is 1/3 slower than deuterium at thermal equilibrium because it is heavier.
2. It loses energy 4x faster than deuterium by Bremsstrahlung radiation.
Being slower means that it will collide much more frequently with D than Deuteriums collide between themselves. Any random 2 deuteriums will most of the time have a much smaller relative radial speed difference than between a D and He3. Losing energy by bremsstrahlung very fast means that it will lose speed faster and this just will increase its probability to collided with a D.
So, He3 has much much more collisions than any Deuterium. Plus, its cross section at 150-200KeV is almost 10x higher than D+D.