Well the problem is that Bussard reactor is spiting out fusion products in all directions, or at least in the six directions permitted by the holes in the toroid magnets of the polygon electrostatic confinement device, so to use a direct converter you would need at least six decelerators sticking out of it like a giant jackstone, or a very large magnetic funnel to aim the particles into a smaller number of decelerators.
