Let us look at that decay in more detail: Before decay, the Uranium-235 nucleus had 92 protons, and (235-92) 143 neutrons.
After decay, the daughter nucleus, Thorium-231, has 90 protons and (231-90) 141 neutrons.
The two protons and two neutrons missing are, of course, now part of the alpha particle.