White,
Something to take into account is that the number designating the caliber does not correspond directly to the recoil experienced by the shooter. Many people believe the .40S&W to recoil harsher than a .45ACP. The cartridge and the platform (round and the gun) make for most of the recoil felt. The rest is up to the shooter. If you have a firm grip and a steady hand you should be able to fire a .44mag without issue. Small people do it often.
Realize this, For most common service calibers, 9mm, .357mag, .40S&W, 357sig, .45ACP there is very little difference in stopping potential with quality defese ammuntition and a good hit. A .500 S&W mag to the leg, while it may ruin his dancing career may not take someone out of the attack. A .32ACP to the skull might make for a fight stopper.
Buy what you feel comfortable with. Shoot as many different guns as you can get your hands on. Take things into account such as availability of ammo, price of ammo and price/availability of parts should you need to get something fixed or buy more (magazines, springs, etc).
It is very subjective, not just based on the round and gun, but the shooter. To sum it up, the 9mm is a quick fairly light snap. The .40 is a quick hardish snap and the .45 is a dull strongish push. At least that's my take on it. I have not fired a 357sig, but I have fired a lightly loaded .357 mag and it was not uncomfortable to shoot. .38 specials in it were downright light. It takes going to hot .357, 44mag, 500S&W and 480 ruger to really be called "harsh," at least by me