First, lay a solid foundation. Get a quality gun belt to support the weight of the gun. Be sure it is thick and sturdy enough to support weight.
Second, a quality holster. Cheap holsters made of nylon or thin leather will let the gun "lean out" way from the attachment point (owb). A quality holster will remain open for one-hand reholstering, have a snug fit, and cover the triggerguard for safety.
IWB - watch the ones with the rough exterior finishes. For those who wear boxer shorts or boxer briefs, the rough surface can "grab" your shorts and move them up or down if you're moving a lot. This can require a mens room visit to adjust certain things.
Metal Clips - look for the metal clips with more than a mere indentation to catch the belt. I've seen people draw the gun and holster at the same time.
J-Clips - work pretty well, but avoid plastic ones if you can. They have to be thicker and are thus harder to get onto a thick belt. They can snap without warning too.
OWB: Take your gun with you to be sure it fits in the mode you want to carry. This is especially true if it can be carried hammer up or down with a safety/thumb strap.
Look for even, straight quality stitching. Avoid holsters showing any kind of plastic/monofilament stitching material, including nylon (these will cut thru the leather and wear out the holster faster).
Fit: Put the holster on the belt where you want to wear it. Relax. Don't hold your stomach in, just relax. A good fit will have the butt tight to your body. If it's sticking out, re-adjust it and try again. Lightly pull up and push down on the gun...it should not move up and down on the belt itself. Push the gun forward and backwards. Does the holster slide fore & aft easily? It should move with some effort or not at all, but should not slide with a light touch.
Mounting: Straight drop or Canted?
Some holsters come with only two belt openings. These will either be vertical carry (muzzle straight down) or or canted with the butt forward. Canted holsters can match the grip to your natural hand angle on the draw. They also rotate the sharp, lower/rear corner of the butt up & forward for less printing. If you will carry forward of your center-line (between 12-3 o'clock or 12-9 for southpaws) you might prefer a straight drop or crossdraw holster. Holsters with 3 slots allow either straight drop or canted carry.
The closer the beltloop area is to the holster itself the tighter the gun fits to your body. Look for a mount in the middle or slightly higher. Low mounts will allow "lean out" unless a metal shank is used to bolster the strength.
Retention systems:
None, Thumb-breaks, tension screws, spring clamps, etc. have all been used. Decide if you want these devices or which one(s) you DON'T want. I like the security of a thumb-break personally. But I do have tension-screw types too.
Other Stuff:
Buy a holster that fits your barrel length if possible. Barrels that peek out the bottom with vertical (Patridge) sights can snag on the draw. Excess length begs for dirt & crap to hide at the bottom. (Yaqui slide types are a different game, of course).
Open Muzzle - at least there should be a small drain hole opening or exit for moisture. This is especially true if your state has heavy rainstorms that form quickly. Closed end holsters can capture moisture and corrode the muzzle end.
Paddle holsters work fine - IF you need to remove and replace your gun frequently during the day. Be prepared for it being less concealable however, due to the paddle.
Shoulder Holsters should be high quality rigs. Avoid those with elastic straps or narrow nylon straps. The harness should lay flat on your weak side shoulder with the gun in place. With the unit adjusted for your size, all straps should lay flat (no flipping over) and any buckles should lay below the rib cage so they don't print thru your jacket. Properly adjusted the gun should not flop around as you walk. Be sure you can easily reach not only the gun, but any release strap. For some women this may not be simple (and for those men with large bellies). For vertical carry, I like a belt attachment at the muzzle end, though this might "flash" the gun in some cases.
(Galco's Jackass/Miami Classic is very comfy, even with a steel framed 1911 Commander and a pair of loaded magazines).
Linings
For blued guns, Sarfailand started using orthopedic elk suede lining decades ago. It does work well to keep leather from abrading your blue finish. If you're carrying a blued gun, it's worth the extra money. The caveat is that if you wear it in dry, dusty climates or in high humidity zones, it's important to clean it at least monthly. Dust & Dirt will act like abrasives. Extra moisture can start rust spots.
For stainless guns, I don't think I'd bother with a lining unless the holster comes very rough on the inside.
Someone else's turn now.