What do you want the alternative cartridge for? It's really hard to beat 5.56/.223 in an AR15 platform IMO. The only alternative cartridge I shoot is .300 BO, and all my AR firearms are shot suppressed.
I did have a .277 Wolverine build going at one time to hunt deer with. I wound up giving away the barrel to a buddy, and just put together another 5.56. The reason is the right .277 bullets are/were difficult to find, and I've been been buying 5.56 cheaper than I could buy the components to manufacture .277 Wolverine.
If you're going to hunt with an AR15 then one of the 6mm and up cartridges might be slightly better than the 5.56/223. However, loaded with a bonded or mono metal bullet the 5.56/223 does a decent job on deer sized game and smaller. The 6mm ARC is interesting if you want to try your hand at steel at ranges beyond 600 yards. But, if you're just going to the range and shooting it's hard to beat the economy of 5.55/223.
The issue with alternative AR15 cartridges is the magazine. It limits you on what you can do with the platform. The 6mm ARC is probably the best optimized alternative cartridge for the AR15 as far as range and terminal performance on deer and smaller.
I've hunted deer a few times with the AR15 in 5.56 and .300 BLK, but I much prefer a bolt action or single shot rifle over the AR for hunting deer. I do like my 5.56 for calling coyotes, but my leg with all its plates and screws doesn't like the cold anymore. I also use the AR15 a bit on prairie dogs, but still prefer a bolt action from the bench.