there will be times that you will see a monster limp away to escape you when you beat the hell out of it so that's the time when you capture one by setting up a trap and wait for the dumb monster to be caught and then hurl those tranq bombs! please refer to the wiki for info on signs on how to spot monsters that are low on health (if you dont have the eagle eye skill/ tranquilizing guru).
better rewards await you if you capture a monster. if you have a mega or ultra lucky cat felyne skill then id suggest that you slay the monster instead since you'll have better luck in getting nice rewards and more items from the carves!