I have been using the 6.5mm WSM for several years in my personal XP-100 and I have never been wanting more but will also say I have only been hunting deer. I have taken several out to 400 yards and a big beyond and everytimes the round and handgun has performed very well.
Ballistically, I would say the 6.5mm and 7mm will be your best options.
Recoil wise, if your not using a brake, all three will be a handful in a hunting weight handgun, the two larger calibers may be more then you want to deal with, in fact they may all be more then you want to deal with without a brake.
The best compromise between the three would be the
7mm WSM. High BC bullets that are still light enough to keep recoil to a somewhat controlable level but with plenty of bullet for big game hunting at longer ranges.
I am not a 30 cal fan for handguns and long range hunting. First off, recoil can get severe with 180 gr class bullets and the bullet flight drops quickly.
Personally, If I were building an XP without a brake, I would not go over the 284 case in capacity. Even this round will not be all that easy to shoot unless handgun weight is up there.
All three will cleanly harvest elk at ranges well past 300 yards with a quality bullet and shot placement so that is not an issue. What is is shootability in my opinion.
I have a 15" Lilja on my 6.5mm WSM XP and personally prefer that length for a hunting handgun. For a dedicated long range XP, 18" is fine but it is pretty long for a packing handgun.
Kirby Allen(50)