Yes, I do it all the time with my Mdl 10 Savage in .308. But watch out for banging those very fine threads on something. You WILL regret it. So I have a thread protector on mine. But COULD just as well use NOTHING on the end.
I had a thread protector made for one gun, Just to be darned sure the threads don't get dinged. A couple others, that are generally only shot off the bench, I just unscrew the brakes and go for it.
If the protector weighs different than the brake, your POI may change when the protector is on. POI may also change just from removing the brake. Then again, it may not on your rifle. Just double check it to make sure.
I shoot my STW with the brake off, and no thread protector. I don't have a protector, so I think I'm going to put either tape around it or a piece of black rubber tuning to protect the threads on an upcoming moose hunt.
The impact changes about 4-5 inches on some loads between having it on and off.