The blind bag test...
A common theme in the threads in the Handloading section is "Will this (fill in the blank) help my accuracy?" Whether it be neck turning, weighing cases, deburring flash holes, getting Redding comp dies, etc., many of us wonder "Is it worth it?"
And the truth will generally be difficult to pin down. Mainly because in some instances __________ will help one's accuracy, and in others--it won't.
Being human, we can fall for the "magic mitt" effect with regard to whether or not something makes a difference. A children's story tells of a little boy who was having trouble catching the baseball during his Little League games until... [img]/ubbthreads/images/graemlins/smile.gif[/img] until his father loaned him his "magic mitt." The child then believes that the mitt has magic, and he then--through his belief that it will work--performs better himself.
In another scenario, it is easy to get drawn into the belief that you must do A, B, C, etc., in order to get even a modicum of accuracy. By skipping any of these "benchrest proven" steps we spoil our chances (so we are taught) to an acceptable outcome with our field rifles. And one need only thumb through the latest flyer from any reloading supply retailer. It's easy enough to see that the industry feeds into and profits from the proliferation of such ideas.
Now. [img]/ubbthreads/images/graemlins/smile.gif[/img] Getting back to what I mentioned earlier about truths. Many presumed truths will not be absolute because they will not apply to every situation. In an extreme example, if one were to neck turn a batch of .223 cases for a Ruger Mini 14, and compare the difference in accuracy to non-neck turned cases in that blunder-buss of a rifle [img]/ubbthreads/images/graemlins/smirk.gif[/img] he would almost certainly see no difference in the accuracy level. I don't think any thinking man would dispute this.
But somewhere along the continuum that runs between Ruger Mini 14 accuracy and the bug-hole accuracy of the winning benchrest rifle, neck turning--assuming the brass cases do not have evenly made necks--begins to make a difference. Flash hole deburring will begin to make a difference (assuming the particular lot of brass shows obstructions here, and the rifle firing the shots is accurate enough to display the difference). Reducing runout to .002" or less may begin to make a difference, again, depending on the abilities of the particular rifle--and this will also be a bullet dependent measurement, so .004" of runout with one bullet might make it misbehave in an extremely accurate rifle, but .006" of runout on another bullet may go unnoticed in that same rifle. Here is where one guy says "Four thousandths of runout makes a difference." And the other guy says "No it doesn't, I tested that theory." Only thing is, they were talking about different bullets in different rifles.
But where (again, on this continuum) do these things actually begin to matter? If, for instance, it could be scientifically proven that a particular rifle could realize an improvement in 600 yard accuracy when the runout amount was reduced from .004" to .002" with a particular bullet, how much difference might this actually make? Almost certainly very little. Maybe--and realistically--so little difference that any accuracy advantage would get "lost in the noise" of other factors like wind, shooter limitations, etc.
In the end, there is only one way to know whether a particular step of match prepping will help you or not. You've got to test the idea--and you have to test the idea correctly.
You cannot simply take two rows of assembled cartridges to the range or field and fire the "improved" group at one target and the "unimproved" group at another. Your own psychology--the placebo effect--will call "advantage improved" before the first shot is fired, and your subtle, unconscious behavior will quite possibly force the outcome that you already suspect, and perhaps want to believe. We're all human, and this effect is alive and well at all times. Scientists know this, which is why they take measures to conceal--often even from themselves--which group is which. A third party will hold the information as to which group of subjects got the medicine, and which got the sugar pills. And only after the test results are in do the scientists ask the third party to reveal which group was which. These scientists realize that their own body language and other behaviors might induce certain outcomes in the study groups, so they do not want to know which group is which as the test is under way.
And this is also how we should conduct tests to decide whether or not performing "presumed improvement A" (on sale from MidwayUSA this month~!) really helps or not.
So. Let's say we want to check the effects of neck turning in a particular rifle with a particular lot of brass. (And remember, we must realize that the final results of any such test will only be applicable to the test rifle, and the test batch of brass. Different rifles and different lots of brass will almost certainly realize different results).
We assemble forty or so cartridges, identical in all respects save the one that we're testing. This means that group A will have the necks turned, and group B will not have, or vice versa.
Once you have completed the forty or so cartridges, the next step would be to examine them all to see if it is obvious by looking at them which is which. If you can tell the difference with the Mark 1 eyeball then you'll have to enlist the help of an assistant when you get to the range. He/she will load the rifle for each shot while you look away. You should also look away as you eject the shell casing, and allow your assistant to pick it up and put it back into the bag.
Oh yeah. The bags. [img]/ubbthreads/images/graemlins/smile.gif[/img] What you're going to do with the group A and group B cartridges--before you head to the range--is you're going to place each group into its own paper bag. Obviously you'll need to use identical bags (or boxes or whatever). Write "group A, turned" inside one bag, and put those shells in there. Write "group B, un-turned" inside the other bag, and put those shells into it.
Somewhere before leaving for the range, you'll deliberately mix these bags up. If you have to, have someone switch them around and give them back to you. The main thing, and the most important thing is that you do not know which bag is which. The human mind is amazing, and can even unconsciously discern little subtleties about one bag, and unconsciously realize which one it is (assuming you put the cartriges into the bags yourself, that is). So be thourough and careful with this "blind bag" test. If possible, have someone else fill and label the bags.
Set up two identical targets at the range. Two bullseyes on the same square of paper should be fine. Shoot two fouling shots from a clean barrel, and make sure those fouling shots are put together with the same powder as you're using in the test. Bullets with the same jacket material should also be used (i.e. moly, naked, etc.)
Next step. Most folks will not have ever considered this, but it is of vital importance to the validity of the test. Fire one shot from the first bag (remember, you don't know which is bag A or B at this time) at the first bullseye, then fire one shot from the second bag at the second bullseye. Keep alternating back and forth in this manner. This will spread the barrel fouling and heating effect evenly across both groups. This will also spread the "damn I'm getting tired and bleary eyed and I gotta take a whizz effect" evenly across both groups. [img]/ubbthreads/images/graemlins/laugh.gif[/img] If you've shot comparisons in the past without doing them in this alternating manner, I would respectfully question your results.
Once you've fired all forty (or so) shots onto the paper, then--and only then--do you reveal to yourself which bag was which.
And of course you'll know--more surely than the average guy doing the average test will know--whether the "improvement" actually helps or not.;)