Blackpilled by Hornaday #50, #52 - Statistics with Ontarget TDS and Taran

[btuley]

Friends,

I was black pilled by the "Your Groups are Two Small Hornaday" podcast series. I have a long set of notes on their points and how they relate to at least my reloading journey (I'm a beginner). For this post, it suffices to say that I'm looking at inferential statistical methods to tell two different reloading treatments apart from each other with the minimum required shots. Essentially proposing the statistical question in the language of stats Ho = Reloading method A results in the same dispersion as Reloading Method B. Ha = Reloading Method A results in a different dispersion from Reloading Method B. P value is, say, 0.05.

Unfortunately, after their black pill podcasts the team didn't leave us in a better place than "just shoot dozens of shots for each load, and only try a few different loads" (a few powder and bullet combinations). Statistics is a powerful tool for measuring and quantifying the difference between two different treatments, such as two different seating depths etc. maximizing what we can confidently say from the data available.

I'm interested in a separate conversation about what figure of merit is the best to use to measure differences in reloading treatments, whether it's radius, x and/or y variance, Circular Error Probable, Weapons Employment Zone, or what if anyone wants to start that conversation (or point me to one here). But for the moment I was looking at the variance of the radius assuming dispersion is randomly distributed in direction as well as that the x and y dispersions are equally distributed (no positive compensation causing different vertical dispersion etc.).

So, that brings me to my point. I found that I can get access to the x and y points in OnTarget TDS (https://ontargetshooting.com/ontarget-tds/) and in Taran (https://taran.ptosis.ch/taran.html) to run the statistics.

I also found that both programs do some calculations of the standard deviation of the dispersion. However, their numbers are way off from what I calculate with the x/y coordinates that they export.

In Ontarget TDS the value they propose for the radius standard deviation is 100% bigger than the number I calculate. When I calculate the horizontal and vertical standard deviations just to check we are off by only ~5%. If I use population standard deviation instead of sample standard deviation we actually match there, so I propose they are using the wrong metric for statistical inferences on their x and y standard deviation. 100% off on radius is crazy and not explained by using a different view of how to calculate standard deviation. Anyone care to weigh in on if I'm doing something wrong or if they've had similar experiences? I show all my work in the attached image. I emailed them ~ a week ago and haven't received a response as of publishing this post.

In Zaran, they don't specify what they are calculating the standard deviation on. When I compare their number to the x, or the y, or the radius that I calculate we are way off. However, if I do a sample standard deviation with all the x and y values then the number is 5% off. So, again rounding error shouldn't be off by 5%, but I assume that is the sample set definition Zaran is using. Any ideas on the potential source of 5% error? Any thoughts on if/why calculating standard deviation on the population of all the x and y points would be more valuable for statistical inference that the on the radius?

 

[Tiny Tim]

Following

 

[RegionRat]

Any chance there is a radius and diameter being swapped by accident somewhere?

Hang in there, double check your version against their (on Target TDS) individual radius values, and be patient while they answer your request. Lets see what they come back with.

In industrial settings, we tend to use CEP at a level set by specs and contracts. However, there is nothing wrong with using Mean Radius to test differences experimentally by using basic statistical tests.

The standard deviation of a sample is amplified by arbitrary convention, not by physics. It isn't real and it can over or under estimate when you extrapolate with it.

As for using SD population versus SD sample, as long as you are not switching back and forth or trying to extrapolate too far ahead, it won't matter which one you use as long as you do it the same for both treatments when comparing them and by using at least 15 or more shots for each.

Be careful when assuming that shot dispersion follows Normal Gaussian distribution. If you are zooming way down, it often doesn't.

That said, you can do just fine learning to shoot/load with normal distribution math. A student still needs to have very good workmanship and shooting skills that don't come from a screen or keyboard, so put an equal effort into the workshop and range time. Good Luck and have fun!

 

[btuley]

Interesting. Yes, St. Dev calculated on the diameter instead of radius was right on (they label it SD Radius, so error there). I checked with another target group and that is off by 17% though, so still issues.

I'm familiar with CEP as a description of system capability. However, I haven't been able to find any methods for comparing two different CEPs to gain confidence that two are statistically different. I understand if you are delivering a weapon system to a customer and you've defined a test and the results are a certain CEP or below you are good. But that's not directly related to the question of load development and two different treatments. I can imagine someone has defined a test for a statistical difference between two CEP's but I haven't been able to find that formulation. Are you aware of one?

 

[RegionRat]

Don't let the CEP treatment distract you.

You still compare the stats using one of two dimensions. You either select the distance of each shot from the aim point, or the distance of each shot to group center.

It is analogous to the difference between Cpk and Cpkm. One is just the process capability, and the other also includes the ability to center on a target tolerance.

One is good enough for talking purposes when there are opportunities for sighters and re-boresighting. The other is point of impact oriented with no sighters.