[aprssig] APRS adoption

[Steve Dimse]

On Feb 17, 2015, at 11:26 AM, Robert Bruninga <bruninga at usna.edu> wrote:

> Regarding my assertion that only 2% of people have APRS was based on dividing the 40 k callsigns on the APRSis (maybe not a recent number) by the 2 million hams worldwide.
> 
> But Frank provided an great analysis of Ham radio in his DELTA Division of the ARRL and it is a GREAT read.  It shows a 28% adoption rate of APRS.
> 
Damn statistics. I used to teach medical students, interns and residents how to evaluate published studies. This one is a great example of doing everything wrong! Lots of impressive numbers, charts and maps, and no statistical validity.

The 28% is of responses, and surveys were only sent to ARRL members. There were 7,167 members and 1,234 valid surveys, so only 17% of ARRL members participated. The report, surprisingly, does not give the total licenses in the region, but I suspect a small minority belong to ARRL. Worse, the survey was only sent to those with registered email addresses. 

If this were a scientific research study it would never get published. There is such a huge selection bias in the survey responses that no realistic conclusions can be drawn other than with the caveat of "ARRL members who registered an email address and cared to respond". In other words, you can say plenty about 1,234 hams, and almost nothing about the rest of the amateur population.

I could make a good case that those hams who either are not ARRL members, didn't give the ARRL their email address (or don't use email), or didn't bother to respond are completely different from the responders. And I'd bet the unmeasured group is far less likely to have involvement in APRS than the response sample. So in the end, 2% may not be that far off!

If you want to get real numbers, the statistically correct way is to develop a random sample of all hams that is large enough to be statistically significant but small enough to be economical, and aggressively seek responses to the survey. You would email those that you had email addresses for, snail mail the others, and then use telephone and in-person methods to get responses from those who didn't answer on their own. The smaller you get the non-response number the smaller the margin of error. It may be counter-intuitive, but you actually get far better results with a small sample, focusing efforts on getting returns from all of your sample, than from surveying everyone.

As this study was so flawed, in my opinion your 40k/2m is a far more useful number, telling the number of hams active on APRS within a certain time frame, with a little allowance for some that are not in areas of APRS IS connection, and assuming you removed tactical and club calls, etc.

Remember, bad data is worse than no data at all!

Steve K4HG