Thank you very much. As a grader and collector who's opinion I respect very highly, this is extremely nice to hear.
One other thing that comes to mind, is that this would make for an interesting article in The Numismatist, once the last coin is posted for the first experiment, especially since it represents original work on a facet of popular subject not yet rigorously explored.
Regarding your 2nd paragraph in your response, and this is just a suggestion. By doing this - I will attempt to pick coins which I think are accurately graded, - my guess is you'll unintentionally skew the results of your efforts. I believe picking the coins at random would be a much better option and provide much more honest and accurate results.
LOL ! C'mon now Paul, you and I have agreed on a great many things over the years, and on a pretty regular basis. But yeah, I get what ya mean
The purpose of the test is to see if a group can arrive at the same conclusion as a TPG. If this test is only going to be done for 25 coins, then the coins need to be selected to be representative of a typical grading event, with outliers or grading anomalies excluded. If a random selection of a coin shows one that looks way off, it should be rejected, as the results of the study will show that for that coin, the hive mind and TPG didn't match, but not give insight as to why. If we were using 2500 coins, we might find that the number of coins for which this happened told us something about TPG tendencies (or Heritage's photography department), but that won't be possible with 25, nor is it the intended purpose of the study. Perhaps rather than stating that he'd be picking coins which he though were accurately graded, he should have said that coins that seemed inaccurately graded or had uncharacteristically poor photos would be excluded.
I could easily agree with automatically excluding coins with poor pics. But by picking coins, for any reason, is going to skew the results because it will bias the test. One way or another, it will bias the test.
It begs the question, is the TPGs “screwing the pooch” 1 out of 25 times a typical grading event? A sample this small is bound to have a higher deviation, that doesn’t mean you should introduce bias to reduce it.
A typical grading event is a coin that is graded more or less correctly. If there is a 50% chance that a group of 25 randomly selected coins will have one that is way off, then there is a 2.7% chance of an atypical grading event for any given coin. For this study, we're going by photos, so we have an incomplete picture. If Jason grabs a random coin and sees that it doesn't make any sense, he won't know if that's on Heritage photo people, the TPG graders, or a "mechanical error." What's to gain by introducing it into the study? It would seem better (necessary, actually) to disclose the selection methodology, including reasons for excluding a coin and how many randomly selected coins were rejected.
So you are saying that any photo that doesn’t mesh with the assigned grade may not be accurately depicting the appearance of the coin and should therefore be excluded. While that seems more reasonable than excluding coins that are definitive outliers, I still don’t think it is necessary. I think he should just pick 25 random examples and live with the results.
Yes, that's what I'm saying. And while it isn't really all that necessary, I'd rather have 25 good data points than 24 and one suspicious one. At the end of the day, however, I'd be surprised if 25 randomly chosen coins had more than one such coin in it.
Another thing we may observe is that The Hive converges on TPG grades with increasing accuracy as this experiment goes on.
I am not a statistics junkie, but this is an excellent thread! And, I thank you for your efforts on this subject. Very nice work! Great reading! One thing you mention is that we should not only post our grade guess, but an explanation of why we came to our conclusion. Then, you also mention that we get better samples and more accurate results if the GTG thread is a poll. The fear I have with the polls is that more people might be inclined to just click the button of their choice, and move on, and we will have even less people give detailed explanations for their grades. If that is the case, we who read these GTG threads will have less analysis to read and learn from. What are your thoughts about that? Maybe that's another study that someone needs to undertake! LOL!