How Good is CoinTalk at Grading?

I'm not sure how you can account for or if it's possible but you must remember TPG's grade by having the coin in hand while CT members grade from a photo.

 

Log in or Sign up to hide this ad.

There is actually a Rule or Law about this but I forget what it is called. There was a man that took a steer to a fair hundreds of years ago and asked as many attendees as he could how much the steer weighed. The average of the guesses was almost dead on.

As for on CT, due to the sample sizes, I would weight my averages. Those posters that I know are better at grading I would weight higher.

 

Food for thought. Some of us post the grade they see vs the grade that the grade that they think the TPG gave it.
I am guilty on both accounts.
As GDJMSP alluded to depending on who the member is the results could get slightly flawed. Gradeflation should be accounted for. Otherwise those members that learned how to grade from the technical aspect won't be heard.

I think it might help to use the poll to give a grade that the TPG gave it and then if a person responds in a post then they give their own opinion and why.
This might help anyone learning what to be looking for when buying appropriately graded coins.

Remember we are the collectors and should be the majority of the market.
The TPG's should be held accountable.

 

While that used to be true, I was inspired to do this study based partly on NGC's new Ebay expert review service. They look at an Ebay listing and offer opinions on authenticity and likely grade range based on the photos. If they can do it, why can't we?

I understand where you're coming from on this, but that would actually be contrary to the idea of the "wisdom of the crowds". Everyone's opinion is weighted equally, and often the consensus comes closer than an expert's guess (as alluded to in your story of the steer). Three expert guesses might be really close but 50 pounds off. The consensus was nearly spot on. If you start to rank guesses based on expertise, you distort that consensus.

 

Great work! I agree that the display could be clearer and condensed to be more readable. Also agree with @physics-fan3.14 that more people reply to a poll where they can anonymously GTG.

 

I guess I'm a little unclear how the polls work.... On that thread, it does not break down who voted for what, and I don't see any option to reveal that. My understanding is "your vote will be publicly visible" means that your vote will be shown tallied in the results. If I'm wrong, what does that toggle actually do?

 

Ok, someone helped me out and explained that option. I will make sure it is anonymous in the future.

 

The people who guess the grades are collectors. While the coins are not their own,
collectors will always grade their own coins higher than the TPGs.
I am a more strict grader and am usually lower than the crowd.

 

My thought is that this is easily the best post I’ve seen in years. I don’t know how long this took you but I thank you for your time, effort, and expertise. This is an amazing thread.

I know that some people like posting GTGs to show the fallibility of the TPGs, but I think most people including myself, enjoy using these threads to challenge people with our NEWPS and see if they think the coin is over or undergraded. It almost like having a Cointalk CAC opinion.

 

Agreed, more often than not the crowd average agrees if I feel a coin I bought is overgraded, or confirms if it’s strong for the grade/under. They are educational in all sorts of ways; I will add polls to mine in the future as well.

Great thread

 

Too....

 

It would have been nice had NGC done an analysis of crowd-sourced grades from their grading contest at FUN.

 

Cool analysis, but I wonder if there is some selection bias. Are users more likely to post a GTG when they think the coin is undergraded and are seeking affirmation of that judgment?

 

That would actually be a fascinating report, but I'm sure they'll never do it.

 

I think that is definitely a factor. However, just as many people post a GTG whenever they buy a new coin (such as myself). As we've mentioned, certain members definitely have a point to make, but just as many post their coins to get genuine opinions of their coins.

I assure you, the coins that will be posted in my experiment over the next few months will not be selected for being undergraded or overgraded. I will attempt to pick coins which I think are accurately graded, to get the best overall assessment of CT's skills.

 

have. Here’s some anecdotal evidence for you:
https://www.cointalk.com/threads/gtg-barber-quarter.274644/

Edit: understood. Good catch.

Thanks for the clarification @physics-fan3.14 It’s past my bedtime.

 

I think many people are also more willing to just click a button than take the time to type a response and post. That's the key difference here - ease of effort.

What Doug is referring to is that this particular poll is not entirely anonymous, as there is a way to see who voted for which grade. My future threads will de-select this option such that voting in the poll is truly anonymous.

 

I'm used to it.