Need a collection of photos of Lincoln Cents for a Neural Network

Hey tlasch, very interesting project, please keep us updated with your progress. Once you get an algorithm trained I think it would be interesting to see how it performs in GTG threads versus the human guessers.

As for the naysayers, everything is impossible until it isn't. There is some pretty powerful open source machine learning software that is freely available. If the technology isn't there now its a matter of time before it will be. I think Doug mentioned the biggest challenges: Who determines the criteria for a given grade?/How do you address inconsistency in grading over time, between TPG's etc.

I think inconsistencies in grading practices would be a bigger challenge to algorithm performance than the ability of the technology to make accurate classifications given enough training images.

 

Log in or Sign up to hide this ad.

Not really. They require the ability to acquire and process a large amount of data that will lead to an appropriate classification of the luster and eye appeal of a coin. The data acquired and processed would be the same as that acquired and processed for every coin that is used to train a neural network (hundreds or thousands per type, being careful not to bias toward a certain grade) to do the grading. The fact that the amount of data per coin is huge and difficult to acquire is what makes it impractical.

 

These really aren't big challenges. You pick out images that you think have reasonable grades associated with them. That becomes the grading standard that is applied to the network. If you can train a DNN to grade to the standard that you want it to learn, you can train it with a different, similar standard. The challenge is the data.

An example of a grading program I've seen in action is an application that scores the quality of transthoracic echocardiograms on a scale of 1 to 5. This was used to guide sonographers to make better quality images for cardiologists. Many TTE images had to be used to train the network, and all had to have scores of 1-5 attached to them. In the end, it was quite precise and quite accurate. Compared with grading coins, the amount of data was negligible. One 2D image was assigned one score that said how good the image was. You might get by with this for assigning a details grade to circulated Barber quarters, or perhaps Lincoln cents, but we're also dealing with more than a 1-5 scale, so the precision of the data will have to be high, as well.

 

I don't have a lot of experience with buying certified coins or submitting coins for grading, but based on the content of numerous threads, it seems that most people accept that standards have changed over time and that the grading services are not perfectly consistent with how grades are assigned hence people resubmitting coins hoping for higher grade. Perhaps, this isn't as much of an issue as I suspected?

 

Shifting grading practices over time may be a reality, and being aware of it is necessary when buying coins, but it's a non-issue when it comes to training a neural network to grade coins now according to some defined standard, good or bad. Attempting to train a neural network to learn how to change its standards over time to react to the market is an entirely different problem.

 

Only if it was an open software not intended for commercial use ( which was the way it originally read by me). We have had so many members who came to get a lot of help on making their website or to test various coin thingees and they try to take it commercial. But most seemed to fail at that , so maybe we are the wrong people to ask Jim

 

No, but there are already loads of machines that can do loads of things that the human brain can't do.

I agree that it's unlikely we'll see robust (good) training from uncontrolled photos, but I won't rule it out completely. Neural nets can do accomplish surprising things.

 

If I were doing this, I would consider the experiment a success if I could use only circulated coins, same number for each grade I wanted to learn, train on one photo of each from a single source, and show that the accuracy is significantly better than a dartboard approach. I would then show the effect of using photos from a different source as my test data. Then train with data from multiple sources and show the effect on convergence and accuracy. On a system trained to grade Barber quarters, I'd additionally try to test with Barber halves, Barber dimes, and Liberty nickels, to show how portable the trained network is across similar types. If the network isn't totally confounded by those, then try Mercury dimes and Buffalo nickels. After all, when we're grading circulated coinage of types we're not familiar with, we need to estimate the amount of detail remaining.

Uncirculated coinage is a different monster, and I would present just why it is.

 

One of the reasons to use a computer is to eliminate subjectivity from the process, or at least make it predictable. Why would you want to reproduce it? A computer learning how to grade will convert the "subjectivity" (and everything else) that went into the ground truth data into a big pile of convolution kernels and biases that will give a deterministic result.

 

In a system where the current subjectivity is learned and made repeatable, you would still have to deal with two things that would perpetuate the need for grading.

Defects. A neural network can only learn to be as good as the ground truth allows. This ground truth is arrived at by a committee of graders, each of which is considered to be doing a good job if they're in agreement with the consensus more than 75% of the time. If the resulting neural network learns how to do a good job, or even a very good job (80%+), it will still be assigning grades that the market will reject. Classifying grades is not the same as the "toy" classification task of identifying hand-written numerals (Google search for MNIST for this example). There is a very fuzzy area of right and wrong with grades that is nearly non-existent with identifying numbers. A postal worker sorting mail with handwritten addresses is right over 99% of the time. If they only had a 75% success rate, then they'd be out of a job (if not for the union, but I digress). The way "wrong" answers are dealt with in a neural network is by some combination of restructuring the network and retraining it with more, and better, data. While this may boost the performance to a slightly higher number, it will potentially produce different results for some coins, putting us right back to the appearance of shifting sands of grading standards.

Technological shifts in data acquisition. When I replaced my Nikon D80 with a Nikon D610, I was able to take slightly better pictures of my coins due to changes in detector technology. Upgrading lights and lenses can have a similar effect, and this is only for static 2D images. For the imagery required for grading, which is at least 3-dimensional, there are many opportunities to devise a way to improve how an attribute is captured, whether it's simple detail, surface qualities, color, luster depth, or detecting problems. This could lead to a difference in what the network sees, thus giving a different result, which would lead to retraining and/or restructuring, which would again lead to resubmissions.

 

This is a fascinating thread and touches on a conversation I had with a collector at our local coin club. The OP's effort to build a neural net to learn how to grade from photos would probably suffer from most if not all the weaknesses previously alluded - you know - the GIGO problem. One of the goals of machine learning is to minimize to near elimination the GI part the equation. Everyone is familiar with the "driverless car" development. Do you think the driving environment is less complicated or with fewer unexpected situations than that of coin grading? The time and expense of training these systems to drive cars is massive.

This, I think, is where the flaw lies in machine grading of coins from photographs. Many of the critical components of grading are hidden by the actual photographic techniques employed, as described by previous posters. And it would not really be practical to take enough photographs to display all the different aspects of a coin under a wide enough spectrum of lighting regimes. But, I think I have at least a potential answer to this problem:

After the fundamentals of a coin grading neural net are built, at least to the point where it can perform the least subjective portions of grading, it would be time to introduce the machine system to an automated physical handling sub-system controlled by the AI and programmed to subject the coin to a specified variety of base and optional angle/rotation/lighting conditions. Properly designed and using conservative values of performance, I can see the entire cycle from loading the coin/slab to removing it being about 5 minutes. This system could probably fulfill a matrix of 25 X 25 or 625 different combinations of angle/rotation/lighting at 1/2 second per shot. It should be possible to train the system to evaluate coins in this manner at least fairly close to the results obtained from good human graders. The task, as with all machine learning environments, is the fairly massive number and type of examples that must be fed into it in order to teach the idiot savant what it's looking at.

I agree that aside from the expense and time of such an effort, it is not in the financial best interest of the TPGs nor of many dealers/collectors to remove or reduce the uncertainty and variability of grading. And as a collector, some of the fun of the hobby would be diminished. As with cars, once the machines are driving, where's the fun?

 

Go have "fun" on a closed track, where the only other humans at risk are those who choose to drive for entertainment.

As for the "fun" of subjective grading -- meh, I guess. I have a hard time enjoying "sports" where the odds are heavily stacked in favor of the house, and that's very much what we have with the current system. Like Doug, though, I don't see it changing.

 

did you mean broken numismatic? here's one:


If it ain't broke, don't fix it & if it's already broken, don't fix it either

 

Sorry for the bump but I find this idea of neural networks very interesting. Is there packaged software available that contains the algorithms with an easy to use GUI that allows most anyone the ability to create such a network? You know something on the level that is similar to learning to use Word, Excel, ect? Or do you need a deep programing background to create one of these networks?

 

You need to have some level of programming background to be able to develop a neural network. There are several online courses in this, and while they have toy demos that don't require much programming, they'll all list some sort of programming background as their prerequisites for accomplishing anything.