Message:

<p>[QUOTE=&quot;Dansco_Dude, post: 25324279, member: 147679&quot;]Hey everyone. As you may remember from my previous <a href="https://www.cointalk.com/threads/i-created-an-ai-to-objectively-grade-coins-give-it-a-try.408631/" class="internalLink ProxyLink" data-proxy-href="https://www.cointalk.com/threads/i-created-an-ai-to-objectively-grade-coins-give-it-a-try.408631/">posts</a>, in late 2023 I developed <b><a href="https://justinhinh.webflow.io/projects/numi" target="_blank" class="externalLink ProxyLink" data-proxy-href="https://justinhinh.webflow.io/projects/numi" rel="nofollow">Numi</a></b>, an AI-powered chatbot that leverages the advanced capabilities of OpenAI&#039;s GPT-4 vision model to assist coin collectors in identifying and grading their coins. It&#039;s been fascinating seeing the exponential growth of Artificial Intelligence, so I created Numi to test AI&#039;s abilities to tackle one of the biggest barriers to new collectors. Throughout Numi&#039;s development, I became more and more convinced that AI is going to fundamentally change the future of the hobby.</p><p><br /></p><p><b>Testing Numi With OpenAI&#039;s Latest GPT-4 Update</b></p><p><br /></p><p>In December 2023, I paused development on Numi as I had maxed out on the AI&#039;s capabilities and the results were not accurate enough to justify further development. Following OpenAI&#039;s recent <a href="https://www.business-standard.com/technology/tech-news/openai-launches-enhanced-gpt-4-turbo-for-chatgpt-plus-users-and-developers-124041100491_1.html" target="_blank" class="externalLink ProxyLink" data-proxy-href="https://www.business-standard.com/technology/tech-news/openai-launches-enhanced-gpt-4-turbo-for-chatgpt-plus-users-and-developers-124041100491_1.html" rel="nofollow">April 2024 update</a> to their GPT-4 model, which powers Numi&#039;s AI capabilities, I conducted another series of tests on Numi&#039;s grading accuracy.</p><p><br /></p><p>I then ran statistical analyses to assess the impact on Numi&#039;s performance and compared its grading accuracy between the December 2023 and April 2024 test results.</p><p><br /></p><p><img src="https://preview.redd.it/revisiting-numi-testing-the-latest-gpt-4-update-v0-aqey3kc1vvvc1.png?width=1041&amp;format=png&amp;auto=webp&amp;s=76e5771ef6ed7c1e773433d2d9ba505a1cf85f92" class="bbCodeImage wysiwygImage" alt="" unselectable="on" /></p><p><br /></p><p><b>Determining the Optimal # of Photos for Accurate Grading</b></p><p><br /></p><p>A key aspect of my analysis focused on identifying the optimal number of coin photos users should upload to achieve the most accurate grading results. In December 2023, my tests indicated that uploading 10 photos yielded the best accuracy across all coin grades. This aligned with my hypothesis that more data = better. However, after the GPT-4 update in April 2024, that number had changed, with just 4 photos now providing the most precise grading outcomes.</p><p><br /></p><p><img src="https://preview.redd.it/revisiting-numi-testing-the-latest-gpt-4-update-v0-104m6y83vvvc1.png?width=1315&amp;format=png&amp;auto=webp&amp;s=b973d7068125525e5a21d655cf0cb74b862163e7" class="bbCodeImage wysiwygImage" alt="" unselectable="on" /></p><p><br /></p><p><b>Just How Much Did Numi Improve?</b></p><p><br /></p><p>To measure Numi&#039;s accuracy and any improvements, I calculated the Mean Absolute Deviation (MAD) – a metric that represents the average deviation between Numi&#039;s predicted grades and the actual, expert-assigned grades. In December 2023, Numi&#039;s MAD was <b>5.39</b>, indicating that, on average, its predictions deviated by approximately 5 grade points from the actual coin&#039;s grade. By April 2024, following the GPT-4 update, Numi&#039;s MAD score decreased to <b>3.64</b>, representing a substantial <b>32.47%</b> increase in overall accuracy.</p><p><br /></p><p>I suspected that Numi would be more accurate given the updates, but I was not expecting this much of a change. While the GPT-4 vision model still struggles immensely with medium-graded coins [Around XF-40], Numi performed exceptionally well for very low and very high graded coins. With the biggest improvements seen for very low-graded coins.</p><p><br /></p><p><b>The Future of AI in Numismatics</b></p><p><br /></p><p>After seeing these results, I am even more convinced that Artificial Intelligence will revolutionize the field of coin collecting. As models like GPT-4 continue to improve, AI tools will become increasingly valuable for collectors seeking to expand their knowledge and make informed decisions about their collections. While Numi itself will most likely not end up being the go-to tool for collectors in the future, it serves as powerful evidence of where the hobby is heading.[/QUOTE]</p><p><br /></p>

Your name or email address:

Do you already have an account?

• No, create an account now.
• Yes, my password is:
• Forgot your password?

Stay logged in