Digital assistants for Arma et Nummi, Bahrfeldt et al

I picked up a new phone last week, and found it had an intriguing new app that wasn't present on my old model phone. When you have a page of text in a different language, you point your phone camera at the text, press a "T" for text button, and it provides you with an immediate OCR rendition on your phone screen, as well as an instant translation. The feature is so seamless that it allowed me to start consuming a densely written book in German with a reading speed almost as fast as if I were reading it in English. You turn a page, press the relevant "T" button which takes all of one second to give you a clear translation on your phone screen, and read in comfort and with clarity on your phone screen until the end of the page, then repeat until the book is finished. In my case the digital assistant was "Bixby" but I assume the same or similar is available from "Cortona", "Alexa", "Siri" etc., allowing you to read directly from a foreign language book without having capture photos permanently or upload anything to a PC.

So, no need to be scared of those foreign language numismatic books any more.

 

Log in or Sign up to hide this ad.

I too have Bixby, but I didn't know it had that capability. For real time translation, I've used the Google Translate app which sounds very similar and probably uses similar technology behind the scenes (Samsung relying on Google APIs). I'm amazed how seamless it looks though, matching fonts, font size, background color.

I've only played with it as a novelty with my kids. But this sounds like a great application!!

 

I bought a new cell phone a week ago since mine wasn't holding a charge & decided on a Google Pixel 3a & I noticed in their Camera app a feature called "Lens" & man it's pretty cool. Even does the text translation.

https://lens.google.com/

 

Is the translation via Google translate or some other program?

 

From Andrew's description, it might happen as in this advertisement of the feature in Bixby:


And here's a advertisement demonstrating Google Translate AR functionality:

 

I think Bixby uses "Google translate" but it does so in such a seamless fashion, as Andrew described, that it's far more useful and elegant. The Bixby method eliminates the need to copy and paste (or, worse, type) blocks of text into the Google translate screen.

 

Wow! One step closer to the babel fish!

 

It does both this (you see the German/whatever changing to English on the page) as well as giving you a clear text on your phone screen in German and English. In practice it's the clear text on my phone that I read as it's … clear in a regular screen font. But seeing the page itself transform into English is really spooky. Like a Harry Potter transformation scene. Unnerving. So here's the transformation picture in mid action (scary and not useful)

​

And here is what I read: one can scroll to hide the other language so the entire screen is filled with English:


Not perfect but given that it appears instantly without any need to cut and paste or save, and assuming we know the subject matter of our numismatic books, easily enough readable. As noted you can see the English in full screen mode and hide the original language if you wish.

 

I discovered that my google translate app on my iPhone does this as well. It is spooky as it translates. I haven’t had the chance to play much but it was a real hit at the brewery last night.

 

Here is my wife’s coffee mug mid-translation into German: