Microsoft Taps Augmented Reality for Bing-Powered Translator App

As the Windows team blog puts it, "Nothing takes the fun out of a foreign vacation faster than an afternoon spent deciphering street signs or pointing randomly at menus." That's why Microsoft has released a new version of Bing's free Translator app for Windows Phone, one that you just point at the foreign-language text on (say) a menu or sign, scan the text and get a translation overlaid on the image in your preferred language, all thanks to augmented reality.

The pitch: "Now you can point your phone camera at printed text—street sign, dinner menu, train schedule, newspaper headline—and the app will translate it. If (you) wanted to ask a stranger for directions to the Eiffel Tower, (you)could also just speak into the phone in English—and wait a moment for the French." It's like “automatic subtitles for everyday life,” as Bing puts it. So should Siri worry?

Bing's announcement details the app's speech recognition: "Want to have a conversation with a local? With the voice translation you can communicate in English (both US and UK), German, French, Italian, and Spanish. Once in the mode and connected online, you simply tap the icon, speak and tap to translate."

The app's AR translation is supported in six languages at the moment: English, French, German, Italian, Simplified Chinese, and Spanish. While machine translation can't handle linguistic nuances, the app is being released in time to help summer travelers get around — and grab the right train or menu item.

The technology has been in development for years, the Bing post notes. "Over the last decade, world class researchers - both at our own research labs and around the world have been working on the problem of language translation, the results of which you enjoy today on Bing. Now you can take that experience with you as you travel across the globe with the new Translator app for Windows Phone."