▲ The Jeju travel app had trouble recognizing voice commands on busy streets. Photo courtesy Brenna McNamara

Jeju Special Self-Governing Province recently released the Jeju Tour ezTalky award-winning mobile application. The ezTalky app is to ease communication for foreign tourists to Jeju and provides voice recognition, automatic translation, and voice output in Korean, English, Japanese and Chinese.

The smartphone app, part of Seoul’s Government 3.0 drive to improve e-governance and data management, also contains over 3,400 exclusive conversations and boasts over 1,000,000 sentences and 3,400 sentence structures, according to Google play. You also have the option to connect via various SNS accounts.

Hearing about its success, I hit the streets to try it out. Supposedly, the tourist asks a question - typed or spoken - into the app, which recognizes it and produces automated Korean. This is played to a helpful local who in turn answers into the app, which produces automated speech in the language of your choice.

▲ Reviews of the app from Google play

Although I experienced some success with specific questions, time-honored grunts and hand gestures often performed better. You see, when I say the app “recognizes” speech, it actually more often struggles. Almost every time it was used outside, the app misheard what I said. It made for some hilarious misinterpretations, however.

The mistakes varied from the minor, “Where is a sauna okay?” to the more significant, “Hey, I bought a medal,” instead of, “Where is the terminal?” Other examples include, “What time does the five-day market close?” coming out as, “What time does the bar 5 days in MARGET clothes?” There were even random cockney-sounding phrases like, “Is a bit more the gum on me England.”

I decided to use the app to direct me to a horse restaurant. After realizing that people reacted awkwardly to having a phone as a third party, I selected victims carefully and preyed on a couple of middle-aged women on a Shin Jeju street.

I asked the app where the bus stop is and it automatically vocalized the translation; on a busy street it was difficult to hear, however, so I showed them the question. They shook their heads when I tried to gesture that they could speak directly into the phone, instead gesturing “straight, right, Emart.” Even with their lack of spoken language, their ideas were conveyed clearly.

▲ The author seeking directions from a local. Photo courtesy Brenna McNamara

At Emart, I got my phone out to ask a young man which bus to take to City Hall. Every time I tried to get near him with my phone, he moved away nervously. He was finally against the glass and I spoke into the phone, trying to maintain eye contact so he knew I was in a conversation with him, not my phone. I showed him the text, which came out as “Which bus do I eat to get to City Hall?” He understood. He spoke Korean back to me and I gestured to speak into the phone instead. He spoke twice before the app recognized on the third attempt: “The room 502.” There I had it. The room 502.

Once in City Hall, I tried to use the app inside to eliminate noise interference. I asked two shop employees, “Where is there a horse restaurant?” Both gestured ‘no.’ I hit the streets again with my patience intact. The next person I asked showed me the general direction on his phone.

Heading in that direction, I asked a middle-aged man smoking a cigarette. The man laughed at my horse hankering, but went inside his warehouse to get his friend, who came out keys in hand. He gestured for me to get into his SUV. He kept repeating “Monday? Afternoon? Horse?” and laughing. I tried to use the app’s list of helpful phrases to express gratitude, but the phone’s loading and searching time did not allow for much communication. Laughing along with him proved the best way forward.

▲ A screen shot of the app translating the voice command "Where is the menu?"

Five minutes later and we had arrived near the bus terminal. He went inside and spoke to the restaurant owner for a moment before waving me in, then waved goodbye. Upon entering, I asked the shop owner what time the restaurant closed. She responded in Korean, so I gestured to the phone. She spoke before the app could register her voice. I pressed ‘Speak’ again, but once more she spoke too soon. I gave her the countdown that I had become accustomed to, but before it had loaded she put her hand up and blurted out: “Eleven!”

Although it was awkward and clumsy at times, it did get me to where I wanted to go. By the end of a week of app experimentation, I was getting the hang of how to utilize it to its full potential. If I were a tourist, I would definitely have the app on my phone for asking specific questions regarding things such as transportation, market opening times, and restaurant recommendations.

Facebook competition for theater tickets