Jun 3, 2019
Qualcomm products mentioned within this post are offered by Qualcomm Technologies, Inc. and/or its subsidiaries.
You’ve spent months dreaming of walking through the famous Jongmyo Shrine and eating fried mandu at Myeong-dong Street Market. Finally, you’re headed to Seoul, Korea. It’s a solo business trip and while you’re excited, you’re also a bit nervous — mostly about the language barrier. Thankfully, your smartphone’s on-device artificial intelligence (AI) could soon make it the perfect travel companion for assuaging those nervous knots and heightening the experience, too.
That’s because on-device AI can make existing user experiences more intuitive now while enabling new ones that could enrich your travel experiences and your life in general. Simply put, on-device AI describes machine learning and other types of AI data processing that occur on the device, rather than solely in the cloud. Localizing AI in this way reduces latency, increases security, and saves network bandwidth and power. Plus, functions run almost instantly because output data doesn’t need to travel to the cloud and back.
See how this technology could be used in your trip to Seoul: Your travel AI companion (your phone!) could create an itinerary catered to your preferences ahead of time. And when you’re there, it can assist you with real-time translation of Hangul. The phone's on-device AI can also help you capture the majesty of Gyeongbokgung Palace and recognize the latest K-pop hit. And because AI is smart about power consumption, you’ll have a constant companion throughout your trip.
Develop a personalized itinerary
The week before you travel to Seoul, you begin to create your itinerary, with the help of AI. Your phone’s digital assistant could create an itinerary that matches your preferences, based on your history. You love to shop, so it suggests a visit to Namdaemun Market and Myeong-dong since you’ll be in the area after a lunch meeting. Knowing you’ve been to Machu Picchu from the photos stored on your device, it recommends a trip to Gyeongbokgung Palace and Bukchon Hanok Village to see traditional Korean homes. And because you enjoy hiking, which your digital assistant knows based on your recent purchases, it incorporates a day at Bukhansan National Park in your schedule.
How it works: With the help of AI, both on-device and from the cloud, digital assistants are becoming increasingly more adept at learning your preferences over time to create personalized experiences for you. It does so by combining data pulled from a variety of sensors — including your phone’s microphone and camera — with on-device (your calendar, messaging, and apps) and off-device (cloud and IoT) data. This gives your digital assistant important background to suggest activities to enrich your trip.
Translate Hangul in real time
When you arrive in Seoul, you’re overwhelmed by the sights and smells … and the language barrier. Street signs, store fronts, and menus are all in Hangul, and your language learning app didn’t prepare you as well as you’d thought. No sweat — with on-device AI, your phone could translates subway instructions in real time, so you’re able to get from Incheon Airport to your hotel easily.
How it works: Many of Qualcomm Technologies’ mobile platforms are packed with advanced AI capabilities that allow you to use your phone for help navigating the bustling city, even if you don’t know Hangul. The Qualcomm Snapdragon 730G Mobile Platform, for example, integrates our 4th generation Qualcomm AI Engine, Qualcomm Hexagon 686 DSP, and Hexagon Vector eXtensions for advanced on-device imaging and computing. Together, they enable the Google Lens feature to run quickly on devices. This application could instantly translate words into your preferred language when you point your phone at a street sign, menu, or anything else.
In the future, you could communicate with business colleagues through video chat as if you were speaking Hangul yourself. Face mapping could create a realistic lip sync that would make it look and sound as if you were actually conversing in the language. Your device’s camera, 5G’s ultra-low latency, and powerful on-device AI would work together to bring this to life.
Learn history with just a tap
Armed with the itinerary your digital assistant created for you, you make your way to the breathtaking Gyeongbokgung Palace. You want to learn more about its history, so you pull out your phone and point it at the palace. Your phone’s on-device AI provides historical facts about the landmark in front of you.
How it works: Snapdragon processors equipped with our Qualcomm AI Engine are able to work with your device’s camera, sensors, modem, and dedicated neural network to identify objects in your environment and share additional context. Google Lens, which can be accessed from the Google Assistant on most Android phones and the camera app on some Android phones, is an application that Google has trained to use all of that data to provide relevant information back to the end user.
Accessing this feature is as easy as opening your phone’s camera app, pointing it at a landmark, and tapping once. Your phone will turn on Google Lens mode, scan the scene, and pull from its database to find information about it.
Capture memories and store them intelligently
With work behind you, you set off to explore Seoul, from the palace to the market to the restaurant that promises the best Korean BBQ. Your AI-powered camera captures your escapades in all their glory, applying the ideal settings based on the subject, lighting conditions, and more. The result: professional-looking photos and videos that accurately and beautifully document the trip of a lifetime.
How it works: When it comes to taking photos, the Qualcomm AI Engine could work with your phone’s camera powered by Snapdragon to automatically apply the best settings, such as auto scene detect and smart cropping. If you’re snapping a photo of the N Seoul Tower at sunset, for example, your AI-powered camera can recognize the low light and adjust accordingly by raising the foreground lighting, bringing out the vivid colors of the scene.
Some mobile platforms, such as the Snapdragon 855 and Snapdragon 730, feature the world’s first announced computer vision image signal processor (CV-ISP), the Qualcomm Spectra 380 ISP. This CV-ISP provides computer vision object classification and processes data 4X faster compared to its predecessor. It also separates elements of a scene and swaps out backgrounds and objects in real time.
When you record video of your journey around Korea’s capital, the CV-ISP captures it all in 4K HDR with portrait mode and achieves depth at 60 fps. It also supports other features that will intelligently and are designed to automatically improve the quality of your footage, including super resolution, super night shot, and object detection. Together, these features help you create an impressive highlight reel of your time in Seoul that you can share with friends and family once home.
All of these photos and footage take up an enormous amount of space on your device. Luckily, the CV-ISP stores photos, videos, and computer vision data in the HEIF format, which saves files at half the size. With all of this newfound space, you have the freedom to capture photos and video to your heart’s content (and way more than any best friend will ever want to watch).
Discover chart-topping hits in other countries
K-pop is everywhere in Korea, and there’s this one hit track that’s played everywhere, even while you were eating at that BBQ restaurant. You want to find out the song’s name, so you can listen to it when you get back home. With on-device AI, your phone would have already identified the track, artist, and album before you prompted it to.
How it works: No need to scramble for your phone when you hear a catchy tune and want to find out its title. Google’s Now Playing feature uses machine learning to recognize when music is playing. Because this audio detection uses on-device AI, it compares a few seconds of the song to those in its internal database without having to send the information to the cloud. It then quickly displays its results on your lock screen, so you won’t have to waste time opening a song-searching mobile app. And it does all of this privately, without consuming a ton of power.
Extend your phone’s battery life
As your trip to Seoul winds down, you savor your last supper of mandu and tteok-bokki. By the end of your packed day of sightseeing, your phone still has plenty of power left for you to stream a movie before bed or share your new memories on social media. While you’re posting, you receive an alert reminding you that your flight home is on time — the last item on the itinerary your digital assistant made for you. Though you’ve traveled solo, you’re not alone: thanks to your travel AI companion, you’ve been on time, fully engaged, and totally immersed in this no-longer foreign city — and your phone’s battery life hasn’t suffered in the process.
How it works: Deep neural networks consume an increasing amount of energy as we develop more AI-powered features. Fortunately, we’ve engineered our chipsets with on-device AI capabilities to intelligently decide how to best use your device’s power. Your phone can efficiently run multiple applications quickly, balancing their workloads to ensure that its battery isn’t drained by the end of a busy day.
Qualcomm Technologies engineers solutions that are integral to your everyday life, and that includes your days off. With on-device AI, we’re helping to make your phone the ideal travel partner, pushing the boundaries of what’s possible for a mobile device.