Getting personal with on-device AI

Welcome to AI on the Edge, our new OnQ series that delivers the latest on-device artificial intelligence insights and trends. Hear from our most active subject matter experts on the dynamic, ever-expanding subject of AI.

Learn more about on-device generative AI by exploring our AI on the Edge posts. If there's a related topic you’d like us to cover, simply send us a note.

In the mid-50’s, Julie Andrews had the world singing, “Getting to know you; getting to know all about you,” and now, this iconic tune could be the theme song for today’s latest tech: artificial intelligence (AI). This cutting-edge technology gets up close and personal because personalization allows AI to better respond to human requests and to anticipate human needs. 

One of the most promising aspects of AI is that it can be used to personalize the interaction between computers and humans using contextual information and personalized fine-tuning of standard models. 

But personalization can often run counter to privacy when the information must travel to the cloud and back. An advantage of on-device AI is that the local AI model can still provide personalized responses, but without sharing that data back to the cloud, therefore enhancing data privacy. 

To achieve personalized AI at the edge, the AI can utilize local data about the individual users and their current environment. For example, edge devices like smartphones, tablets and PCs, can utilize user data from built-in sensors like cameras, microphones, accelerometer, gyroscope, GPS, Wi-Fi and Bluetooth. For the cloud, this can be costly, inefficient, impractical and sometimes even impossible, because uploading data like video data to the cloud can take a lot of bandwidth and easily drain the battery life. This local sensor data can then be used for on-device inference to give more relevant and personalized responses.

While many privacy laws grant users the legal right to control their personal data, on-device AI grants users more control of their data and whether they want to enable this personalization capability. By keeping the data on the device in a controlled environment as set and managed by the users themselves, no additional copy of the data needs to be created and stored in a remote place that is technically out of the user’s control. Moreover, data exposure is minimized, because no data is transmitted outside the device. This reduces the risk of data breaches during transit and with no additional data copies being stored on the cloud.

Running the personalized AI on the edge devices does not require continuous access to cloud resources. This ensures low latency, enhances data privacy, and provides reliable offline capabilities.

What are the advantages of personalized AI? 

A personalized AI system is designed to understand and adapt to individual preferences and behaviors. The goal is to provide an enhanced, more enjoyable and efficient user experience, with the AI providing content, recommendations and interactions that are specifically relevant to the user. Personalized AI can engage users more effectively by presenting them with content and experiences that resonate with their interests and increase engagement. Below are a few examples of personalized on-device AI experiences.

Workflow optimization with personalized on-device generative AI 

Using personalized AI, users become more productive by automating repetitive tasks, providing customized workflow recommendations, and offering timely reminders or suggestions based on the user's work habits and preferences.  

By understanding user preferences and habits, personalized AI can automate routine decisions and actions, saving users time. For example, it can automate creating shopping lists, to-do lists and provide meal recommendations based on user patterns stored on device.

Health benefits of personalized on-device generative AI 

There can be health benefits to personalized AI. Health and fitness apps can provide customized workout plans, dietary recommendations and health tracking based on an individual's goals, health history and progress. Personalized on-device AI can react quicker to health changes, even without cloud access. The AI can monitor changes in biometric readings and detect anomalies based on known health conditions.

Improved accessibility with personalized on-device generative AI 

There’s an opportunity for personalized AI to enhance accessibility features for users with disabilities by adapting interfaces, content and interactions to individual accessibility needs and preferences. Personalized AI could be used to translate speech for those people with disabilities.

Increased productivity with on-device personal assistant 

Personalized on-device generative AI could filter and prioritize information and notifications, reducing the overwhelming volume of data and content users encounter daily. This helps users focus on what matters most to them. Knowing a user’s local context, a personalized AI would be able to alert the user to important emails, texts and social media posts without setting these filters in the cloud and with a fully integrated approach to alerts.

Leveraging personalization without compromising AI privacy

With on-device AI, users may be more comfortable with using their data for personalization knowing that their data can remain on the device and not go to the cloud. Users can still choose what information to provide as input and how it's used to personalize their experiences, providing a sense of empowerment. Privacy is still in the control of the user and security measures available on device can protect AI models, prompts, outputs and user data.

Personalization with on-device learning

Building AI models that are specific to the individual user or use case involves machine learning techniques, such as retraining from scratch, fine-tuning, reinforcement learning or transfer learning. Looking forward, on-device personalization can be achieved through on-device learning or adaptation, such as by fine-tuning a pre-trained AI model on the device with user data. 

A benefit of on-device learning is that the model can continuously improve with more user data over time and does not drift from the user’s personal preferences. As an example, a personalized AI could be fine-tuned on device to recognize a user’s particular accent or speech patterns for more accurate responses and adjust as a user’s voice changes over time.

Generative AI can also be personalized by fine-tuning large language models (LLMs) on a user’s specific data. As an example, an LLM can be fine-tuned on a user’s writings to better mimic the user in new writings and automated — but contextually-aware — responses. A user’s personal collection of images could be used to synthesize new images or videos for the user to watch. The issue today is that LLMs are extremely large and difficult to fine-tune on an edge device — however, techniques like low-rank adaptation are making it more feasible. In the future, models will be more compact and devices more powerful to close that gap. 

Overall, personalized on-device AI can enhance user satisfaction, engagement and productivity by tailoring experiences and solutions to everyone’s unique characteristics and preferences. It's essential to balance personalization with user privacy and data protection to maintain trust and transparency in AI systems — on-device personalization provides this.