Samsung Research Institute, (Jan 2021 - Aug 2022)

Designing AI and Data Privacy Solutions for Smartphones

The overarching brief was enhancing productivity by automating menial everyday tasks to create a seamless experience for users while ensuring data privacy. The touch points covered a broad range from keyboard suggestions to the share panel to device search.

View Press Release for One UI 5 AI and Privacy Features Here
Role and Team

Senior Interaction Designer

My role included user research, conducting competitor analysis and creating and pitching design concepts to Samsung HQ.

Upon finalisation, I worked on the UX flow and final UI design for the features.

For each project, I collaborated with a Product Manager, other Product Designers from the On-Device AI team and with the AI engineering team.
Tools and Methods

UX Research Methods and Product Design Tools Used

User Research -
User Surveys, Interviews, Usability Testing, Competitor Analysis
‍
Product Design -
Prototyping, Information Architecture, UX Flows, UI Design (Sketch)
Disclaimer

Non-Disclosure Agreement

Due to the NDA, adding images or details of unreleased features is prohibited. This document contains some of the major features worked on that have been released, the user problem and solution.

I. Device Intelligence - Personalisation and automation

The On-Device AI branch of Samsung Research deals with identifying user needs for personalisation and automation in various areas of the phone.
Process Map
#1 Smart Suggestions and Smart Widget

Suggestions for useful actions across core apps based on the user's context

*Released in 2022, as part of One UI 5.

The Problem

‍Users often find themselves burdened with numerous manual tasks.

Tasks such as inputting information and transferring data between different applications leading to inefficiencies and user frustration.
The Solution

Intelligent suggestions, such as creating reminders, scheduling events, and text prediction.

‍Leveraging the wealth of data collected regarding user patterns, we recognized the potential to extend its utility beyond simple text input. This data can be harnessed for a wider array of intelligent suggestions aimed at streamlining user experiences across various touchpoints.
#2 Image Text Extraction using Vision AI

Copying text from images and the users surroundings using the camera

*Released in 2022, as part of One UI 5.

The Problem

‍Users have to manually remember and type out text from images.

Extracting text from images so far has been a laborious process, requiring users to manually remember and type out the text, which was both time-consuming and tedious.
The Solution

Using a simple copy and paste function to extract text from images.

The team achieved high accuracy with Optical Character Recognition (OCR), an early form of computer vision. The solution became user-friendly by using the familiar keyboard as the entry point for pasting the extracted information.
#3 Bixby Routines Plus

Helping users create custom automations on their smartphones

*Bixby Routines Plus is available on the Samsung testing platform for advanced users

The Problem

Simplifying the UX for advanced controls and actions on Routines Plus for advanced users.

Users rely on Bixby Routines to create simple sequences for custom automations using their smartphones and various IoT devices. Samsung created an advanced version called Routines Plus to give users more control. The task was to ensure simplicity of the interface.
The Solution

Breaking down actions into small steps to enable users to create complex routines.

Routines uses a simple if else structure to display the overall routine created. In order to allow user to use different in-app actions, a step-by-step action selection process using pop-up cards was recommended.
#4 Intelligent Keyboard

Contextual pasting suggestions via the keyboard

*New features unreleased. Basic smart paste displayed here.

The Problem

Switching between apps to copy different content to share.

To quickly copy and paste items is important to maintain the flow of conversation. Normally users have to switch between apps to copy different text and media and links to share.
The Solution

Expanding on the smart paste for screenshots feature.

Based on the context and usage patterns among other things, the keyboard would be able to suggest input, supporting different forms of media for pasting.

II. Data privacy solutions

In the last few years, privacy and security concerns have snowballed and now become one of the most important factors when it comes to personal devices. The proof is evident in the widespread discussions about privacy and the significant efforts by most tech companies to reassure users that their privacy is a top priority.
Process Map
#1 Security and Privacy Dashboard

Increased transparency around app permissions

*Released in 2022, as part of One UI 5.

The Problem

Users want to be better informed about how and when their camera, mic and location is being accessed by different apps.

In the world of smartphones, a significant concern revolves around the trustworthiness of these devices. Users often feel uncertain about whether their microphone or camera is being accessed without their knowledge, or if their location is being shared without permission.
The Solution

A centralized location for accessing and managing app permissions.

Leveraging Android 12's microphone and camera indicators, we introduced a concept aimed at simplifying the management of permissions for both third-party apps and Samsung apps. With just a single tap, users can regain control over their privacy, addressing these pervasive concerns.
#2 Private Share

Accidental sharing of sensitive information through images

*Released in 2022, as part of One UI 5.

The Problem

Users can unknowingly share sensitive personal information while sharing images.

While sharing images, users might accidentally share private and personal information like ID numbers, credit card info, etc. which can be potentially dangerous for the user.
The Solution

Alerting the user to potential exposure.

Using OCR, sensitive information can be identified based on the understanding that ID numbers, credit card numbers, etc. have specific patterns and character numbers. In the Share Panel, the product provides a simple alert bringing to the attention of the user that they might be sharing personal information.
Let's connect!

Reach out to me at srishti.malaviya@gmail.com if you would like to chat about anything related to design, product or tech!