Smart home gadget demo

Role

Conversation Designer

UX Designer

Deliverables

UX Design

Use case design
Haptic interaction suggestions

Conversation Design

Competitive analysis
User research
User journeys and conversation flows
Sample dialogues
Error handling design
Intent design and prompt design
Usability tests and findings

Project Specifications

Duration

4 WEEKS

Company

Nuance (Cerence)

Background

Our client Lenovo planned to showcase a smart home gadget demo during an international smart devices conference. At that time, Google and Amazon had been working on their own smart home devices, but nothing had been released yet. Lenovo wanted to showcase their bleeding edge capabilities of their own smart home devices. They asked for help on voice experience design. I led their design team and provided strategy and design suggestions.

Goals

Become a starting point for Lenovo to break into the smart home market
Showcase the bleeding edge capabilities of conversational design with smart home devices
Use the latest technology to support innovative features

Requirements

From Lenovo

Lenovo planned to create features in two main fields: home control and connecting services. Only single turn conversations were planned for this project, except for some cases which needed disambiguation.

Audience: Professionals in IT field, media, competitors, and other attendants of the conference.

The goals for the demos were usually very different from the goals for customer-facing products. The main goal is to showcase Lenovo adopting the latest technology and designing more friendly user experiences. The focus was to develop and showcase the capabilities of certain features of the products.

Constraints

Time constraints
Technical constraints
Lack of VUI design experience on the Lenovo design team

Dependencies

Data sourcing

Brainstorming

First, I listed out speech technology, current voice supported domains, and some ideas for use cases.

Evaluation and Finalizing use cases

After brainstorming, I evaluated the potential use cases by creating a Optimal outcome VS. Effort matrix.

Effort includes time effort and technical effort. Some use cases require longer time to tune the computational engine, some require wide-range-covered real-time data, and some take longer to be tested. As a result, selecting the low hanging fruit use cases was more realistic for this demo.

I worked closely with speech engineers from Nuance, software engineers from Lenovo, and a PM to finalize the best use cases for the demo by considering the ratio of effort to outcome. In the end, we finalized about 30 use cases/intents within these 7 topics: light control, AC control, air humidifier control, weather, music, traffic information, and taxi calling.

Most of the conversations are one-off tasks. The plan was for me to design a few dialog samples and to provide design methodology & principles for the Lenovo design team, and they would finish the rest of the design. After that, I would review the designs and provide further suggestions for refinement.

Design

VUI design

Here is an example of one of the dialogs I wrote: Turn on/off light in some location.

Wake-up Word

How to start a conversation? Among all the methods, Lenovo decided to use a Wake-up Word to invoke the system. They proposed three candidates:

hello, cookie
cookie
cookie, cookie

Since the second candidate only has two syllables which might be too short as a good wake-up word, I suggested Lenovo to pick the first or the third candidate depending on their product positioning.

After waking up the system, the user could start a conversation.

The happy path for this one turn conversation is like this:

User: “Turn off the light in the bedroom”
System: “Alright”,”ok”,beep

Slots

Here we see that there are two slots in the user’s phrase:

Phrases + Grammar

Other than slots, there are also many ways to say for the same intent.

INTENT: Turn on light in the kitchen

Turn on the light in the kitchen
Turn the kitchen light on
Can you turn the light in the kitchen on?
Can you turn on the kitchen light?
Can you turn on the light in the kitchen?
Kitchen light on
Turn the light on in the kitchen
…

System Response

System response can be verbal response or a simple earcon or even a visual cue depending on the specific use case and the device’s physical look & product positioning. I asked the Lenovo team to make their own decision on this, since the final product positioning and physical look were not yet done.

The first design strategy I gave to Lenovo was to design one-off conversations by defining the phrases and slots, and system responses. This covered the most of the conversations. However, there were more topics which needed to be carefully taken care of.

Error-Handling

Besides the happy paths, the system also needed to be able to recover from a failed conversation.

NO INPUT

After waking up the system, the user doesn’t say any command, or their voice is too soft. What is the system supposed to say?

NO MATCH

If after waking up, the system does not understand what the user said, how should the system should recover from this?

COMMAND CANNOT BE ACCOMPLISHED

Error types need to be collected, such as no internet connection, or some electronics type is not supported,…

For each case, how should the system respond?

After the the design guidelines and design tasks were ready, I handed them over to the Lenovo design team to finish the rest of the design.

Review & Testing

When I received the design from the Lenovo design team, I reviewed the design documents and refined a few versions. During each iteration, I was testing the prototype and gave feedback on how to improve the design. I also did some usability testing in the office.

Before the conference, I helped Lenovo draft the demo script to make sure that they showed interesting use cases with the most innovative conversational experience.

Results

The results of this project exceeded expectations of the Lenovo team and their leadership. The demo won a lot of attention and was successful in the international conference from the feedback I received from Lenovo a few months later. Here is a video by Lenovo showcasing the features of Cookie that I worked on: