Opus: Adaptive Storytelling
Opus: Adaptive storytelling
Summary:
Opus is an adaptive storytelling platform that consists of an e-paper bound book for storytelling, and an Augmented Reality (AR) ‘magnifying glass’.
OPUS facilitates richer, more personalized, and meaningful two-way communication between parents and their child that is based on the child’s day. Children play with the ‘magnifying glass’ during their day, collecting interactive experiences with multiple items that appear during story time later that night, to create personalized story-time moments rich in 2-way content.
The adaptive storybook provides prompts (fill in the blanks with items the child has collected) to create the story imaginatively in real time. It listens to the conversation between parent and child and integrates the topics into the plot of the story.
Project Brief:
Sponsored by Microsoft Design Expo 2016, The prompt was to explore near-future situations where a product, service or solution demonstrates the value and differentiation of Conversational User Interface (CUI) in the context of Human-Machine Symbiosis.
Team:
Sarah Foley, Dixon Lo, & Jiyoung Ko
Semester Long Project
My Role and Contribution:
Concept Development, Primary and Secondary Research, UX Flow, Illustrations, Project Management.
Exploratory Research:
Touchstone Tours:
2 Preschools
1 Family house
1 Local Baby Product Company.
For the Preschools we tried to understand the type of communication they had with their children’s parents. An Example, if a child took their first steps while in the care of the preschool, they would not communicate this to the parent as they felt that the parent would feel robbed of that moment.
In the Family House, there was an example of a picture of the family smiling. When the child was in trouble, he would try to bring his mom to the picture and ask why she wasn’t smiling like she was in the picture.
Interviews:
10 Parents
3 Professionals.
Interviews were semi-structured. Questions focused on hardships and new identities formed during the first few years as parents.
Exploratory Research Synthesis
Themes and Insights from Exploratory Research:
New Parent Presence
Physical and Emotional Presence is challenging in the first couple years after becoming a parent.
New Parent Information Overload
The source of where information came from was important when deciding to integrate it into one's parenting methodology When information came from someone the parent admired, it meant more.
Conversations
Conversation was important when looking for information, asking for confirmation and to build/ maintain relationships
Framing Questions:
“How can we provide mentorship and community involvement to isolated parents to lessen mental anxiety?”
“How do we make short bursts of mundane time more meaningful and memorable for busy parents?”
Generative Research:
Magical device workshops
4 Individual Workshops
2 Group Workshops
Focus on daily family rituals, allowed for the insight that initiation of communication to gain insight into their children’s life was difficult.
More directed questions were asked about task delegation to a CUI (Conversational User Interface) within these rituals. Parents tended to delegate busy work to the CUI such as ‘do the dishes’. Some parents felt comfortable with the CUI helping with tasks such as homework as long as the help was targeted to exactly that child needed.
Generative Research Synthesis:
8 Additional Interviews
Themes emerged where parents wished they knew exactly what happened during the day while they were not there. Some parents used words such as “Sunny, Cloudy, Rainy” or “Happy, Crabby, Funny” to prompt their children to share what happened that day.
Insights:
No Information is given
“I point the Lens of Truth at the baby, and it tells me what the baby wants.”
Part of the information is given
“I ask my kids to describe their day as Sunny, Cloudy or Rainy. Because I need to know how difficult the rest of my day with them will be.”
Generative Phase Round 2:
Speed Dating:
12 Individual Sessions
2 Group sessions
100 ideas with different inputs and outputs.
Participants were given money to ‘purchase’ their top choices. This showed that participants liked and disliked certain concepts in theory, but when shown specifics of the scenario, their feelings usually flip-flopped.
Guiding Insights:
Intention of Technology
“I don’t want to be a robot parent.”
Education
“I tell stories to teach my children lessons.”
Autonomy
“I want my child to decide what to play with.”
Resulting Directions
What is considered acceptable technology to parents for their children to play with?
If a computer knew ways to create richer interaction through conversation or activity, how would this information be delivered?
What are these activities or conversations that could be made richer?
Evaluative Research:
CUI Experience Prototype:
Storytelling with non-scripted responses through Text-To-Speech.
The pause in conversation when the human CUI typed in a response was almost like loading time in real CUI. We found that without an indicator of “I’m thinking” or “I’m typing”, the human actor would re-enter the conversation too quickly, not knowing if there was a response coming.
In addition to the Text-to-Speech method, we sat in a circle and took turns prototyping different responses/ personalities to test which felt best for a semi-private bonding moment between parent and child. Resulting personality was a ‘caring grandmother’.
Low-Fi Prototypes:
4 Parents and Children.
Understanding what a CUI coupled with machine learning or AI would say in a certain situation requires understanding the nuance of what is being asked. Because of this for some tests we were forced to create custom scenarios for specific people. This required a pretest research phase that in the real world would have been done through big data.
Parents also told stories from scratch. When creating a story parents asked their children to fill in the blanks and lead the story.
Focus was put on how the stories were being delivered. What type of personality does the CUI have? Do the characters talk to you or do they just respond to your voice? And, what degree of freedom the storyteller would want to have?
Key Insights from Research:
Interviewed parents wanted interactions with their child to be technology free, as they felt the technology-mediated their relationship. This sentiment dissipated since OPUS facilitates better 1:1 interaction.
Parents want clues about their child's day to have richer conversations. When asked 'How was your day?' children didn't know how to answer and gave brief vague answers.
Storytime is a daily ritual where families connect.
Final Research Question:
“How can we enable parent and child to ask better questions and engage in deeper conversations about the child’s life experiences?”
Design Principles:
A range of freedom in storytelling is needed.
Product can assist the stories parents create already.
Technology delivered through a book form is OK.
Option to review the story before reading it.
Opus works in the following way:
1. DISCOVER:
The child uses the interactive magnifying glass to discover items in their vicinity. When the child looks at an object through the magnifying glass, the object is recognized and comes alive with stories about its adventures and history, eventually asking the child to look for it later that night in the bedtime story.
2. CO-CREATE:
Opus provides a storytelling structure that allows stories to be created through blanks. This feature was modeled after parental behaviors observed in our research when parents were asked to make up stories with their children.
3. EXPERIENCE:
Opus listens to the story-time conversation and constructs the story in real time based on what it hears. OPUS guides the story using a three-act story structure, which allows for cohesion between real-time changes and the theme pre-chosen by the parent.
4. SHARE AND LEARN:
Objects gathered by the child during the day, prompts the parent to have a conversation. Prompts allow parents to have more directed questions about the child’s day.
5. REFLECT AND PLAN:
After the story the parent can plan the theme for the next time OPUS is used. Here OPUS allows parents to choose what values they want the next story to focus on. A summary of the next story is provided along with an empirical reasoning behind each plot point.
Validation of Parent-Child interviews:
“The best way for a kid to learn from a story is to see themselves in it”
“It doesn’t only help me, but it also helps my kid recollect their experiences”
Extended Family:
- Allow extended family to contribute to story-time
- Schools can utilize OPUS for educational purposes
Medical:
- Create a different way for heath care providers to communicate with their pediatric patients
- Expand use to Children Psychologists for diagnostic purposes. Conversation then become diagnostic, giving insights to the way the child thinks.
Selected to represent CMU at the Microsoft Design Expo, July 2016.
Our team won “Most engaging”.
