‍

‍

Design Process

Background Research

Defining the Problem Space

Our initial research into the topic revealed that audio diaries have not become widely adapted due to security, logistical, and cost problems. Through our deeper investigations of this space, including interviews with researchers, interviews with patients, user testing, surveys, and further discussions with our industry sponsor, we were able to narrow and clarify the specific problem we hoped to help solve:

‍With current technology, researchers do not have a suitable means for collecting strong data from studies employing audio diaries, neither as EMA (Ecological Momentary Assessment) nor as a daily diary method.

We learned that the root of this issue stems from:

• A lack of personal mobile applications that successfully account for privacy and security concerns impedes researchers from successfully collecting audio diary data from a large sample of participants.
• A lack of effective strategies to engage participants enough to provide consistent, valid data results in data lapses.
• Accessibility issues in recording methods for diverse populations and diverse recording contexts further increase the difficulty of applying the audio diary method.

‍

‍

Defining User Needs and Design Implications

As our team conducted three distinguishable research activities throughout the initial data collection procedure, findings from each method contributed to the project in the following areas:

• Semi-structured interviews drove the primary creation of found issues through thick and qualitative data. Main themes were derived across researchers’ and participants’ experiences and pain points directly interacting with existing platforms and methods.
• Our Probe Study returned first-time users’ experiences with recording themselves in a research study context and competitive comparison with written diaries, the audio diary’s primary counterpart.
• Competitive Analysis verified existing features of apps and interfaces that both worked and didn’t. Analyzing the current state of technologies in the field allows us to reflect on what areas are lacking and produce interventions.

‍

Methods for Identifying the Problem Space

Method 0: Initial Research

• Literature Review
• Stakeholder Identification

‍

‍

Method 1: User Interviews

Information Goals: In the first stage of our research, we sought to understand the entire user journey of both researchers and participants in audio diary studies for emotion assessment through semi structured interviews. We wanted to understand the overall procedure of audio diary studies and detailed user needs & pain points at different stages throughout the study process. Through interviews, we specifically wanted to dig deeper into the reasons behind researchers’ decision to adopt audio diary studies, participants’ data privacy concerns, comfortability during audio recording, and what might cause inconsistent or invalid data.

Method: Semi-structured

Participants: 3 researchers, 2 prior audio-diary study participants

‍

‍

‍

Interview Data Analysis: Affinity mapping of qualitative data

‍

‍

‍

‍

Key findings and initial design implications from interview data:

• Help participants feel motivated and engaged to complete their recordings
• Minimize privacy and security concerns (technical or perceived)
• Increase comfortability and ease of recording process
• Effective notifications to remind participants
• Foster effective communication about the setup, procedure, etc.
• Ensure the data quality and validity of the report
• Incorporate Objective Reflections to Track Personal Development During Longitudinal Study
• Address Technical Accessibility and Improvements Outside of Recording Process

‍

Method 2: Probe Study Utilizing Ethica App

Through preliminary research, we identified that first time audio diary users found it hard to get used to audio recording. Awkwardness, nervousness, difficulty in reflection and organizing words, are common pain points according to the research participants we interviewed. To further understand the strengths and weaknesses of audio diary methods, we probed daily diary experience using both audio recording and written text, since written text is the most widely-used alternative logging method and has distinct characteristics that can provide different recording experiences. Through the probing study, we wanted to understand how first-time users feel about the audio recording method in daily diary studies in different contexts, the pros and cons of it in comparison to written text, and possible strategies they would use to progress in keeping audio diaries. Following the study, a survey was completed by each participant and the data was compiled and analyzed using Qualtrics.

Method Details:

• Platform: Ethica
• 2 separate study designs
• 6 Participants

‍

‍

Method 3: Competitive Analysis

Our team conducted a general competitive analysis with a focus on mobile applications for logging and tracking the users’ daily mood and wellness. After an initial meeting with Dr. Kaplan, our team was introduced to three major EMA softwares/platforms: Life Data, ExpiWell, and Ilumivu. We developed a brief competitive analysis board using Miro to analyze the following criteria: ease of use, UI components, level of customization provided, quantity of features, and level of technical support.

‍

‍

‍

Interpretation of Findings

Design Implications

Following analysis, we identified 7 primary themes to address:

1. Participants’ engagement and motivation for recording
2. Privacy and security concerns
3. Smooth user experience and comfortableness in recording process
4. Effective notifications for participants
5. Communication between researchers and participants
6. Data quality and validity in reported data
7. Technical accessibility issues and further improvements

‍

‍

Concept Mapping

‍

‍

Personas and Jouney Maps

To understand the characteristics and needs of our main user groups, we constructed three personas and journey maps. Two personas identify participants in audio diary studies for emotion regulation and one persona identifies a researcher in the space. One Example is shown below.

‍

‍

‍

‍

Design and Prototyping

Sketched Concepts

Keeping our focus on our list of design implications and primary research findings, we then moved on to the ideation phase. We began with a group brainstorming session focused on generating a large quantity of ideas to address our findings. We knew from the beginning that our goal was to design a mobile application solution so we kept our scope within this realm. As a group, we spent 30 minutes individually coming up with as many ideas as we could and sketching them out on note cards, then after 30 minutes, convened to share and discuss our ideas.

‍

‍

As a team, we ultimately decided on two design themes to further develop. The first was a diary app built around the metaphor of a record collection, focused on improving user engagement and effective metric tracking. The second was an app which capitalizes on the assistance of a virtual voice agent, focused on ease of recording process, comfort, perceived security, and an overall high quality user-experience.

‍

Concept 1: Record Collection

‍

‍

‍

‍

Concept 2: Virtual Agent

‍

‍

‍

‍

Concept Feedback

After the first iteration of our two designs, we needed user feedback to assist with proceeding on a decision. Our team constructed an outline of the broad questions we were seeking to have answered and the justification for these decisions. For our Record Case design, we wanted feedback to center around changes in motivation to record, and for the Virtual Agent design, we wanted feedback to center around comfort and engagement. For both designs, we wanted feedback on accessibility considerations and users' natural understanding of how to use the app and its various features.

Method: Structured Interview

Participants: 5 college students, mixed gender, ethnicity, and prior experience with diary recording apps

‍

Feedback Session Findings

After completing our initial round of feedback sessions, we pulled quotes and information from the notes and organized in Miro. We then sorted these notes into categories based on the aspect of the design that they each addressed. As these distinct categories took shape, we were able to recognize recurring themes in the feedback from which we developed design ideas and requirements to be implemented in the next phase.

‍

‍

Wireframe

After analyzing and discussing the insights obtained through feedback, we proceeded with wireframing a new design centered in the idea of the virtual agent concept though combining strengths from both ideas. Some of the wireframes are shown below.

‍

‍

‍

‍

Feedback

After completing our wireframes, we proceeded with another round of feedback conducted in a similar manner as the concept sessions. Participants were able to explore the design as we guided them, sharing their unfiltered thoughts aloud as they navigated and completed tasks.

‍

‍

Prototype

Using our feedback findings, we identified areas of improvement, continued iterating our design, and began building a functional prototype in Figma. Some of the prototype screens are shown below.

‍

‍

‍

‍

‍

‍

‍

‍

‍

Evaluation

In developing an evaluation plan, we first constructed a set of evaluation goals that we wanted to receive feedback on. These included understanding if our solution meets the needs of our users, understanding if our solution is easy to use, understanding if our solution is enjoyable to use, and understanding of our solution meets the expectations of our users. In order to assess these questions, we broke the process into two separate evaluations; expert-based evaluation and user-based evaluation.

‍

‍

For the user-evaluation, we chose to implement both Think-Aloud usability testing and semi-structured interview evaluation methods. We recruited 5 researchers across various disciplines interested in implementing our app for audio diary research. First, we introduced the scope of the app and described the tasks we had for them to complete. Then, the participants proceeded to complete tasks while speaking aloud their thoughts and impressions. After the think-aloud was finished, we completed the semi-structured interview to probe specific topics.

For the expert evaluation, we chose to implement a heuristic evaluation where we had three 2nd year MS-HCI students at Georgia Tech engage in task-specific flows and critique based on the Nielsen Norman’s 10 Usability Heuristics.

‍

User-Based Evaluation

Methods: Think-aloud usability testing, Semi-structured interview

Participants: 5 researchers across various diciplines

‍

Expert Evaluation

Methods: Heuristic evaluation using Nielsen Norman 10usability heuristics

Participants: 3 second-year MS-HCI students at Georgia Tech

‍

‍

Findings and Design Recommendations

In sorting and analyzing the feedback we received, we were able to distill the most crucial findings and suggest design changes that should be implemented in the final application. There were some issues that presented themselves as being most important due to the severity ratings provided, number of times the issue was raised, and the passion with which our users and experts shared the feedback.

• There need to be more ways for collecting quantitative data.
• The app has great flexibility but still needs more options for researcher customization as to make it effective for a broad range of research applications.
• Feedback on voice agent is positive but we need to allow more customization options to suit different populations.
• Functions such as deleting should not be hidden (swipe) as to make the app accessible to all populations including those who have less experience with mobile apps and technology in general.

‍

‍

‍

Final Design Renders

‍