Internship Story
10 min read
Nielsen Norman Group (NN/G) has traditionally focused on UX consulting and instructor-led Live Online Training. Now, NN/G is introducing an asynchronous learning format called Self-Paced Training. To support the development and iteration of this new offering, I was brought on as a UX Research Assistant (Qualitative) to evaluate the learning experience and guide product decisions through research. I conducted research across multiple phases phases to understand learning behaviors, engagement patterns, assess course effectiveness and identify improvement opportunities.
CONTEXT
Stakeholder alignment and research kickoff.
My first efforts were dedicated to refining the research goals and scope with the team. During our kick-off call, NN/G had already completed exploratory and market research that outlined user needs, early content strategy and market opportunities in the e-learning space. Based on these studies, the Instructional Design and Product team created early prototypes for Alpha testing. However, many decisions about course components, content, and structure were still driven by assumptions and secondary data. Thus, ownership of Self-Paced Training research was transitioned to me and a Senior UX Researcher (my supervisor) to lead and execute research across multiple phases of the product lifecycle.
Instructional Design Team
Defines how course content should be structured and delivered.
RESEARCH PLAN
Defining the research questions based on current knowns, unknowns, and stakeholder assumptions.
Research Goals
Goal
Understand how learners experience NN/G's course content and components in the new self-paced format.
Goal
Identify strengths and opportunities for improvement across course components (e.g., videos, activities, written content, assessments, and downloads).
Research Questions
Practicality
Which parts of the course do learners perceive as most valuable? How do those experiences influence perceived practicality?
Content Consumption
How are people engaging with our courses? Which course components do people skip?
Engagement Patterns
How quickly are people moving through the courses? How well are learners able to stay oriented in the courses when navigating through different sessions?
Utility
How do learners perceive the usefulness of each course components? What format best support their effectiveness?
METHODOLOGY
Understanding content consumption and engagement patterns over time through a diary study.
The Instructional Design team based early course components on competitor examples, but we needed to validate whether those patterns translated to NN/g’s content, audience, and learning objectives. Additionally, because our goal was to observe content consumption patterns, we needed to study learners in a self-paced context where progress unfolds over time. Thus, a longitudinal diary study was used to evaluate whether benchmarked patterns translated into effective learning experiences, and how learner engagement and perceptions changed across sessions.
RECRUITMENT
Recruited participants based on self-paced experience and interest in course topic.
Participants were selected based on inclusion criteria relevant to self-paced learning. Within this pool, participants were intentionally balanced across the sample composition below. Standard exclusion criteria (e.g., non-UX practitioners, technical incompatibility) were applied to ensure reliability and validity.
Recruitment Plan
Recruitment Source
Research panel for current customers
Recruit through User Interviews for non-customers
Sample Size
15 participants
5 participants for each course
1 pilot for each course
Sample Composition
NN/G Familiarity
10 customers and 5 non-customers to capture differences in prior brand familiarity.
Self-Paced Experience
11 existing SP course learners and 4 new learners to compare experienced vs first-time self-paced learners.
SP Provider Exposure
Understand how prior norms shape perceptions and expectations of learners.
Market Representation
Account for regional and contextual differences.
SESSION APPROACH
Created discussion guides to support reliability, validity and reduce the Hawthorne effect.
Intro Interview
I used a semi-structured, funneling approach to build rapport and allow participants to speak broadly before probing into specifics. When possible, participants shared their screens to walk through competitor courses they had taken, which helped ground expectations in concrete examples. The final 10–15 minutes were used to observe initial reactions to the overview page and their approach to the first module.
Contextual Inquiry
For the contextual inquiry, I asked participants to pause at a predefined point in the course (e.g., Lesson 4.2). I observed two consecutive lessons, starting with a standard content lesson and then moving into the associated practice. I did this so participants could first get comfortable thinking aloud before reaching the practice activity, which was the main focus of observation. Observing two lessons also helped prevent signaling which part of the course I was most interested in, resulting in more natural behavior. Facilitation was split with my supervisor, and we aligned in advance on when to observe versus when to probe to ensure consistency across sessions.
Final Interview
The final interview was more exploratory and reflective, focusing on participants’ overall assessment of the learning experience. I asked participants to reflect on their diary entry submissions, which course components felt most and least valuable, and how the experience compared to other self-paced courses they had taken.
CHALLENGE
Pilot testing showed that the study duration was not enough for participants to complete the course.
I discussed this with stakeholders and explained that shortening the study risked losing high-quality data, since meaningful feedback depended on participants fully progressing through later modules and completing the course. We considered removing the mid-course contextual inquiry, but this was not viable because a key research goal was to understand how learners engaged with practice activities in their natural context, which was also a priority for the instructional design team.
Additionally, to address timeline concerns, I proposed analyzing data in parallel while the study was ongoing and coordinating facilitation responsibilities with my supervisor to ensure synthesis could be completed by the final deadline. By clearly articulating these tradeoffs and presenting a concrete plan to manage timelines, I was able to align stakeholders and secure buy-in to extend the study.
Personal Takeaways
I strengthened my decision-making skills by weighing tradeoffs, and learned that clearly explaining my rationale, along with a concrete plan for managing timelines, helped me secure stakeholder buy-in for an adjusted budget.
Celine sent a message.
1m
Here's a sneak peak on how I created the tagging system for analysis on Dovetail!
FINDINGS
Framing insights into actionable, team-specific recommendations.
This study generated many findings across course components. To keep this case study focused, I present one representative course-component finding and show how it was translated into a concrete recommendation for a specific team.
Videos are easier to comprehend and retain when they are paired with examples, visuals, and practical guidelines.
Do's
Simplified visuals are clearer and faster to understand because they isolate the element being discussed.
Do's
Real-world examples add context once the basics are clear. Well-known products make it easier to understand and follow for international audiences.
Don'ts
Avoid small, unreadable fonts in screen recordings. The text needs to be readable from a smaller screen size without expanding it to the full screen.
Personal Takeaways
I learned to frame findings and recommendations with a clear audience in mind. By always asking which team would use a given insight, I was able to make recommendations more actionable and easier to adopt, while avoiding over-solutioning or stepping into others’ ownership.
WORKSHOP
Created a prioritization workshop to re-align on unknowns or knowns for next phase of research.
I chose to run a prioritization workshop to move from sharing findings to deciding what we should study next. I introduced a set of draft research questions based on unanswered questions from earlier studies. From there, stakeholders were prompted to add, refine, or challenge these questions together, resulting in a shared set of research questions created collaboratively. Once finalized, we mapped and prioritized the questions using a quadrant exercise to identify which areas were most critical to address next. Additionally, this format worked particularly well because NN/G has a high level of UX maturity and many stakeholders are experienced UX practitioners. Participants were comfortable generating and forming a discussion on research questions, which made the workshop more productive and focused on decision-making.
Personal Takeaways
Always tailor your workshop design to stakeholder expertise and organizational maturity!
IMPACT
Translating insights into actionable recommendations that help inform data-driven decisions.
Wrote self-paced course creation guidelines now used across teams.
I updated the existing instructional design guidelines to replace assumptions with evidence-based best practices. I authored the updates and collaborated with the Instructional Design Lead for review and refinement. The revised guidelines are now used by instructors, visual designers, UI designers, and the video production team to create more consistent and effective self-paced courses.
Informed design decisions in NN/G's actual platform.
These findings helped the product design and instructional design teams understand which components supported learning, which were frequently skipped, and how different activities influenced engagement. Because the courses were tested in Articulate Rise as a prototype, the insights also directly informed how course components and structures should be designed in NN/G’s own platform.
RETROSPECTIVE
Final reflections and learnings.
Learning to make decisions across competing priorities and constraints.
This project reinforced the importance of making timely decisions under real constraints and clearly weighing what would be gained or lost with each option. In my case, I had to assess whether shortening the study would compromise data quality and whether extending the timeline and increasing incentives was justified. I also learned to communicate clearly to gain stakeholder buy-in.
Facilitating shared decision-making through prioritization workshops.
I learned that workshops are effective for bringing stakeholders into a shared discussion to evaluate the findings together, surface assumptions, and contribute their perspectives to the decision-making process. This workshop also gave my research greater visibility across teams, which was important for ensuring insights were understood, trusted, and are used to make decisions. Additionally, I was able to align on next steps and scope future research phases.
Celine sent a message.
1m
I am currently working on Phase 5 of this study. Feel free to reach out if you’d like to learn more about my work at Nielsen Norman Group!
