Enterprise Product Design & Research

MedBridge Learning Management System

  • Timeframe: 10 Months (2019-2020)
  • Personal role: Lead product designer and researcher
MedBridge LMS

The Learning Management System team at MedBridge provides education and training tools for medical professionals. These tools help subject matter experts create educational experiences using their own custom content as well as MedBridge’s library of original instructional videos.

My Role

I led the design for the LMS product line, from discovery research through execution, across multiple projects. I collaborated with engineering leaders and program management on product strategy and execution.

People and Products

An Ecosystem of Legacy Products

The learning management system is primarily used by large medical providers to manage the training of their staff. From physical therapists to nurses, these clinicians must complete periodic training to satisfy licensing requirements. This product is also used to onboard new employees and get them up to speed on their company’s best practices. Administrators at these healthcare companies manage the training and licensing across thousands of employees and are responsible for clinician education.

The learning management system includes multiple products to address each part of the learning experience. Each tool is an independent part with distinct user groups. Use is primarily split between people creating content, people consuming that content, and the people managing the process.



Business Context

Beyond LMS, MedBridge has multiple product lines that include original educational content for both clinicians and patients. There are additional products for delivering, managing, and analyzing patient and clinician progress.

The learning management system helps clients deliver MedBridge’s core product of educational content to thousands of their clinicians. MedBridge sees this product as a hub for customer interaction. As LMS becomes an integral part of the customer’s business operations, we are better able to retain customers' subscriptions across complementary product offerings.

The Challenge

The Design Challenge: Scale & Impact

The learning management system needed an update. The product suite was built early in the company’s startup phase, when time and resources were scarce. Usability issues, technical bugs, and an accretion of features has led to a cumbersome and dated system. As clients began switching to competitor’s products, MedBridge chose to invest in a complete overhaul of the product offering in pursuit of three goals:

  • Increase utilization of the platform. Grow the number of Knowledge Tracks (our term for collections of educational content) created in the platform. Increase the rate of completion among all participants.
  • Support larger clients who operate at scale. Tools intended to manage 25 learners are insufficient when managing thousands. Our existing product did not support the kinds of bulk automation needed at large organizations.
  • Provide a technical and experiential foundation for future expansions of the product line. There were many planned feature additions, but first we had to create an extensible system to support basic features while keeping focused growth areas in mind.

Discovery Research

Understanding Education’s Role In Healthcare

As a team, we began the project without clear expectations or priorities. Stakeholders had varied and conflicting assumptions, and there was no cohesive product vision. This left an opportunity for me to influence the product strategy by bringing user-oriented evidence to the discussion.

Discovery Research Goals
  • Learn more about how people use LMS products
    • What are people using Knowledge Tracks for?
    • What business purposes do learning experiences serve?
    • Where do people spend time in our tools, and where is time wasted?
  • Understand the larger context of use
    • What are the roles involved with designing, creating, and managing education content?
  • Discover specific issues and needs
    • What are the motivations behind existing feature requests?
    • Identify confusing concepts usability issues
Research Methods

My research focused primarily on remote interviews and user testing of the existing product. This was supplemented with customer requests that had been gathered from feedback forms embedded in the existing product.

When conducting research interviews, I approached education administrators at both our major clients who operate at scale, as well as administrators at smaller companies who represented the long tail of our customer base. These administrators are not only the users of the content authoring tools that comprise the bulk of the software suite, but are often the purchase decision makers for their company. Including them in research served a secondary purpose of reassuring frustrated customers that progress is being made.

Following the interviews, a knowledge board was created to track evidence and synthesize insights.

Insights

Managing Education At Scale

Primary Insights

LMS = Content x People x Outcomes. Each learning experience requires creating content, selecting learners, and tracking their outcomes. No part of the process stands in isolation, and a deficit in any part quickly renders the learning experience meaningless. All aspects of the LMS ecosystem must be considered in product decisions.

The product equation: Every use case comes down to combining content, people, and outcomes

Unsupported processes require time-consuming and error prone manual work. Based on expert interviews and user testing, we know that savings in time are highly valued. Several tasks like scheduling and followup are not supported and must be manually completed using offline processes like spreadsheets and calendar reminders. There is a tremendous opportunity to create savings in time for customers with large teams by addressing our assignments and outcomes tools.

Knowledge Track Creator usability issues create a perception of low quality. Our content creation tool is buggy and tedious to use, but fundamentally sound in its structure. As the tool our customers spend much of their time in, usability issues lead to perceptions of low quality that have an outsized impact on subscription renewals. MedBridge’s urgent desire to rebuild this feature is largely based on remedying perceptions of poor quality and retaining customers, rather than on creating user impact.

Deeper insights

While some conclusions quickly became apparent, synthesis and followup interviews revealed deeper insights around organizations’ behavior.

Chain of Command: Administrators typically identify learners who require followup. This is often due to missed deadlines or low test scores. Administrators will forward this information to the learner’s direct manager, who then follows up either in person or by email. This is a time consuming, manual process that does not scale as more learners are added to the organization.

Supporting Scale: Manual tasks that work for 25 learners can’t scale to 1000 learners. At scale, it’s not possible to manually check whether each participant is on track to complete their training on time. Policies for alerts and reporting could automatically identify students who need followup and send them appropriate reminders.

"It's hard to police 150 new grads." followup has to be delegated to another clinician. "It should be automated."
-P.I.

The student groups tool helps manage complex assignment tasks, but is not enabled for most administrators. An updated groups tool could help administrators quickly assign relevant training to students and analyze their performance.

Assignments: Having too much assigned work overwhelms learners. This causes administrators to stagger assignments, a manual process that is coordinated with pen and paper or spreadsheets. Basic scheduling tools could automate this work, saving time and decreasing errors.

Two Needs: Two primary use cases for Knowledge Tracks emerged: Single courses and onboarding. The LMS was designed for one-time learning experiences that are created, assigned once, and discarded completely. For organizations onboarding new employees weekly, Knowledge Tracks are continuously updated and assigned to new learners. This rolling update schedule created issues when tracking which employees had completed a given version of the content.

“We have to spend a lot of time trying to go back and figure out [completions] when hopefully there's a way for the system to just produce that for us.”
-P.M.

Recommendations

From these insights came a set of high level recommendations.

  • Holistically address the entire process of creating, assigning, and following up on Knowledge Tracks.
  • Support manual processes like scheduling and auditing to reduce the most time-consuming activities.
  • Modernize existing tools with better usability and clear workflows.
Reframing the problem

The redesign of the platform was reframed holistically using a “How might we…” statement that provided a high level design prompt.

How might we help businesses create, deliver, and assess their educational content?




Product Strategy

A Major Change In Direction: Redesigning Knowledge Track Creator For Content Authoring

A product strategy decision

Given the evidence that assigning Knowledge Tracks was the primary pain point and highest impact opportunity for the effort, the team investigated this challenge first. But several weeks into the redesign, a major technical dependency was discovered that required engagement from another team. Because of roadblocks in obtaining this resource, we decided to find another opportunity to impact this space.

While the assignment aspect of the LMS was causing pain for administrators, some of that difficulty came as a result of the Knowledge Track Creator. While its usability issues were highly visible, smaller process issues like lack of reusability or ability to manage updates made the system inflexible and created manual tasks for administrators.

We chose to pivot to a rebuild of the existing Knowledge Track with three main goals in mind: refactor the underlying technology, rebuild existing functionality with improvements, and improve overall usability and visual design.

What are knowledge tracks?

Our customers want to create learning experiences for their employees. Knowledge Tracks are containers for educational content. Education administrators can add content from MedBridge’s own library of training videos, as well as include their own custom content in several formats.

Addressing user pains in the old Knowledge Track builder

Several issues were apparent and needed to be addressed in the rebuild.

  • The old Knowledge Track builder used a monolithic code base, requiring a complete technical rebuild before making significant changes.
  • Content was created with a use-once mentality. In the old Knowledge Track builder, as content was created, each section was given a due date. This worked for one-time content, it was impossible to have multiple cohorts using the same Knowledge Track on different schedules. This led to administrators creating multiple copies of the same knowledge track that could be assigned to different cohorts.
  • It was difficult to manage changes and their impacts. In the old tool, a Knowledge Track was a record of completed content. Any change made to that content marks previous assignments as incomplete. While sometimes desirable, many administrators either did not understand this change or were not adequately communicating the change to their learners.
  • The old quiz builder couldn’t reuse content, so quizzes needed to be rebuilt manually, question by question, when reused in a different knowledge track. Usability issues made it difficult to create and organize content.

 

Design Direction

My research showed that while the existing content authoring tool was buggy and confusing, its foundational patterns were sound and worth preserving. The redesign focused on improving usability, removing clutter, and improving reusability and update management.

I began designing the Knowledge Track Creator by defining an architecture that could support and integrate all future products. Working backwards from an aggressive development start date, I first specified a general plan for the app’s flow that the backend developers could begin implementing. This was followed by specific UI that the front end engineers could execute in the following sprints.

Core Features

Content Authoring: Knowledge Track Creator

Knowledge Tracks allow education administrators to build educational experiences and content and assign them to learners. Knowledge Track Creator is a content authoring tool that helps administrators use the MedBridge content library as well as their own custom content and learning assessments.

Browsing Content

The course catalog helps people add content from MedBridge’s library of training videos. Rather than using a modal window or a drawer, this content can be added to the Knowledge Track through an inline browser. This preserves the context of use and shows where the new content is being added. This is especially important on the low-resolution displays that are common among our customers.

Reordering

Research showed that building content is a creative process, not data entry. Creating content is an iterative process, often with major reorganizations. User testing showed that reorganizing content was either tedious or impossible, leading to frustration and poorly organized content.

In response, two modes of reordering were added. Content can be reordered through a drag and drop interface and placed into sections for easier organization. Since drag and drop is cumbersome when moving multiple items or when changing their position within a long Knowledge Track, sections can be reordered using arrow buttons on mobile devices.

Quiz Builder

Quizzes are essential for assessing learning and certifying comprehension. A custom quiz builder module was created to help administrators build large assessments quickly and manage them over time. Borrowing UI conventions from the rest of the application, questions are managed like content. Each question has an open and closed state, allowing quick editing while reducing unneeded controls once a question is complete. This addresses complaints about the former quiz builder with users not being able to find existing questions because too much content is hidden when not in edit mode. With necessary features available at hand, content creation is sped up. Inline validation catches issues immediately, rather than only checking when saved.

Quiz Reuse

The quiz authoring tool is used in the majority of Knowledge Tracks to assess student learning. These quizzes can take hours to create, but the content is not portable or reusable.

Anecdotally, we knew of administrators spending hours recreating long quizzes each time they created a new version of a Knowledge Track. Unsure of a need, During usability testing I added a button labeled “Quiz Bank” to a prototype. The button didn’t lead anywhere, but I received comments on this from nearly all of the participants. This proved the need and value of a quiz bank feature.

I designed a quiz bank to allow reuse in the most common use scenarios. Educators and administrators can now select content from anywhere in their organization and reuse entire quizzes or individual questions as desired. This UI reuses the course catalog browser, saving developer time and creating a more familiar tool for users.

Communicate Changes At Scale

Research interviews showed that when making major edits to knowledge tracks with participants, administrators would often send an email to participants explaining those changes. This required exporting the name of each participant, correlating that to an email address, and manually sending a notification email. In the new Knowledge Track Creator, attempting to save changes will show the administrator how many people are being impacted by the changes and offer to send participants a customizable email before saving the changes.

Integration With The Ecosystem

Assignments and outcomes, the other main pillars of Knowledge Tracks, are now available in the main navigation, rather than being buried in a linear authoring process. The ability to quickly switch between areas allows important features like assignments to appear where they make sense, rather than being shoehorned into Knowledge Track creation for quick access.

Formerly, due dates were added while authoring the content. This is acceptable for a one-time use knowledge track, but makes it difficult to reuse Knowledge Tracks at scale when multiple groups of students are engaging with the content concurrently. The new design leaves scheduling features in the assignments tab so that knowledge tracks are dedicated to content, not to scheduling. The flexibility afforded by separating content authoring from assignment helps manage student expectations at scale.

User Testing

A Rebuild that Feels Familiar

Several rounds of user testing were completed with customers in administrator roles who use the legacy application. For the initial release, this project aimed to recreate a tool with a new UI, but with limited changes to the underlying process of creating a course.

A surprising theme emerged in qualitative feedback: The new application didn’t feel like a major change.

"I like the look of it but I'm not sure what's different necessarily"
-L.S.

Given the application’s complete redesign and rebuild, this result was initially surprising. This revealed the expert audience’s focus on their business process for developing learning content and how the application’s UI was only one small part of their professional role. This validated my strategy of keeping the underlying process stable during the transition to a new UI so there would be less for users to relearn or adapt to.

Usability tests exposed small optimization issues, but did not reveal a need for substantial changes in UI direction. Testing with our panel of experts found no unintentional process change or breaking use cases, a major concern going into testing. This success gave both the product team and the business confidence to expand our roadmap and take on further challenges.

Results

A Solid Foundation for Growth

The new Knowledge Track Creator launched in 2021 alongside a new learner experience for consuming content.

From a user perspective, anecdotal evidence shows that we are saving our customers hours at a time through simple automation features that allow reuse of content. By removing unnecessary tedium from the process, our most critical customers have more time to spend on driving their own business results.

While the initial release aimed for feature parity with the old product, the team also created a platform for future growth. From both a design and technical perspective, more opportunities are queued up and ready to execute. With fewer bugs and lower support costs, more resources are available for feature additions and adjacent product rebuilds.