Conducting Usability Testing for eLearning Platforms: 9 Steps

Usability testing for eLearning platforms sounds intimidating until you’ve actually run one session. I’ve been on both sides of this—watching learners struggle with a “simple” navigation flow and then realizing the issue wasn’t the content at all. It was the interface, the labels, the missing step, the tiny button that blends into the page.

So if you’re working on an LMS or LXP (or even a mobile learning app), this is for you. I’m talking about the real scenarios: logging in, finding a course, starting a lesson, submitting an assignment, checking grades, and getting help when something goes wrong. In my experience, those are the moments where usability either saves the learner… or loses them.

In a recent LMS redesign I helped with, we found that learners could “complete” tasks on paper, but they couldn’t find the next action. After we fixed 3 navigation issues and clarified 2 labels, support tickets dropped noticeably and course sessions became way more consistent.

By the end of this post, you’ll have a practical, step-by-step usability testing process you can run yourself—plus clear metrics and decision rules so you’re not guessing what to change.

1. Conduct Usability Testing to Improve eLearning Platforms

Usability testing is how you stop debating and start seeing what learners actually do. Not what they should do. What they do when they’re confused, tired, on a phone, and trying to meet a deadline.

Here’s how I set it up in a way that’s useful (and not just “we watched someone click around”).

Define goals that map to real learner tasks

Instead of “test usability,” pick 2–4 goals tied to specific workflows. For example:

• Navigation: Can learners find the next lesson within 60 seconds?
• Assignments: Can they locate “Assignment 2,” upload a file, and submit?
• Support: If they fail, do they know where to get help?
• Mobile usability: Can they complete the same tasks on iOS/Android without breaking flow?

Use representative users (and be honest about who they are)

I usually aim for two groups when possible:

• New learners: first-time users who don’t know your terminology.
• Returning learners: people who’ve used an LMS before, so we can see what’s “obvious” vs what’s truly confusing.

In many projects, that split alone reveals half the issues.

Run the session with a task script (example included)

During the test, don’t just ask them to “explore.” Give them a reason to act. Example task script for a typical course page:

• Task 1 (Start learning): “Log in and find the course ‘Intro to Data Literacy’. Start Lesson 1.”
• Task 2 (Submit work): “Find Assignment 2. Upload a file and submit it.”
• Task 3 (Check progress): “Where would you look to confirm your submission was received?”
• Task 4 (Get help): “If you had a question about grading, where would you go?”

Then measure what matters:

• Task success rate: Did they complete it without help?
• Time-on-task: How long did it take?
• Error rate: How many wrong clicks or dead ends?
• Assistance needed: Did they need hints? How many?

Capture evidence (and don’t rely on memory)

In my sessions, I record screen + audio (Zoom/Meet works fine). I also note:

• Where they hesitated
• What they said they expected to happen
• Any “I thought this was…” moments

Using screen recording helps you replay exactly where the confusion started—and it’s way easier to explain findings to stakeholders when you can show the clip.

Make a decision rule before you start

Otherwise you’ll end up with a pile of feedback and no clear next steps. Here’s a simple rule I’ve used:

• Fix immediately if >30% of users fail a critical task (like submitting an assignment) or if average time-on-task is more than 2x your target.
• Fix in next sprint if users succeed but show repeated confusion (e.g., 2+ navigation detours).
• Monitor if issues are rare, cosmetic, or users recover quickly.

2. Understand the Importance of Usability in eLearning

Usability isn’t “nice to have” in eLearning. It’s the difference between learners staying in flow and learners hitting friction at every step.

When a platform is easy to navigate, learners spend more time on learning and less time figuring out where things are. And when learners can’t complete tasks (or don’t know what to do next), motivation drops fast.

What I look for (because it affects outcomes)

• Clarity: Do labels match how learners think? (For example, “Submit” vs “Upload” vs “Turn in”)
• Feedback: After submission, do users get a clear confirmation?
• Recovery paths: If something fails (file upload error), can they fix it without starting over?
• Consistency: Are navigation patterns the same across lesson pages, assignments, and grade views?

About those big completion/engagement percentages you sometimes see online—those numbers vary by industry, course type, and measurement method. I don’t like using them blindly. Instead, I prefer to base decisions on your funnel metrics (more on that in Step 3) and what learners experience during testing.

Mobile isn’t a separate project—it’s part of usability

If you support mobile, usability testing should include it. I’ve seen “desktop-friendly” layouts fail on mobile because of:

• scroll depth issues (important buttons are too far down)
• too-small tap targets
• poor handling of file uploads

Even if your course content is great, mobile friction can break the learning rhythm.

3. Identify Key Quality Measures for Usability Testing

If you don’t define quality measures, usability testing turns into vibes. You’ll hear “it felt confusing” and “the layout is fine,” but you won’t know what to fix first.

Here are the measures that consistently show up in eLearning usability work.

User satisfaction (but make it actionable)

Use a short post-task rating like:

• Overall ease (1–5)
• Confidence (“I know I completed the task correctly,” 1–5)
• Frustration (1–5)

Then connect it to tasks. If satisfaction is low specifically around assignments, you’ve got a targeted improvement area.

Task success rate (your “must not fail” metric)

For each task, define success clearly. Example:

• Success: learner uploads a file and sees a “Submission received” confirmation
• Failure: learner cannot submit, or confirmation is unclear

Decision rule: if task success is <80% on a critical task, treat it as high severity.

Time-on-task (and what “too long” means)

Timing is useful when you set expectations. Example targets:

• Start lesson: < 60 seconds
• Find assignment + submit: < 3 minutes
• Locate help/support: < 45 seconds

In my experience, learners tolerate small delays. They don’t tolerate uncertainty. Time-on-task helps you spot both.

Error rate and “wrong path” counts

Instead of counting every click, focus on meaningful errors:

• clicking the wrong navigation item repeatedly
• dead ends (pages that don’t lead anywhere)
• misinterpreting labels (e.g., “Grades” vs “Assessments”)

Navigation efficiency (especially in course flows)

Track:

• number of page transitions needed to complete a task
• how often learners backtrack
• how many times they look for the same button

Quant data from analytics (use it to validate usability findings)

Here’s what I’d pull from analytics tools like Google Analytics (or your product analytics platform):

• course entry → lesson start drop-off
• assignment page views → submission events
• help button clicks and support article views
• file upload failures (if you track them)

Then compare it to usability test observations. If analytics show a drop after “assignment page,” and learners in testing can’t find the upload button—now you’ve got a strong, evidence-based story.

4. Choose the Right Methods for Usability Testing

Picking the right usability testing method is basically picking the right kind of evidence.

Moderated testing (best for “why”)

Moderated sessions work great when you need depth—especially for tricky workflows like grading, submission rules, or navigating between modules. I like using moderated testing early in the process because you can probe:

• “What were you expecting to happen here?”
• “What made you click that?”
• “If you had to teach someone this step, what would you say?”

Typical size: 5–8 participants per round is often enough to uncover the majority of major issues.

Unmoderated testing (best for “how often”)

Unmoderated tests are useful when you want more data quickly—especially for comparing variants. Users complete tasks on their own time, and you analyze recordings afterward.

Typical size: 15–30 participants for unmoderated rounds, depending on how many tasks you include.

Remote testing (best for real-world diversity)

Remote testing lets you include people across devices and locations. If your platform is global, this matters. In my experience, remote testing also surfaces device-specific issues faster (keyboard behavior, mobile upload quirks, browser differences).

Optional: A/B testing for UI changes

If you have two layout or label options, A/B testing can confirm what usability testing suggested. Example: testing two versions of an “Assignments” navigation label—“Assignments” vs “Due Today.”

Decision rule: pick a success metric like assignment page-to-submission conversion and run until you have enough data to be confident.

5. Set Up Effective Usability Tests

Setup is where most teams either save time… or accidentally create a test that can’t produce useful results.

Write task scripts like you’re training a new tester

For each task, include:

• the exact starting point (e.g., “logged in, on course home”)
• the goal (“submit Assignment 2”)
• the allowed help (none, or “you can ask once”)
• the success confirmation (what screen proves success)

Example task definition (copy/paste ready):

• Task: “Submit Assignment 2.”
• Start: user is on course page after login.
• Success: sees “Submission received” confirmation and the submission appears under “My Submissions.”
• Allowable hint: if stuck > 2 minutes, tester can say “Try looking for the upload or submit button.”

Choose your success criteria before the first session

Decide what you’ll call “good enough” for each task. Example:

• Critical task success: ≥ 80% completion without help
• Time-on-task: median ≤ 3 minutes for assignment submission
• Confidence score: average ≥ 4/5 after submission

Prepare the testing environment

Make sure the test environment matches real conditions as much as possible:

• disable unrelated notifications
• use a realistic course structure (modules, assignments, due dates)
• test on the target browsers/devices ahead of time

I also do a quick “dry run” with one colleague. If the tester can’t find the “next step” quickly, the real participant won’t either.

Plan incentives and logistics

Compensation matters. It improves participation and reduces drop-offs. Even small incentives help. I usually budget for:

• time (30–60 minutes per participant)
• incentive delivery (gift card or payment)
• backup scheduling (because people miss sessions)

6. Conduct the Usability Test with Real Users

Once the test is set up, the session itself needs structure. Otherwise, you’ll get recordings that are hard to interpret.

Recruit with clear criteria

Don’t just recruit “students.” Define it. Example criteria:

• Role: student / instructor / admin (pick the one that matches your testing goals)
• Experience: novice vs LMS-experienced
• Device: at least some mobile users if mobile matters
• Accessibility needs: include users who use assistive tech if that’s relevant to your audience

Use a think-aloud approach (with guardrails)

In moderated sessions, I ask participants to “talk through what they’re thinking.” If they go quiet, I prompt with neutral questions like:

• “What are you expecting to see next?”
• “What makes you think this is the right option?”
• “Is this what you thought it would look like?”

It’s not about leading them. It’s about capturing their mental model.

Take notes without steering the outcome

During the session:

• note hesitation points (timestamps help)
• record what they tried right before the issue
• avoid saying “you’re supposed to click…”

If you need to intervene, do it consistently (for example, after 2 minutes of no progress).

What I usually measure in-session

• time-to-first-action (do they know where to start?)
• number of navigation detours
• assistance count (how many hints were needed)
• final success and confidence

7. Analyze Results and Implement Changes

Here’s where most teams fall apart: they collect feedback but don’t translate it into decisions.

When I analyze usability tests, I organize findings by:

• Severity: how badly it affects completion
• Frequency: how many users hit it
• Impact: what funnel step or task it blocks
• Evidence: clips, timestamps, and quotes

Prioritize with a simple scoring model

You can do this in a spreadsheet. Example:

• Severity (1–5)
• Frequency (1–5)
• Impact (1–5)
• Total score = priority

Decision rule: fix anything scoring above your threshold (for example, 12+), especially if it blocks critical tasks like submission.

Document each change and re-measure

Don’t just “make improvements.” Track outcomes.

• Before: task success rate for “submit assignment” (e.g., 60%)
• After: same task success rate (e.g., 85%)
• Before/after median time-on-task (e.g., 4:30 → 2:45)

Even one usability round after changes can validate you didn’t introduce new confusion.

Communicate findings in a way stakeholders can act on

I always include:

• 1–2 sentence summary of the issue
• the exact task it affects
• what users expected vs what happened
• recommended fix (and why)

8. Test Performance for Platform Scalability

Usability and performance are tied together more than people think. A slow page doesn’t just feel bad—it changes behavior.

During peak usage (enrollment surges, cohort starts, assignment deadlines), performance issues can look like usability problems because learners can’t find buttons that aren’t loading.

Run load tests and watch the right metrics

Start by simulating high traffic using tools like Apache JMeter. During those runs, monitor:

• response times (p95 is often more useful than averages)
• error rates (HTTP 4xx/5xx)
• session timeouts
• server downtime

Validate with a small real-user check

After load testing, I like doing a quick usability sanity test with a small group under similar conditions. Even 3–5 users can confirm:

• does the “submit assignment” flow still work?
• does the confirmation page show up reliably?
• can users recover if an upload fails?

Infrastructure changes should have a usability outcome

If you update caching, databases, or CDN settings, verify that learners feel the improvement in the exact workflows you tested.

9. Maintain Usability Over Time for Continuous Improvement

Usability doesn’t “finish” after launch. New features ship. Labels change. Content structures evolve. And learners adapt… until they don’t.

Set a cadence (and stick to it)

In practice, I recommend:

• Lightweight checks: every 4–6 weeks (quick review of analytics + a few support tickets)
• Full usability rounds: every quarter or after major UI changes
• Targeted retests: whenever you change critical flows (login, assignments, progress tracking)

Use thresholds to trigger a retest

Don’t wait for “something feels off.” Trigger a retest when metrics cross a threshold. Examples:

• assignment submission conversion drops by >10% week-over-week
• median time-on-task increases by >20%
• help article views spike for the same topic

Build a feedback loop inside the platform

Instead of relying only on surveys, add in-context feedback:

• “Was this step clear?” after assignment submission
• “Report a problem” on course navigation
• quick “what went wrong” prompts when errors happen

Train your team—so usability stays consistent

Make usability part of your workflow: design reviews, QA checklists for task success, and accessibility checks. Otherwise, usability regressions creep in quietly.

FAQs