DEV Community

Cover image for Automated Testing for SCORM E-Learning Packages Using Playwright — A Step-by-Step Guide
Aditya Tiwari
Aditya Tiwari

Posted on

Automated Testing for SCORM E-Learning Packages Using Playwright — A Step-by-Step Guide

#testing #playwright #lms #scorm

Most testing tutorials ignore e-learning completely. Here's how to build a Playwright test suite that validates your SCORM packages actually work across LMS platforms.

Why E-Learning Testing Is Different

If you've ever published a SCORM package to an LMS and watched it silently fail — no completion recorded, quiz scores vanishing, navigation broken — you know the pain. E-learning content doesn't behave like a typical web app. It runs inside an LMS-provided iframe, communicates through a JavaScript API (the SCORM Runtime), and its behavior changes depending on which LMS hosts it.

Manual QA across even 3-4 LMS platforms is slow and error-prone. In this tutorial, I'll walk you through setting up Playwright to automate SCORM package testing — from basic content loading to verifying API calls and completion status.

Prerequisites

Before we start, make sure you have:

  • Node.js 18+ installed
  • Playwright (npm init playwright@latest)
  • A SCORM 1.2 or 2004 package (a .zip file containing your e-learning content)
  • A local LMS for testing — we'll use SCORM Cloud (free tier) or a simple SCORM API shim

Step 1: Set Up a Local SCORM Runtime Shim

Testing SCORM content requires an API that mimics what an LMS provides. Rather than spinning up a full Moodle instance, we'll create a lightweight shim.

Create a file called scorm-api-shim.js:

// scorm-api-shim.js
// Mimics the SCORM 1.2 Runtime API that an LMS would expose

window.API = {
  _data: {},
  _initialized: false,
  _calls: [],

  LMSInitialize: function(param) {
    this._initialized = true;
    this._calls.push({ method: 'LMSInitialize', param, timestamp: Date.now() });
    console.log('[SCORM] LMSInitialize called');
    return "true";
  },

  LMSGetValue: function(key) {
    this._calls.push({ method: 'LMSGetValue', key, timestamp: Date.now() });
    return this._data[key] || "";
  },

  LMSSetValue: function(key, value) {
    this._data[key] = value;
    this._calls.push({ method: 'LMSSetValue', key, value, timestamp: Date.now() });
    console.log(`[SCORM] SetValue: ${key} = ${value}`);
    return "true";
  },

  LMSCommit: function(param) {
    this._calls.push({ method: 'LMSCommit', param, timestamp: Date.now() });
    return "true";
  },

  LMSFinish: function(param) {
    this._calls.push({ method: 'LMSFinish', param, timestamp: Date.now() });
    this._initialized = false;
    return "true";
  },

  LMSGetLastError: function() { return "0"; },
  LMSGetErrorString: function(code) { return "No error"; },
  LMSGetDiagnostic: function(code) { return ""; }
};
Enter fullscreen mode Exit fullscreen mode

This gives us a mock API that logs every SCORM call — which becomes our test assertion layer.

Step 2: Serve the SCORM Package Locally

Unzip your SCORM package and serve it with a simple HTTP server. Create serve-scorm.js:

// serve-scorm.js
const express = require('express');
const path = require('path');
const app = express();

// Serve the SCORM API shim at the parent level (LMS frame)
app.get('/lms', (req, res) => {
  res.send(`
    <!DOCTYPE html>
    <html>
    <head>
      <title>Test LMS</title>
      <script src="/scorm-api-shim.js"></script>
    </head>
    <body>
      <iframe id="content-frame" 
              src="/scorm-content/index.html" 
              width="100%" 
              height="600px">
      </iframe>
    </body>
    </html>
  `);
});

app.use('/scorm-api-shim.js', express.static(path.join(__dirname, 'scorm-api-shim.js')));
app.use('/scorm-content', express.static(path.join(__dirname, 'unzipped-scorm-package')));

app.listen(3000, () => console.log('Test LMS running at http://localhost:3000/lms'));
Enter fullscreen mode Exit fullscreen mode

Run it:

npm install express
node serve-scorm.js
Enter fullscreen mode Exit fullscreen mode

Visit http://localhost:3000/lms — you should see your SCORM content loaded inside the iframe, with API calls logging in the console.

Step 3: Write Your First Playwright Test — Content Loads and Initializes

Create your test file at tests/scorm-basic.spec.js:

// tests/scorm-basic.spec.js
const { test, expect } = require('@playwright/test');

test.describe('SCORM Package - Basic Validation', () => {

  test('should load content and call LMSInitialize', async ({ page }) => {
    await page.goto('http://localhost:3000/lms');

    // Wait for the content iframe to load
    const frame = page.frameLocator('#content-frame');
    await frame.locator('body').waitFor({ state: 'visible' });

    // Check that LMSInitialize was called
    const initCalled = await page.evaluate(() => {
      return window.API._calls.some(c => c.method === 'LMSInitialize');
    });

    expect(initCalled).toBe(true);
  });

  test('should set lesson_status to incomplete or browsed on load', async ({ page }) => {
    await page.goto('http://localhost:3000/lms');

    const frame = page.frameLocator('#content-frame');
    await frame.locator('body').waitFor({ state: 'visible' });

    // Give the content a moment to make its initial API calls
    await page.waitForTimeout(2000);

    const lessonStatus = await page.evaluate(() => {
      return window.API._data['cmi.core.lesson_status'];
    });

    // SCORM content typically sets status to 'incomplete' or 'browsed' on launch
    expect(['incomplete', 'browsed', 'not attempted']).toContain(lessonStatus);
  });

});
Enter fullscreen mode Exit fullscreen mode

Run the test:

npx playwright test tests/scorm-basic.spec.js
Enter fullscreen mode Exit fullscreen mode

Step 4: Test Quiz Interaction Tracking

If your SCORM package has a quiz, you need to verify that interaction data is being recorded correctly. This is where most LMS compatibility bugs live.

// tests/scorm-quiz.spec.js
const { test, expect } = require('@playwright/test');

test.describe('SCORM Package - Quiz Interactions', () => {

  test('should record interaction data when answering a quiz question', async ({ page }) => {
    await page.goto('http://localhost:3000/lms');

    const frame = page.frameLocator('#content-frame');

    // Navigate to the quiz slide (adjust selectors to your content)
    // This will vary based on your authoring tool's output
    await frame.locator('[data-slide="quiz-1"]').click();

    // Select an answer
    await frame.locator('.answer-option').first().click();

    // Submit the answer
    await frame.locator('.submit-btn').click();

    // Verify interaction was recorded via SCORM API
    const interactions = await page.evaluate(() => {
      return window.API._calls.filter(c => 
        c.method === 'LMSSetValue' && 
        c.key.startsWith('cmi.interactions')
      );
    });

    // Should have at least one interaction recorded
    expect(interactions.length).toBeGreaterThan(0);

    // Verify the interaction has a valid type
    const interactionType = interactions.find(i => i.key.includes('.type'));
    expect(['choice', 'true-false', 'fill-in', 'matching', 'sequencing'])
      .toContain(interactionType?.value);
  });

  test('should calculate and commit score after quiz completion', async ({ page }) => {
    await page.goto('http://localhost:3000/lms');

    // ... navigate through quiz and answer all questions ...

    // After completing the quiz, check for score
    const scoreRaw = await page.evaluate(() => {
      return window.API._data['cmi.core.score.raw'];
    });

    const scoreMax = await page.evaluate(() => {
      return window.API._data['cmi.core.score.max'];
    });

    // Score should be a valid number
    expect(Number(scoreRaw)).not.toBeNaN();
    expect(Number(scoreMax)).toBeGreaterThan(0);
    expect(Number(scoreRaw)).toBeLessThanOrEqual(Number(scoreMax));

    // Verify LMSCommit was called after scoring
    const commitCalls = await page.evaluate(() => {
      return window.API._calls.filter(c => c.method === 'LMSCommit');
    });
    expect(commitCalls.length).toBeGreaterThan(0);
  });

});
Enter fullscreen mode Exit fullscreen mode

Step 5: Test Completion and Finish Sequence

The most common SCORM bug: content that never calls LMSFinish or sets lesson_status to completed/passed. This breaks LMS reporting.

// tests/scorm-completion.spec.js
const { test, expect } = require('@playwright/test');

test.describe('SCORM Package - Completion Flow', () => {

  test('should set lesson_status to completed/passed after full navigation', async ({ page }) => {
    await page.goto('http://localhost:3000/lms');

    const frame = page.frameLocator('#content-frame');

    // Navigate through all slides (adjust to your content structure)
    const nextButton = frame.locator('.next-btn, [aria-label="Next"]');

    let hasNext = true;
    while (hasNext) {
      try {
        await nextButton.click({ timeout: 3000 });
        await page.waitForTimeout(500);
      } catch {
        hasNext = false;
      }
    }

    // Check final lesson status
    const finalStatus = await page.evaluate(() => {
      return window.API._data['cmi.core.lesson_status'];
    });

    expect(['completed', 'passed']).toContain(finalStatus);
  });

  test('should call LMSFinish on content exit', async ({ page }) => {
    await page.goto('http://localhost:3000/lms');

    // Navigate through content...
    // Then close or navigate away
    await page.evaluate(() => {
      // Simulate unload event
      window.dispatchEvent(new Event('beforeunload'));
    });

    await page.waitForTimeout(1000);

    const finishCalled = await page.evaluate(() => {
      return window.API._calls.some(c => c.method === 'LMSFinish');
    });

    expect(finishCalled).toBe(true);
  });

});
Enter fullscreen mode Exit fullscreen mode

Step 6: Generate a SCORM API Call Report

The real power of this approach is the ability to dump every SCORM API call for debugging. Add this utility:

// tests/helpers/scorm-report.js
async function generateSCORMReport(page) {
  const calls = await page.evaluate(() => window.API._calls);
  const data = await page.evaluate(() => window.API._data);

  console.log('\n=== SCORM API Call Log ===');
  calls.forEach((call, index) => {
    const time = new Date(call.timestamp).toISOString().split('T')[1];
    if (call.key) {
      console.log(`${index + 1}. [${time}] ${call.method}("${call.key}"${call.value ? ', "' + call.value + '"' : ''})`);
    } else {
      console.log(`${index + 1}. [${time}] ${call.method}()`);
    }
  });

  console.log('\n=== Final SCORM Data Model ===');
  Object.entries(data).forEach(([key, value]) => {
    console.log(`  ${key}: ${value}`);
  });

  return { calls, data };
}

module.exports = { generateSCORMReport };
Enter fullscreen mode Exit fullscreen mode

Use it in your tests:

const { generateSCORMReport } = require('./helpers/scorm-report');

test.afterEach(async ({ page }) => {
  await generateSCORMReport(page);
});
Enter fullscreen mode Exit fullscreen mode

Step 7: Run Across Multiple Browser Contexts (Simulating Different LMS Environments)

Different LMS platforms use different iframe embedding strategies. Test your content across configurations:

// playwright.config.js
module.exports = {
  projects: [
    {
      name: 'chromium-default',
      use: { browserName: 'chromium' },
    },
    {
      name: 'firefox',
      use: { browserName: 'firefox' },
    },
    {
      name: 'webkit-safari',
      use: { browserName: 'webkit' },
    },
  ],
};
Enter fullscreen mode Exit fullscreen mode

Run all:

npx playwright test --reporter=html
Enter fullscreen mode Exit fullscreen mode

This gives you a visual HTML report showing pass/fail across browsers — which maps roughly to how your content will behave across different LMS platforms that use different embedded browser engines.

What This Gets You

With this setup, you can now:

  • Catch SCORM API bugs before publishing — missing LMSInitialize, broken completion triggers, invalid interaction data
  • Debug LMS-specific failures by examining the full API call log
  • Run regression tests every time content is updated or the authoring tool ships a new version
  • Automate cross-browser validation that would take hours to do manually

In our team, this approach cut QA time for SCORM packages by roughly 70% and caught a category of bugs — specifically, race conditions in LMSSetValue calls during quiz scoring — that manual testing had missed for months.

Next Steps

From here, you can extend this framework to:

  • Test SCORM 2004 packages (replace window.API with window.API_1484_11)
  • Test xAPI (Tin Can) statements by intercepting fetch/XMLHttpRequest calls to the LRS endpoint
  • Integrate into CI/CD with GitHub Actions so every content build gets validated automatically
  • Add accessibility checks using @axe-core/playwright alongside SCORM validation

If you're building or testing e-learning content and you've run into weird LMS bugs, I'd love to hear about them — the edge cases in this space are wild.

I'm a senior software engineer with 11 years in e-learning technology. I write about the tools and techniques behind enterprise content authoring. Find me on LinkedIn.

Top comments (1)

Collapse
 
alexshev profile image
Alex Shev •

SCORM testing is a great niche for Playwright because the important behavior is workflow state, not just DOM rendering. Launch, progress tracking, completion, suspend data, and LMS communication are all easy to miss with manual spot checks. Automating that path probably catches the failures users actually feel.