Test Fixing Complete

A systematic skill for identifying, diagnosing, and fixing failing tests. Covers common failure patterns, smart grouping strategies for batch fixes, and techniques for resolving flaky, outdated, and misconfigured tests.

When to Use This Skill

Choose this skill when:

• Multiple tests are failing and need systematic triage and fixing
• Tests broke after a refactoring and need bulk updates
• Flaky tests are undermining CI/CD reliability
• Test environment configuration issues cause failures
• Migrating test framework or updating test dependencies

Consider alternatives when:

• Writing new tests from scratch → use a TDD or testing patterns skill
• Need to detect which test framework to use → use a test detect skill
• Running performance tests → use a performance testing skill
• Debugging application code → use a debugging skill

Quick Start

Copy
# Step 1: Run all tests and capture output
npm test 2>&1 | tee test-output.txt

# Step 2: Categorize failures
# Parse output to group by:
# - Same error message → likely same root cause
# - Same file → related changes needed
# - Same fixture/setup → environment issue
# - Timeout errors → async/performance issue

# Step 3: Fix by group, not individually
# Fix the root cause → re-run → verify multiple tests pass

Core Concepts

Test Failure Categories

Category	Symptoms	Root Cause	Fix Strategy
Assertion	Expected vs actual mismatch	Code behavior changed	Update assertion or fix code
Import	Module not found	File renamed/moved	Update import paths
Timeout	Test exceeds time limit	Async not awaited, slow I/O	Add await, increase timeout
Type	TypeScript compilation error	Interface changed	Update types and test code
Setup	beforeAll/beforeEach fails	DB/service unavailable	Fix environment config
Flaky	Passes sometimes	Race condition, timing	Add waits, fix concurrency
Snapshot	Snapshot mismatch	Intentional UI change	Update snapshots

Smart Grouping Strategy

Copy
// Group failures by root cause for batch fixing
interface TestFailure {
  file: string;
  testName: string;
  error: string;
  stackTrace: string;
}

function groupFailures(failures: TestFailure[]): Map<string, TestFailure[]> {
  const groups = new Map<string, TestFailure[]>();

  for (const failure of failures) {
    // Group by error pattern
    const key = categorize(failure);
    if (!groups.has(key)) groups.set(key, []);
    groups.get(key)!.push(failure);
  }

  return groups;
}

function categorize(failure: TestFailure): string {
  if (failure.error.includes('Cannot find module')) return 'import-errors';
  if (failure.error.includes('Timeout')) return 'timeout-errors';
  if (failure.error.includes('toMatchSnapshot')) return 'snapshot-mismatches';
  if (failure.error.includes('is not a function')) return 'type-errors';
  if (failure.error.includes('ECONNREFUSED')) return 'connection-errors';
  return `assertion:${failure.file}`; // group assertions by file
}

Fixing Patterns

Copy
// Pattern 1: Bulk update assertions after intentional refactor
// Before: function returned string, now returns object
// Find: expect(result).toBe('success')
// Replace: expect(result).toEqual({ status: 'success', code: 200 })

// Pattern 2: Fix import paths after file restructure
// Automated fix with regex:
// find . -name "*.test.ts" -exec sed -i 's|from "../old-path/|from "../new-path/|g' {} +

// Pattern 3: Fix async test patterns
// Before (missing await):
it('fetches data', () => {
  const result = fetchData(); // missing await
  expect(result).toBeDefined(); // tests Promise object, not resolved value
});
// After:
it('fetches data', async () => {
  const result = await fetchData();
  expect(result).toBeDefined();
});

// Pattern 4: Fix environment-dependent tests
// Before (depends on system timezone):
expect(formatDate(date)).toBe('2024-01-15');
// After (timezone-independent):
expect(formatDate(date)).toBe(new Date(date).toISOString().split('T')[0]);

Configuration

Parameter	Type	Default	Description
failureGrouping	string	'smart'	Grouping: smart (auto), by-file, or by-error
fixBatchSize	number	10	Max tests to fix per batch
rerunAfterFix	boolean	true	Re-run tests after each fix batch
snapshotUpdate	string	'ask'	Auto-update snapshots: ask, yes, or no
timeoutMultiplier	number	2	Multiply timeout for slow environments
parallelFix	boolean	false	Fix independent groups in parallel

Best Practices

1. Fix root causes, not individual test symptoms — If 15 tests fail because a function's return type changed, fix the function (or update all 15 assertions) rather than patching tests one by one. Group by error pattern to find root causes.

2. Run tests after each fix group to prevent cascading issues — A fix for one group of failures might break other tests. Verify after each batch before moving to the next group. This prevents the "fix one, break two" spiral.

3. Update snapshots deliberately, not blindly — npm test -- -u updates all snapshots without review. Instead, review each snapshot change to verify it's intentional. Blindly updating snapshots can hide regressions in the updated output.

4. Address flaky tests as a separate category — Flaky tests need different fixes than consistently failing tests. Identify them by running the test suite multiple times. Common causes: shared state, timing dependencies, and non-deterministic data.

5. Fix environment issues at the infrastructure level — If tests fail because a database isn't running, fix the test setup (use Docker Compose, in-memory databases) rather than skipping the test. Skipped tests accumulate into untested code.

Common Issues

Tests pass locally but fail in CI — Environment differences: different Node.js version, missing environment variables, no database connection, or file system differences (case sensitivity on Linux vs macOS). Standardize with Docker and .env.test files.

Fixing one test breaks others due to shared state — Tests that share global state (module-level variables, database records, mock configurations) interact unpredictably. Add beforeEach cleanup, use unique test data per test, and reset mocks between tests.

Snapshot tests produce huge diffs after dependency updates — Component libraries update HTML output between versions, causing massive snapshot diffs. Use targeted assertions (toHaveTextContent, toHaveAttribute) instead of full-component snapshots for stable tests.