fix: match tests by title instead of line number in reporter

[maieutiquer]

Summary

Playwright's test.location.line in reporters may point to the describe block instead of the actual test line. This causes "Cannot find bddTestData" errors in the cucumber reporter because the line number doesn't match the pwTestLine in bddFileData.

Root Cause

When a test is inside a test.describe() block, Playwright's reporter API reports test.location.line as the line where the describe block starts, not where the actual test() call is. For example:

// Line 4: test.describe('API Check', () => {   ← Playwright reports this line
// Line 5: 
// Line 6:   test('My Test', async () => {      ← bddFileData has this line

The cucumber reporter tries to find bddTestData where data.pwTestLine === test.location.line, but this fails because 4 !== 6.

Solution

This fix adds the test title to bddFileData and uses it as the primary matching criterion, with line number as a fallback for backward compatibility:

1. Add testTitle field to BddTestData type
2. Store test title in BddDataRenderer during generation
3. Match by test title first, then by line number in MessagesBuilder

Changes

• src/bddData/types.ts - Add testTitle: string field
• src/bddData/renderer.ts - Store test.testTitle in bddFileData
• src/reporter/cucumber/messagesBuilder/index.ts - Match by title first, line number as fallback

Testing

Tested locally with 49 BDD tests - all pass without reporter errors.

[noor-tg]

I see the same error. thanks for fixing it

[vitalets]

Playwright's test.location.line in reporters may point to the describe block instead of the actual test line.

Could you provide a reproducible demo of this behavior?
I do not encounter it.

Example test:

import { test } from '@playwright/test';

test.describe('suite 1', () => { // <-- line 3

  test('test 1', async () => { // <-- line 5

  });

});

Custom reporter:

import type { Reporter, TestCase } from '@playwright/test/reporter';

export default class MyReporter implements Reporter {
  onTestEnd(test: TestCase) {
    console.log(test.title, test.location);
  }
}

Output - test location is on line 5:

$ npx playwright test
test 1 {
  file: '/Users/vitalets/projects/playwright-issues/test/sample.test.ts',
  line: 5,
  column: 7
}

[samixchoumi]

Hello, after updating my Playwright repository from 1.58.0 to 1.59.1 without changing anything on my current test, it looks like I've the same issue (but I don't seem to understand the error message properly as when I take a look at the file in the .features-gen repo, they're all present).
Using the fix from the MR to patch it directly from node-modules, it works like a charm.

Is their any news regarding this topic ?

[maieutiquer]

I still haven't had time to provide a reproducible demo, I'm working in a private repo and I also use the fix as a pnpm patch, this is in my file ./patches/playwright-bdd@8.4.2.patch in my monorepo:

diff --git a/dist/bddData/renderer.js b/dist/bddData/renderer.js
index 25f233750e3bf85e54b19494e3b8e96d638abc40..b649435e674a6b9734590f4ff99cfef873d7a53b 100644
--- a/dist/bddData/renderer.js
+++ b/dist/bddData/renderer.js
@@ -42,6 +42,7 @@ class BddDataRenderer {
         return {
             pwTestLine: this.sourceMapper.getPwTestLine(test.pickle),
             pickleLine: test.pickle.location.line,
+            testTitle: test.testTitle, // store test title for reliable matching in reporter
             skipped: test.skipped || undefined,
             timeout: test.ownTimeout,
             slow: test.slow || undefined,
diff --git a/dist/reporter/cucumber/messagesBuilder/index.js b/dist/reporter/cucumber/messagesBuilder/index.js
index 6547cab43fc6d0eb93d6ab0e7f3db861357977b0..d210d158f1e9385682f60cafa0e1c0a633c76121 100644
--- a/dist/reporter/cucumber/messagesBuilder/index.js
+++ b/dist/reporter/cucumber/messagesBuilder/index.js
@@ -52,11 +52,14 @@ class MessagesBuilder {
         if (!bddConfig)
             return;
         const { bddData, featureUri } = this.testFiles.getBddData(test.location.file);
-        // todo: move these line somewhere else
-        const bddTestData = bddData.find((data) => data.pwTestLine === test.location.line);
+        // Match by test title (reliable) with fallback to line number (legacy).
+        // Note: Playwright's test.location.line in reporters may point to describe block
+        // instead of actual test line, so we prefer matching by title.
+        const bddTestData = bddData.find((data) => data.testTitle === test.title) ||
+            bddData.find((data) => data.pwTestLine === test.location.line);
         if (!bddTestData) {
             const filePath = (0, paths_1.relativeToCwd)(test.location.file);
-            throw new Error(`Cannot find bddTestData for ${filePath}:${test.location.line}`);
+            throw new Error(`Cannot find bddTestData for ${filePath}:${test.location.line} (title: ${test.title})`);
         }
         // Important to create TestCaseRun in this method (not later),
         // b/c test properties can change after retries

and in my package.json:

{
  "pnpm": {
    "patchedDependencies": {
      "playwright-bdd@8.4.2": "patches/playwright-bdd@8.4.2.patch"
    }
  }
}

It might be because I'm doing mostly API tests with playwright?

@vitalets this is my playwright config, if that helps in any way:

import { execSync } from 'node:child_process'
import { defineConfig, devices } from '@playwright/test'
import { cucumberReporter, defineBddConfig } from 'playwright-bdd'

export default defineConfig({
  timeout: 120000,
  expect: {
    timeout: 20000, // 20 seconds for all expect assertions
  },
  testDir: defineBddConfig({
    features: 'features/**/*.feature',
    steps: ['features/steps/**/*.ts'],
    outputDir: '.features-gen',
  }),
  /* Global setup script to run once before all tests */
  globalSetup: './global-setup.ts',
  /* Run tests in files in parallel */
  fullyParallel: true,
  /* Fail the build on CI if you accidentally left test.only in the source code. */
  forbidOnly: !!process.env.CI,
  /* Retry on CI only */
  retries: process.env.CI ? 1 : 0,
  /* Allow 1 worker on CI for stability, 2 locally for balance. */
  workers: process.env.CI ? 1 : 2,
  /* Reporter to use. See https://playwright.dev/docs/test-reporters */
  reporter: [
    ['list'],
    ['html', { open: 'never' }],
    cucumberReporter('html', { outputFile: 'cucumber-report/index.html', externalAttachments: true }),
  ],
  /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
  use: {
    /* Base URL to use in actions like `await page.goto('')`. */
    baseURL: process.env.PLAYWRIGHT_BASE_URL || 'http://localhost:3000',

    /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
    trace: process.env.CI ? 'on-first-retry' : 'on',
    video: process.env.CI ? 'on-first-retry' : 'on',
  },

  /* Configure projects for major browsers */
  projects: [
    {
      name: 'API and UI tests',
      use: {
        ...devices['Desktop Chrome'],
        channel: 'chromium',
      },
      /* Additional information for the report page */
      metadata: {
        gitHubRunId: process.env.GITHUB_RUN_ID ?? 'local',
        gitHubRepo: process.env.GITHUB_REPOSITORY ?? 'local/local',
        gitHubRunUrl: process.env.GITHUB_RUN_ID
          ? `https://github.com/${process.env.GITHUB_REPOSITORY ?? 'local/local'}/actions/runs/${process.env.GITHUB_RUN_ID}`
          : 'local run',
        gitHubCommit: process.env.GITHUB_SHA ?? 'local',
        gitBranch: process.env.GITHUB_REF_NAME ?? execSync('git rev-parse --abbrev-ref HEAD').toString().trim(),
      },
    },
  ],
})

I haven't tried updating to the latest versions yet, I'll get back to this when I can, unless anyone else would have time to provide a reproducible demo?

[vitalets]

Hi @maieutiquer , I'll make another approach to investigate.
I also updated the pnpm branch of the playwright-bdd-example, could you please try to build the minimal demo over it? You can keep it private, just share the results.

[nmokkenstorm]

running into the same issue, would you (both?) be helped with a reproducible/anonymous example?

[vitalets]

@nmokkenstorm could you please try on a fresh clone of https://github.com/vitalets/playwright-bdd-example? If it does not reproduce, add more stuff making it closer to your project where it reproduces.

[IanGraingerGMSL]

I'm seeing this issue as well. 👍 v8.5.0

[vitalets]

@IanGraingerGMSL could you try to reproduce it on playwright-bdd-example?