Skip to content

chore(tests): Functional tests polish and skipping passwordless#20262

Open
nshirley wants to merge 1 commit intomainfrom
nshirley/functional-test-ignore-case
Open

chore(tests): Functional tests polish and skipping passwordless#20262
nshirley wants to merge 1 commit intomainfrom
nshirley/functional-test-ignore-case

Conversation

@nshirley
Copy link
Copy Markdown
Contributor

@nshirley nshirley commented Mar 25, 2026

Because:

  • Passwordless tests are continuing to fail on stage and production
  • And, there is a flaky test looking for a specific string that can be modified outside of FxA code

This commit:

  • Skips passwordless tests until they're ready
  • Updates enableRecoveryKey to check text with ignored case

Checklist

Put an x in the boxes that apply

  • My commit is GPG signed.
  • If applicable, I have modified or added tests which pass locally.
  • I have added necessary documentation (if appropriate).
  • I have verified that my changes render correctly in RTL (if appropriate).
  • I have manually reviewed all AI generated code.

How to review (Optional)

  • Key files/areas to focus on:
  • Suggested review order:
  • Risky or complex parts:

Screenshots (Optional)

Please attach the screenshots of the changes made in case of change in user interface.

Other information (Optional)

Any other information that is important to this pull request.

Because:
 - Passwordless tests are continuing to fail on stage and production
 - And, there is a flaky test looking for a specific string that can be
   modified outside of FxA code

This commit:
 - Skips passwordless tests until they're ready
 - Updates enableRecoveryKey to check text with ignored case
@nshirley nshirley marked this pull request as ready for review March 26, 2026 14:26
@nshirley nshirley requested a review from a team as a code owner March 26, 2026 14:26
Copilot AI review requested due to automatic review settings March 26, 2026 14:26
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Polishes functional test behavior by making one recovery-key assertion less brittle and by conditionally skipping passwordless API tests in environments where the feature isn’t enabled (stage/production), reducing noise from known failures.

Changes:

  • Make enableRecoveryKey’s initial status assertion case-insensitive to reduce flakiness from text casing changes.
  • Add a conditional skip for passwordless tests on stage/production targets.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
packages/functional-tests/tests/resetPassword/resetPasswordRecoveryKey.spec.ts Makes the “Not set” recovery-key status assertion case-insensitive.
packages/functional-tests/tests/passwordless/passwordlessApi.spec.ts Adds conditional skip for stage/production and applies formatting-only tweaks in the test file.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

// TODO: Remove when passwordless is enabled in production
test.beforeEach(({}, { project }) => {
test.skip(
['production', 'stage'].includes(project.name),
Copy link

Copilot AI Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The conditional skip checks project.name against stage/production, but this file will still run under the stage-chromium and production-chromium projects (and any other suffixed project names). Consider basing the condition on the targetName worker fixture (stage/production) or matching via startsWith('stage') / startsWith('production') so all related projects are skipped consistently.

Suggested change
['production', 'stage'].includes(project.name),
project.name.startsWith('production') ||
project.name.startsWith('stage'),

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems a bit excessive


test.describe('severity-2', () => {
// TODO: Remove when passwordless is enabled in production
test.beforeEach(({}, { project }) => {
Copy link

Copilot AI Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test.beforeEach(({}, { project }) => { ... }) uses an empty destructuring pattern ({}), which is flagged by ESLint's no-empty-pattern rule (enabled via eslint:recommended). Use an unused identifier instead (e.g. (_, testInfo) or (_fixtures, { project })) to avoid a lint failure.

Suggested change
test.beforeEach(({}, { project }) => {
test.beforeEach((_fixtures, { project }) => {

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Meh

Copy link
Copy Markdown
Contributor

@vbudhram vbudhram left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nshirley The passwordless test in stage failed because of an invalid config, which has since been updated. So maybe this just needs to skip for production. There are also some incoming changes in #20258, that will modify how the tests get skipped or ran.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants