Environment & Configuration

Best Practices Guide for Environment and Configuration

1. Align Test Environment with Purpose

• Site-Specific Feature Tests

If you are running feature tests tailored to your institution’s REDCap instance (e.g., custom External Modules, hooks, or unique configurations), your test environment should mirror production as closely as possible:

• Same REDCap version

• Matching PHP, MySQL/MariaDB, and web server configurations. Exception might be authentication. Likely want to use database accounts.

• Similar External Modules and settings enabled.
  This ensures tests accurately reflect real-world behavior and reduces false positives/negatives. NOTE: RCTF support for global external module enable / disable not yet supported.

• Consortium-Wide RSVC Tests
  If your goal is to contribute to RSVC (REDCap Standardized Validation & Compliance) tests, you do not need a production-like environment.
  Instead, use the official redcap_docker repository maintained by Vanderbilt:

• https://github.com/vanderbilt-redcap/redcap_docker standardized REDCap instance ideal for writing and validating RSVC tests without institutional dependencies.

---

2. Managing Test Environments

• Use Dedicated Test Instances
  • Never EVER run Cypress tests against production REDCap environments.
  • Maintain separate environments for development, staging, and CI/CD pipelines.
• Automate Environment Setup
  • Use scripts or containerization (e.g., Docker) to provision REDCap test environments consistently.
  • Make sure to point Cypress at the correct MySQL instance.
• Version Alignment
  • Ensure REDCap version in test environments matches the target deployment version.
  • More version guidance located here
• Environment Isolation
  • For parallel test execution, consider ephemeral (i.e. throw away) environments (e.g., Docker + CI runners).

---

3. Handling Secrets and Environment Variables Securely

• Use .env Files with Caution
  • Store sensitive values (API tokens, REDCap credentials) in .env files, but never commit them to Git.
  • Add .env to .gitignore.
• Leverage Cypress env Configuration
  • Define non-sensitive defaults in cypress.config.js.
  • Inject secrets at runtime via CI/CD environment variables.
• CI/CD Secret Management
  • Use your CI/CD platform’s secret vault (e.g., GitHub Actions Secrets, GitLab CI Variables).
  • Rotate credentials regularly and enforce least privilege.
• Avoid Hardcoding
  • Never hardcode credentials or URLs in test specs or configs.
  • Use Cypress.env('VAR_NAME') for dynamic access.
• Audit and Logging
  • Ensure test logs do not expose secrets.
  • Mask sensitive values in screenshots or video recordings.

---

4. Recommended Folder Structure

Institution Specific Tests or RSVC Feature Tests

redcap_cypress_docker/
├── redcap_cypress/
├── ├── cypress/
│   ├──   ├── features/           # Default test feature folder
│   ├──   ├── fixtures/           # Test data
│   ├──   ├── support/            # Custom commands
│   ├──   ├── cypress.config.js   # Cypress configuration
│   ├──   ├── cypress.env.json    # Environment-specific configs
│   ├── redcap_rsvc/              # RSVC Feature tests are contained here

External Module Tests

redcap_cypress_docker/
├── modules/                                 # External Modules mounted for testing inside the container
│   ├── some_module_v1.0.0/                  # One EM version per folder (pins config & behavior for reproducible tests)
│   ├──   ├── automated_tests/               # RCTF-specific test features that live alongside the module
│   ├──   ├──   ├── commands/                # Custom Cypress commands (e.g., login helpers, reusable UI actions)
│   ├──   ├──   ├── files/                   # Static test data files consumed by feature tests
│   ├──   ├──   ├──   ├── cdisc_files/       # Domain: CDISC reference files used for validation (ODM XML files)
│   ├──   ├──   ├──   ├── dictionaries/      # Controlled data dictionaries that are imported within feature tests
│   ├──   ├──   ├──   ├── import_files/      # Input payloads for REDCap imports (records, metadata, instruments)
│   ├──   ├──   ├── step_definitions/        # Custom Step Definitions for RCTF: bind Gherkin steps to Cypress implementations

5. Additional Tips

• Use Feature Flags / Action Tags for running specific types of features OR toggling REDCap modules or EMs in tests

Command Run Example:

cypress run --env tags="@important"

Sample Feature with flag:

@important
Feature: Very Important REDCap Feature

  As a REDCap end user
  I want to see this very important feature is functioning as expected

  Scenario: Important Functionality in Action
    Given I login to REDCap with the user "Test_Admin"
    When I click on the link labeled "Control Center"
    And I click on the link labeled "Administrator Privileges"
    Then I should see "Set administrator privileges"
    And I should see a table header and rows containing the following values in the administrators table:
      | Administrators          |
      | Test_Admin (Admin User) |

We could also use to disable and enable features within REDCap ...

No tag on this run:

cypress run

We could configure RCTF to detect EMs as enabled or disabled globally:

@enableEM
Feature: External Module Behavior

  Scenario: EM is active
    Given the EM is enabled
    When I perform an action that depends on the EM
    Then I should see EM-specific behavior

@disableEM
  Scenario: EM is inactive
    Given the EM is disabled
    When I perform the same action
    Then I should NOT see EM-specific behavior

• Consider snapshot Testing for REDCap UI changes to detect regressions.
• Consider network stubbing for external API calls to avoid dependency on external systems. Example: https://github.com/vanderbilt-redcap/redcap_docker/blob/main/docker-compose.yml