</>

Technology

Cucumber

Difficulty

Advanced

Interview Question

How do you maintain a growing Cucumber suite of 1000+ scenarios to prevent test rot?

Answer

Maintaining 1000+ Cucumber Scenarios

The Problem: Test Rot

Test rot happens when:

  • āœ“Old scenarios test removed features
  • āœ“Step definitions drift from actual UI
  • āœ“Duplicate scenarios cover the same case
  • āœ“@wip scenarios accumulate and never get finished
  • āœ“No one knows which tests are worth keeping

Strategy 1: Quarterly Test Audit

Bash
# Find scenarios not tagged with a required tag (no owner)
grep -rn "Scenario:" src/test/resources/features/ | wc -l

# Find scenarios missing required tags
grep -rL "@P1\|@P2\|@P3" src/test/resources/features/**/*.feature

# Find @wip scenarios older than 30 days (Git blame)
git log --follow -p src/test/resources/features/checkout/payment.feature \
  | grep "@wip"

Quarterly review checklist:

CODE
For each feature file:
☐ Does this feature still exist in production?
☐ Are all scenarios still relevant?
☐ Are any scenarios duplicates?
☐ Do all @wip scenarios have a JIRA ticket?
☐ Have @flaky scenarios been fixed or removed?

Strategy 2: Automatic Flakiness Tracking

Java
// Track scenario flakiness in a results database
@After
public void trackResults(Scenario scenario) {
    dbUtils.insertResult(
        scenario.getName(),
        scenario.getStatus().toString(),
        LocalDateTime.now(),
        System.getenv("BUILD_NUMBER")
    );
}
SQL
-- Find scenarios that failed > 20% of runs in last 30 days
SELECT scenario_name,
       COUNT(*) total_runs,
       SUM(CASE WHEN status = 'FAILED' THEN 1 ELSE 0 END) failures,
       ROUND(SUM(CASE WHEN status = 'FAILED' THEN 1.0 ELSE 0 END) / COUNT(*) * 100, 1) flakiness_pct
FROM cucumber_results
WHERE run_date > NOW() - INTERVAL 30 DAY
GROUP BY scenario_name
HAVING flakiness_pct > 20
ORDER BY flakiness_pct DESC;

Auto-tag scenarios with > 20% flakiness as @flaky and move to quarantine suite.

Strategy 3: Step Definition Governance

CODE
Rules enforced via CheckStyle + Code Review:

1. No Thread.sleep() in step definitions
   → CheckStyle rule: IllegalMethodCallCheck

2. No WebDriver calls in step definitions
   → All locator code must be in Page Objects

3. Step definitions max 5 lines
   → Anything longer = extract to Page Object method

4. One step definition file per feature domain
   → LoginSteps.java, not AllSteps.java

Strategy 4: Scenario Naming Convention

Gherkin
-- Good naming — searchable, self-documenting
Scenario: Login - valid credentials - admin user
Scenario: Login - invalid password - standard user
Scenario: Checkout - payment - credit card - success
Scenario: Checkout - payment - credit card - expired
Scenario: Checkout - coupon - valid 10% discount - applied

-- Bad naming — vague, unsearchable
Scenario: Test login
Scenario: Verify payment
Scenario: Check coupon

Strategy 5: Ownership via Tags

Gherkin
@owner-payments-team @regression @P1
Scenario: Credit card payment processes correctly

When a scenario fails:

Bash
# Find owner of failing scenario
grep -r "@owner-" failing_scenario.feature
# → @owner-payments-team → alert payments team Slack channel

Strategy 6: Sprint Allocation for Maintenance

CODE
20% of every sprint = test maintenance:
  Week 1: Fix @flaky scenarios (top 10 by flakiness rate)
  Week 2: Remove obsolete scenarios for deprecated features
  Week 3: Update step definitions for UI changes
  Week 4: Add missing coverage for new features

Strategy 7: Feature File "Last Updated" Header

Gherkin
# Feature: User Login
# Last reviewed: 2024-Q4 by QA Team
# Owner: @auth-team
# Linked story: PROJ-1234

Feature: User Login
  ...

KPIs for Healthy Suite

CODE
Target metrics for 1000+ scenarios:
  Flakiness rate:      < 2%   (< 20 flaky scenarios)
  Execution time:      < 30m  (parallel, not serial)
  Coverage gap:        < 10%  (known uncovered flows)
  @wip backlog:        < 15   (open WIP scenarios)
  Step def reuse rate: > 60%  (shared steps, not duplicates)
  Build green rate:    > 97%  (36 days of 37 pass)

Follow AutomateQA

Related Topics