Answer
Maintaining 1000+ Cucumber Scenarios
The Problem: Test Rot
Test rot happens when:
- āOld scenarios test removed features
- āStep definitions drift from actual UI
- āDuplicate scenarios cover the same case
- ā@wip scenarios accumulate and never get finished
- āNo one knows which tests are worth keeping
Strategy 1: Quarterly Test Audit
Bash
# Find scenarios not tagged with a required tag (no owner)
grep -rn "Scenario:" src/test/resources/features/ | wc -l
# Find scenarios missing required tags
grep -rL "@P1\|@P2\|@P3" src/test/resources/features/**/*.feature
# Find @wip scenarios older than 30 days (Git blame)
git log --follow -p src/test/resources/features/checkout/payment.feature \
| grep "@wip"
Quarterly review checklist:
CODE
For each feature file:
ā Does this feature still exist in production?
ā Are all scenarios still relevant?
ā Are any scenarios duplicates?
ā Do all @wip scenarios have a JIRA ticket?
ā Have @flaky scenarios been fixed or removed?
Strategy 2: Automatic Flakiness Tracking
Java
// Track scenario flakiness in a results database
@After
public void trackResults(Scenario scenario) {
dbUtils.insertResult(
scenario.getName(),
scenario.getStatus().toString(),
LocalDateTime.now(),
System.getenv("BUILD_NUMBER")
);
}
SQL
-- Find scenarios that failed > 20% of runs in last 30 days
SELECT scenario_name,
COUNT(*) total_runs,
SUM(CASE WHEN status = 'FAILED' THEN 1 ELSE 0 END) failures,
ROUND(SUM(CASE WHEN status = 'FAILED' THEN 1.0 ELSE 0 END) / COUNT(*) * 100, 1) flakiness_pct
FROM cucumber_results
WHERE run_date > NOW() - INTERVAL 30 DAY
GROUP BY scenario_name
HAVING flakiness_pct > 20
ORDER BY flakiness_pct DESC;
Auto-tag scenarios with > 20% flakiness as @flaky and move to quarantine suite.
Strategy 3: Step Definition Governance
CODE
Rules enforced via CheckStyle + Code Review:
1. No Thread.sleep() in step definitions
ā CheckStyle rule: IllegalMethodCallCheck
2. No WebDriver calls in step definitions
ā All locator code must be in Page Objects
3. Step definitions max 5 lines
ā Anything longer = extract to Page Object method
4. One step definition file per feature domain
ā LoginSteps.java, not AllSteps.java
Strategy 4: Scenario Naming Convention
Gherkin
-- Good naming ā searchable, self-documenting
Scenario: Login - valid credentials - admin user
Scenario: Login - invalid password - standard user
Scenario: Checkout - payment - credit card - success
Scenario: Checkout - payment - credit card - expired
Scenario: Checkout - coupon - valid 10% discount - applied
-- Bad naming ā vague, unsearchable
Scenario: Test login
Scenario: Verify payment
Scenario: Check coupon
Strategy 5: Ownership via Tags
Gherkin
@owner-payments-team @regression @P1
Scenario: Credit card payment processes correctly
When a scenario fails:
Bash
# Find owner of failing scenario
grep -r "@owner-" failing_scenario.feature
# ā @owner-payments-team ā alert payments team Slack channel
Strategy 6: Sprint Allocation for Maintenance
CODE
20% of every sprint = test maintenance:
Week 1: Fix @flaky scenarios (top 10 by flakiness rate)
Week 2: Remove obsolete scenarios for deprecated features
Week 3: Update step definitions for UI changes
Week 4: Add missing coverage for new features
Strategy 7: Feature File "Last Updated" Header
Gherkin
# Feature: User Login
# Last reviewed: 2024-Q4 by QA Team
# Owner: @auth-team
# Linked story: PROJ-1234
Feature: User Login
...
KPIs for Healthy Suite
CODE
Target metrics for 1000+ scenarios:
Flakiness rate: < 2% (< 20 flaky scenarios)
Execution time: < 30m (parallel, not serial)
Coverage gap: < 10% (known uncovered flows)
@wip backlog: < 15 (open WIP scenarios)
Step def reuse rate: > 60% (shared steps, not duplicates)
Build green rate: > 97% (36 days of 37 pass)
