Exploratory testing. A skilled tester with curiosity will find bugs that no automated suite catches. They try weird input combinations. They interrupt processes mid-flow. They behave like confused users, impatient users, and users who read the interface in a different language. You can't script curiosity.
At Globalbit, our exploratory testers dedicate 20% of each sprint to unscripted investigation. In the past 12 months, exploratory testing found 35% of our severity-1 bugs across all projects, all of them bugs the automated suites had missed.
First-time user experience. The onboarding flow, the first purchase, the initial setup. These need fresh eyes. Automated tests verify functionality. Manual testers verify comprehension. "Does this flow work?" is automation. "Does this flow make sense to someone seeing it for the first time?" is manual.
Accessibility with assistive technology. Automated scanners catch issues like missing alt text, low color contrast and missing ARIA labels. They miss a lot. When the UK Government Digital Service tested 10 tools in 2017 on a page with 143 planted accessibility failures, the best single tool found 41% of them. Screen reader navigation, keyboard focus order and cognitive load need a human using assistive technology.
Cross-functional workflows. When a user's journey spans multiple products or services (start on the web app, continue in email, complete on mobile), the orchestration of this journey needs human evaluation. The individual pieces might pass automated tests while the connected experience fails.
The AI-shifted middle ground
These categories were firmly "manual" until 2025. AI testing agents are making them at least partially automatable:
Visual [regression testing](/blog/fix-slow-regression-testing). AI agents compare screenshots across releases and flag unintended visual changes. Tools like Applitools and Percy do this well. What AI still misses: whether a visual change is an improvement or a regression. It catches differences; humans judge intent.
Cross-browser testing. AI agents now run the same test suite across 20+ browser-device combinations in parallel, detecting rendering differences automatically. What AI misses: browser-specific interaction patterns that users actually care about.
Performance baseline testing. AI agents establish performance baselines and alert when response times deviate. They're good at detecting regressions but poor at diagnosing root causes. A 200ms slowdown on the search API might be a database index issue, a cache miss, or an infrastructure problem. The agent flags it; a human investigates it.
Test case generation from requirements. Given a user story, AI agents generate 80-120 test cases covering happy paths, edge cases and boundary conditions. The generated tests need human review. Roughly 70% are useful, 20% are redundant, and 10% test the wrong thing. Starting with 70% useful tests still beats starting from zero.
The decision flowchart
For any test, run through these questions:
1. Does this test change between runs? If no (same inputs, same expected outputs, every time): automate it. If yes (different evaluation criteria each time): manual.
2. Does this test require judgment about quality? If it's binary (pass/fail based on expected output): automate. If it requires opinion (does this feel right? is this confusing?): manual.
3. How often does this test run? If daily or per-commit: must be automated. If monthly or per-release: consider manual if the judgment requirement is high.
4. What does it cost when this test misses a bug? If the missed bug costs $10,000+: automate AND run exploratory manual testing on the same area. If the missed bug costs under $1,000: automate alone is sufficient.
The cost reality
Here's what testing typically costs at different automation ratios, based on our planning estimates:
| Automation ratio | Monthly cost (20-dev team) | Bug escape rate | Detection speed |
|---|
| 0% (all manual) | $18,000-$25,000 | 25-35% | 3-5 days |
| 30% automated | $14,000-$20,000 | 18-25% | 1-3 days |
| 60% automated | $12,000-$16,000 | 10-15% | 4-24 hours |
| 80% automated | $13,000-$18,000 | 8-12% | 1-4 hours |
| 95%+ automated | $16,000-$22,000 | 10-15% | Minutes |
Notice the cost increase at 95%+. That's because the last 15-20% of test automation is the most expensive to build and maintain. You're automating complex scenarios that change frequently, require sophisticated visual comparison, and break often enough that maintenance eats the savings.
The sweet spot for most teams: 60-70% automation. Maximum bug detection improvement per dollar invested.
Why manual-only testing stops scaling
Manual testing grows in step with your features. A new app with 50 test cases needs one tester for one day per release. A year later, 300 test cases need three testers for a week, and tired people miss things. Automated suites don't grow that way.
Here is one calculation we ran for a mid-size enterprise client with 400 test cases and releases every two weeks:
| Manual | Automated |
|---|
| Setup | None | $120,000 for the framework and first tests |
| Ongoing | 3 QA engineers at $6,000/month plus about $4,000/month in escaped defects | 1 automation engineer at $7,500/month |
| Year one | $268,000 | $210,000 |
| Years two and three | $268,000 each | $90,000 each |
| Three-year total | $804,000 | $390,000 |
Over three years, automation cost 52% less, and the gap widens each year.
Where to spend automation effort
Follow the test pyramid:
- Unit tests, about 70% of the effort. Fast and cheap, and they catch bugs early. Every function with business logic needs them.
- API and integration tests, about 20%. They catch what unit tests miss: wrong data formats, misconfigured endpoints and broken authentication flows.
- UI and end-to-end tests, about 10%. The most expensive to write and maintain. Limit them to critical paths such as login, checkout and core workflows.
Two mistakes undo most automation programs. The first is automating everything at once: a team writes 500 tests in a month and then can't maintain them. Start with 50 critical tests and add 10-20 per sprint. The second is shared ownership. When everyone owns the tests, nobody does, so assign one automation owner.
How to get from 0% to 60% in 90 days
Days 1-14: Foundation
Set up your test framework (we recommend Playwright for web, Detox or Appium for mobile) and integrate it with your CI/CD pipeline. Write smoke tests for the 5 most critical user flows. By day 14, every commit triggers these tests automatically.
Days 15-45: Core automation
Automate regression tests for your 20 most important features. Focus on the features that change least frequently (they'll be the most stable automated tests) and break the hardest (they'll deliver the most value).
Days 45-75: API and data layer
Add API contract tests for all public endpoints. Add data validation tests for any data pipelines. These tests are cheap to write, fast to run, and catch entire categories of bugs.
Days 75-90: Tune and establish manual testing protocol
Review which automated tests are flaky (fix or delete). Establish a structured exploratory testing cadence: 20% of each sprint, focused on areas with recent changes. Train your manual testers on the new areas: AI-shifted categories where they should focus their judgment rather than routine checks.
Frequently asked questions
Can we automate everything and eliminate manual testing?
Technically yes, practically no. Full automation misses entire categories of bugs: usability issues, accessibility problems with assistive technology, and business logic that's technically correct but doesn't match user expectations. Teams that try 100% automation typically see their bug escape rate increase for exactly these reasons.
What automation framework should we use?
For web: Playwright. It's faster than Selenium, more stable than Cypress at scale, and supports multiple browsers natively. For mobile: Detox for React Native, XCUITest for native iOS, Espresso for native Android. For API: a combination of Jest/Vitest and Pact for contract testing.
How do we handle flaky tests?
Quarantine them. Move flaky tests to a separate test suite that runs but doesn't block deployments. Assign one engineer per sprint to investigate and fix quarantined tests. If a test has been flaky for more than 3 sprints, delete it and rewrite from scratch. Flaky tests erode trust in the entire suite.
When is a product too early for automation?
When you're still searching for product-market fit and the UI changes weekly. Automate once your core workflows stabilize, usually after the MVP.
We have zero automated tests. Where do we start?
Start with 5 smoke tests for your most critical flows (login, main page load, primary action, payment if applicable, data submission). Run them on every commit. This takes one engineer about a week and immediately catches the bugs that cause the most damage. Need help getting started? We've done this for teams of every size.
