TL;DR: The answer to "should we automate testing?" is "yes, but not everything." Automate regression, smoke tests, and API contracts. Keep manual testing for exploratory, UX, and first-time-use scenarios. The right ratio for most teams: 60-70% automated, 30-40% manual. AI testing agents are changing this math by making previously manual categories (visual regression, cross-browser) automatable. But business logic and security judgment remain human.
The automation trap that wastes money
"We need to automate all our testing." We hear this from CTOs at least twice a month. It's the wrong goal.
100% test automation is technically possible and economically wasteful. Some tests cost more to automate than to run manually. Some tests lose their value when automated because the automation tests the expected flow, while the manual tester discovers unexpected flows.
The real question isn't "manual or automated?" It's "which tests belong in which category, and how does AI change the allocation?"
Here's the framework we use at Globalbit after setting up testing programs for over 150 products.
The testing spectrum: what goes where
Automate (always)
Regression testing. Every feature that worked yesterday should work today. Humans are terrible at running the same checks 500 times without getting sloppy. Machines are perfect at it. If you automate only one thing, automate regression.
Smoke tests. Can users log in? Does the main page load? Do critical API endpoints respond? These binary checks should run on every commit. They take minutes to write and seconds to run and catch the most embarrassing bugs.
API contract tests. If your backend serves multiple clients (web app, mobile app, third-party integrations), contract tests verify that API changes don't break consumers. These are cheap to maintain and catch integration-breaking changes before they reach staging.
Data validation. Database migrations, ETL pipelines, data transformations. Anywhere data moves between systems. Automated validation catches corruption that manual testing almost never finds because the data volumes are too large for human inspection.
Keep manual (always)
Exploratory testing. A skilled tester with curiosity will find bugs that no automated suite catches. They try weird input combinations. They interrupt processes mid-flow. They behave like confused users, impatient users, and users who read the interface in a different language. You can't script curiosity.
At Globalbit, our exploratory testers dedicate 20% of each sprint to unscripted investigation. In the past 12 months, exploratory testing found 35% of our severity-1 bugs across all projects. None of these would have been caught by the automated suites.
First-time user experience. The onboarding flow, the first purchase, the initial setup. These need fresh eyes. Automated tests verify functionality. Manual testers verify comprehension. "Does this flow work?" is automation. "Does this flow make sense to someone seeing it for the first time?" is manual.
Accessibility with assistive technology. Automated accessibility scanners catch 30-40% of WCAG issues (missing alt text, color contrast, ARIA labels). The other 60-70%, like screen reader navigation flow, keyboard focus order, cognitive load, requires a human using assistive technology.
Cross-functional workflows. When a user's journey spans multiple products or services (start on the web app, continue in email, complete on mobile), the orchestration of this journey needs human evaluation. The individual pieces might pass automated tests while the connected experience fails.
The AI-shifted middle ground
These categories were firmly "manual" until 2025. AI testing agents are making them at least partially automatable:
Visual regression testing. AI agents compare screenshots across releases and flag unintended visual changes. Tools like Applitools and Percy do this well. What AI still misses: whether a visual change is an improvement or a regression. It catches differences; humans judge intent.
Cross-browser testing. AI agents now run the same test suite across 20+ browser-device combinations in parallel, detecting rendering differences automatically. What AI misses: browser-specific interaction patterns that users actually care about.
Performance baseline testing. AI agents establish performance baselines and alert when response times deviate. They're good at detecting regressions but poor at diagnosing root causes. A 200ms slowdown on the search API might be a database index issue, a cache miss, or an infrastructure problem. The agent flags it; a human investigates it.
Test case generation from requirements. Given a user story, AI agents generate 80-120 test cases covering happy paths, edge cases, and boundary conditions. The generated tests need human review. Roughly 70% are useful, 20% are redundant, and 10% test the wrong thing. But starting with 70% useful tests beats starting from zero.
The decision flowchart
For any test, run through these questions:
1. Does this test change between runs? If no (same inputs, same expected outputs, every time): automate it. If yes (different evaluation criteria each time): manual.
2. Does this test require judgment about quality? If it's binary (pass/fail based on expected output): automate. If it requires opinion (does this feel right? is this confusing?): manual.
3. How often does this test run? If daily or per-commit: must be automated. If monthly or per-release: consider manual if the judgment requirement is high.
4. What does it cost when this test misses a bug? If the missed bug costs $10,000+: automate AND run exploratory manual testing on the same area. If the missed bug costs under $1,000: automate alone is sufficient.
The cost reality
Here's what testing actually costs at different automation ratios:
| Automation ratio | Monthly cost (20-dev team) | Bug escape rate | Detection speed |
|---|---|---|---|
| 0% (all manual) | $18,000-$25,000 | 25-35% | 3-5 days |
| 30% automated | $14,000-$20,000 | 18-25% | 1-3 days |
| 60% automated | $12,000-$16,000 | 10-15% | 4-24 hours |
| 80% automated | $13,000-$18,000 | 8-12% | 1-4 hours |
| 95%+ automated | $16,000-$22,000 | 10-15% | Minutes |
Notice the cost increase at 95%+. That's because the last 15-20% of test automation is the most expensive to build and maintain. You're automating complex scenarios that change frequently, require sophisticated visual comparison, and break often enough that maintenance eats the savings.
The sweet spot for most teams: 60-70% automation. Maximum bug detection improvement per dollar invested.
How to get from 0% to 60% in 90 days
Days 1-14: Foundation
Set up your test framework (we recommend Playwright for web, Detox or Appium for mobile) and integrate it with your CI/CD pipeline. Write smoke tests for the 5 most critical user flows. By day 14, every commit triggers these tests automatically.
Days 15-45: Core automation
Automate regression tests for your 20 most important features. Focus on the features that change least frequently (they'll be the most stable automated tests) and break the hardest (they'll deliver the most value).
Days 45-75: API and data layer
Add API contract tests for all public endpoints. Add data validation tests for any data pipelines. These tests are cheap to write, fast to run, and catch entire categories of bugs.
Days 75-90: Tune and establish manual testing protocol
Review which automated tests are flaky (fix or delete). Establish a structured exploratory testing cadence: 20% of each sprint, focused on areas with recent changes. Train your manual testers on the new areas: AI-shifted categories where they should focus their judgment rather than routine checks.
Frequently asked questions
Can we automate everything and eliminate manual testing? Technically yes, practically no. Full automation misses entire categories of bugs: usability issues, accessibility problems with assistive technology, and business logic that's technically correct but doesn't match user expectations. Teams that try 100% automation typically see their bug escape rate increase for exactly these reasons.
What automation framework should we use? For web: Playwright. It's faster than Selenium, more stable than Cypress at scale, and supports multiple browsers natively. For mobile: Detox for React Native, XCUITest for native iOS, Espresso for native Android. For API: a combination of Jest/Vitest and Pact for contract testing.
How do we handle flaky tests? Quarantine them. Move flaky tests to a separate test suite that runs but doesn't block deployments. Assign one engineer per sprint to investigate and fix quarantined tests. If a test has been flaky for more than 3 sprints, delete it and rewrite from scratch. Flaky tests erode trust in the entire suite.
We have zero automated tests. Where do we start? Start with 5 smoke tests for your most critical flows (login, main page load, primary action, payment if applicable, data submission). Run them on every commit. This takes one engineer about a week and immediately catches the bugs that cause the most damage. Need help getting started? We've done this for teams of every size.