Issue #319
Not Every Test Should Be an AI Agent 🤖
| |||
| Welcome to the 319th issue! I really enjoyed reading through Oleksii Samara's detailed story on their approach to agentic QA: Not Every Test Should Be an AI Agent. Highly recommended. Just as the rest of the news in this issue. Happy testing! 🙂 | |||
| | |||
| |||
| |||
| | |||
| |||
| How do I know my testers are doing a good job? Joep Schuurkes explains why testing metrics, such as defect counts and escaped bugs, can be misleading and how to think about tester performance without relying on numbers. Moreover, David Ingraham gives advice on How to Be a 10x Engineer from an SDET perspective, while Betty Lin points out that Where QA Sits Decides What QA Can Say. | |||
| It Isn't Testing that Doesn't Scale; It's Prudence Should testing keep up with AI code generation? Michael Bolton has a sharp reply, stating that the output should be tested at a pace that allows your team to understand and address risks. At the same time, Jitesh Gosai advises to Start with what changed. | |||
| Small team, no QA? Be your own tester Peter Brumby shares a practical guide covering browsers, accessibility, Lighthouse, user journeys, and how to balance testing effort with risk. Moreover, Katja Obring contemplates: Should the tester role even exist? | |||
| The Quality Quadrant Matrix: A Framework for Deciding Who Should Own Testing on Every Project Reza Dhanu Pramitra shares a framework for splitting testing responsibility between developers and testers based on risk and complexity. He then Made ~100 Developers Own Their Testing And Quality Standards Didn't Drop. | |||
| Three Years of Building Agents in Production (Part 1) This is a rare insight into what it takes to ship AI agents into a testing product. Lauren Leidal shares a story of three years of experiments at mabl, explaining why their agent architecture needed five iterations. You can also read the second part. Moreover, Demi Van Malcot wrote A practical introduction to testing LLMs. | |||
| Yes you can run exploratory testing with AI If you wonder whether AI can help you with exploratory testing, Callum Akehurst-Ryan has a few tips, including identifying risks, writing test charters and then running them with AI in a browser or codebase. What's more, Ercan Semerci says: We Didn't Put AI in Place of QA. We Put It Beside QA. | |||
| | |||
| |||
| Agentic QA: Letting AI Author Your Tests Without Losing Control Vibin Wilson describes a practical agentic QA workflow with specialised agents for spec writing, test generation and triage. I also recommend two related articles by Julia Pottinger: | |||
| Automating mobile tests with AI Giulia Brocchi describes a mobile end-to-end test framework where tests are stored in Markdown files for an AI agent to drive the simulator via MCP. Furthermore, Russell Morley put together Five Characteristics of a Good Goal-Based Mobile Test. | |||
| Page Object Model Is Not Enough: The 4 Layers Real Frameworks Use If your test framework is getting hard to maintain, Pranta Kundu shows how separating driver code, page objects and business workflows into four layers makes scaling and maintenance much easier. Moreover, Joseph Ward explains a few reasons Why Simple UI Tests Become Slow. | |||
| Test Pyramid for AI Agents Wondering how we should test AI agents? Angie Jones rethinks the test pyramid into four layers — from deterministic unit tests to LLM-as-judge evaluations. At the same time, Aastha Singh shares that My "Improved" Prompt Made the Agent Worse, So I Built an Eval Loop. | |||
| Your AI Generated 400 Tests. They All Pass. None of Them Catch the Bug That Ships to Production. Dan Stauffer makes the case for pairing AI test generation with mutation testing in order to verify whether your test suite would be able to catch a defect. Also, Swati Seela describes why Your Test Automation May Be Revealing Team Problems. | |||
| | |||
| |||
| |||
| | |||
| |||
| How to Test POST API Requests with SuperTest in Node.js If you write API tests in Node.js, Mohammad Faisal Khatri has a practical guide to SuperTest POST requests covering JSON, form data, XML, file uploads and dynamic data generation. Also, Gil Zilberfeld wrote a follow-up article showing how to Test The Feature: Now With Examples using the API. | |||
| Looking Behind Playwright's Magic Curious about how Playwright's auto-waiting works? Joseph Ward demonstrates it step-by-step from element checks and scrolling to CDP network event handling. Moreover, Kailash Pathak shares the process of Building a Self-Healing Playwright MCP Multi Agent Framework with Integrating Angular Dashboard. | |||
| Playwright in Pictures: How Fixtures Work If you want to understand fixture scopes, dependencies and overrides in Playwright, Vitaliy Potapov has a great visual guide with timeline charts that show what happens before, during and after each test. Similarly, Laxminarayana Boga describes using Playwright Fixtures as a Runtime System: Five Patterns Beyond page. | |||
| The Best Test Management Tool Is the One You Build Yourself An interesting story of how Kubra Sayli got tired of juggling Cypress Cloud, TestRail and Jira and built her own test management app in two weeks instead. | |||
| | |||
| |||
| AI App Testing: A Core QA Skill Just Like Selenium & Playwright An interesting, 11-minute take by Karthik KK on how AI testing skills might become a must-have for testers. | |||
| AI Code Review Toolkit Comparison In this 21-minute video, Alan Richardson compares AI code review tools to help you automate pull request workflows and catch bugs before they reach production. | |||
| | |||
| |||
| | |||
| Thanks for reading! If you enjoy this newsletter and find it helpful in becoming a better tester, please consider sharing it with others. |