Welcome to the 329th issue!

What happens to testing when AI speeds up development?

This week, I want to share with you recent insights from Anthropic and Spotify:

• Agentic coding is straining CI. Here's how we scaled test impact analysis at Anthropic
• AI Changed How Spotify Builds. What We Learned (and Fixed) About Quality at Higher Velocity

In both cases, merge requests and CI jobs multiplied exponentially, and they had to adjust their testing and verification to keep up.

This is another confirmation of a major challenge for all companies adopting agentic development — just as I highlighted three weeks ago.

Happy testing!

Dawid Dylowicz

If you're planning to become a QA Lead, Oksana Kravets shares her journey, including how she learned to do less hands-on work, lead the team and make risks visible to the business.

Moreover, Jean-Christophe Jamet shares a lead's view: I Told My QA Engineer: "You're About to Become a Manager — Just Not of People".

Valid question, especially in the AI era where testing has been augmented further. But does it work? Joep Schuurkes shares an opinion on why some companies may need testers more than others.

This Reddit discussion also got my attention: I work on the business side of a QA company. Here's what I've noticed about how testing actually gets bought.

Ever felt like running out of time to test ahead of the release? Martin Cáceres González advises on how to protect that time before the pressure comes.

Also, here's a Reddit thread that many of us can relate to: Why does everything work perfectly until QA starts testing it?

Good read from James Bach on why quality engineering isn't the same as testing. Also explains why an AI tool can't be fully responsible for quality and replace humans.

Similarly, Neeru Gupta shares a personal view on "Building Quality In, Rather Than Testing It In" — A Thought That Stayed With Me.

Testing financial or compliance-sensitive products? Karina Neklyudova describes why a risk matrix helps and how an AI agent can review features against policies.

At the same time, Janet Gregory and Lisa Crispin share an important update to Agile Testing Principles in the age of AI.

Now that AI can write tests in minutes, Julia Pottinger shows how to review them against risk and keep the suite small and stable.

Additionally, Gary Worthington explains The test pyramid: what it actually means now that AI writes the code.

Running tests in parallel can make a suite much faster. But it also exposes hidden dependencies between tests. Ankita Bhattacharjee describes three kinds of conflicts and how a @sequential tag solved them.

Visual testing is great for verifying UI changes but it doesn't deal well with dynamic content. Alexey Sumin and Oleg Anisimov share how they built an AI-powered solution for mobile apps to address this challenge.

Furthermore, Tate G. Wilks wrote a helpful article on Implementing Simulation Testing.

Passing tests don't necessarily mean that everything works correctly. Serhii Fedorenko explains why a big test suite missed 15 bugs and how they found them.

Speaking of that, Muskan Agarwal has a point on why Your Code Coverage Number Is Lying to You.

Michael Bromley shares why lots of AI-generated tests may not give us the confidence we expect at first and suggests having a separate AI agent to review them. Why? Because in their case, a separate review found issues in about 30% of PRs.

In relation to that, Gil Zilberfeld looks at the same problem in We Finally Got the Tests. Now We Don't Trust Them.

Marc Rasmussen describes an interesting approach to using an AI tool that replays a recorded user journey with VoiceOver. It shows which steps a screen reader user can complete and where they get stuck.

What's more, Oscar van der Leij explains Accessibility Testing: What axe-core Cannot See.

Oskar Kwaśniewski's team has just released an open-source AI testing framework that allows you to test API, web and mobile apps with agents. The idea is familiar: we describe the goal in English and the agent handles the actions, locators and deterministic assertions. But it also remembers and replays them without calling the model again. You can learn more from the documentation.

Good read from Jaison Thomas on using Claude to compare test builds with the design, including a tip on how to log issues faster.

On top of that, Amruta Lohabare shares another way to log bugs faster: My Bug Tickets Used to Take 20 Minutes. Now Claude Files Them in 2.

Heard of Jev yet? Jason Arbon paired it with Playwright to choose test steps and judge results, explaining why he wouldn't trust its accuracy yet.

Moreover, Fardin Faruk tried a different setup: Planner, Generator, Healer: How I Ran a Full Playwright Suite With Three AI Agents And Playwright MCP

David Mello gives an overview of how to run Playwright tests that change global state in parallel with a demo repo for each option.

Also, Ekki Syam Sugiardi shares what happened when I Asked Codex to Build a Disposable Playwright CI Platform While I Was Away

Testing LLM-based apps? Well, sit down comfortably, because James Bach explains why it's not easy and suggests an approach called metamorphic testing, including how to run thousands of variations effectively.

Once again, Alex Khvastovich is analysing the QA job market using historical and current data to see whether the trend has changed. Quite important, given how much we hear about AI replacing roles and testers being laid off or struggling to find new jobs.

Bug in the web... 😂

Thanks for reading!

If you enjoy this newsletter and find it helpful in becoming a better tester, please consider sharing it with others.

Dawid Dylowicz