|
Something fascinating about the use of agents to take over tasks from us, including mobile development tasks, is that it presents opportunities to rethink our workflows. It also makes more apparent the challenges that have always been painful for humans—challenges that are even more painful for agents. Take testing, for example. It's an automation tool to ensure the code we write does what it's supposed to do at runtime. Tests range from unit tests, which are fast, reliable, and easier to develop and maintain, to acceptance tests, which are slow, unreliable, and more painful to maintain because they examine the application as a whole. A good balance of both is useful because unit tests, which mock the dependencies of the subject under test, might make incorrect assumptions about integrations, causing the tests to be unrealistic. But acceptance tests are quite costly—very costly. The alternative is manual QA, which is financially expensive and doesn't scale well, especially as applications become more complex with numerous possible navigation flows. For many years, we and Apple have tried to reduce the costs of writing and maintaining these tests. For instance, in Xcode 26, you can now generate test code as you navigate the app yourself. But still, once the code is written, you become its maintainer. If something breaks, you'll have to dive deep into code that wasn't written by you, that interacts with other code likely not written by you either, and understand test scenarios that might not be obvious from looking at the test. Doesn't sound like the best experience, does it? What if we've been looking at the problem through the wrong lens? LLMs and the concept of agents present a perfect opportunity to challenge this approach. Think of it as QA that can dynamically test particular scenarios based on inferred context (for example, from a PR description) and provided context (for example, through a context file). So instead of developing and maintaining a test suite, imagine tests happening on the fly, with the agent given all the information necessary to make decisions and to collect and export the diagnostic information developers need to understand what happened. We believe ideas and problems should be continuously challenged because the environment and technological capabilities evolve. This is certainly the case for app testing, where we might be cargo-culting ideas, costs, and challenges that could easily be swept away. The content has been written by a human and the grammar reviewed with Claude Sonnet 4
|
