I can say my company stopped contracting for test system design, and we use a mix of models now to achieve the same results. Some of these have been running without issue for over a year now.
As in writing test cases? I’ve seen devs write (heavily mocked) unit tests using only AI, but these are worse than no tests for a variety of reasons. Our company also used to contract for these tests…but only because they wanted to make the test coverage metric to up. They didn’t add any value but the contractor was offshore and cheap.
If you’re able to have AI generate integration level tests (ie call an API then ensure database or external system is updated correctly - correctly is doing a lot of heavy lifting here) that would be amazing! You’re sitting on a goldmine, and I’d happily pay for these kind of tests.
Amazingly, there is industry outside tech that uses software. We are an old school tangible goods manufacturing company. We use stacks of old grumbling equipment to do product verification tests, and LLMs to write the software that synchronizes them and interprets what they spit back out.
