An AI product can pass its checks and still be wrong.
State could return valid structured output, complete a request successfully, and still represent something the project had never decided. That made ordinary “did it run?” testing insufficient.
I used different kinds of testing for different kinds of confidence.
My strongest QA experience is manual. I repeatedly used State end to end, reproduced strange behavior, and challenged things that technically worked but felt wrong for the product.
My project-management experience gave me familiarity with the broader testing approaches production software needs. I used AI to implement automated coverage I could not have built as comprehensively myself.
I used eval-style cases focused on State's most important failure risks. AI accelerated case creation; I reviewed the cases and used the results to make product decisions.
The automated coverage gave me execution leverage. I do not present its existence as evidence that I personally wrote the test automation.
Three failures looked like “AI problems.” They came from different layers.
An unresolved automation target became a confident 0%. The output fit the schema, but it stated a decision the project had never made. This exposed the gap between schema-valid and product-correct behavior and reinforced the need to preserve uncertainty and keep consequential changes behind review.
Ask returned the wrong billing contact. Rewriting the prompt would have treated the symptom. Investigation showed the model had reasoned from the context it received; the wrong context had been routed into the answer. Fixing retrieval fixed the result.
Streaming looked like an obvious improvement to a slow AI interaction. The first implementation produced corrupted, broken-up text. I pulled it rather than keep an experience that looked faster but was less trustworthy, then brought it back only after the implementation was reliable.
Find the failing layer before deciding what to change.
I learned to work through the system rather than reach immediately for prompt changes. The same bad-looking answer can come from the model, the context it received, deterministic logic, an integration, or the interface.
I stopped treating model quality as the same thing as product reliability.
I started State thinking mostly about whether AI could interpret project information accurately. Building and testing it pushed me to think about the whole system around the model. Reliability depended just as much on preserving uncertainty, enforcing the right boundaries, and investigating failures at the layer where they actually occurred.
What I did vs. what AI did
I decided what behavior mattered, tested State hands-on, reproduced failures, challenged plausible explanations, and decided whether the resulting behavior was acceptable for the product.
I used AI heavily to expand test coverage and accelerate debugging. That increased what I could execute without replacing the product judgment about whether State was actually correct.