Test Automation
Check that your web app and your AI assistant still work, every time you change them.
Every release can break something that worked yesterday, and an AI assistant can start giving different answers after a prompt or model change without a line of code changing. Checking by hand is slow and easy to skip. Test Automation is MAICEM's testing platform: Playwright browser tests for your web apps and scripted conversations for your AI assistant, run from one console against the environment you choose.
Why it exists
Manual testing does not keep up with frequent releases, and ordinary tests cannot tell whether an AI answer is any good. Test Automation covers both: browser tests that click through your app the way a user would, and conversation tests that check what your assistant says, turn by turn, with an optional quality score for each answer.
What you can see
- The pass rate of each run and each test plan, by conversation and by turn
- Quality scores for each answer, such as relevance, completeness and usefulness, rated by an AI model
- Response times and, where your app reports them, tokens and cost for each run
- Every past run, with each conversation's turns, checks and failures, and screenshots of failed browser tests
What you can do
- Create browser tests three ways: snapshot a whole screen, record yourself clicking through, or describe the test in plain English and let AI write it
- Test cases are generated in Playwright, so any test case can be selected and its code updated, or copied as the base of the next test and edited. A trained assistant lets you create and edit tests with goal-based natural language
- Group test cases into plans and run them as a suite
- Run the same tests against development or production by switching the environment
- Re-run just the failed conversations, and update screen baselines when a change is intended