Sandbox
The Sandbox is your private workspace for drafting and trying out test cases before they go into the shared Library. Use it to iterate on the instructions until the agent does what you mean and judges the result the way you would; only then promote the test case, so the Library contains tests the whole team can rely on.
How the Sandbox works
Every project member has their own Sandbox in each project. Test cases and collections you create there are visible only to you; other members, including Managers, do not see them. Everyone with access to the project can use their Sandbox, including Viewers. See Roles.
A Sandbox test case differs from a Library test case in two ways:
- It stores its own Device Preset and Source, so you can launch it on its own without creating a run. See Test case fields.
- Only its own Test Instructions (Prompt) and attachments are sent to the agent. There are no plan or run instructions in the Sandbox.
To experiment with an existing Library test case, use Send to Sandbox on it in the Library: this places a copy in your Sandbox and leaves the original unchanged. When a test case works the way you want, promote it.
Collections, search and filters work as in the Library. Deleting a collection permanently deletes all test cases inside it, and deleting a Sandbox test case is permanent.
Launching
A test case can be launched once it has a Device Preset and a Source; until then Launch is disabled.
To run several sessions of the same test case at once, click the arrow next to Launch and set Number of sessions (1 to 100, default 2). All sessions are started together; when the platform is busy, they wait in the queue and start as capacity frees up.
Viewing results
Open Execution History from the row menu, or History in the detail panel. Each execution shows the device, source, timing, who launched it, the exact instructions and attachments it used, the agent’s Expected Behavior, Actual Behavior, Failure Reason and Steps to Reproduce, and any Requested Output. The verdicts are explained in Results.
Promote to Library
Promoting copies a Sandbox test case into the project’s Library (Promote to Library in the row menu).
| Mode | Description |
|---|---|
Update Existing Test Case | Offered only when the test case came from the Library with Send to Sandbox, and selected by default then. Saves your changes as a new version of the original Library test case. |
Create New Test Case | Creates a new Library test case in the Target Collection you pick. New test cases start as Draft. |
In update mode, if someone changed the Library test case after you copied it, a note says so: your version is saved on top of the latest one, so review the Library test case afterwards.
Promoting copies the Device Preset but not the Source URL; a run supplies the URL. The copied device is not shown in the Library and takes priority over a run’s device until the test case is next edited in the Library. If it is your private device or simulator, the test case is Skipped when other people run it.
Promoting requires the Tester or Manager role. Your Sandbox copy stays in the Sandbox after promoting.
Jira issues
Any project role can link a Jira issue to their own Sandbox test case with + Link Issue in the detail panel, once Jira is set up for the project. See Jira integration.