Skip to main content

Quick Start

Run your first AI-powered test flow from the Rock Smith desktop app in a few minutes.

Windows Availability

The Windows desktop app is coming soon. You can follow this Quick Start today on macOS and Linux.

Prerequisites​

  • Rock Smith desktop app installed (Installation Guide)
  • Active Rock Smith account with available credits
  • Internet connection (required to connect to the server)
  • A website URL to explore and test (public, staging, or localhost)
  • Optional: A saved browser profile for testing authenticated pages
Alternative: Create Flows Manually

This guide uses Discovery Sessions to auto-generate flows. You can also create flows manually:

  • New Flow: Create a flow step-by-step in the flow editor.
  • Generate Flow: Describe what you want to test and let AI generate the flow.

Navigate to Flows > New Flow or Generate Flow to try these options.

Step 1: Launch and Sign In​

Open the desktop app and sign in with your credentials (or continue with GitHub). If you don’t have an account, use the Sign up link from the sign-in screen.

Step 2: Connect to the Server​

  • If you see a banner titled "Not Connected to Server", click Connect to Server.
  • The connection status updates automatically. When connected, you can start discovery sessions and execute flows.
  • If connection fails, ensure you're signed in and online, then try again.

Step 3: Start a Discovery Session​

Discovery Sessions let the AI explore your website and automatically generate test flows based on what it finds.

  • From the sidebar, click Discovery Sessions.
  • Click New Discovery Session.
  • Configure the session:
    • Target URL: Enter the website URL to explore (e.g., https://example.com).
    • Browser Profile (optional): Select a profile to test authenticated pages. Learn more
    • Custom Instructions (optional): Guide the AI's exploration (e.g., "Focus on the checkout flow").
  • Click Start Discovery.

The AI agent will launch a browser and begin exploring your site, identifying pages, forms, buttons, and user interactions.

Step 4: Monitor Discovery Progress​

  • Watch real-time progress as the AI explores your site.
  • The discovery panel shows:
    • Discovered URLs: Pages the AI has found and analyzed.
    • Page States: Different states of each page (e.g., form empty, form filled).
    • Generated Flows: Test flows being created from discovered patterns.
  • The browser window shows what the AI is doing. Keep it open during discovery.

Step 5: Review Generated Flows​

When discovery completes:

  • Navigate to Flows from the sidebar.
  • You'll see flows generated from the discovery session, organized by your project.
  • Each flow contains:
    • Steps: Actions the AI will perform (navigate, click, type, etc.).
    • Assertions: Checks to verify expected behavior.
    • Target Elements: Semantic descriptions of UI elements (e.g., "the Submit button below the form").
  • Review the flows and approve the ones you want to keep.
tip

Flows use semantic element targeting—describing elements by their appearance and context rather than brittle CSS selectors. This makes tests resilient to UI changes.

Step 6: Execute a Flow​

  • Select a flow from the Flows list.
  • Click the Run button.
  • Configure execution options:
    • Browser Profile: Select a profile for authenticated testing.
    • Headless Mode: Run without a visible browser window.
  • Click Start Execution.

The AI executes each step in the flow, capturing screenshots and validating assertions along the way.

Step 7: Monitor Execution​

  • Watch live progress as the AI executes each flow step.
  • The execution view shows:
    • Step Progress: Current step and pass/fail status.
    • Screenshots: Captured at each step for visual verification.
    • Agent Reasoning: Why the AI made specific decisions (helpful for debugging).
  • Keep the browser window open during execution (unless running headless).

Step 8: Review Execution Results​

When execution completes:

  • View the execution summary with pass/fail metrics.
  • Examine individual step results:
    • Screenshots showing what the AI saw.
    • Assertion results (passed or failed).
    • Agent reasoning for each action.
  • Review execution history to track trends over time.

Common Errors and Fixes​

  • Authentication required: Sign in and retry.
  • Not connected to server: Click Connect to Server and wait for status to show Connected.
  • Execution already running: Wait for the current execution to finish, then start a new one.
  • Connection failed: Check your internet connection and retry.
  • 401 Unauthorized: Your session may have expired; sign in again.
  • 403 Access denied: Your account lacks permission to run flows.
  • 500 Server error: Try again later.
  • Validation error: Fix highlighted fields (e.g., missing or invalid URL).
  • Insufficient credits: Purchase more credits or upgrade your plan.

Understanding Credit Usage​

Credits are consumed during discovery and flow execution:

  • Agent Steps: AI agent actions during discovery or execution (1 credit each)
  • Test Steps: Individual flow step executions (1 credit each)
  • Total Credits: Sum of agent steps and test steps

Check your credit usage in the execution details panel or your account dashboard. Learn more about credits.

Next Steps​

Get Help​

See the Troubleshooting Guide or email support@rocksmith.ai.