Skip to main content

Observe

See how to use observe() to discover actionable elements and analyze web page structure

​
Method Signatures

// String instruction only
await stagehand.observe(instruction: string): Promise<Action[]>

// String instruction with options
await stagehand.observe(instruction: string, options: ObserveOptions): Promise<Action[]>
ObserveOptions Interface:
interface ObserveOptions {
  model?: ModelConfiguration;
  timeout?: number;
  selector?: string;
  page?: PlaywrightPage | PuppeteerPage | PatchrightPage | Page;
  serverCache?: boolean;
}

// ModelConfiguration can be either a string or an object
type ModelConfiguration =
  | string  // Format: "provider/model" (e.g., "openai/gpt-4o", "anthropic/claude-sonnet-4-6")
  | {
      modelName: string;  // The model name
      apiKey?: string;    // Optional: API key override
      baseURL?: string;   // Optional: Base URL override
      // Additional provider-specific options
    }

​
Parameters

​
instruction
string
required
Natural language description of elements or actions to discover. If not provided, defaults to finding all interactive elements on the page.
​
model
ModelConfiguration
Configure the AI model to use for this observation. Can be either:
  • A string in the format "provider/model" (e.g., "openai/gpt-4o", "anthropic/claude-sonnet-4-6")
  • An object with detailed configuration
​
timeout
number
Maximum time in milliseconds to wait for the observation to complete. Default varies by configuration.
​
selector
string
Optional XPath selector to focus the observation on a specific part of the page. Useful for narrowing down the search area.
​
page
PlaywrightPage | PuppeteerPage | PatchrightPage | Page
Optional: Specify which page to perform the observation on. Supports multiple browser automation libraries:
  • Playwright: Native Playwright Page objects
  • Puppeteer: Puppeteer Page objects
  • Patchright: Patchright Page objects
  • Stagehand Page: Stagehand’s wrapped Page object
If not specified, defaults to the current “active” page in your Stagehand instance.
​
serverCache
boolean
Override the instance-level serverCache setting for this request. When true, enables server-side caching. When false, disables it.
Only applies when env is "BROWSERBASE". Has no effect in local environments.
Defaults to the value set on the Stagehand constructor (which itself defaults to true).

​
Returns Promise<Action[]>

Array of discovered actionable elements, ordered by relevance.
​
selector
string
XPath selector that precisely locates the element on the page.
​
description
string
Human-readable description of the element and its purpose.
​
method
string
Suggested interaction method for the element (e.g., "click", "fill", "type").
​
arguments
string[]
Additional parameters for the suggested action, if applicable.
Action Interface: 
interface Action {
  selector: string;        // XPath selector to locate element
  description: string;     // Human-readable description
  method?: string;         // Suggested action method
  arguments?: string[];    // Additional action parameters
}
Example Response: 
[
  {
    "selector": "/html/body/div[1]/header/nav/button[1]",
    "description": "Login button in the navigation bar",
    "method": "click",
    "arguments": []
  },
  {
    "selector": "/html/body/main/form/input[1]",
    "description": "Email input field in the login form",
    "method": "fill",
    "arguments": []
  }
]

​
Built-in Support

Iframe and Shadow DOM interactions are supported out of the box. Stagehand automatically handles iframe traversal and shadow DOM elements without requiring additional configuration or flags.

​
Code Examples

import { Stagehand } from "@browserbasehq/stagehand";

// Initialize with Browserbase (API key and project ID from environment variables)
// Set BROWSERBASE_API_KEY and BROWSERBASE_PROJECT_ID in your environment
const stagehand = new Stagehand({ env: "BROWSERBASE" });
await stagehand.init();
const page = stagehand.context.pages()[0];

await page.goto("https://example.com");

// Basic element discovery
const buttons = await stagehand.observe("find all clickable buttons");
const formFields = await stagehand.observe("locate form input fields");

// Working with results
const [loginButton] = await stagehand.observe("find the login button");
if (loginButton) {
  console.log("Found:", loginButton.description);
  console.log("Selector:", loginButton.selector);
  await stagehand.act(loginButton); // Execute the action
}

​
Integration Patterns

// Observe → Act workflow
const actions = await stagehand.observe("find checkout elements");
for (const action of actions) {
  await stagehand.act(action);
  await page.waitForTimeout(1000);
}

// Observe → Extract workflow
const tables = await stagehand.observe("find data tables");
if (tables.length > 0) {
  const data = await stagehand.extract({
    instruction: "extract the table data",
    selector: tables[0].selector,
    schema: DataSchema
  });
}

// Element validation
const requiredElements = await stagehand.observe("find the login form");
if (requiredElements.length === 0) {
  throw new Error("Login form not found");
}

​
Error Types

The following errors may be thrown by the observe() method:
  • StagehandError - Base class for all Stagehand-specific errors
  • StagehandDomProcessError - Error occurred while processing the DOM
  • StagehandEvalError - Error occurred while evaluating JavaScript in the page context
  • StagehandIframeError - Unable to resolve iframe for the target element
  • ContentFrameNotFoundError - Unable to obtain content frame for the selector
  • XPathResolutionError - XPath does not resolve in the current page or frames
  • StagehandShadowRootMissingError - No shadow root present on the resolved host element
  • LLMResponseError - Error in LLM response processing
  • MissingLLMConfigurationError - No LLM API key or client configured
  • UnsupportedModelError - The specified model is not supported for this operation
  • InvalidAISDKModelFormatError - Model string does not follow the required provider/model format