Skip to main content

​
Surveillance & Robotics

Pi exposes two purpose-built APIs for physical intelligence workloads: video surveillance for programmable scene understanding and robotics for agentic robot control. Both follow the same OpenAI-compatible async job pattern and support SSE event streams for real-time delivery.

​
Video surveillance

Configure what to watch for, which behaviors trigger alerts, and how you want incidents delivered — Pi handles the perception pipeline internally.

​
Submit a video analysis job

POST /api/v1/surveillance/streams creates an async job (202 Accepted) that runs perception, evaluates inline behaviors and stored policies, optionally narrates incidents with Gemini, and publishes incidents to SSE subscribers. 
export BASE="https://api.example.com"
export API_KEY="pi_live_***"

job_id=$(curl -sS "$BASE/api/v1/surveillance/streams" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "stream_id": "cam-entrance-1",
    "source": { "url": "rtsp://camera.local/stream", "type": "rtsp", "fps_cap": 15 },
    "profile": "warehouse_safety",
    "detect": ["person", "forklift"],
    "behaviors": [
      { "type": "loitering", "zone": "loading_dock", "seconds": 90 },
      { "type": "intrusion", "zone": "restricted", "classes": ["person"] }
    ],
    "anomaly": { "enabled": true, "sensitivity": 0.72 },
    "outputs": { "delivery": ["sse", "webhook"], "format": "detailed", "webhook_url": "https://hooks.example.com/alerts" },
    "alerts": { "min_severity": "warning", "cooldown_seconds": 45, "group_by": "zone" },
    "context": {
      "site": "Warehouse 7",
      "zones": {
        "loading_dock": { "xyxy": [0.1, 0.2, 0.5, 0.9] },
        "restricted": { "xyxy": [0.55, 0.1, 0.95, 0.5] }
      }
    },
    "input": { "data": "<base64 or data: URL>", "mime_type": "image/jpeg" },
    "output": { "locale": "en", "include_diagnostics": false, "format": "json" }
  }' | jq -r '.data.job_id')

# Poll for result
curl -sS "$BASE/api/v1/jobs/$job_id?wait_for_completion=true&timeout_seconds=30" \
  -H "Authorization: Bearer $API_KEY"
Response (202) 
{
  "id": "req_pi_<uuid>",
  "object": "job",
  "status": "queued",
  "created_at": 1711700000,
  "data": { "job_id": "<uuid>" }
}

​
Create and manage behavior policies

POST /api/v1/surveillance/policies stores a reusable policy scoped to a stream or globally (when stream_id is empty). Policies are evaluated alongside inline behaviors on every stream job. 
curl -sS -X POST "$BASE/api/v1/surveillance/policies" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "stream_id": "cam-entrance-1",
    "name": "night_perimeter",
    "type": "intrusion",
    "condition": { "zone": "yard" },
    "action": { "severity": "critical", "cooldown_seconds": 120 },
    "enabled": true
  }'
List policies for a stream: 
curl -sS "$BASE/api/v1/surveillance/policies?stream_id=cam-entrance-1" \
  -H "Authorization: Bearer $API_KEY"

​
Subscribe to the incident SSE stream

GET /api/v1/surveillance/events emits incident events as JSON and : heartbeat comments every 15 seconds. Filter by severity with ?severity=warning. 
curl -N "$BASE/api/v1/surveillance/events?stream_id=cam-entrance-1" \
  -H "Authorization: Bearer $API_KEY"

​
Built-in profiles

Pass a profile to apply sensible defaults for your deployment type. Your explicit behaviors override the preset when non-empty.
ProfileTypical use
retail_securityShop floor loitering, checkout crowding, unattended object proxy
warehouse_safetyForklift speed, one-way lanes, restricted storage
smart_cityCrosswalk crowding, arterial speed, bus lane direction
residential_perimeterNight intrusion, driveway loitering
construction_siteHazard zone breach, scaffold crowding
parking_lotRow loitering, after-hours intrusion, lot speed
school_campusCafeteria crowding, parking loitering, staff-only intrusion
healthcare_facilityPatient wing dwell time, ER crowding, pharmacy perimeter
Treat video streams as sensitive PII. Use TLS in transit, least-privilege API keys, and clear data retention policies. Incident narration is assistive — verify alerts against operational procedures and local regulations.

​
Robotics

The Robotics API lets you run programmable robot behaviors (patrol, observe, follow) with HTTP calls. Pi orchestrates perception, behavior evaluation, and decisions via a ROS2 bridge sidecar.

​
Start an agentic robot run

POST /api/v1/robots/run queues an async job (perception + behaviors + decisions). 
robot_job_id=$(curl -sS -X POST "$BASE/api/v1/robots/run" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "robot_id": "robot_123",
    "task": "patrol",
    "profile": "patrol_security",
    "zones": [
      { "name": "entrance", "type": "patrol", "frame": "map", "polygon": [[0,0],[10,0],[10,10],[0,10]] }
    ],
    "behaviors": [
      { "type": "intrusion", "zone": "entrance" }
    ],
    "actions": [
      {
        "on": "intrusion",
        "do": [
          { "type": "alert", "severity": "critical" },
          { "type": "command", "command": { "command": "announce", "message": "Restricted area." } }
        ]
      }
    ],
    "perception": {
      "input": { "data": "<base64_or_data_url>", "mime_type": "image/png" },
      "frame_index": 0,
      "detect": ["person"]
    },
    "outputs": { "delivery": ["sse"] }
  }' | jq -r '.data.job_id')

echo "Robot run job: $robot_job_id"
Response (202) 
{
  "id": "req_pi_...",
  "object": "job",
  "status": "queued",
  "created_at": 1711900000,
  "data": { "job_id": "uuid" }
}
Poll via GET /api/v1/jobs/:id. Failures are captured in jobs.error_log.

​
Check robot status

GET /api/v1/robots/:id/status returns robot metadata and current state from the ROS2 bridge sidecar when available. 
curl -sS "$BASE/api/v1/robots/robot_123/status" \
  -H "Authorization: Bearer $API_KEY"

​
Send a direct command

POST /api/v1/robots/:id/command sends a command directly to the robot without queuing a full run. 
curl -sS -X POST "$BASE/api/v1/robots/robot_123/command" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "command": {
      "command": "move_to",
      "target": { "frame": "map", "x": 1.2, "y": 3.4 }
    }
  }'

​
Subscribe to robotics SSE events

GET /api/v1/robots/events streams real-time incident events from robot runs. 
curl -N "$BASE/api/v1/robots/events" \
  -H "Authorization: Bearer $API_KEY"

​
Stored configuration

Store reusable zones, behaviors, and action templates at the org level:
MethodPathPurpose
GET / POST/api/v1/robots/zonesDefine patrol and watch zones
GET / POST/api/v1/robots/behaviorsRegister reusable behavior rules
GET / POST/api/v1/robots/actionsRegister response action templates
Pass stored resource IDs in your run payloads instead of inlining the full config each time.