# FirstCall WebSocket API

The WebSocket API gives you real-time meeting data for a specific bot — participants, transcripts, audio, screenshots, and more. You also send commands through the same connection.

## Connecting

```
wss://api.firstcall.dev/v1/bots/{bot_id}/ws?api_key={your_api_key}
```

- **bot_id**: The bot ID returned from `POST /v1/bots`
- **api_key**: Your API key from the dashboard
- The bot must belong to your account
- Connect after creating the bot — you'll receive events as the bot joins and during the meeting

### Node.js

```javascript
const WebSocket = require('ws');

const botId = 'bot-abc123def456';
const apiKey = 'ak_live_xxxxxxxxxxxxxxxx';

const ws = new WebSocket(
  `wss://api.firstcall.dev/v1/bots/${botId}/ws?api_key=${apiKey}`
);

ws.on('open', () => {
  console.log('Connected to bot', botId);
});

ws.on('message', (data) => {
  const event = JSON.parse(data);
  console.log(event.type, event);
});

ws.on('close', (code, reason) => {
  console.log('Disconnected:', code, reason.toString());
});
```

### Python

```python
import asyncio
import json
import websockets

async def connect():
    bot_id = "bot-abc123def456"
    api_key = "ak_live_xxxxxxxxxxxxxxxx"
    uri = f"wss://api.firstcall.dev/v1/bots/{bot_id}/ws?api_key={api_key}"

    async with websockets.connect(uri) as ws:
        async for message in ws:
            event = json.loads(message)
            print(event["type"], event)

asyncio.run(connect())
```

## Connection Errors

| Close Code | Meaning |
|------------|---------|
| 4000 | Invalid URL format |
| 4001 | Missing or invalid API key |
| 4003 | Bot does not belong to your account |
| 4004 | Bot not found |

## Events You Receive

### Meeting Events

#### `meeting.participant_joined`

A participant joined the meeting.

```json
{
  "type": "meeting.participant_joined",
  "participant": { "id": "p-1", "name": "Alice" },
  "participants": [
    { "id": "p-1", "name": "Alice" },
    { "id": "p-2", "name": "Bob" }
  ],
  "timestamp": "2026-03-21T14:31:20.000Z"
}
```

#### `meeting.participant_left`

A participant left the meeting.

```json
{
  "type": "meeting.participant_left",
  "participant": { "id": "p-2", "name": "Bob" },
  "participants": [
    { "id": "p-1", "name": "Alice" }
  ],
  "timestamp": "2026-03-21T14:45:10.000Z"
}
```

#### `meeting.active_speaker`

The active speaker changed.

```json
{
  "type": "meeting.active_speaker",
  "speaker": { "id": "p-1", "name": "Alice" },
  "timestamp": "2026-03-21T14:31:25.000Z"
}
```

`speaker` is `null` when no one is speaking. During brief pauses, the last known speaker is retained to prevent flickering (sticky speaker attribution).

#### `meeting.chat_message`

A chat message was sent in the meeting.

```json
{
  "type": "meeting.chat_message",
  "sender": "Alice",
  "message": "Can everyone hear me?",
  "message_id": "msg-123",
  "timestamp": "2026-03-21T14:32:00.000Z"
}
```

#### `meeting.mic.result`

Result of a `meeting.mic` command. Sent after the bot attempts the mic toggle.

```json
{
  "type": "meeting.mic.result",
  "muted": false,
  "label": "Turn off microphone",
  "noop": false,
  "warning": null,
  "request_id": "optional-correlation-id",
  "timestamp": "2026-03-21T14:32:00.000Z"
}
```

**Fields:**
- `muted` — final mic state after the operation (`true` = muted)
- `label` — the button's `aria-label` at the final state (platform-specific)
- `noop` — `true` if the requested action already matched current state (no click fired)
- `warning` — non-null in edge cases (e.g. Zoom mode 4 SS audio loopback conflict)
- `request_id` — echoed from the command if provided

If the toggle fails, an `error` event is sent instead, with `command: "meeting.mic"` and the matching `request_id`.

### Transcription Events

Requires `transcription: true` when creating the bot.

#### `transcript.partial`

In-progress transcription (updates as the speaker talks). Replace previous partial with the latest one.

```json
{
  "type": "transcript.partial",
  "text": "So the quarterly numbers",
  "speaker": { "id": "p-1", "name": "Alice" },
  "is_final": false,
  "timestamp": "2026-03-21T14:32:05.000Z"
}
```

#### `transcript.final`

Completed utterance (speaker paused for ~600ms). This is the final text — store this.

```json
{
  "type": "transcript.final",
  "text": "So the quarterly numbers show a fifteen percent increase in revenue.",
  "speaker": { "id": "p-1", "name": "Alice" },
  "is_final": true,
  "timestamp": "2026-03-21T14:32:08.000Z"
}
```

**Notes:**
- Speaker attribution uses the current active speaker. During brief silences, the last speaker is retained.
- Bot speech is automatically filtered — you won't receive transcripts of audio you injected.
- Empty transcripts (silence chunks) are dropped automatically.

### Audio Events

Requires `audio_streaming: true` when creating the bot.

#### `audio.chunk`

Raw meeting audio in real-time.

```json
{
  "type": "audio.chunk",
  "data": "base64-encoded-pcm-data...",
  "timestamp": "2026-03-21T14:32:10.000Z"
}
```

**Audio format:**
- PCM signed 16-bit little-endian (s16le)
- 16 kHz sample rate
- Mono channel
- 20ms chunks (640 bytes per chunk, ~50 chunks/second)

### Screenshot Events

Available in **all modes** (1-4).

#### `screenshot.result`

Response to a `screenshot.take` command.

```json
{
  "type": "screenshot.result",
  "data": "base64-encoded-jpeg...",
  "width": 1920,
  "height": 1080,
  "request_id": "req-123",
  "timestamp": "2026-03-21T14:33:00.000Z"
}
```

### Capture Events (Periodic Screenshots)

#### `capture.started`

Periodic capture has begun.

```json
{
  "type": "capture.started",
  "interval_ms": 1000,
  "timestamp": "2026-03-21T14:34:00.000Z"
}
```

#### `capture.frame`

A periodic capture frame.

```json
{
  "type": "capture.frame",
  "data": "base64-encoded-jpeg...",
  "frame_number": 5,
  "timestamp": "2026-03-21T14:34:05.000Z"
}
```

#### `capture.stopped`

Periodic capture has stopped.

```json
{
  "type": "capture.stopped",
  "total_frames": 30,
  "timestamp": "2026-03-21T14:34:30.000Z"
}
```

### Screenshare Events

Only available in **mode `webpage-av-screenshare`**.

#### `screenshare.started`

Screenshare is active in the meeting.

```json
{
  "type": "screenshare.started",
  "url": "https://your-app.com/slides",
  "timestamp": "2026-03-21T14:35:00.000Z"
}
```

#### `screenshare.stopped`

Screenshare has stopped.

```json
{
  "type": "screenshare.stopped",
  "timestamp": "2026-03-21T14:40:00.000Z"
}
```

#### `screenshare.error`

Screenshare failed to start.

```json
{
  "type": "screenshare.error",
  "message": "Failed to load URL",
  "timestamp": "2026-03-21T14:35:02.000Z"
}
```

### System Events

#### `ack`

Confirmation that your command was received and forwarded to the bot.

```json
{
  "type": "ack",
  "command": "meeting.send_chat",
  "request_id": "req-456",
  "timestamp": "2026-03-21T14:36:00.000Z"
}
```

#### `error`

Something went wrong with your command.

```json
{
  "type": "error",
  "message": "Bot container not connected",
  "command": "meeting.send_chat"
}
```

## Commands You Send

### Audio

#### `audio.send`

Inject audio into the meeting (the bot "speaks").

```json
{
  "type": "audio.send",
  "data": "base64-encoded-pcm-s16le-16khz-mono..."
}
```

Audio format must match: PCM s16le, 16 kHz, mono.

#### `audio.clear`

Stop any currently playing injected audio (barge-in).

```json
{
  "type": "audio.clear"
}
```

### Meeting Actions

#### `meeting.send_chat`

Send a chat message in the meeting.

```json
{
  "type": "meeting.send_chat",
  "message": "Hello everyone!"
}
```

#### `meeting.raise_hand`

Raise the bot's hand in the meeting.

```json
{
  "type": "meeting.raise_hand"
}
```

#### `meeting.leave`

Make the bot leave the meeting gracefully.

```json
{
  "type": "meeting.leave"
}
```

#### `meeting.mic`

Mute, unmute, or toggle the bot's microphone in the meeting. Works across Google Meet, Microsoft Teams (consumer + corporate), and Zoom Web Client.

```json
{
  "type": "meeting.mic",
  "action": "on" | "off" | "toggle",
  "request_id": "optional-correlation-id"
}
```

**Fields:**
- `action` — `"on"` unmutes, `"off"` mutes, `"toggle"` flips current state
- `request_id` — optional string echoed back on the result event for correlation

**Response:** emits a `meeting.mic.result` event on the WebSocket when the toggle completes (see [Meeting Events](#meeting-events) below).

If the action matches the current state (e.g. `"on"` when already unmuted), no click is fired and the response will include `"noop": true`.

**Zoom mode 4 caveat:** `webpage-av-screenshare` mode uses a PulseAudio loopback to route screenshare audio into the mic. Manual toggles still work but may conflict with the loopback. The response will include a `warning` field in this case.

**Errors:**
- `action must be 'on', 'off', or 'toggle'` — validation failure
- `Zoom mic button not found (audio not joined yet?)` — bot has not completed audio-join on Zoom
- `Click fired but mic state did not change within 2s` — click didn't take effect (DOM changed, or selector drift)

### Screenshots

#### `screenshot.take`

Take a single screenshot of the meeting view.

```json
{
  "type": "screenshot.take",
  "request_id": "req-123"
}
```

The `request_id` is returned in the `screenshot.result` event so you can match request to response.

### Periodic Capture

#### `capture.start`

Start capturing screenshots at a regular interval.

```json
{
  "type": "capture.start",
  "interval_ms": 1000
}
```

- Minimum interval: 500ms
- Must be a multiple of 250ms
- Each frame arrives as a `capture.frame` event

#### `capture.stop`

Stop periodic capture.

```json
{
  "type": "capture.stop"
}
```

### Screenshare

Only available in mode `webpage-av-screenshare`.

#### `screenshare.start`

Start screensharing a webpage URL into the meeting.

```json
{
  "type": "screenshare.start",
  "url": "https://your-app.com/slides"
}
```

#### `screenshare.stop`

Stop screensharing.

```json
{
  "type": "screenshare.stop"
}
```

## Complete Example: Transcript Bot

```javascript
const WebSocket = require('ws');

const API_KEY = process.env.FIRSTCALL_API_KEY;
const BOT_ID = process.env.BOT_ID;

const ws = new WebSocket(
  `wss://api.firstcall.dev/v1/bots/${BOT_ID}/ws?api_key=${API_KEY}`
);

const transcript = [];

ws.on('message', (data) => {
  const event = JSON.parse(data);

  switch (event.type) {
    case 'meeting.participant_joined':
      console.log(`${event.participant.name} joined`);
      break;

    case 'meeting.participant_left':
      console.log(`${event.participant.name} left`);
      break;

    case 'transcript.final':
      transcript.push({
        speaker: event.speaker?.name || 'Unknown',
        text: event.text,
        time: event.timestamp,
      });
      console.log(`[${event.speaker?.name}] ${event.text}`);
      break;

    case 'transcript.partial':
      // Show live typing indicator
      process.stdout.write(`\r[${event.speaker?.name}] ${event.text}...`);
      break;

    case 'error':
      console.error('Error:', event.message);
      break;
  }
});

ws.on('close', () => {
  console.log('\n--- Meeting transcript ---');
  transcript.forEach(t => {
    console.log(`[${t.speaker}] ${t.text}`);
  });
});
```

## Complete Example: AI Meeting Assistant

```javascript
const WebSocket = require('ws');

const ws = new WebSocket(
  `wss://api.firstcall.dev/v1/bots/${BOT_ID}/ws?api_key=${API_KEY}`
);

ws.on('message', async (data) => {
  const event = JSON.parse(data);

  if (event.type === 'transcript.final') {
    // Send transcript to your AI for processing
    const aiResponse = await processWithAI(event.text, event.speaker?.name);

    if (aiResponse) {
      // Send AI response as chat message
      ws.send(JSON.stringify({
        type: 'meeting.send_chat',
        message: aiResponse,
      }));

      // Or inject AI voice response
      // ws.send(JSON.stringify({
      //   type: 'audio.send',
      //   data: base64AudioFromTTS(aiResponse),
      // }));
    }
  }
});
```

## Rate Limits

- Maximum **100 commands per second** per WebSocket connection
- Exceeding the limit returns an error: `"Rate limit exceeded. Max 100 commands/second."`
- Maximum message payload: **1 MB**

## Supported Platforms

| Feature | Google Meet | Microsoft Teams | Zoom |
|---------|-----------|----------------|------|
| Participants | Yes | Yes | Yes |
| Active speaker | Yes | Yes | Yes |
| Chat messages | Yes | Yes | Yes |
| Transcription | Yes | Yes | Yes |
| Audio streaming | Yes | Yes | Yes |
| Screenshots | Yes | Yes | Yes |
| Screenshare (mode 4) | Yes | Yes | Yes |

## Bot Modes

| Mode | Description | Use Case |
|------|-------------|----------|
| `audio-ws` | Audio capture + injection via WebSocket | Transcription bots, AI voice agents |
| `webpage-audio` | Webpage with audio routing | Audio-only web apps in meetings |
| `webpage-av` | Webpage with audio + video injection | Slides, dashboards, video in meetings |
| `webpage-av-screenshare` | Full AV + screenshare | Complete meeting participation with screen sharing |