This API provides streaming speech-to-text transcriptions using WebSockets.

Endpoint:
wss://api.sully.ai/v1/audio/transcriptions/stream?account_id=1234567890&api_token=1234567890&sample_rate=16000&language=en

Headers

X-API-KEY

string

The API key to use for authentication. Required if X-API-TOKEN is not provided.

X-API-TOKEN

string

The API token to use for authentication. Required if X-API-KEY is not provided.

X-ACCOUNT-ID

string

The account ID to use for authentication.

Query parameters

language

string

default:"en"

The language of your submitted audio. See our Supported Languages documentation for a complete list of language options.

sample_rate

string

The sample rate of your submitted audio.

encoding

string

Specifies the encoding format of the audio being sent.

Important: This parameter is required when transmitting raw, unstructured audio packets without headers. If the audio data is encapsulated within a container format, this parameter should be omitted.

Supported formats:

linear16: 16-bit, little-endian PCM audio
flac: Free Lossless Audio Codec (FLAC)
mulaw: Mu-law encoded WAV
amr-nb: Adaptive Multi-Rate, narrowband
amr-wb: Adaptive Multi-Rate, wideband
opus: Ogg Opus codec
speex: Speex codec
g729: G729 codec (usable with raw or containerized audio)

account_id

string

The account ID to use for authentication. Required if X-ACCOUNT-ID is not provided.

api_token

string

A temporary authentication token. Required if X-API-KEY is not provided.

When to use

The Speech-to-Text Websockets API is designed to generate text from partial audio input. It’s well-suited for scenarios where the input audio is being streamed or generated in chunks.

Protocol

The WebSocket API uses a bidirectional protocol that encodes all messages as JSON objects.

Connection Status Messages

Upon successful connection, the server sends a status message:

{
  "status": "connected",
}

When the connection closes:

{
  "status": "disconnected",
}

Important: Wait for the “status”: “connected” message before sending audio data. This ensures the server is ready to process your stream.

Streaming input audio

The client can send messages with audio input to the server. The messages can contain the following fields:

{
  "audio": "Y3VyaW91cyBtaW5kcyB0aGluayBhbGlrZSA6KQ=="
}

audio

string

required

A generated partial audio chunk encoded as a base64 string.

Browser MediaRecorder Notice: When using Chrome’s MediaRecorder API, the first audio chunk contains critical header information. Always send this first chunk for proper audio processing. Failing to include header information may result in transcription errors or complete failure.

Streaming output audio

The server will always respond with a message containing the following fields:

Response message

{
  "type": "transcript",
  "audio_start": 0,
  "audio_end": 3.2,
  "duration": 3.2,
  "text": "Hello, world",
  "isFinal": false, 
  "is_final": false,
  "words": [
    {
      "word": "Hello",
      "start": 0.2,
      "end": 0.7,
      "confidence": 0.9856743,
      "punctuated_word": "Hello,"
    },
    {
      "word": "world",
      "start": 0.9,
      "end": 1.2,
      "confidence": 0.9978341,
      "punctuated_word": "world"
    }
  ],
  "timestamp": "2023-08-15T14:32:07.123Z"
}

type

string

The type of response, will be “transcript” for transcription results.

audio_start

number

Start time of the audio segment in seconds.

audio_end

number

End time of the audio segment in seconds.

duration

number

Duration of the audio segment in seconds.

text

string

The processed text sequence.

isFinal

boolean

deprecated

Indicates if the generation is complete. Deprecated: Use is_final instead.

is_final

boolean

Indicates if the generation is complete.

words

array

Array of word objects with text content and timing information:

word: The raw word as recognized
start: Start time of the word in seconds
end: End time of the word in seconds
confidence: Confidence score between 0-1 for the word recognition
punctuated_word: The word with proper capitalization and punctuation

timestamp

string

ISO-formatted timestamp when the response was generated.

Delete Audio Transcription Create Streaming Token

On this page

Headers
Query parameters
When to use
Protocol
Connection Status Messages
Streaming input audio
Streaming output audio

This API provides streaming speech-to-text transcriptions using WebSockets.

Endpoint:
wss://api.sully.ai/v1/audio/transcriptions/stream?account_id=1234567890&api_token=1234567890&sample_rate=16000&language=en

Headers

X-API-KEY

string

The API key to use for authentication. Required if X-API-TOKEN is not provided.

X-API-TOKEN

string

The API token to use for authentication. Required if X-API-KEY is not provided.

X-ACCOUNT-ID

string

The account ID to use for authentication.

Query parameters

language

string

default:"en"

The language of your submitted audio. See our Supported Languages documentation for a complete list of language options.

sample_rate

string

The sample rate of your submitted audio.

encoding

string

Specifies the encoding format of the audio being sent.

Supported formats:

linear16: 16-bit, little-endian PCM audio
flac: Free Lossless Audio Codec (FLAC)
mulaw: Mu-law encoded WAV
amr-nb: Adaptive Multi-Rate, narrowband
amr-wb: Adaptive Multi-Rate, wideband
opus: Ogg Opus codec
speex: Speex codec
g729: G729 codec (usable with raw or containerized audio)

account_id

string

The account ID to use for authentication. Required if X-ACCOUNT-ID is not provided.

api_token

string

A temporary authentication token. Required if X-API-KEY is not provided.

When to use

The Speech-to-Text Websockets API is designed to generate text from partial audio input. It’s well-suited for scenarios where the input audio is being streamed or generated in chunks.

Protocol

The WebSocket API uses a bidirectional protocol that encodes all messages as JSON objects.

Connection Status Messages

Upon successful connection, the server sends a status message:

{
  "status": "connected",
}

When the connection closes:

{
  "status": "disconnected",
}

Important: Wait for the “status”: “connected” message before sending audio data. This ensures the server is ready to process your stream.

Streaming input audio

The client can send messages with audio input to the server. The messages can contain the following fields:

{
  "audio": "Y3VyaW91cyBtaW5kcyB0aGluayBhbGlrZSA6KQ=="
}

audio

string

required

A generated partial audio chunk encoded as a base64 string.

Streaming output audio

The server will always respond with a message containing the following fields:

Response message

{
  "type": "transcript",
  "audio_start": 0,
  "audio_end": 3.2,
  "duration": 3.2,
  "text": "Hello, world",
  "isFinal": false, 
  "is_final": false,
  "words": [
    {
      "word": "Hello",
      "start": 0.2,
      "end": 0.7,
      "confidence": 0.9856743,
      "punctuated_word": "Hello,"
    },
    {
      "word": "world",
      "start": 0.9,
      "end": 1.2,
      "confidence": 0.9978341,
      "punctuated_word": "world"
    }
  ],
  "timestamp": "2023-08-15T14:32:07.123Z"
}

type

string

The type of response, will be “transcript” for transcription results.

audio_start

number

Start time of the audio segment in seconds.

audio_end

number

End time of the audio segment in seconds.

duration

number

Duration of the audio segment in seconds.

text

string

The processed text sequence.

isFinal

boolean

deprecated

Indicates if the generation is complete. Deprecated: Use is_final instead.

is_final

boolean

Indicates if the generation is complete.

words

array

Array of word objects with text content and timing information:

word: The raw word as recognized
start: Start time of the word in seconds
end: End time of the word in seconds
confidence: Confidence score between 0-1 for the word recognition
punctuated_word: The word with proper capitalization and punctuation

timestamp

string

ISO-formatted timestamp when the response was generated.

Delete Audio Transcription Create Streaming Token

On this page

Headers
Query parameters
When to use
Protocol
Connection Status Messages
Streaming input audio
Streaming output audio

Streaming

Headers

Query parameters

When to use

Protocol

Connection Status Messages

Streaming input audio

Streaming output audio

Getting Started

Notes

Note Styles

Audio Transcriptions

Codings

Macros

Utils

Events

Schemas

Streaming

Headers

Query parameters

When to use

Protocol

Connection Status Messages

Streaming input audio

Streaming output audio

​Headers

​Query parameters

​When to use

​Protocol

​Connection Status Messages

​Streaming input audio

​Streaming output audio

Getting Started

Notes

Note Styles

Audio Transcriptions

Codings

Macros

Utils

Events

Schemas

​Headers

​Query parameters

​When to use

​Protocol

​Connection Status Messages

​Streaming input audio

​Streaming output audio

Headers

Query parameters

When to use

Protocol

Connection Status Messages

Streaming input audio

Streaming output audio

Headers

Query parameters

When to use

Protocol

Connection Status Messages

Streaming input audio

Streaming output audio