Multi-Language Support

Handle international and multilingual clinical encounters with 50+ supported languages

Overview

Sully.ai supports transcription and note generation in over 50 languages, enabling healthcare providers to document clinical encounters in the patient’s preferred language. The API offers two modes for language handling:

Mode	Best For	Behavior
Single-Language	Known language encounters	Audio in other languages is filtered out
Multilingual	Mixed-language conversations	Automatic language detection and transcription

Generated clinical notes are produced in the same language as the transcript, ensuring consistency throughout the documentation workflow.

---

Supported Languages

Sully.ai supports the following languages using BCP47 language tags:

Language	BCP47 Tags
Bulgarian	bg
Catalan	ca
Chinese (Mandarin Simplified)	zh, zh-CN, zh-Hans
Chinese (Mandarin Traditional)	zh-TW, zh-Hant
Chinese (Cantonese)	zh-HK
Czech	cs
Danish	da, da-DK
Dutch	nl
English	en, en-US, en-AU, en-GB, en-NZ, en-IN
Estonian	et
Finnish	fi
Flemish	nl-BE
French	fr, fr-CA
German	de, de-CH
Greek	el
Hindi	hi
Hungarian	hu
Indonesian	id
Italian	it
Japanese	ja
Korean	ko, ko-KR
Latvian	lv
Lithuanian	lt
Malay	ms
Norwegian	no
Polish	pl
Portuguese	pt, pt-BR, pt-PT
Romanian	ro
Russian	ru
Slovak	sk
Spanish	es, es-419
Swedish	sv, sv-SE
Thai	th, th-TH
Turkish	tr
Ukrainian	uk
Vietnamese	vi

---

Single-Language Mode

When you know the language of the clinical encounter in advance, specify it explicitly. This improves transcription accuracy and filters out audio in other languages.

File Upload

Specify the language when uploading audio files:

TypeScript

import SullyAI from '@sullyai/sullyai';
import * as fs from 'fs';

const client = new SullyAI();

// Transcribe Spanish audio
const transcription = await client.audio.transcriptions.create({
  audio: fs.createReadStream('patient-visit.mp3'),
  language: 'es',
});

console.log(`Transcription ID: ${transcription.transcriptionId}`);

Regional Variants

Use regional variants when relevant for better accuracy with regional accents and terminology:

TypeScript

// British English
const transcription = await client.audio.transcriptions.create({
  audio: fs.createReadStream('uk-patient-visit.mp3'),
  language: 'en-GB',
});

// Brazilian Portuguese
const transcriptionBR = await client.audio.transcriptions.create({
  audio: fs.createReadStream('brazil-patient-visit.mp3'),
  language: 'pt-BR',
});

// Canadian French
const transcriptionCA = await client.audio.transcriptions.create({
  audio: fs.createReadStream('quebec-patient-visit.mp3'),
  language: 'fr-CA',
});

When a specific language is set, audio in other languages will be filtered out or ignored. This is useful for ensuring clean transcripts when the encounter language is known.

---

Multilingual Mode

For clinical encounters where multiple languages are spoken (such as with an interpreter or bilingual patients), use multilingual mode with language=multi.

When to Use Multilingual Mode

• Patient and provider speak different languages
• Interpreter-assisted visits
• Bilingual patients who switch between languages
• Family members speaking different languages during the visit

Supported Languages for Multilingual Mode

Multilingual mode works well with the following languages:

• Dutch
• French
• German
• Hindi
• Italian
• Japanese
• Portuguese
• Russian
• Spanish

File Upload with Multilingual Mode

TypeScript

import SullyAI from '@sullyai/sullyai';
import * as fs from 'fs';

const client = new SullyAI();

// Transcribe a multilingual encounter
const transcription = await client.audio.transcriptions.create({
  audio: fs.createReadStream('interpreter-visit.mp3'),
  language: 'multi',
});

console.log(`Transcription ID: ${transcription.transcriptionId}`);

// Poll for completion
let result = await client.audio.transcriptions.retrieve(
  transcription.transcriptionId
);

while (result.status === 'STATUS_PROCESSING') {
  await new Promise((resolve) => setTimeout(resolve, 2000));
  result = await client.audio.transcriptions.retrieve(
    transcription.transcriptionId
  );
}

// Transcript includes all detected languages
console.log('Multilingual Transcript:', result.payload?.transcription);

---

Language in Streaming

When using real-time WebSocket streaming, specify the language as a URL parameter.

WebSocket URL Parameters

wss://api.sully.ai/v1/audio/transcriptions/stream?sample_rate=16000&account_id={id}&api_token={token}&language={language}

Single Language Streaming

TypeScript

// Get streaming token first
const token = await getStreamingToken();
const accountId = process.env.SULLY_ACCOUNT_ID!;

// Connect with Spanish language
const ws = new WebSocket(
  `wss://api.sully.ai/v1/audio/transcriptions/stream?sample_rate=16000&account_id=${accountId}&api_token=${token}&language=es`
);

ws.onmessage = (event) => {
  const data = JSON.parse(event.data);
  if (data.text) {
    console.log('Spanish transcript:', data.text);
  }
};

Multilingual Streaming

TypeScript

// Connect with multilingual mode
const ws = new WebSocket(
  `wss://api.sully.ai/v1/audio/transcriptions/stream?sample_rate=16000&account_id=${accountId}&api_token=${token}&language=multi`
);

ws.onmessage = (event) => {
  const data = JSON.parse(event.data);
  if (data.text) {
    // Automatically handles multiple languages
    console.log('Transcript:', data.text);
  }
};

---

Best Practices

Follow these guidelines to optimize language handling in your integration:

Use Specific Languages When Known

When you know the language of the encounter in advance, always specify it explicitly rather than using multilingual mode. Single-language mode provides:

• Better transcription accuracy
• Faster processing
• Reduced false positives from background noise in other languages

Use Regional Variants When Relevant

Regional variants improve accuracy for:

• Accents: en-GB for British accents, en-AU for Australian
• Medical terminology: Regional differences in drug names and procedures
• Spelling conventions: en-US vs en-GB spelling in generated notes

Reserve Multilingual Mode for True Multilingual Encounters

Only use language=multi when the conversation genuinely involves multiple languages:

• Interpreter-assisted visits
• Bilingual patient-provider conversations
• Family discussions involving multiple languages

Using multilingual mode when only one language is spoken may reduce transcription accuracy. Always prefer single-language mode when the encounter language is known.

Note Language Consistency

Generated clinical notes are produced in the same language as the transcript:

• Spanish transcript produces Spanish notes
• Multilingual transcripts produce notes in the dominant language of the conversation
• No separate language parameter is needed for note generation

---

Next Steps

Audio Transcription

Learn about file upload and real-time streaming options

Clinical Notes

Generate structured notes from multilingual transcripts

TypeScript SDK

Full SDK reference for Node.js applications

Python SDK

Full SDK reference for Python applications