Speech Transcription
Redactor example guide
Generate speech transcription data through the Redactor API, optionally alongside detection and render operations.
Transcript output
Save `speech.json` with the output state by enabling speech transcription data in the request body.
Language control
Specify a supported language code when the source media is not using the English default.
API-only mode
Run transcription by itself when you only need structured speech data and minimal metadata.
A basic speech transcription feature is available as part of the Redactor API. Adding the SPEECH_TRANSCRIPTION feature to the request, either by itself or with along with other features, will analyze the video and attempt to detect the words spoken. The speech transcript will be available in the Redactor UI for use in the editor and is also available as a JSON file as part of the API.
To output the speech.json file, set outputContext.speechTranscriptionData: true in your request body, and it will be saved along with the other standard redaction state data. Be sure to provide a path to a folder in the outputUri, and ensure the path ends in a trailing slash.
curl --location --request POST 'http://localhost:9000/api/v1/videos:process' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer YourRedactorApiToken' \
--data-raw '{
"inputUri": "https://example.com/path/to/input.mp4",
"features": ["HEAD_DETECTION", "SPEECH_TRANSCRIPTION", "MEDIA_RENDERING"],
"outputUri": "file:///path/to/output/folder/",
"outputContext": {
"speechTranscriptionData": true
}
}'
By default, English is used for the transcription. If a different language is being spoken in the media, specify that language code in videoContext.speechTranscriptionConfig.languageCode. The currently available language codes are:
| Code | Language | Word timing |
|---|---|---|
en-us |
English (default if none specified) | Forced alignment (ships built-in) |
cs |
Čeština (Czech) | Forced alignment (on-demand download) |
de |
Deutsch (German) | Forced alignment (on-demand download) |
es |
Español (Spanish) | Forced alignment (on-demand download) |
fr |
Français (French) | Forced alignment (on-demand download) |
it |
Italiano (Italian) | Forced alignment (on-demand download) |
nl |
Nederlands (Dutch) | Forced alignment (on-demand download) |
pl |
Polski (Polish) | Forced alignment (on-demand download) |
pt |
Português (Portuguese) | Forced alignment (on-demand download) |
ru |
Русский (Russian) | Forced alignment (on-demand download) |
cn |
中文 (Chinese) | Offsets only |
ja |
日本語 (Japanese) | Offsets only |
ko |
한국어 (Korean) | Offsets only |
Word-level timing comes from a per-language forced-alignment model. English ships with the server; the other alignment-capable languages are fetched on demand (see On-demand alignment models). The CJK languages (cn, ja, ko) have no forced-alignment model and are transcribed with offsets-only timing.
curl --location --request POST 'http://localhost:9000/api/v1/videos:process' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer YourRedactorApiToken' \
--data-raw '{
"inputUri": "https://example.com/path/to/input.mp4",
"features": ["HEAD_DETECTION", "SPEECH_TRANSCRIPTION", "MEDIA_RENDERING"],
"outputUri": "file:///path/to/output/folder/",
"outputContext": {
"speechTranscriptionData": true
},
"videoContext": {
"speechTranscriptionConfig": {
"languageCode": "es"
}
}
}'
{
"inputUri": "https://example.com/path/to/input.mp4",
"features": ["HEAD_DETECTION", "SPEECH_TRANSCRIPTION", "MEDIA_RENDERING"],
"outputUri": "file:///path/to/output/folder/",
"outputContext": {
"speechTranscriptionData": true
},
"videoContext": {
"speechTranscriptionConfig": {
"languageCode": "es"
}
}
}
If a speech transcript is all that's needed, this can be obtained without performing any redactions on the media. In the following example, we are only running the SPEECH_TRANSCRIPTION feature and configuring the outputContext to save the transcription data and to skip saving the full redaction state. The speech.json will ultimately be saved to the output location along with some minimal metadata. Note that this additional metadata may not be output by default in future releases.
curl --location --request POST 'http://localhost:9000/api/v1/videos:process' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer YourRedactorApiToken' \
--data-raw '{
"inputUri": "https://example.com/path/to/input.mp4",
"features": ["SPEECH_TRANSCRIPTION"],
"outputUri": "file:///path/to/output/folder/",
"outputContext": {
"speechTranscriptionData": true,
"state": false
},
"videoContext": {
"speechTranscriptionConfig": {
"languageCode": "en-us"
}
}
}'
On-demand alignment models
Word-level timing requires a per-language forced-alignment model. English (en-us) ships with the server; the other alignment-capable languages listed above are downloaded on demand.
For API and in-process callers, alignment models are resolved automatically at transcription time:
- If the requested language's alignment model is already installed, transcription proceeds with word-level timing.
- If the model is missing but downloadable, the server fetches it inline (verifying its checksum) and then transcribes. Only one model download runs server-wide at a time.
- If the language is alignment-capable but the model is not downloadable (the deployment has on-demand download disabled, or no archive is published for it), the request errors instead of silently degrading.
- The CJK languages (
cn,ja,ko) have no alignment model and are always transcribed with offsets-only timing: words are returned without tight per-word start/end alignment. No download is attempted.
Pre-installing languages
To avoid first-request download latency, a deployment can pre-install alignment models so the language is already present when the request arrives.
Model source URL
The server fetches alignment models using a pointer file that lists download URLs, sizes, and checksums. The pointer URL is resolved with the following precedence:
- The
REDACTOR_SPEECH_MODELS_URLenvironment variable (operator override). - The brand's
features.speechDetection.pointerUrlconfiguration. - The built-in default (
https://sys.redactor.com/speech-models.json).
Set REDACTOR_SPEECH_MODELS_URL (or host a brand pointer file) to serve models from your own repository — useful for air-gapped or bandwidth-controlled deployments.
Speaker data in speech.json
When speaker diarization runs, each transcribed word in speech.json can carry a speaker object identifying who spoke it:
id— the speaker number, assigned0, 1, 2, …in the order each distinct speaker first appears.certainty— a0..1confidence score for the assignment.manual— present andtrueonly when a user reassigned the word in the editor; pipeline-assigned words omit it.
Human-readable speaker names are stored separately in speakerLabels, a per-language map of speaker id to display name. Speaker ids are assigned independently for each language, so the labels are nested by language code to keep ids from colliding across languages. A word with no speaker, or a language with no speakerLabels entry, simply means diarization produced no speaker for it.
{
"currentLanguageCode": "en-us",
"version": "1.0",
"items": {
"en-us": [
{ "region": [400, 700], "text": { "word": "Hello", "speaker": { "id": 0, "certainty": 0.97 } } },
{ "region": [1100, 1400], "text": { "word": "Officer", "speaker": { "id": 1, "certainty": 0.88 } } }
]
},
"speakerLabels": {
"en-us": { "0": "Reporter", "1": "Subject" }
}
}
Each item's region is an [start, end] pair in milliseconds. Speaker data is additive: existing consumers that read only word and region continue to work unchanged.