Live transcription and intelligence

Live transcription processes a conversation while it is happening. Audio streams over WebSocket, transcript segments arrive as they are recognized, and optional intelligence features turn the running conversation into alerts, metrics and guidance that agents and supervisors can act on.

Live features

Live Transcription is enabled per account. Each live feature below is then switched on individually for the account:

Feature	What it adds	Requires
Keyword Detection	Detects your configured keywords and phrases as segments arrive.	—
Alerts & Rules	Turns detections and frustration into alerts using alert rules.	Keyword Detection
Metrics	Talk time, silence, interruptions, response time, speaking rate, questions and filler words.	—
Sentiment & Intent	Customer sentiment, frustration (0–100) and call intent.	—
Conversation State	A running view of stage, issues, entities, requested actions and open items.	Sentiment & Intent
Summary	A concise running summary and the customer's current request.	Conversation State
Agent Coaching	Suggested responses, next best action, questions to ask, opportunities and compliance reminders.	Conversation State
Live Language Detection	Records each speaker's language and enables multilingual and automatic translation modes.	—
Events API	Exposes everything above as a machine-readable event stream.	—

Changes apply to calls started afterwards.

Cost and latency

Sentiment & Intent, Conversation State, Summary, Coaching and translation use an AI model during the call. Metrics and keyword detection are computed directly from the transcript and are inexpensive.

Session lifecycle

Live sessions are started and stopped through the API; see Start a live transcription. While a session runs, the call's status is 8 (Live).

The pinCode returned when a session starts lets a viewer follow the live transcript, optionally translated, in the Speechlytics live viewer without an account. Coaching is never shown to PIN viewers.

Mono and stereo

	Stereo	Mono
Agent	Second channel, returned as Left	Shared Both channel
Customer	First channel, returned as Right	Shared Both channel
Per-speaker metrics	Available	Not separated
Per-speaker translation	Available	Single shared target

Speechlytics does not estimate speaker-specific values when mono audio does not preserve speaker separation.

Live capabilities

Keyword detection

Live keyword detection matches the account's configured keywords and phrases as finalized segments arrive. It is independent of post-call keyword matching, which still runs when the call ends.

Alert rules

Alert rules are configured per account. Each rule has a name (the alert's headline), a keyword (a specific one or any keyword), a trigger with a threshold, and a severity of Info, Warning or Critical.

Trigger	Fires when
On first detection	The keyword is spoken for the first time.
On reaching a count	The keyword has been detected threshold times.
On frustration level	Customer frustration reaches threshold (0–100). Needs Sentiment & Intent.
When not said in time	The keyword has not been said within threshold seconds of the call start, for example a mandatory disclosure. Needs a specific keyword.

Each rule fires at most once per call. Alerts are delivered as AlertTriggered live events.

Conversation metrics

Metrics are calculated from the live transcript and audio duration: talk share per speaker, silence, interruptions, longest monologue, response time, words per minute, agent and customer questions, unanswered questions and filler words (English, German, Spanish, French and Italian).

Mono conversations do not report speaker-separated metrics, and unanswered questions remain zero because there is no reliable second channel to inspect.

Sentiment, intent and conversation state

Sentiment & Intent classifies the customer's sentiment (positive, neutral, negative), a frustration score from 0 to 100, and the call intent: cancellation, complaint, billing, technical, sales, retention, information or other. It is refreshed roughly every 20 seconds, starting after the first 15 seconds of conversation.

Conversation State adds a running structured view: the stage (greeting, discovery, resolution, closing), issues raised, entities, requested actions and resolved or unresolved items.

Summary and agent coaching

Live Summary maintains a concise account of the conversation and the customer's current request, refreshed about every minute.

Agent Coaching adds a suggested response, a next best action, up to three recommended questions, opportunities and compliance reminders. It also raises rule-based warnings, for example when the agent dominates the conversation, silence is high, frustration is high or several issues remain unresolved.

Coaching is agent-facing and is never returned to a public PIN viewer.

Language detection and translation

A session can translate to a fixed target language (translationLanguage).

With Live Language Detection enabled, two more modes are available when starting the session:

• multilingualDetection: true re-detects the spoken language on every segment, for calls that switch languages;
• autoTranslationDirection: true translates each speaker into the language detected for the other speaker, falling back to the configured target until both languages are known.

Consuming live events

With the Events API enabled, every detection, alert, sentiment change, entity, summary, coaching suggestion and translation is available by cursor or WebSocket. See Live events.

When a live call ends

Stopping the session finalizes the stored call. Enabled post-call features then run as for any recording: sensitive-content masking, keyword matching, translation, summaries, analytics and custom questions. Retrieve the result with GET /api/v1/calls/{id}/status.

• On accounts with sensitive-content protection, the call stays In Progress until masking is finished.
• Otherwise, on accounts with polishing enabled, the transcript is polished automatically first; keywords and analytics run on the polished text.
• The recording is stored at 16 kHz, like any other call.

Troubleshooting

Symptom	What to check
Start returns 403	Live Transcription is not available for the account.
No recognition messages	Confirm authentication, the WAV header, 16-bit PCM, sample rate, channel count and real-time pacing.
Speakers are reversed	The agent must be on the second channel and the customer on the first.
No alerts fire	Turn on both Keyword Detection and Alerts & Rules, and make sure the rule is enabled.
Live events return 403	The Events API is not enabled for the account.
An event type never appears	Its feature, or a feature it depends on, is not enabled for the account.
Translation starts with the fallback target	Automatic direction has not detected both speakers' languages yet.