Skip to content
Docs menu

ElevenLabs, Fish Audio and BytePlus Seed Speech

Route text to speech and transcription to ElevenLabs, Fish Audio or BytePlus Seed Speech through the orchestrator's OpenAI-shaped audio API.

View .md

Three cloud speech providers have upstream kinds of their own: elevenlabs, fishaudio and byteplus-speech. Clients keep using the OpenAI audio endpoints, POST /v1/audio/speech, GET /v1/audio/voices and POST /v1/audio/transcriptions, and the orchestrator translates each request to the provider's API. Aliases, fallback, breakers, key budgets and the request log work as they do for other models. The text and audio you send to these providers leave your machine.

Configure the upstreams

These providers don't list models, so name the ones you use:

upstreams:
  - name: elevenlabs
    kind: elevenlabs
    base_url: https://api.elevenlabs.io/v1
    api_key_env: ELEVENLABS_API_KEY
    models: [eleven_flash_v2_5, eleven_multilingual_v2, scribe_v2]
  - name: fish
    kind: fishaudio
    base_url: https://api.fish.audio
    api_key_env: FISH_API_KEY
    models: [s2.1-pro, transcribe-1]
  - name: byteplus
    kind: byteplus-speech
    base_url: https://voice.ap-southeast-1.bytepluses.com
    api_key_env: BYTEPLUS_SPEECH_API_KEY
    models: [seed-tts-2.0, seed-asr-fast]
routes:
  - alias: venom-speak
    targets:
      - {upstream: elevenlabs, model: eleven_flash_v2_5}
pricing:
  models:
    elevenlabs/eleven_flash_v2_5: {per_1m_chars: 50}

None of the three has a list price in the bundled price table, because they bill through plans and credits. Set per_1m_chars (USD per 1M input characters) for speech and per_second (USD per second of audio) for transcription so that spend views and key budgets count them.

What each provider supports

Kind Auth Speech formats Voice Transcription
elevenlabs xi-api-key header mp3, opus, wav, pcm (8–48 kHz) required, a voice id Scribe (scribe_v2)
fishaudio bearer token mp3, opus, wav, pcm (8–44.1 kHz) optional, a voice model id transcribe-1, transcribe-1-pro
byteplus-speech X-Api-Key header (a Seed Speech key, not a ModelArk key) mp3, opus, pcm required, a speaker id the flash recognizer (seed-asr-fast), wav, mp3 or ogg files

If you ask for a format the provider can't produce, or leave out a voice that the provider requires, the orchestrator answers 400 with the code unsupported_by_provider and never calls the provider. A pcm answer gives its sample rate in Content-Type: audio/pcm;rate=… and in x-venom-sample-rate. Ask for another rate with venom_options.sample_rate.

The OpenAI fields are mapped to each provider's own options:

  • speed becomes ElevenLabs' voice_settings.speed, Fish Audio's prosody.speed, or BytePlus's speech_rate.
  • venom_options.language becomes language_code (ElevenLabs), language (Fish Audio) or explicit_language (BytePlus).
  • venom_options.provider is merged last into the provider's request body, so any other option is reachable.
  • With stream: true, ElevenLabs uses its streaming endpoint. BytePlus streams base64 chunks, which the orchestrator decodes to raw audio as they arrive.

Voices

GET /v1/audio/voices?model=<alias> lists:

  • ElevenLabs: the voices in your library.
  • Fish Audio: your own voice models, or the most used public ones if you have none.
  • BytePlus: a fixed subset of the official Seed Speech TTS 2.0 voices, because BytePlus has no voice-list API.

Each voice carries provider (the upstream name) and model.

Transcription

A transcription through any of the three answers response_format json, text, verbose_json, srt or vtt. The orchestrator builds each format from the provider's timed transcript and logs the billed audio as audio_ms. None of the three serves /v1/audio/translations. BytePlus's streaming recognition and bidirectional speech run over WebSocket and aren't offered here.