Senior Machine Learning Engineer (Voice Agents) Hugging Face
Community-driven platform for ML models.
Permanent
Paris
Fully-remote
Salary: Not specified
> 5 years
Skills we're looking for
Communication skills
TICK
Llama
WebRTC
Python
You'll be part of
667 employees
Logiciels
Intelligence artificielle / Machine Learning
Our take
Large language models, which recognise and generate text, are already being used to auto-complete sentences, power chatbots, moderate content, summarise news articles, and translate text online. However, many of these advances are happening behind closed doors, stifling innovation and raising ethical concerns. To combat this, Hugging Face is building the "GitHub of machine learning". It’s a community-driven platform with a ton of repositories, where developers can create, discover and collaborate on ML models, datasets and ML apps. This work led to the creation of BLOOM in 2022, an open-access LLM with similar architecture to OpenAI’s GPT, but that is multilingual by design. The company also launched the Hugging Chat Assistant, an open-source tool creating personalised chat assistants, in direct competition with OpenAI's paid service ChatGPT Plus. By continually creating value based on a global community, not just for Microsoft or Google, HuggingFace should be able to carve out its own path for success.
You'll be working on
- We are building the open voice-agent stack for Hugging Face, and we are looking for a senior engineer to own a large part of it.
- Two things sit at the centre of this role
- The first is , our open-source library for realtime voice agents. The second is hf-voice, a new product that will let any developer build and deploy voice agents with their Hugging Face account
- Take architectural ownership of large parts of speech-to-speech: pipeline design, latency budget, and the reliability of the realtime loop
- Integrate new ASR, TTS and end-to-end speech models as they land, and keep the abstractions clean while the model landscape keeps moving
- Review community PRs, triage issues, cut releases, and grow the group of contributors around the project
- Design the developer API and the streaming protocol: session lifecycle, transport (WebSockets/WebRTC), authentication, error semantics, versioning
- Build the serving side: realtime inference on GPU, concurrency, autoscaling, observability, and cost per session
- Work with the Hub and inference teams so that a working voice agent is easy to integrate into products and demos
- Take the product from demo to production: load testing, SLOs, graceful degradation when a model or a network path misbehaves
- Write the docs, examples and templates that get a developer from zero to a running agent in minutes
- Support the deployments already relying on the stack, starting with the Reachy Mini fleet
- Talk about the work publicly if you enjoy it: blog posts, demos, conference talks. We cover the travel and the prep time
Benefits
- Flexible Work Environment
- Health Insurance
- Unlimited PTO
- Equity
- Growth, Training, & Conferences
- Generous Parental Leave
We're looking for
- Practical experience with LLMs or multimodal models in production. Clear written communication and a habit of collaborating async and in public
- You have shipped something realtime: streaming, WebSockets or WebRTC, audio or video pipelines, live inference
- Experience building developer-facing infrastructure at an AI or developer-tools company: inference APIs, agent infrastructure, or something comparable
- Senior engineer, able to own a substantial part of an architecture and drive it forward autonomously
- Substantial open-source contributions to a Python library. Comfortable with async Python and distributed systems, including their failure modes
- Motivated by voice and conversational AI
- If you’re interested in joining us, but don’t tick every box above, we still encourage you to apply! We’re building a diverse team whose skills, experiences, and backgrounds complement one another. We’re happy to consider where you might be able to make the biggest impact
- GPU serving, quantization, or on-device inference experience
- Experience shipping to embedded or robotics targets
- Contributions to llama.cpp or another low-level inference runtime
- A public track record: talks, blog posts, demos
- Hands-on work with ASR, TTS or end-to-end speech models, including evaluation of latency and quality trade-offs
- Contributions to a voice-agent framework such as speech-to-speech, pipecat, LiveKit Agents, Vocode or TEN
- Audio pipeline knowledge: VAD, echo cancellation, jitter buffers, barge-in and turn detection