We Built an AI Phone System in Minutes with xAI's New Tool
xAI announced Voice Agent Builder, in beta version, on 1 July 2026. The company, founded in 2023 by Elon Musk and Igor Babuschkin, presents it as a way to "create a personalised voice agent in under 2 minutes, without writing a single line of code". "No code" means that the basic setup can be done in the interface. The idea is to describe in natural language the expected behaviour of the agent, then provide it with the necessary documents and actions.
To achieve this, the Grok Voice Agent API is integrated with LiveKit Agents, the open source platform dedicated to real-time audio and video applications. xAI's no-code builder uses Grok Voice with a speech-to-speech approach. That is, the system processes the user's voice directly to produce a spoken response. Grok's voice models support expressive markers. In particular, they allow the addition of laughter, sighs, pauses or whispered passages to the voice response. However, connecting to an internal CRM, an order database or an existing telephone system may require configuring an API, an MCP connector or the help of a technical team.
LiveKit, for its part, offers a more software development-oriented approach. With the open source LiveKit Agents framework, developers can programme a voice agent in Python or Node.js. LiveKit also allows the three stages of a voice pipeline to be built separately: STT, for Speech-to-Text, LLM, to understand the request and generate a response, and TTS, for Text-to-Speech.
xAI offers a price of around 0.08 dollars per minute of audio for the grok-voice-think-fast-2.0 model. On top of this, there may be the cost of telephony, tools and storage.
Practical example: being precise in instructions
To assess the truth of these promises, we design a no-code voice agent through a concrete use case. We create a fictitious grocery shop, named for the occasion "Epicerie Bruno". We avoid exposing personal data or immediately connecting a real payment system.
To do this, we go to https://x.ai/voice. We try the tool for free by clicking on the "Try it free" button at the top left of the interface. We click on "Create agent". In the chatbot, as advised by the bot, we describe our project. Our instructions define the role, tone, conversational flow and transfer conditions. While these reduce the risk of inappropriate responses, they do not guarantee the absence of errors. In production, prices, stock and orders should be checked in a business source using a tool or API.
I want to create a voice agent for a grocery shop. The agent must greet customers in French, answer questions about products, prices, opening hours, address and item availability. It must help customers prepare an order by asking for items and quantities, then repeat the order to obtain confirmation. It must speak in a natural, polite, warm and concise manner. It must never invent a price, stock level or information about the shop. If it does not know the answer, it must say so clearly and offer to transfer the call to an employee. For refunds, payments or complex complaints, it must also direct the customer to a human employee.
We then provide information about the imaginary grocery shop, directly in the chat.
Available products:
- Semi-skimmed milk 1 L - Wholemeal bread: €2.50 - Bananas, 1 kg: €2.20 - Free-range eggs, box of 6: €2.90 - Ground coffee, 250 g: €4.80 - Water, pack of 6 bottles: €3.60 - Rule: never invent a product, price or availability. If the information is not in the catalogue, say that a human check is necessary.
We also provide the delivery rules:
- Delivery from Monday to Saturday, 9 am to 7 pm. - Free delivery from €30. - Below €30, the fee is €3. - Usual delivery time is 45 to 90 minutes. - The agent must never ask for the full bank card number. - The agent must never confirm a paid order: it only prepares a basket and transfers validation to a human.
It is possible to add the knowledge base from "File Collection", at the bottom of the Agent Builder interface. This section is used to provide information that the agent can retrieve during the call. It can use it as a form of RAG (Retrieval-Augmented Generation). However, this requires credits. After importing, check that the files are properly processed and associated with the correct agent. Once validated, these instructions are directly mentioned and structured in the Agent Builder.
A fairly conclusive test
After choosing the voice, language, speed and expressiveness of the AI, we carry out our test. It is possible to test the agent in the browser. We first ask it questions about the products, then we carry out error tests. For example, we ask it for a product that is not available. We call with slight background noise to see if the agent correctly detects our requests.
We see that the agent responds correctly according to the information provided. But the voice is a bit "metallic". The tone lacks a little warmth. The latency is acceptable, sometimes a bit long. It tends to repeat the same things. It should have requested a transfer to an employee more quickly. You can view the conversation in the tab of the same name. The logs help identify errors. The main area for improvement is to make the responses more useful, while maintaining caution with unknown information.
Note: the real phone number is preferably added after configuring the agent. xAI states that a Voice Agent Builder account includes a number that can be used for testing and then for production traffic. This is done in the "Setup a phone number" section. In our case, a message indicates that xAI cannot directly assign a new number in our region. We therefore connect a number using Direct SIP or Twilio. Direct SIP links our telephone operator to the xAI agent. Twilio can be chosen if you already have an account there. Once one of the two options is selected, a closed pilot is carried out, then the service is gradually opened up.
Some challenges to manage
This test shows that Voice Agent Builder does indeed simplify the creation of a first voice agent. Instructions, voice, documents, tools, safeguards and telephony can all be configured from a single interface. But "no code" does not mean no integration, no cost, or no human oversight. For a business, the real challenge lies in connecting to real data, securing actions, transferring to employees and validating before going live to the public.
This article was automatically translated from French. Read the original version: https://www.journaldunet.com/intelligence-artificielle/1554569-comment-creer-des-agents-vocaux-sans-code-avec-xai/