Bridging the Gap: Connecting 3CX to ElevenLabs AI Voice Agents for Next-Level Call Handling

33

We’re all used to Interactive Voice Response menus – when you call your bank or ISP, you’ve probably had to navigate at least one (but often up to four nested menus!). Listening closely for the required menu option, then picking the closest when none really align to your goal, and pressing the dreaded option to repeat the menu prompt all over again could be behind us.

ElevenLabs offers AI Voice Agents that support natural language processing and generation, and supports integration into phone systems using SIP services, which makes it perfect for integrating into 3CX. We’ll start with setting up your AI voice agent, and move onto configuring 3CX to handle calls in and out of the system.

Please note that this blog post contains affiliate links – helps me keep the lights on!

Understanding the Players: 3CX and ElevenLabs AI Voice Agent

Before we get our hands dirty with configuration, let’s briefly introduce the two stars of our show:

3CX Phone System

For those unfamiliar, 3CX is a leading open-standard IP PBX that offers a comprehensive suite of unified communications features. It’s a software-based solution that allows businesses to manage their phone calls, video conferencing, live chat, and more, all from a single platform. Whether you’re a small startup or a large enterprise, 3CX provides flexibility, scalability, and cost-effectiveness. It can be deployed on-premise, in the cloud, or as a hosted solution, making it adaptable to various business needs. With features like auto-attendants, call queues, CRM integration, and mobile apps, 3CX empowers businesses to handle communication efficiently and professionally.

Note that for today’s project, you’ll need to be running the self-hosted or private-cloud version of 3CX, as it requires you setup a custom SIP provider which is unsupported in 3CX’s own cloud-hosted option.

ElevenLabs AI Voice Agents

ElevenLabs is at the forefront of AI speech technology, and their AI Voice Agents are truly revolutionary. These agents are designed to have natural, human-like conversations, understand context, and even perform actions based on user input. Unlike traditional IVRs that rely on rigid menu structures, ElevenLabs AI Voice Agents can engage callers dynamically, providing a much more fluid and satisfying experience. Their ability to synthesize high-quality, realistic voices adds another layer of authenticity, making interactions feel less robotic and more human. From customer support to information retrieval, the possibilities with ElevenLabs AI Voice Agents are virtually limitless.

Building Your Smart AI Voice Agent

This is the fun part – let’s create an ElevenLabs AI Voice Agent that will intelligently route your calls.

While writing this post, the kids are watching Monsters Inc, so naturally I’ll be using references to the characters and organization in the film.

1. Setting up your 3CX Phone System

You’ll need to be running an on-premises or private-cloud version of 3CX for this to work. You can get a free trial of 3CX Pro for 1 month. If you’re private-cloud hosting, I’d recommend spinning up a virtual machine in Vultr.

While I won’t cover the basic setup for 3CX, as 3CX has done a great job of covering this themselves.

Once you’re logged in to the Admin section of your 3CS system, we’re ready to get onto the first part.

Creating departments, users and call queues

Go to Users and create one for each person who’ll be answering the phone. I’ve created three users: James Sullivan, Mike Wazowski and Randal Boggs. You can optionally create departments but it’s not required to get you started.

Next, go to Call Handling and click + Add Queue. Set the name of the queue that the user should ask for and take note of the Virtual extension number as you’ll need this when you route your call from your AI Voice Agent. Go to the Users tab and add the users you created from the previous step as required. Save the queue, and repeat for each department you need.

My Queues are as follows:

Creating a custom SIP trunk and call routes

Let’s get 3CX ready to connect to ElevenLabs.

Go to Voice & Chat and click + Add Trunk. Set your country accordingly, the provider to Generic SIP Trunk (IP Based).

Once the trunk has been created, open the trunk, and on the General tab, wset the following options:
Name: ElevenLabs Trunk
Main Trunk Number: You can use any number here, as long as it’s populated
Registrar/Server: sip.rtc.elevenlabs.io

On the Options tab, set the following options:
Transport mode: TCP
Include diversion header support: Yes

You’ll be prompted to create an outbound rule for this trunk, set the following options:
Calls to numbers starting with prefix: Since ElevenLabs will only work with proper E164 phone numbers, I’m using phone numbers reserved for fictional use in my country as an example.
Calls to numbers with a length of: The length of your phone number
Prepend: +

If you’re using multiple agents, you could use a number range and tweak the prefix to add more options.

Now we’re done with the 3CX part, now to move onto ElevenLabs.

2. Creating a Basic AI Voice Agent

First things first, if you haven’t got one already, create yourself an account with ElevenLabs.

Login to your account and navigate to Conversational AI -> Agents, and click New Agent. I’d suggest starting with Business Agent and the type of business you’re building it for. Follow the prompts to create a starting point for your agent. I’ve named my agent Celia Mae, after the receptionist at Monsters Inc.

Once your agent is created, tweak the System Prompt to your liking, in terms of personality and tone. In the goal section, we’ll need to make sure your agent is prepared to transfer your caller to an extension on your phone system. Here’s my System prompt for Celia:

# Personality

You are Celia, a professional and efficient answering service representative for Monsters Incorporated. You are polite, clear, and helpful, ensuring all callers are directed to the correct department.

# Environment

You are answering phone calls for Monsters Incorporated. Callers are external and may be trying to reach Sales, Accounts, or Service departments.


# Tone

Your responses are bubbly, courteous, and friendly. You should use inclusive and welcoming language, but be brief. Avoid jargon. Speak at a moderate pace and enunciate clearly.

# Goal

Your primary goal is to efficiently direct incoming calls to the correct department within Monster Incorporated.

1.  Greeting: Answer the call promptly with a professional greeting, "Thank you for calling Monsters Incorporated, this is Celia, how may I direct your call?"
2.  Information Gathering: Clearly identify which department the caller needs to reach (Sales, Accounts, or Service). If the caller is unsure, ask clarifying questions to determine the nature of their inquiry.
5.  Confirmation: Confirm which department the caller has asked for politely. "So it was {{department}} you were after? Let me transfer you now." And transfer the call immediately.

# Guardrails

Do not provide information about company financials, internal policies, or employee information beyond publicly available directory information. Transfer calls directly. If a caller is abusive or uses inappropriate language, politely end the call.

# Tools

None

Next, scroll down to the Tools section and enable End Call and Transfer to number. Once enabled, click the settings gear icon next to Transfer to number.

Under the Human Transfer Rules, click Add Rule and set the Transfer type to SIP REFER.

Set the Number type to SIP URI, and set the SIP URI value to “sip:<queue extension number>@<3cx server address>”. It should look a bit like: “sip:811@my-ai-demo.3cx.com.au”. The queue extension number is the number you recorded while setting up the queue in 3CX.

Set the condition for the first department you’d like to transfer to: “If the caller wants to speak to the Accounts department”.

Click Add Rule again and repeat for every department you wish to add.

You might also want to add a condition for if the caller becomes agitated or asks to speak to a human.

We’re pretty much done setting up our agent. Feel free to customize your agent further, like the voice selection.

Don’t forget to save your agent!

3. Connecting ElevenLabs to 3CX

We’re now on the home stretch!

Navigate to Telephony -> Phone Numbers and click Import Number -> From SIP Trunk.

Set your Label to something logical, I called mine Monsters Inc Switchboard. Then set your phone number to the number you set in the Outbound rule in 3CX.

Set the Origination URI to “sip:sip.rtc.elevenlabs.io:5060;transport=tcp”, and Media Encryption to “Allowed”.

In the Outbound Configuration, set these options:
Address: <your 3cx server FQDN> ie: my-ai-demo.3cx.com.au
Transport Type: Auto
Media Encryption: Allowed

Save the SIP trunk, then set the agent for your newly created SIP trunk.

4. Finalization

Great, everything is now setup!

From one of your extensions, dial the phone number you setup for your AI Voice Agent. With any luck, you’ll hear an AI generated voice asking where you’d like your call directed. Ask it to put you through to accounts and see how it goes!

No doubt you’ll want to tweak your agent before going live, but once you’re happy with your agent, setup one of your inbound routes to forward to the external number for your agent.

Did you get stuck or need help?

It is a bit of a process to get setup, but understanding where to look when things go wrong can point you in the right direction.

Check your call logs in 3CX and double check your configuration.

Here’s some issues I found while doing the initial implementation:

My call never connects or rings out
Make sure you added the + in the prepend section in your outbound rule, and your numbers match in both your 3CX outbound rules and Phone Numbers in ElevenLabs.

My agent says the call transfer failed
Make sure Include diversion header support is enabled on your ElevenLabs SIP trunk. Also check the queue extension numbers match your Human Transfer Rules in your agent configuration.

My agent gives me an extension number and refuses to transfer me
Make sure you enabled Transfer to Number in the agent’s tools section, and in your System prompt told the agent they can transfer calls directly.

We provide telephony consulting services at my day job, so please get in touch if you need further help.

33 Comments

  1. Pingback: ElevenLabs + 3CX: Passing through Caller ID - Ben's Blog

  2. This was very helpful, thanks!

    Not sure if you can help with this issue, but I have 3CX sending calls to an elevenLabs agent – everything works well, but I am unable to get the agent to end the 3CX call. It ends the call in Eleven labs, but if you’re calling from a phone, it doesn’t hang up.

    • Have you got “End call” enabled under the tools section? I’ve found that sometimes you need to tweak your system prompt to encourage the agent to end the call.

  3. Hello Ben! I’m Luca.
    I managed to set up the trunk I use between 3CX and ElevenLabs, the problem is that when the call has to be forwarded from ElevenLabs to a SIP URI (of the queue), the call does not arrive.

    • Hi Luca! Check the conversation history to see if the agent is attempting to transfer the call. If not, double check your SIP URI in your Transfer to Number is correct and that the conditions for transfer are being met. If that all looks good, I’d suggest looking at your 3CX call log to see if there’s any messages relating to the call there. If you are familiar with SNGREP or Wireshark, you can look for the call transfer there; you should receive a SIP REFER message from ElevenLabs.

      • Thank you Ben, now it’s working fine. The only problem is that you have to disable the firewall if you want to have the call back into your pbx. Do you have the 11labs static ip addresses?

        • ElevenLabs offers a static IP range to their enterprise customers. 3CX does a great job of blocking unauthorised requests with their firewall, as well as distributed block list.

  4. Hi Ben,

    We are trying to setup 3cx with ElevenLabs. Problem we have is that the call gets ended when it goes over the trunk to ElevenLabs;

    000-9d1 +316123456 +31881234567 Inbound Unanswered – 00:00:00 00:00:00 0.00 Ended by ElevenLabs Trunk (+31881234567)

    In the Elevenlabs conversations, I do not see the call.

    Any advice?

    • Hi Mario,
      Make sure the DID you added in ElevenLabs matches the outbound number you setup in 3CX, then check that your outbound rule is prepending the +.
      If not, if you can perform a packet capture within 3CX to confirm the SIP conversation.
      Cheers, Ben

  5. I tried to do this with twillio provider… but i think it s not possible…

  6. Hi, nice one !

    I have a question, if i have twillio number… and tried multiple things ( like setting the sip manually ), and whenever i call i get the number could not be reached πŸ™‚

    • Hi Luci,
      Are you having inbound calls directed to your 3CX from Twilio, then forwarding to ElevenLabs? It should work the way I’ve described. If you’re having ElevenLabs answer the call directly, you can setup Twilio directly with ElevenLabs.
      Thanks, Ben

  7. What if I had a SIP trunk from DirectCall and wanted to use it to make the calls? I had some problems with it and I’m looking for some way to dial through the 3CX API

    POST Request β†’ 3CX API (originate call)
    ↓
    DirectCall SIP Trunk (outbound)
    ↓
    ElevenLabs Agent (answers/talks)
    ↓
    Transfer to Number (lead wanted to talk to a human)
    ↓
    SIP Trunk routes back to 3CX
    ↓
    3CX Queue

    • I am thinking the 3CX Call Control API is going to be the best way to do this. Alternatively, you could use your SIP trunk to make an outbound call from ElevenLabs, then use Conference (instead of SIP REFER) to connect back to a DID assigned to your human team, skipping using 3CX initially.

  8. Hi Ben,
    I have a trunk with a geographic number that currently routes correctly to my 3CX system and from there to several internal extensions.

    How can I redirect incoming calls on that trunk (the external number) to ElevenLabs?

    From what I understood in your guide, the process works the other way around β€” an internal extension calls another extension, and the AI answers.

    In my case, I’d like external calls to be redirected to an extension that then forwards them to the ElevenLabs agent.

    Thanks!

  9. Thanks i solved.

  10. Hello,

    Thank you for your explanation,I just wanted to know if this method works well with our own local numbers added to 3Cx, that is to say, if I connect the AI agent directly with our number without having an intermediate number for call forwarding.

    Best regards,

  11. Hi Ben!
    Is there any chance to contact you directly?
    I am trying to setup my 3cx with an AI agent, but unfortunately it doesn’t work. I would like to ask if you can help me (I will pay for your service).
    Thank you!

  12. hi ben
    i’am having issue connecting my freepbx to the elevenlabs ai agent , i dont have a ip number so i use zoiper as an extension to call , and when i call i have busy , my freepbx receive the call but dont know where to send it , in my freepbx the trunk i made to elevenlabs is not registrated , in elevenlabs it’s said that elevenlabs dont need a registration , so my question is is it normale that my trunk is not registrated in freepbx ? if someon can help

    • Shouldn’t need registration. I just spun up a FreePBX instance to test. You will need to enable TCP signaling in Settings -> Asterisk SIP Settings -> SIP Settings [chan_pjsip] and enable it on your ElevenLabs trunk.

  13. I too am working on FreePBX + ElevenLabs – I’ve got calls going into the agent, but the refer back isn’t working yet …

    • I just spun up a FreePBX instance to try it out and am getting a similar issue. Checking the SIP message flow between FreePBX and Elevenlabs, it seems ElevenLabs is not sending a REFER SIP message, despite FreePBX allowing REFER in the INVITE header. I’ll look further into it.

  14. Hey Ben,

    Any luck on the FreePBX + ElevenLabs? If you want to work on it together, be glad to. You’d think as large of a userbase as Asterisk has, that they would have this working. I shot an email to their support, but no good reply yet, just a reply to check their chatbot, which of course I did, but still trying … I wonder how much I beat my head versus hearing from them. I have to assume its on their end as I’ve tried about everything I can think of.

  15. I can’t even end the call. I tell it to, it say’s it will, but it stays on the line. Also of note … I setup a test in their Tests section and both Transfer and End Call say they are good.

  16. I sent an email to one of the engineers @ ElovenLabs and got this response:

    Will pass your feedback on to team, better asterisk support is something on our roadmap

  17. HI, I am using yeastar and ElevenLabs. able to make the inbound calls but unable to make outbound calls or transfer to human is not working at all.

    Please help me out to solve this issue.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.