How to Build a Vapi Voice Agent with Google Calendar Integration

What you'll build

A Vapi scheduling assistant that acts like a receptionist - this is the first demo/close-to-production agent I built for my first client, an acupuncture clinic. It answers the call as "Sarah," figures out whether you're a new patient or a follow-up, collects name and phone number, checks real Google Calendar availability, offers two time slots, collects a spelled-out email, books the event, and sends the calendar invite - all in-house on Vapi, no n8n or Make.com required.

In the demo, the caller asked for January 2nd, got offered 11 a.m. or 3 p.m., picked 11, and the calendar event + email invite were created on the spot.

What you'll need

A Vapi account (bare assistant with no tools runs about 10 cents per minute - cost and latency climb as the system prompt and tools grow)
A Google Calendar to book into
Optional: an ElevenLabs voice ID if you want a more human or bilingual voice
Optional: a knowledge-base doc (FAQs) from the business you're building for

Step 1: Create the assistant and set the first message

Create a new assistant in Vapi. We use GPT-4o cluster - OpenAI's model that's very good at conversation. Latency is a little higher, but the reasoning on conversations is usually better.

First message:

Thank you for calling Best Acupuncture. This is Sarah. How can I help you?

Step 2: Paste in the system prompt and understand its sections

This is really where the money is made - the system prompt is key. The video shows the full prompt on screen - rebuild yours section by section and tweak it. Walkthrough of what each section does:

Identity - tell it who it is. Works for anything: professional data extraction agent, real estate cold caller, lead qualifier. Here: a scheduling assistant for an acupuncture clinic.
Current date - very important. Vapi disappointingly has no built-in function for this, so a snippet in the prompt lets the agent know "today is December 22nd."
Voice/personality - friendly, good at handling customers, not just a scheduling robot.
Opening line - redundant since it's already the first message, but I keep it in so the whole script lives in one place.
Spanish handling - this client deals with a lot of Spanish-speaking callers, so the agent recognizes Spanish and transfers to someone who can handle it.
Appointment type - classify new patient vs. follow-up. This flows into the Google Calendar event title ("new patient") so the client knows which it is at a glance.
The scheduling flow, steps 1-8 - the exact route the agent takes when someone's booking vs. just asking a question: determine appointment type → collect name and phone → check availability with the tool → offer times → confirm → collect email → create the event → close.
Business hours - Monday through Friday, 9 a.m.-6 p.m. Eastern.
Only offer two options - keeps the conversation short. Every extra back-and-forth increases call time, and the cost of the call becomes exorbitant. Two options, caller picks one, move on.
Confirm the selected time and date - voice agents mishear things; make it repeat the selection back so everything's recorded accurately.
Collect email - ask the caller to spell everything before the @, and assume gmail.com unless they say otherwise.
Date rules - always interpret a weekday as the NEXT upcoming occurrence of that weekday, and don't ask for the numeric date. If the caller says "next Monday," it just knows that's the 29th.
Knowledge base instructions - point it at the FAQ doc for cost, insurance, conditions, treatment questions (Step 4).
Hard rules - never offer weekends, never give actual medical advice, always close with "have a great day."
Fallback - if anything goes wrong, transfer immediately to a live human receptionist.

ChatGPT or Gemini can give you a really good baseline system prompt for a voice agent, but it will never be perfect. Expect errors - from not calling the tool properly to not knowing the date - and iterate. That's the actual prompt-engineering work.

Step 3: Create the two Google Calendar tools

Vapi has Google Calendar integrated natively - no n8n or Make needed. Go to Tools → Create tool and you'll see Check Availability and Create Event already there. For each one, all you really do is add the specific Google account email you're booking with and a short description:

Check availability: Use this tool to check calendar availability for a specific date.
Create event:       Use this tool to create a calendar event booking.

Then go back to your assistant's Tools section and attach both.

Step 4: Add the knowledge base

On the assistant you'll see an Add knowledge base option; under Files you can upload a document and attach it. For the clinic we uploaded their existing phone script - FAQs covering cost, insurance, conditions, and treatment explanations - so the agent quotes the exact answers the clinic wants clients to hear. Whatever agent you're building, you'll probably want one of these.

Step 5: Pick the voice

Under voice configuration, choose the voice so it sounds like a Sarah and not an Elliot. This is also where you can paste in an ElevenLabs voice ID for a more human sound or bilingual capability.

Step 6: Publish and test

Publish the assistant and run a test call. You should see: new-patient classification, name + number collected, real availability offered, email spelled out, and the event + invite landing on the calendar.

Step 7 (bonus): Connect a phone number

No Twilio needed. In Vapi's phone number section, type in your area code, hit create - bang, working phone number. Connect your assistant to it and you're taking real calls.

That's pretty much it: assistant prompt + the two calendar tools + a knowledge base = a working demo and a pretty close version of a production-level assistant.

Resources

Vapi - https://vapi.ai
ElevenLabs - https://elevenlabs.io (voice IDs, multilingual voices)