An AI meeting agent is an AI participant that joins a Google Meet, Zoom, or Microsoft Teams call and does a defined job in it. It listens, speaks, asks questions, presents, and hands the humans a written record when the call ends. A lot of products now use the name for notetakers that only record and summarize, and that is where most of the confusion starts.
Recording and summarizing is useful. It is not what most people mean when they ask how to build an AI meeting agent. They want something that can run a discovery call while the sales rep is on another call, screen a candidate at 2 AM, or walk a customer through a product demo, and then tell a human what happened.
This guide covers both halves of that work. The first half is plumbing: how a bot actually gets into a Google Meet or Zoom call, what the official APIs allow, and what you would have to build yourself. The second half decides whether anyone wants the agent in the room: the job you give it, the context it knows, and how you judge its calls. We build and deploy these agents at Tough Tongue AI, so I will try to be concrete about both.
What is an AI meeting agent?
An AI meeting agent is software that attends a live meeting as a participant and performs a defined job in it. The word “defined” matters. A generic chatbot dropped into a Google Meet does not help anyone. A useful meeting agent has a role, an objective, a conversation flow, source material, and success criteria.
Three kinds of meeting AI tend to share the name, and it helps to pull them apart.
AI notetakers join the call, record it, transcribe it, and send a summary. They never speak. Most meeting tools people already know by name sit here, along with the summaries built into Zoom and Google Meet.
AI meeting assistants help the humans who are running the meeting. They answer a question when someone asks, suggest what to say next, pull up a document, or draft the follow-up email. A person is still driving the call.
AI meeting agents run part of the meeting themselves. They introduce themselves, ask the questions, present slides, handle objections, and decide what to do next, then write up the call against a rubric. If a notetaker is a recorder left on the table, an agent is a teammate you sent to the call.
Depending on how you configure it, the agent might run a sales discovery call, demo a product, ask interview questions, collect customer requirements, coach a trainee, or sit silently and score a rep’s performance against a rubric. In Tough Tongue AI these come in two shapes. Streaming agents show a video avatar and take part in the conversation: they talk, listen, ask follow-up questions, and use tools like slides, whiteboards, and browser automation. Notetaker agents join silently and score the call against a rubric you define. That second shape still counts as an agent in the sense above. It does not speak, but it has a job.
If you want a face on a streaming agent, this is where a live avatar provider comes in. Tough Tongue AI works with Anam, HeyGen’s LiveAvatar, Avatario, and Protoface. For how the avatar layer fits into meeting agents more broadly, see my post on Tavus PALs joining Google Meet.
How an AI meeting agent works
Whether you build it yourself or use a platform, every meeting agent needs the same six layers. It is worth knowing them, because each one is a place where the agent can feel broken.
-
Meeting access. A bot has to get into the call as a participant: join at the right time, get past the waiting room or the “ask to join” screen, and stay connected for the whole session. This is the least glamorous layer and the one that needs the most upkeep.
-
Real-time voice conversation. The agent needs to hear participants, understand speech, decide what to say, and speak back naturally through the meeting’s audio. Latency matters. A response that takes three seconds makes the agent feel broken. Turn-taking and interruption handling matter too, because real conversations overlap.
-
Scenario instructions. The agent needs a job description. This includes its persona, objective, conversation phases, tone, guardrails, and what to do when it does not know an answer. Without this, you get a chatbot that wanders.
-
Business context. A useful meeting agent needs context: pitch decks, product docs, pricing, qualification criteria, objection handling notes, interview rubrics, customer intake forms, or training material. The more specific the context, the more useful the agent.
-
Tools and actions. Depending on the use case, the agent may need tools. A sales demo agent might present Google Slides or use browser automation to walk through a live product. A coach might use a notepad to capture structured observations. A support agent might trigger a webhook to log information in a CRM.
-
Recording, transcript, and analysis. After the meeting, the system should produce a transcript and structured analysis. For sales teams, that might include qualification scores, objections raised, next steps, and follow-up context. For training, it might include rubric scores with specific quotes as evidence.
The second layer is the one people underestimate. Every time someone in the meeting stops talking, the agent runs a loop: decide whether it is really its turn, turn the speech into text, let the model pick a reply using the scenario and context, turn that reply into speech, and play it into the call. Each step adds a little delay, and the delays stack. Speech-to-speech models fold some of these steps together, but the agent still has to decide when it is its turn, and still has to stop talking when someone cuts in.
How to build a meeting bot from scratch
If you are building a meeting agent yourself, the first real problem is not the AI. It is getting a bot into the call and getting audio in and out of it. Each platform has a different answer, and none of them is a single REST call.
Google Meet: no supported way in, so bots drive a browser
Google does not offer a supported API that lets a bot join a Meet call and talk. So most Google Meet bots work the same way. A headless Chrome, driven by Puppeteer or Playwright, signs in to a Google account, opens the meeting link, clicks through the join screen, and captures the meeting audio from the page. To speak, the bot plays generated audio into a virtual microphone. An avatar needs a virtual camera as well.
This works, and it is how you would build a Google Meet bot from scratch today. The cost is upkeep. You are automating a web page that Google changes whenever it wants, so selectors break, join screens change, and a bot that worked on Friday can get stuck on Monday. You also need one isolated browser per meeting, because bots that share audio devices can leak audio from one call into another.
Google does have two official APIs worth knowing about. The Meet REST API gives you the recordings, transcripts, and participant lists that Meet itself created, after the meeting ends. That is enough for a simple notetaker. The Meet Media API gives an app real-time audio and video from a conference, but as of September 2026 it is still in Developer Preview, every participant in the call has to be enrolled in that preview program, and it only receives media. An agent cannot talk through it. I go deeper on the Google Meet side, including private meetings and admission, in how to build a Google Meet bot.
Zoom: an SDK for bots, and RTMS for listening
Zoom gives developers more to work with. The Meeting SDK lets a bot join as a participant and work with raw audio and video, which is what an agent needs to hear and speak. Zoom also offers Realtime Media Streams (RTMS), which streams a meeting’s audio, video, and transcript to your app over a WebSocket without putting a bot in the room. RTMS is a good fit for a notetaker or a live coaching tool. It only receives, so an agent that talks still needs the SDK route.
Microsoft Teams: real-time media means C# on Windows
Teams has the most formal path and the heaviest one. A bot that needs live audio has to be an application-hosted media bot built on Microsoft’s Graph Communications media library. That means C# or .NET, deployed on Windows Server in Azure. Microsoft’s own documentation now points people building AI agents for Teams meetings toward Copilot Studio instead. If your backend is in Python or Node, this is usually where teams stop and look for a meeting bot API.
Meeting bot APIs: one integration for every platform
Because the three platforms are so different, a category of meeting bot APIs has grown up to hide them. Recall.ai is the best known. You send it a meeting link, it sends a bot, and it streams you each participant’s audio, video, and transcript in real time. Its Output Media feature can also stream audio and video from your own web app back into the call, which is how people build talking agents on top of it. Attendee is an open-source alternative you can self-host, with Zoom, Google Meet, and Teams support and audio in both directions.
A meeting bot API solves the access layer. It does not give you an agent. You still build the voice loop, the conversation logic, the tools, and the post-call analysis on top of the audio it hands you.
If you only need a meeting notetaker
Some people who search for a meeting agent really need a notetaker, and that is a much smaller build. Nothing has to happen live, so you can skip the voice loop entirely.
- Get the audio. Send a meeting bot, or skip the bot and pull the recording after the call: the Meet REST API for Google Meet, or cloud recordings for Zoom.
- Transcribe with speaker labels. Without speaker labels (diarization), a summary cannot say who agreed to what.
- Summarize against a template. Ask the model for decisions, action items with owners, and open questions, plus whatever matters for that type of meeting. For interviews, the template is your rubric: each competency, the evidence for it, and a recommendation.
- Send it where people work. A CRM record, a Slack channel, or an email to the host.
Whatever you build, make the bot’s presence obvious. People should know a bot is in the call and that it is recording.
Build vs buy: should you build from scratch?
You can build an AI meeting agent from scratch if you have the engineering team for it. But the scope is larger than it looks.
You need meeting access and scheduling on every platform you support. Real-time audio with low latency. Turn-taking and interruption handling. Speech recognition and text-to-speech. A conversation engine that can follow a scenario and adapt to what the other person says. Tool use for slides, browsers, and note-taking. Recording and transcript processing. Post-call analysis with rubric evaluation. Memory across sessions. Authentication and error recovery. Monitoring and analytics.
Each of these is its own engineering problem. Together, they add up to months of work before you have something reliable enough to put in front of a real prospect or candidate. The access layer alone is a real team: Recall.ai, which sells that layer, says building and operating meeting bots takes three to five engineers. If you are about to hire a freelancer to build a meeting bot, use the list above to scope the work. Getting a bot into a call is the first item, not the whole job.
The alternative is to start with the scenario layer. Define what the agent should do, give it the right context, test the conversation, deploy it, and review the results. Tough Tongue AI handles the infrastructure, the meeting access, the real-time conversation, the tools, and the post-call analysis. You focus on what matters: what the agent should actually say and do.
“Thanks for making time. What made you book this call?”
Talk to ParisHear the latency, the turn-taking, and the tone on a live call, then decide what your own agent should do.
- Talk to Paris, our AI SDR, from the home page.
- Create a scenario with your own pitch, pricing, and objections, in the app or from Claude or ChatGPT.
- Send it into a Google Meet or Zoom call from the Meeting Bot page.
How to build an AI meeting agent in Tough Tongue AI
Here is the practical workflow. None of it needs code. You can do every step in the web app, through the API, or from Claude or ChatGPT over MCP, which I cover further down.
Step 1: Decide what the agent should do
Start with the job, not the technology. A good meeting agent has one clear purpose.
“Qualify inbound demo requests for our admissions team.” That is a clear job. “Run a first-round candidate screen for product managers.” That is a clear job. “Coach new managers through difficult feedback conversations.” Also clear.
“Be helpful in meetings” is not a clear job. It creates an agent that talks too much, measures nothing, and adds noise to conversations that already have enough of it.
Write down what the agent should accomplish, who it is speaking with, and what a successful meeting looks like. This becomes the foundation of your scenario.
Step 2: Create a scenario
In Tough Tongue AI, go to the Scenario Library and click Create New Scenario.
A scenario defines three things. First, the agent’s role: who it is and how it should present itself. Second, the agenda: what it should do during the conversation, including questions to ask, information to present, and how to handle objections. Third, the evaluation rubric: how performance should be scored afterward, with dimensions, weights, and scoring bands.
You write all of this in natural language. Here is an example:
Create an AI meeting agent for our sales discovery calls.
The agent represents [company]. It should join a Google Meet with a warm lead,
introduce itself, ask discovery questions, explain our product briefly, handle
basic objections, and book a follow-up meeting with a human rep if the lead is
qualified.
The agent should be concise, friendly, and consultative. It should not pretend
to be human. If it does not know an answer, it should say so and offer to have
a human follow up.
After the call, analyze the lead's pain, urgency, budget signal, objections,
and recommended next step.For notetaker scenarios, toggle on Notetaker Mode and Audit Mode in the meeting configuration. This tells the agent to observe and score rather than participate.
Step 3: Add your context
The quality of the agent depends on the context you give it. A meeting agent without context is like a new hire on their first day with no onboarding materials.
For a sales meeting agent, add your pitch, ICP and qualification rules, pricing guidance, common objections with approved answers, demo flow, competitor notes, and escalation rules. The agent should know enough to run the conversation confidently, and it should know when to say “let me have someone on the team follow up on that.”
For an interview agent, add the role description, interview rubric, required skills, question bank, scoring criteria, and red flags to watch for.
For a training or coaching agent, add training objectives, the coaching framework, example answers, the scoring rubric, and the feedback style you want.
The more specific the context, the better the conversations. Vague context produces vague agents.
Step 4: Test the scenario before sending it into a meeting
Before deploying the agent into a live Google Meet or Zoom call, run the scenario as a normal practice session within the platform. Click Join on the scenario details page to enter a session directly.
Check whether the agent introduces itself clearly. Does it ask the right first question? Does it stay concise or ramble? Does it handle interruptions gracefully? Does it avoid making unsupported claims? Does it collect the information you need? Does the post-session analysis match your rubric?
This is the cheapest place to tune the prompt, tools, and rubric. Fix problems here, not in a live meeting with a real prospect.
Step 5: Deploy the meeting agent
Once the scenario is ready, open the scenario details page and find the Channels section. Click the Meeting Bot card to open the Meeting Bot Integration page. The meeting bot is included on the Premium plan and uses your existing platform minutes.
You have two deployment options.
One-Off Meeting. Use this when you have a specific meeting to deploy the agent into. Select your platform (Google Meet or Zoom), paste the meeting URL into the Meeting URL field (you can add up to 5 URLs and the bot joins each one), optionally set meeting security and invitees, optionally schedule for a future time, and click Dispatch Bot. The bot joins within seconds.
This works well for individual sales calls, candidate screens, coaching sessions, customer research calls, and team training exercises.
Calendar Integration. Use this when you want the agent to join meetings automatically. Connect your Google Calendar and the bot joins your scheduled meetings without manual dispatching. You can set keyword filters so different agents handle different types of meetings. A screening agent joins anything with “interview” in the title. A sales coach joins “discovery call” meetings. Each scenario has its own calendar connection and keyword filters.
Here is a walkthrough of the deployment flow:
How the agent joins Google Meet and Zoom
This is where most meeting bots fall short. They work fine for open meetings but cannot get into anything with restricted access.
Tough Tongue AI agents join as authenticated, signed-in Google accounts, not anonymous link-clickers. This is an important distinction.
For open meetings where anyone with the link can join, the agent walks right in. No extra steps.
For private meetings where only invited participants can join, you have two options. The first is manual admission: the agent knocks on the meeting door, a participant sees the “someone wants to join” notification in Google Meet, and clicks Admit. This works fine for notetaker scenarios where someone in the meeting expects the bot.
The second option is auto-join. Add ttai@toughtalkai.com as an attendee on the calendar event. The system recognizes the bot as an invited participant and it joins without knocking. No human intervention needed.
The auto-join approach matters for autonomous agent scenarios. If your agent is conducting a first-round interview and the candidate is the only human in the room, there is nobody to click Admit. Adding the email to the calendar invite solves this cleanly.
For Zoom meetings, paste the Zoom URL directly in the One-Off Meeting form. The agent joins the call using the provided link. For a screen-by-screen version of this setup, see the Google Meet bot walkthrough.
Is there an AI meeting agent API?
Yes, at two levels, and it is worth knowing which one you need before you start.
A meeting bot API, like Recall.ai or the open-source Attendee, gives you access. It puts a bot in the call and streams you audio, video, and transcripts. Everything the agent does with that audio is still yours to build.
An agent API gives you the whole participant. In Tough Tongue AI, you build the agent once as a scenario, then send it into any meeting with one request to the meeting bots endpoint:
curl -X POST https://api.toughtongueai.com/api/public/v2/meeting-bots \
-H "Authorization: Bearer $TTAI_API_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"scenario_id": "YOUR_SCENARIO_ID",
"meeting_url": "https://meet.google.com/abc-defg-hij",
"meeting_provider": "google-meet",
"scheduled_ts": null,
"bot_name": "Paris from Acme"
}'The response returns a bot_id and a session_id for each bot. Set scheduled_ts to null to send the agent in about a minute, or to a timestamp to book it for a later call, and use zoom as the provider for Zoom links. When the call ends, the session holds the transcript and rubric analysis, and you can pull both through the same API.
The meeting bot is included on the Premium plan, so any Premium account can call this endpoint. The API docs also cover listing and cancelling scheduled bots.
Run it from Claude, ChatGPT, Codex, or Cursor
You do not have to write that request yourself. Tough Tongue AI has an MCP server, which is the standard way AI assistants call outside tools. It works with Claude, ChatGPT, OpenAI Codex, Cursor, GitHub Copilot, Gemini CLI, and other MCP clients, and you sign in with your Tough Tongue account the first time you use it. In Claude Code or Codex, you add it from the terminal:
# Claude Code (then run /mcp to sign in)
claude mcp add --transport http ttai https://api.toughtongueai.com/api/public/mcp
# Codex
codex mcp add ttai --url https://api.toughtongueai.com/api/public/mcp
codex mcp login ttaiOn ChatGPT, install the Tough Tongue AI app from the app directory. On Claude web and desktop, add our plugin, which bundles the MCP server with our open-source skills. Then you can build and run a meeting agent from a chat window:
Create a discovery call agent for our admissions team. Use the pitch,
pricing, and objections in the attached doc, and score each call on
need, budget, timeline, and next step.
Send it to https://meet.google.com/abc-defg-hij at 3 PM today
as "Paris from Acme".After the call, ask the same chat to pull the session and summarize the lead’s pain, budget signal, and next step. The server covers the rest of the platform too, from editing the scenario to placing outbound phone calls to running analytics across every call, so you can build, deploy, and review an agent without leaving the assistant you already use.
One limit worth knowing: if your company connects ChatGPT to the server as its own custom app instead of installing ours, ChatGPT only allows write actions, like creating a scenario, on Business, Enterprise, and Edu plans. Setup for every client is on our agents page.
What happens after the call
Once the meeting ends, the platform processes the recording and produces several outputs.
A full transcript of the conversation is generated automatically. The agent runs AI analysis against the rubric you defined in the scenario, with each dimension scored, evidence quoted from specific moments in the call, and coaching recommendations provided. The scores and analysis are emailed to the scenario admin automatically. Everything is accessible in the Sessions page with recordings, transcripts, and evaluations.
The analysis is not generic “good job” feedback. For a sales discovery call, you might see something like:
Discovery depth: 8/10. At 4:32, the rep asked “what does that cost you in engineering hours?” This turned a surface-level complaint into a quantified $180K/year pain point. Strong probe.
Closing: 5/10. When the prospect said “we already have something for that,” the rep moved to the next feature instead of asking what specifically their current solution handles well. Missed opportunity to dig into competitive positioning.
You can point the same analysis at whatever your team cares about. A common request from sales teams is buying signals: a budget mentioned, a timeline, another stakeholder named, a competitor in play, or a next step agreed. Write each one into the rubric as its own dimension, with a one-line definition and an example quote, and the post-call summary will flag it with the moment it happened. The more concrete the definition, the fewer false alarms you get.
For teams that need the data in their own systems, the same API returns session results programmatically, and there is an Apps Script integration that pipes them into Google Sheets. From there, updating CRM notes after every call is a small script.
Example AI meeting agents you can build
Sales discovery agent. The agent joins a call with a warm inbound lead, often someone who filled out a form an hour earlier, asks qualification questions, explains the product, handles basic objections, and books a follow-up with a human closer. After the call, it produces a summary with pain points, urgency signals, budget indicators, and a recommended next step.
Product demo agent. The agent joins a Google Meet, presents slides from a Google Slides deck, walks through the product using browser automation, answers common questions, and captures objections or missing requirements. This is particularly useful when demo requests outnumber the available sales reps.
Candidate screening agent. The agent conducts a structured first-round interview, asks follow-up questions when answers are thin, and summarizes strengths, concerns, and a recommended next step. The hiring manager reviews the analysis instead of spending 30 minutes on every initial screen.
Customer intake agent. The agent collects requirements before a human consultant joins, reducing the amount of repetitive discovery work. It asks the standard questions, captures the answers in structured form, and flags anything unusual for the human.
Sales coaching observer. A notetaker agent joins discovery calls or demos silently. It evaluates opening hooks, personalization, discovery depth, talk-to-listen ratio, objection handling, and closing strength. Each dimension gets a score with specific quotes as evidence.
Beyond meetings: other deployment channels
Meeting deployment is one channel. Tough Tongue AI also supports two other ways to deploy the same scenario.
Website embedding. Embed the agent directly into your website or app using an iFrame. Customize the design from the left sidebar in the platform, preview how it looks, and grab the embed snippet with the Show Code button. This is useful for onboarding assistants, product demo agents, or in-app support experiences. It is also the answer when someone asks whether an AI agent can guide a customer through a web session in real time: when the agent lives inside the page, it can talk the user through onboarding or a demo as they go.
Phone calls. Deploy the agent as a phone-based experience. Outbound calls let the agent call leads back, qualify them, and book a meeting with a human rep. Inbound calls let the agent handle incoming calls from a connected number. You connect a calling provider with your SIP endpoint, authentication, and phone number.
All three channels use the same scenario. You define the agent’s behavior once and deploy it wherever your users already are.
FAQ
What is an AI meeting agent?
An AI meeting agent is an AI participant that joins a Google Meet, Zoom, or Teams call and does a defined job in it, such as running a discovery call, a product demo, or a screening interview. Unlike a notetaker, it takes part: it speaks, asks follow-up questions, and presents, then writes up the call for a human.
How can I build an AI meeting agent?
An AI meeting agent needs a way into the meeting, a real-time voice loop, instructions for its job, business context, tools, and post-call analysis. You can assemble these yourself on top of a meeting bot API, or use a platform like Tough Tongue AI: describe the job as a scenario, test it in a practice session, then send it into Google Meet or Zoom.
How do I build a meeting bot from scratch?
It depends on the platform. Google Meet has no supported API for a bot that joins and talks, so most Meet bots drive a headless Chrome with Puppeteer or Playwright. Zoom offers a Meeting SDK for bots that join as participants, and Microsoft Teams needs a C# or .NET media bot running on Windows Server. Meeting bot APIs such as Recall.ai or the open-source Attendee wrap all three behind one API, and you build the agent on top.
Is there an AI meeting agent API?
Yes, at two levels. Meeting bot APIs like Recall.ai and the open-source Attendee put a bot in the call and stream you the audio, and you build the agent on top. Tough Tongue AI’s API sends a complete agent: post a scenario ID and a meeting URL to the /v2/meeting-bots endpoint, and the agent joins, runs the conversation, and returns a transcript and analysis.
Can I build an AI meeting agent from Claude or ChatGPT?
Yes. Tough Tongue AI has an MCP server that works with Claude, ChatGPT, OpenAI Codex, Cursor, and other MCP clients. From a chat, you can create the agent’s scenario, send it into a Google Meet or Zoom call, and pull back the transcript and scores after the call, without opening the dashboard.
Can an AI agent attend a meeting for me?
Yes, for meetings with a clear and repeatable job, such as a discovery call, a first-round screen, or a customer intake call. Give the agent your pitch or rubric and it can run the call and send you a summary afterward. It works best when the other side knows they are talking to an AI and a human follows up on anything it cannot answer.
What is the difference between an AI notetaker and an AI meeting agent?
A notetaker listens, records, and summarizes, and never speaks. A meeting agent, sometimes called a voice agent, takes part in the conversation: it talks, asks follow-up questions, presents, and uses tools. Both can send a summary after the call, but only the agent can run the meeting.
Can an AI meeting agent join Microsoft Teams?
Yes, but Teams is the hardest platform to build for yourself. Bots that handle live audio must be written in C# or .NET and run on Windows Server in Azure, so most teams use a meeting bot API such as Recall.ai instead. Tough Tongue AI agents join Google Meet and Zoom today, with Teams support on the way.
Can I use an AI meeting agent for sales calls?
Yes. Sales teams use meeting agents to run discovery calls and product demos for warm leads, qualify them, handle common objections, and book the next meeting with a human rep. After the call, the analysis flags pain points, buying signals, objections, and next steps, which you can push into your CRM.
Get started
The best AI meeting agents are specific. Give the agent a clear job, the right context, and a rubric for success. Test it like you would test a new teammate: run a practice session, read the transcript, tune the instructions, and iterate.
If you are building the plumbing yourself, start with a meeting bot API and spend your time on the voice loop and the conversation, because that is what the people in the call will notice. If you would rather skip the plumbing, build the scenario in Tough Tongue AI, deploy it from the Meeting Bot integration, and review the analysis after each call, or do all three from Claude or ChatGPT through the MCP server. Either way, the agent becomes useful the same way a new hire does: by knowing exactly which conversation it is responsible for.
Links:
