Confgraph: Did I reinvent the wheel while building an AI secretary for video calls

I occasionally catch myself thinking “what if someone has already done this?” — after I’ve just written another piece of code. With confgraph, my AI secretary for video conferences, I decided not to guess but to honestly figure out how novel the project is and whether it’s worth continuing.

What’s the project about

Confgraph is a server-side bot based on LiveKit that joins a video call as a participant in listen-only mode (no microphone, usually hidden in the participant list), listens to each speaker separately, and:

• performs speech recognition locally, via faster-whisper on the CPU, without sending audio outside;
• uses an LLM (via OpenRouter) to assign phrases to topics and builds a discussion graph from them — not a flat list of tags, but topic-nodes with connections between them;
• saves the transcript, embeddings, and graph in SQLite — raw audio is not stored in the database;
• answers questions about the call content via a RAG endpoint;
• after the call compiles summaries: by topic, off-topic items, decisions, deadlines, and tasks for the next call.

Interface — Meet, a fork of livekit-examples/meet with a “Secretary” panel: a list of topics, a discussion tree with auto-refresh, and a summary button. All of this runs in its own docker-compose (LiveKit + confgraph + meet) without external cloud services, except for calling the LLM via API. You can try it live at meet.regionruza.ru.

For those interested in details — some real configuration parameters:

ACTIVE_SPEAKER_MODE=replace   # phrase boundaries = active speaker + hangover 2s
CLASSIFY_TOPIC_CONFIDENCE=0.9 # threshold below which a short "ok" doesn't switch the topic
LIVEKIT_HIDDEN=true           # bot is not shown in the participant list
CHAT_LOG_ENABLED=false        # don't spam the meeting's general chat

Checking for reinventing the wheel

Before writing another line, I looked at what’s already on the market.

Commercial SaaS

Otter.ai, Fireflies, Fathom, tl;dv, Grain, Read.ai — this whole category works on the same principle: a guest-bot joins someone else’s call (Zoom, Google Meet, Teams) as a separate participant, audio is sent to the cloud, and in return you get a transcript and a summary. Pricing ranges from limited free tiers to paid subscriptions per team member. Some services (Fireflies, MeetGeek) have “topic tracking”, but that’s analytics of recurring topics — a flat list, not a graph with connections between topics.

Self-hosted and open-source alternatives

This area is more interesting: these are direct competitors in the sense of “free and without subscription”:

• Meetily — open source under the MIT license, local recognition via Whisper or Parakeet, summaries via Ollama or any API of choice. This is the closest free analogue to confgraph in terms of “local recognition and summarization.” But it’s a desktop app for a single user on their own computer, not a service with an API for a room with multiple participants and not part of any video conferencing platform.
• Vexa — an open API for participant-bots that you can deploy yourself: bots join Google Meet, Teams, and Zoom and relay transcriptions in real time and via webhooks. Architecturally (own server, API at the core) this is closest to confgraph, but it’s a developer-focused toolkit designed to connect to other people’s calls, not its own meeting platform. It doesn’t provide a topic graph or room-level RAG.
• LiveKit Agents — the framework on which confgraph is built. It provides basic building blocks for voice agents — speech recognition and synthesis, voice activity detection — but it’s building material, not a ready-made secretary. Using LiveKit doesn’t mean reinventing the wheel; it’s applying an existing component for its intended purpose.

So it’s not entirely reinventing the wheel

If you reduce the task to “local transcription and LLM summarization without a subscription,” that part is already solved — Meetily is proof, and I’m not the first.

But I couldn’t find the three things combined anywhere:

1. The secretary is integrated into its own calling platform, not a guest joining someone else’s. No permission is needed to let the bot in, and you don’t have to trust someone else’s infrastructure with your audio — the secretary joins my own Meet.
2. A topic graph with connections, not a flat list. Also, a short “yes” or “ok” doesn’t switch the conversation to another topic if the classifier isn’t confident. None of the tools I found have that.
3. RAG queries tied to a specific room via a private API — not a general search across the account history, but the context of a single conversation.

Conclusion

Partly yes, this is reinventing the wheel: I didn’t invent the basic “listen and transcribe” part — it existed before me. But the whole construction — its own platform, topic graph, and room-level RAG — I didn’t find anywhere for my use case. And the main practical advantage for me isn’t architectural uniqueness, it’s that it’s mine: no subscription for a seat in a team of one, full control over call data, and the ability to change whatever I don’t like at any time instead of writing to support.