70 million Americans show symptoms of insomnia

Evolve is a YC-backed company that's been building products for about nine years. It started with Podcast App, one of the most-downloaded podcast apps on iOS with more than 15 million downloads, and they're continuing to innovate today.

The path into sleep came from an unexpected signal in their own product data: usage of Podcast App spiked late at night. Digging into why, the team found that people weren't listening to catch up on episodes. They were using podcasts to fall asleep. That observation sent Evolve down a rabbit hole into sleep science and into a market they hadn't set out to enter.

What they found: roughly 70 million Americans show symptoms of insomnia, with many more worldwide. The clinically recommended first-line treatment, ahead of medication, is cognitive behavioral therapy for insomnia (CBT-I), and it's highly effective. But access to a professional is scarce. Even Evolve's own clinical partner, Ashley Mason, a CBT-I researcher and practicing therapist at UCSF, has an 18-month waitlist, sees patients in groups of around eight, or charges thousands of dollars for individual care. Evolve saw an opening to deliver CBT-I through voice AI, at a scale and cost no human-delivered program could match.

Rest: a sleep expert that never sleeps

Rest is Evolve's AI sleep coach that runs people through a CBT-I program. The program, which was built with sleep experts, typically runs over about eight weeks.

Tracing a voice-and-text agent end to end

All of Rest's AI usage is traced in Langfuse. The voice experience runs on Vapi as the orchestration layer, with ElevenLabs for voice synthesis, Deepgram for transcription, and OpenAI as the underlying LLM. Vapi has a native Langfuse integration, so voice conversations are traced automatically at the click of a button. The text-chat experience is built and traced directly on OpenAI LLM APIs in Evolve's own backend. The text experience, and also non-conversational LLM work such as background extractors that process conversations after the fact, are traced through Langfuse as well.

Error analysis as a product strategy tool

For both Ignacio and Martin, tracing their agents in Langfuse is the foundation for their most important eval practice: error analysis. It's how they find failure modes and act on them, either by implementing and testing a fix directly or by setting up an evaluator to monitor for it going forward.

Martin was clear that looking into traces himself functions as a product strategy exercise. Reviewing real conversations in annotation queues, based on the traces in Langfuse, surfaces which product gaps are quick fixes and which are "a whole big component we need to build... a multi-week, maybe even a multi-month exercise." Staying close to those real conversations is one of the most important inputs into where the team invests next.

On top of manual review, Evolve runs LLM-as-a-judge evaluators in Langfuse for specific known issue types, including voice and audio quality problems (noise, cutoffs, echo, largely on the Vapi side) and memory and forgetting issues.

Deep dive: fixing the AI's memory

The clearest example either of them gave of error analysis driving product change was memory.

Users were telling Rest's coach things, and the AI would respond in the moment as if it understood and would carry that forward, "sure, I'll take that into account," but then fail to retrieve it in a later conversation. Users noticed. And said so directly. And often.

That kind of failure hits trust hard, even when the underlying product is otherwise working. For Martin, Evolve's founder, it reads as a broken promise rather than a minor bug. Investigating it, the team realized memory wasn't a single problem with a single fix. Fixing it properly turned into a big project: a genuinely multi-month engineering effort, only now wrapping up. Getting there took the right examples from production data. To continuously broaden their understanding of how these failures happen, the team set up an LLM-as-a-judge in Langfuse to catch the moments where users are frustrated specifically because of memory issues. That gives them high-signal input to better understand how to fix it at scale. It also lets them track whether that frustration is actually going down over time.

Prompt management at scale

Beyond tracing, Evolve manages more than 100 prompts in Langfuse Prompt Management. These aren't static prompts. They are composed on the fly, and Langfuse makes that composition easy to pull off: a foundational prompt sets up the overall conversation, and on top of that sits an agenda of two to four items, each its own prompt that can embed further prompts, including locale-specific variants (English and Spanish, adjusted for users in Argentina, Colombia, and Spain).

The main win, per Ignacio, is that this gives non-engineers, people writing and refining prompts who don't have (or want) deploy access, a safe way to edit and ship prompt changes without touching code or GitHub. Staging and production labels let a prompt be tested before it's promoted live.

Why Langfuse

Ignacio's team found Langfuse through Evolve's YC network, and it stuck.

Ignacio credits responsive support and steady, visible product improvement. Martin, running on Langfuse Cloud, is just as confident handling sleep-related conversations there: "we also know that you have very high standards." Self-hosting isn't a strict compliance requirement for Evolve, but he'd still like it as a future option, for more direct control over personal data.

What's next

Deeper eval coverage. The product has grown substantially in complexity over the past year, and Martin wants confidence that core conversation flows are properly regression-tested before shipping changes. For this the team plans to start building out more datasets in Langfuse.

More proactive issue detection, including using the Langfuse API, MCP server, and CLI inside Evolve's own internal tools and admin dashboard, to pull trace data into context faster and catch problems earlier.

Possibly bringing the agent in-house. Ignacio said Evolve is considering building a custom agent harness for more control over performance and latency of voice interactions. Observability in Langfuse stays on regardless - a key advantage of Langfuse's vendor-agnostic tracing.

Impact

Full visibility into a voice-and-text agent. Every Vapi voice trace and direct OpenAI text-chat trace lands in one place, alongside background extractor calls, instead of scattered across two stacks.

Non-engineers can safely ship prompt changes. 100+ dynamically composed prompts, including locale variants, are edited and staged in Langfuse without code deploys.

A systematic fix for the AI's biggest user trust issue. Trace-driven error analysis identified and helped resolve a memory issue that was actively eroding user trust.

Automated coverage of known failure modes. LLM-as-a-judge evaluators now flag voice-quality and memory issues without a human reviewing every conversation manually.