AI is transforming how companies manage customer interactions, but building AI-powered systems that work reliably at scale is no easy feat. As companies grow, so does the volume and complexity of customer needs. Behind every interaction is an opportunity to use AI to better understand what needs attention, prioritize responses, and deliver a more seamless customer experience.
So what does it actually take to build the intelligence layers that make this possible?
That's what we explored at our latest Tech @ Ro Talk, Ghosts in the System: How AI Powers Scale.
The evening kicked off with a deep dive from Dan Cadden, Senior Software Engineer at Ro, into AI Ro'sponder, the LLM-powered intelligence layer behind Ro's patient messaging. Ro'sponder analyzes and classifies every incoming patient message in real time, helping care teams identify urgent needs, prioritize clinical work, and respond more efficiently. Dan walked through how the system powers everything from adverse event detection and AI-generated draft responses to intelligent task routing and clinical safety guardrails. He also shared how the system has evolved, the technical challenges encountered along the way, and what the team has learned about deploying AI safely at scale.

The evening continued with a panel of engineers from Ro, Decagon, and Nayya exploring the technical decisions, tradeoffs, and lessons learned from building reliable AI systems at scale.
Here are some of the key themes we explored:
How to make AI systems reliable at scale. Moving an AI system from an early prototype into production introduces an entirely new set of engineering challenges. As customer interactions grow in volume and complexity, teams must think beyond whether a system works in isolation and consider how it performs consistently under real-world conditions. Reliability, accuracy, and the ability to adapt to changing needs are critical to delivering a customer experience that’s seamless and high-quality.
The intelligence behind the scenes can make all the difference. Not every impactful application of AI is customer-facing. Behind each customer message is valuable context that can help teams understand what needs attention, how urgently to respond, and what should happen next. There are a lot of teams that can build behind the scenes to improve operational workflows. At Ro, Ro'sponder is an example of how we're building intelligence into patient messaging workflows to support more effective care operations. These systems have the potential to make customer experiences more responsive, efficient, and personalized.
The path to production. Building effective AI systems requires experimentation, iteration, and a willingness to rethink early assumptions. From deciding which problems are worth solving with AI to navigating architectural tradeoffs and learning from approaches that don't work, engineering teams must continuously refine their systems as they scale. Understanding what didn't work can be just as valuable as understanding what did.
A huge thank you to our speakers from Ro, Decagon, and Nayya for sharing their experiences, and to everyone who joined us!

At Ro, we're helping patients achieve their health goals so they can chase their life goals. We're doing that by building a patient-centric healthcare system from the ground up.
This is a once-in-a-generation time to build in health. What used to feel like science fiction now just feels like science. Our engineering teams are at the forefront of this shift, tackling complex challenges at the intersection of AI and healthcare to make care more proactive, personalized, and effective for patients.
If this type of problem-solving piques your interest, come build with us. Explore open roles on Ro's engineering and technology teams: https://ro.co/tech/.