The Blueprint: How Voicify makes AI-enabled ordering a delight for customers

The challenge: When we started Voicify in 2018, our vision was to help organizations build confident, pragmatic, and technically grounded voice-driven assistants for any channel, including phones and chat. But the pandemic changed everything. We shifted our focus to telephone use cases primarily in the restaurant and healthcare sectors where, at the time, call volume and staffing posed significant challenges.

Restaurants could potentially miss up to 20% of their calls and lose orders as a result, and healthcare providers struggled to keep up with call volume with the required 100% accuracy when integrating appointment information into a practice management system. We realized that specialized, purpose-driven AI assistants were the key to businesses maintaining excellent service at scale. To succeed, we had to overcome four primary challenges: Transactional precision: Our voice assistant needed to reason with complex customer requests against point of sale and practice management systems with 100% accuracy.

Traffic spike management: Our LLM usage needs to be provisioned accurately to keep costs down and maintain customer services in spite of the common (and extreme) spikes in traffic seen in restaurants and healthcare organizations. Latency: Any delay in the assistant's response can cause customers to hang up. We needed superfast time to first token, with minimal delay from when a user sends a voice or text request to when the AI model generates its first piece of output.

Security and compliance: Since our founding in 2018, we've ensured that we're HIPAA, SOC2, ISO27001, and PCI-compliant, and that our security is enterprise-grade. We needed architecture and infrastructure that employs all possible safeguards to safeguard data integrity and security. The solution: Our conversational orchestration platform builds and validates restaurant orders against a point-of-sale system before submission to ensure accuracy.

Under the hood, Gemini Flash , served via Gemini Enterprise Agent Platform , vastly improves latency, minimizing user wait times and preventing hang-ups. With it, we also see approximately 25% to 30% savings compared to our previous use of other LLMs, and with greater reliability too. To grow the business - and call volume - and to handle traffic spikes, we switched from Google AI Studio to Vertex AI and its current incarnation in Gemini Enterprise.

We wanted the enterprise guarantees the latter provided, which we needed for scaling as well as for security and compliance for our healthcare clients. Specific Gemini Enterprise Agent Platform features help us manage high call volumes without experiencing service interruption or dropped responses. These enterprise-grade services may have carried an increased cost over AI Studio, but they were well worth it to ensure reliable uptime, and the premium pay-as-you-go feature made scaling much easier for us.

For example, we used a combination of provisioned throughput and premium pay-as-you-go with Vertex AI to accommodate all-time high usage the day before Thanksgiving, and we saw no rate limiting issues. The architecture: The outcome: Gemini has reduced the burden on our in-house programmatic tools for pulling context and building menus. We've seen great improvements in performance and reliability, with lower latency and greater reliability with Gemini.