Case Study: Pushpay's Journey to Reliable Agentic AI with Amazon Bedrock
Coverage of aws-ml-blog
In a recent post, the AWS Machine Learning Blog details how Pushpay leveraged Amazon Bedrock to build a reliable agentic AI search feature, transforming how non-technical users access data insights.
In a recent post, the AWS Machine Learning Blog presents a case study on Pushpay, a digital engagement platform, and their development of an agentic AI solution using Amazon Bedrock. As enterprises move beyond initial generative AI experiments, the focus is shifting toward "Agentic AI"-systems capable of reasoning, planning, and executing complex tasks rather than merely generating text. This publication explores how Pushpay navigated this transition to solve a specific, high-value user problem.
The Context: Democratizing Data Access
For many SaaS platforms, a significant gap exists between the data stored in the system and the user's ability to extract actionable insights from it. Traditionally, answering questions like "Who are the active members that haven't donated this year?" required either technical proficiency with reporting tools or manual data sifting. This friction slows down decision-making and limits the utility of the data.
The industry is currently witnessing a surge in "Text-to-Insight" applications, where Large Language Models (LLMs) act as intermediaries between natural language and structured databases. However, the challenge for engineering teams is not just building these agents, but ensuring they are reliable, secure, and accurate enough for production environments. An agent that "hallucinates" financial data or membership statistics is worse than useless-it is a liability.
The Gist: From Natural Language to Actionable Insights
Pushpay utilized Amazon Bedrock to construct a search feature designed specifically for ministry staff and community leaders. The goal was to allow users to ask questions in plain English and receive real-time, accurate data regarding their community's engagement and giving patterns.
According to the post, the implementation of this agentic workflow significantly reduced the time-to-insight. Tasks that previously took minutes of manual configuration or required support tickets can now be executed in seconds. The solution leverages the capabilities of Amazon Bedrock to interpret user intent and interact with the underlying data structure.
While the technical brief highlights the outcome, the original article is particularly valuable for its focus on the "GenAI evaluation" journey. Building a prototype is straightforward; refining it to handle the nuances of human language reliably requires a robust evaluation framework. Pushpay's experience offers a roadmap for other organizations attempting to bridge the gap between raw data and non-technical users.
Why This Matters
This case study serves as a practical example of vertical-specific AI application. Rather than a general-purpose chatbot, Pushpay built a domain-specific agent that understands the context of church management and donor engagement. For technical leaders, this underscores the importance of scoping AI agents to specific domains to ensure high reliability and utility.
We recommend reading the full article to understand the specific methodologies Pushpay employed to evaluate their agents and the architectural decisions involved in deploying this solution on Amazon Bedrock.
Read the full post on the AWS Machine Learning Blog
Key Takeaways
- Pushpay developed an agentic AI search feature using Amazon Bedrock to assist non-technical ministry staff.
- The solution translates natural language queries into data insights, reducing time-to-insight from minutes to seconds.
- The project highlights the critical shift from experimental GenAI to reliable, production-grade agentic workflows.
- The post emphasizes the importance of 'GenAI evaluation' strategies in ensuring the accuracy of AI-generated data insights.