Beyond the Chatbot: A Hybrid Approach to Building Smarter Applications with LLMs

So here’s something I’ve been mulling over lately. It’s more of a hypothesis than a definitive statement, but I think it might be an interesting way to look at how we use large language models in our work. We often box them in as tools for conversation, as chatbots. But I’ve been playing around with the idea that their potential is far greater, especially if we rethink how they integrate into our line-of-business applications.

Imagine you’re developing a standard application. You build your services, your business logic, and a user interface with forms and buttons. This is the traditional path. But what if we added another layer? I’ve been thinking of this as a ‘Model Context Protocol’ (MCP) — essentially an interface designed specifically for an AI agent to interact with your application’s core functions. Suddenly, your application isn’t just something a user clicks through; it’s something an agent can reason with.

This opens up some fascinating possibilities for reducing the code we have to write. Think about all the boilerplate logic we build for handling various user scenarios. A classic example is bulk data import. Instead of building a dedicated feature that requires users to format a CSV file perfectly, what if they could simply instruct an agent: “Here’s a list of three new employees, please add them to the payroll system.” The LLM, interacting through that MCP, could parse the request and make the necessary API calls. The complex use case is handled with a simple conversation.

The real magic here is the LLM’s ability to handle ambiguity and diverse formats. You’re no longer forced to write separate parsers for JSON, XML, CSV, or even image-based data. A user could, in theory, paste a screenshot from an old system, and the LLM would perform the OCR, understand the context, and map the data to the correct API endpoints. Or consider an even more unstructured example: a user receives an email from a candidate saying, “Yes, I accept your offer.” They could just forward that email to the system, and the LLM would create the new employee record. Your APIs become dramatically more powerful because the LLM acts as a universal, intelligent adapter. Even if it only gets things 80% right, prompting the user to confirm or fix the remaining 20% is still a monumental leap in efficiency, achieved without writing a single line of custom import logic.

But as I explored this idea, a practical challenge emerged. What about high-volume, repetitive tasks? Take a financial application I’m working on, for instance. A core requirement is to import months of bank transactions and identify recurring patterns, like subscriptions, to project future expenses. Throwing thousands of transactions at an LLM every time a user uploads a statement seems wildly inefficient. The computational cost and data transfer would be significant. It felt like using a sledgehammer to crack a nut, over and over again. This is where the initial idea evolved into a more sustainable, hybrid approach.

What if, instead of having the LLM do the heavy lifting every single time, we use it to teach our system how to do the work itself?

Let’s walk through that financial application scenario again with this hybrid model. A user uploads their transactions for the first time. The system can first apply any rules it already knows. Then, the user could click a button like, “Analyze for new patterns.” This is the moment we engage the LLM. It scans the data and might come back with an insight: “I’ve noticed three payments of $15.99 to ‘Netflix.’ This appears to be a recurring monthly expense. Would you like me to create a rule to automatically categorize this in the future?”

If the user approves, the LLM doesn’t just categorize those three transactions. It generates the rule itself — perhaps as a JSON configuration or a simple script — that defines how to identify a Netflix subscription from now on. That rule is then saved locally within your application. The next time the user uploads their statement, the system identifies the Netflix payment on its own, instantly and efficiently, without ever calling the LLM.

This pattern feels powerful because it’s so adaptable. It can be applied to almost any domain. Imagine a system monitoring telemetry events. The LLM could analyze a stream of alerts and suggest, “This pattern of events seems critical. Should I create a rule to notify all administrators whenever it occurs?” With user approval, the rule is baked into the system. You get the intelligence of the LLM to establish the pattern, and the efficiency of local code to execute it.

The cycle doesn’t have to end there. You can collect telemetry on how these rules are performing. Over time, you could even use an LLM to audit them. It might suggest, “These three rules are very similar; I can combine them into a single, more efficient rule,” reducing the cognitive load on users who have to manage them. Or it might point out, “This rule hasn’t been used in six months. Perhaps it’s obsolete.” The LLM becomes not just a creator, but a curator, helping to maintain the system it helped build.

So, this is the thought I’m exploring. It’s a shift from viewing LLMs as just conversationalists to seeing them as collaborative partners in the development process. By using them to generate logic, not just execute it, we can create applications that are more flexible, learn over time, and ultimately allow us to build more powerful systems by writing significantly less code. It’s just a hypothesis, but it’s an exciting one to consider.