Solving Context Before Adding Infrastructure

I keep seeing the same technologies in AI engineering roles: LangChain, LangGraph, RAG, vector databases and agent orchestration.

They solve real problems. After building extensively with ICM, I've started wondering how often we introduce that infrastructure before deciding whether the problem actually requires it.

ICM approaches the problem from the context side.

A workflow is divided into stages. Each stage has one job, known inputs, expected outputs and defined context. The agent reads what that stage requires and produces an artifact that the next stage can work from.

Context selection becomes part of the design.

That distinction has changed how I think about RAG.

RAG is extremely useful when the system cannot know in advance which information will be relevant. If someone can ask arbitrary questions across thousands of changing documents, retrieval has to determine what information should reach the model.

That creates a second problem alongside the original AI task: retrieval itself has to work well.

Documents need to be chunked. Search and ranking need to return the right material. Retrieval quality has to be evaluated. If an answer is wrong, you may need to determine whether the model reasoned poorly or whether it never received the right information in the first place.

There are many applications where that complexity is justified.

But consider a structured workflow where the current stage already tells you which references are relevant.

If I'm performing a security review and the workflow already defines the security requirements, architecture references and files that should be inspected, I'm not convinced semantic retrieval should automatically sit between the agent and that information.

The architecture already knows what context is required.

In that situation, retrieving the "most relevant" context at runtime may be less useful than explicitly providing the correct context by design.

The failure modes change too.

With retrieval, a poor result can come from chunking, indexing, query formulation, ranking or reasoning.

With explicit context routing, I remove several of those variables. I can inspect exactly what the agent was told to read.

That doesn't eliminate context errors. I can still design a stage badly and give it the wrong references. The difference is that the mistake is explicit and inspectable rather than emerging from a retrieval pipeline.

I've reached a similar conclusion with agent orchestration.

Before creating another agent, I now ask whether I need another independent reasoning process or whether the existing model needs a better-defined job.

A single capable model can behave very differently when its role, context, tools and expected output change from one stage to another. Adding more agents also adds coordination, state and handoff problems that eventually need to be understood and debugged.

This is one reason I like having state and context represented as ordinary files.

I can inspect them. A human can edit them. Git can version them. The next session can read the previous session's log instead of depending on hidden conversational memory.

There are clear boundaries to this approach.

I use ICM to structure how I build systems. It is not the runtime architecture of my production applications.

If I need dynamic search across a large knowledge base, RAG may be exactly the right tool. If I need persistent runtime execution, concurrent users, dynamic routing or complex application state, dedicated infrastructure has a reason to exist.

So I've stopped thinking about this as ICM versus LangChain or ICM versus RAG.

The more useful question for me is whether context should be selected at design time or discovered at runtime.

When the workflow already knows what information a task requires, I prefer to make that knowledge explicit.

When it doesn't, retrieval and orchestration earn their place.

That changes the order in which I design AI systems. I define the work, context boundaries, sources of truth, human checkpoints and outputs first. Infrastructure comes later, once I can point to the problem it needs to solve.

A surprising amount of complexity disappears when context architecture is treated as an architecture problem before it becomes a retrieval problem.

9 comments

Solving Context Before Adding Infrastructure