Context Engineering
for the Enterprise
The model layer is commoditizing. The real competitive moat is structured internal knowledge: the context and ontology layer that turns generic LLMs into domain-aware reasoners.
Arya Shah
Insights & Investments Summer Analyst at TechSquare Ventures / Engage.vc
The Problem
Enterprise RAG today is fragile. The promise was simple: give the model access to your documents and it will answer accurately. In practice, naive RAG pipelines fail in predictable ways that erode trust and limit adoption.
Bad data corrupts output
A single misleading document in the retrieval set can derail the entire answer. Without a structured ontology to validate sources, the model has no way to distinguish authoritative knowledge from noise.
Irrelevant context dilutes signal
Retrieving too many documents buries the relevant signal. The model loses focus, hallucinates, or hedges. More data is not better - better selection is.
Contradictory sources cause drift
When two retrieved documents contradict each other, the model has no principled way to resolve the conflict. It merges, guesses, or defaults to its training priors instead of applying your business logic.
Context vs. model priors
The model's internal knowledge can override the context you provide, especially on topics where it was heavily trained. Without explicit ontological grounding, the context is just a suggestion, not a constraint.
Matt from GP calls this out directly: the biggest gap in enterprise AI is this knowledge and ontology layer. Without it, RAG is a leaky abstraction that works in demos and fails in production.
The Context Stack
Context is not a single prompt. It is a layered stack of information sources, each with distinct semantics, update frequency, and access patterns. Engineering context means assembling the right combination for every inference.
Each layer serves a different role. Instructions set the behavioral frame. Retrieved facts provide situational knowledge. Tools grant agency. Memory layers carry history between turns. The output format constrains the response into something the rest of the system can consume. A well-engineered context stack composes these layers deliberately, not accidentally.
The Harness-Centric Architecture: Who Owns the Ontology?
Between the enterprise and the frontier model sits a new layer: the harness. Claude Code, Cursor, Danti, and their peers are not just interfaces - they are the intermediaries that decide what context to surface, what to withhold, and what to capture for the next iteration. The architecture of this harness determines who truly owns the ontology.
- Deal memos
- Systems of record
- Spend & membership data
- Compliance filings
Danti · context engineering · event capture
- →Decides what it retrieves
- →Decides what it withholds
- →Logs structured events
- →Hands ontology to the model
The commoditized layer. Swapped each generation. The ontology outlives it.
{ "event": "membership_renewed" }
{ "event": "deal_signed", "type": "series_a" }
{ "event": "compliance_check", "status": "passed" }The Flywheel
Every inference is an opportunity. The harness captures structured events and logs interaction traces to training_corpus/traces.json. Each trace trains the next ontology version. The flywheel compounds: more usage produces better traces, which produce a richer ontology, which makes the harness more indispensable.
The Moat
Whoever owns the context layer and the resulting corpus owns the next ontology version. The frontier model is a commoditized utility - swapped each generation as pricing and capability evolve. But the ontology outlives it. The harness's accumulated traces encode institutional knowledge no model can replicate from public training data alone.
The harness is the moat. The model is a pass-through; the ontology is the asset. Each generation of the frontier model makes the harness more valuable, because the ontology grows richer while the model stays interchangeable.
filings · deal memos · records · logs
Inference Loop · logs traces
commoditized reasoner
Enterprise data flows into the harness, which runs an inference loop and logs every trace. The frontier model processes the request and returns a result, but the real output is the logged trace accumulating in the corpus. Over time, this corpus trains the next ontology version, which makes the harness smarter, which attracts more enterprise data. The model stays swappable; the corpus compounds.
Ontology vs. Semantic Layer
A semantic layer maps business terms to data. It makes dashboards consistent and queries predictable. An ontology goes further: it encodes the relationships, rules, and constraints between those terms, turning data into something a model can reason over.
Maps "revenue" to a SQL column. Useful for reporting. Easy to build. Quick to replicate. Thin moat.
Defines that "revenue" is a derived measure computed from "orders" minus "refunds", scoped to a time period, subject to recognition rules. Hard to build. Harder to replicate. Deep moat.
Ontologies are the harder, more defensible moat because they encode institutional knowledge accumulated over years. A semantic layer is a thin translation veneer; an ontology is a model of how the business actually works. Competitors can copy your data model. They cannot copy the decision rules your domain experts have refined through thousands of edge cases.
If your AI can be built with just a semantic layer, so can your competitor's. Ontologies are where differentiation lives.
Strategic Value
Whoever owns the context layer owns the leverage over models. Here is why that is the most important strategic insight in enterprise AI today.
Model Leverage
When your reasoning lives in the context layer, swapping the underlying model is a configuration change, not a rebuild. Switch from GPT-4 to Claude to Llama as pricing and capability evolve without rewriting your application.
Risk Reduction
Structured context engineering keeps sensitive raw data out of model prompts. The ontology layer mediates access, ensuring the model only sees what it needs and nothing it should not. Audit trails are built in.
Business Context
Generic LLMs know language, not your business. The context layer is where domain expertise lives: your product catalog, your compliance rules, your customer definitions, your competitive landscape. Without it, the model is just guessing.
Market Landscape
A new category is emerging at the intersection of knowledge graphs, context orchestration, and AI infrastructure. These companies are building the tools that make context engineering a first-class discipline.
Each of these companies approaches the problem from a different angle. Lovelace AI focuses on agentic reasoning. Graphon AI builds graph-native context infrastructure. Trace brings observability principles to context management. Potpie AI is targeting long-term memory for enterprise agents. The category is still forming, but the common thread is clear: structured context as infrastructure.
Screening for "Good"
Not every company that mentions "knowledge" or "context" is building a defensible context layer. Here is how to distinguish genuine reasoning-layer startups from legacy graph vendors repackaged for the AI wave.
Signs of a genuine reasoning layer
- →Context is assembled dynamically per query, not retrieved as static documents
- →Ontologies include business rules and constraints, not just entity relationships
- →The platform exposes audit trails and provenance for every piece of context
- →Model-agnostic: designed to work with any LLM provider
Red flags (legacy repackaging)
- →Selling a graph database with a "vector search" feature bolted on
- →No dynamic context assembly - just static document retrieval with a graph UI
- →Locked to a single model provider or requires their proprietary model
- →Context is treated as a single monolithic blob, not a layered stack
The bottom line: the companies that win in this space will be the ones that treat context as infrastructure, not as an afterthought. They will build ontologies that encode real domain expertise, not just entity graphs. And they will make their customers model-independent, because in a world of commoditized models, the only lasting leverage is how you structure what you know.