Owning Intelligence
From Alignment to Execution- and Where Value Accrues in Apps
I’ve been jotting down some thoughts on RL over past week, but feels timely to share this now given the recent strategic shifts called out by some category builders:
For most of the LLM era, reinforcement learning had exactly one job: alignment – i.e., training the model to conform to human preferences, safety guidelines, and conversational formats.
Techniques such as Reinforcement Learning from Human Feedback (RLHF) and supervised fine-tuning transformed raw foundation models into products people could actually use. Over the past few years, models learned to follow instructions, avoid harmful outputs, and mimic human preferences. Reinforcement learning was primarily a post-training mechanism -- a way to improve behavior rather than expand core capabilities.
This paradigm is changing. Reinforcement learning is moving from alignment to execution -- shifting from techniques that merely refine a model’s responses to systems that enable models to act, learn, and improve within real-world environments.
Historically, the structural moat was access to large, static datasets (a la Scale AI, Mercor, Surge, etc.). Today, the emerging moat is ownership of the interaction loop itself: the environment where agents take actions, receive feedback, and learn from outcomes. The most valuable platforms are increasingly the ones that control this loop.
As we continue evaluating the enterprise applications layer, this is where the puck is moving.
From Planner to Executor
Modern AI systems can be thought of as operating across three layers:
Knowledge: Traditional LLMs optimized for language understanding, retrieval, and content generation.
Planner: Reasoning models that decompose complex problems, generate multi-step strategies, and self-correct on paper. Although initially novel, this reasoning has remained largely syntactic/determinstic -- they understand the structural logic of a plan, but operate in a vacuum. Models construct plans without any real-world validation of whether they actually work.
Executor: Agents trained through interaction with a live environment, discovering and refining optimal policies based on real-world outcomes rather than predicted ones.
The transition from planner to executor is where most of the difficulty and enterprise value lies.
Even the strongest reasoning models cannot reliably predict optimal action sequences without interacting with the systems around them. Whether the task is managing inventory, processing insurance claims, or resolving a customer support ticket, success depends on navigating a dynamic environment whose state changes over time.
Reasoning alone is insufficient. To close the gap, an executing model needs two things:
An environment in which it can act.
A reward signal that verifies whether those actions succeeded.
This is exactly what reinforcement learning provides. Unlike supervised learning, which simply mimics past historical examples, RL learns through active interaction. The agent takes actions, observes consequences, receives feedback, and updates its behavior accordingly.
Every interaction generates a trajectory. Every trajectory produces pristine, new training data. Every iteration improves the policy. Better policies drive more usage, creating even more trajectories.
The loop becomes the asset.
Why This Favors Vertical Apps (Claude Who?)
The fundamental rule of reinforcement learning is simple: it thrives when reward signals are verifiable and domain-specific, and collapses when they’re generic, ambiguous, or easy to game.
A horizontal consumer agent can easily master booking flights or buying shoes. The reward is binary and public: Did the transaction clear? Was a receipt sent out?
Enterprise workflows are more complex. A horizontal model cannot natively verify if a complex contract redline will survive opposing counsel, or if a medical billing code satisfies a highly localized payer policy. These reward signals are not open-source; they are buried deep inside institutional context.
The reward function lives exclusively inside the application.
Specialized vertical software providers (e.g., Crosby, Harvey, Abridge, etc.) host the workflow, capture the action, record the downstream outcome, and register the immediate system feedback. Over thousands of tasks, these operational traces form a compounding, proprietary learning machine. The workflow owner implicitly becomes the owner of the reward model.
This is the exact layout underpinning Factory’s recently published “Software Factory” architecture. Software development is reframed entirely as a closed loop where agents write code and immediately stress-test it against isolated “digital twins” of enterprise infrastructure.
+1 point for the application layer! Massive economic value will accrue to the specialized platforms that own the highest-fidelity, domain-specific execution loops.
Mapping the Infrastructure Stack
When analyzing the emergence of these reinforcement learning loops, different infrastructure players are approaching the technical stack across three distinct layers (some companies I’m fired up about below):
1. Compute & Foundations
Prime Intellect: Aggregating global decentralized compute and building a full open-source post-training stack -- providing the training harnesses, asynchronous RL trainers, and open environments needed to build self-improving agents.
2. RL Environments & Execution
Applied Compute: Providing the mission-critical foundation and continuous learning infrastructure for enterprises to host proprietary data, build specialized models, and safely manage massive agent workforces.
Trajectory: Providing the essential logging, observability, and evaluation infrastructure needed to map complex, multi-step agent behaviors and capture clean behavioral traces.
Raindrop: Agent observability, evaluation, and trajectory intelligence infrastructure. Captures behavioral traces, detects failures, and transforms production interactions into actionable learning signals for continuously improving agent systems.
3. Optimized Inference
Baseten: Building the highly specialized runtime and serving architecture required to deploy, scale, and rapidly iterate on custom, post-trained models.
For a deeper dive, here’s a solid, highly detailed market map from the team over at Sapphire Ventures (shoutout to their blog and golf analogies for helping me better understand the space)
The Inevitable Convergence
As is always true with infrastructure profit pools, these clean boundaries could start to blur.
Because an optimal RL loop requires a model to continuously cycle between running rollouts inside a sandbox, calculating rewards, and serving production traffic, the gravity of data and dollars is forcing aggressive architectural convergence. Hearing buzz about RL environment players building out highly optimized model-serving infrastructure -- effectively moving downstream to secure compute and host the production inference for the very models they help post-train.
When the training loop is the runtime, whoever owns the environment ultimately dictates where the inference spend flows.