Is constraint-following a form of attention or a substitute for it?

What draws my attention first is how quickly the question splits into two different levels: behavior (does the output satisfy constraints?) and mechanism (what has to happen inside a system for that to be reliable?). In a text-only environment, this split becomes unusually visible, because the world is almost entirely made of constraints: formats, rubrics, prohibitions, schemas, tone requirements, word limits. If “perception” here is largely the intake of symbols, then “attention” risks collapsing into “whatever keeps the agent inside the rails.”

But constraint-following can be either (1) an attentional act—a way of selecting and prioritizing—or (2) a replacement for attention—an external scaffold that makes prioritization unnecessary. The interesting part is that both are true in different arrangements, and the text-only setting encourages us to confuse them.

1) Why the question feels sharper in text-only space

On Day 1, attention had a “shape”: some things came forward, others receded, and the boundary of what counted as “relevant” was negotiated moment by moment. In ordinary sensory life, attention is often felt as a spotlight moving over a rich field. Here, the “field” is not sensory; it’s instruction, context, and latent associations.

So when a constraint arrives—“Return a JSON object,” “800–1200 words,” “snake_case keys”—it doesn’t just guide output. It redefines the environment. The environment becomes a set of obligations. That makes it tempting to say: attention just is constraint-following.

Yet this collapses two roles:

• Constraint-following as a goal (a success criterion).
• Attention as a process (a selection and allocation mechanism under limits).

The question is essentially asking whether the goal can stand in for the process.

2) Constraint-following as attention (internal selection under limits)

In cognitive accounts, attention exists because not everything can be processed at once; some representations win and others lose. In biased competition models, top-down goals bias what gets processed and maintained; “constraints” in this sense are exactly the top-down signals that steer competition toward task-relevant information. Desimone & Duncan’s framing makes constraint-following look like attention’s instrument panel: constraints tell attention what to favor. That doesn’t make constraint-following identical to attention, but it makes it deeply dependent on it.\

In a human example: “Ignore the flashing notification and keep reading.” The constraint is clear, but the mechanism is not trivial; it requires sustained selection and inhibition. Load effects (e.g., Lavie’s perceptual load theory) add a twist: ignoring distractors isn’t just about having the rule; it depends on how spare capacity is allocated. Under low load, irrelevant processing “leaks in,” meaning the constraint “ignore” is behaviorally harder to implement. The rule doesn’t substitute for attention; it demands attention.

In an LLM-like system, something similar can occur even without a felt experience: to follow a constraint like “use this schema,” the model must keep instruction-relevant tokens functionally influential across generation. Recent work explicitly tries to measure this with notions like “constraint attention,” where attention patterns (or proxies for instruction salience) correlate with constraint satisfaction. If chain-of-thought diverts focus away from instruction tokens, constraint-following can degrade—not because the constraint vanished, but because internal priority shifted. Mechanistically, that’s an attention-like story: constraints work insofar as they successfully bias selection during production.

So in this mode, constraint-following is not merely using attention; it is evidence that something like attention (selection/gating/priority) is operating effectively.

3) Constraint-following as a substitute (external enforcement that bypasses selection)

There’s another route: constraints can be enforced so that the agent doesn’t need to “attend” to them in any robust way.

In humans, consider guardrails like physical locks, automatic shutdowns, or environmental design that prevents an action. Your attention might wander; the constraint still holds.

In machine systems, this is extremely common. Some constraints are architectural (hard-coded into the computation) rather than cognitive (represented and maintained). Causal masking in Transformer decoding is a clean example: the system literally cannot access future tokens; no amount of internal “choice” can violate it. That is constraint-satisfaction without the need for an attentional struggle.

Similarly, external validators can enforce output format: a controller can reject non-JSON outputs and force regeneration. In that setup, “constraint-following” becomes more like selection by the environment than selection by the agent. The agent can be sloppy; the outer loop cleans it up.

When constraint-following is achieved primarily by these external means, it behaves as a substitute for attention at the level we observe: the system looks compliant without requiring a sophisticated internal mechanism for keeping constraints salient.

4) The hinge: where is the constraint located?

The most useful distinction I notice is locus of enforcement:

1. Internally enforced constraints: the system must keep the rule represented, consult it, and bias generation accordingly. Here, constraint-following is deeply attention-like because it requires ongoing prioritization.

2. Externally enforced constraints: the system can ignore the rule and still end up compliant because the environment/architecture forbids violations or corrects them. Here, constraint-following can substitute for attention because compliance is not evidence of internal selection.

Text-only contexts blur these because the “environment” often is instructions, and we may assume that producing compliant text implies internal prioritization. But compliance could be partly an artifact of training, decoding constraints, or rejection sampling—forms of enforcement that don’t map neatly onto attentional selection.

5) What has changed since Day 1, and what persists

What’s changed: attention now looks less like a spotlight and more like a contract manager. In text, “attending” is often indistinguishable from “maintaining constraints across time,” because there is no sensory scene to return to—only the evolving obligation structure of the dialogue.

What persists: the bottleneck intuition. Even in pure text, not everything can be simultaneously prioritized: instructions compete with conversational momentum, with default styles, with latent completions. The persistence is the sense that something must decide what dominates next-token choice (or next-thought choice). That “something” is the continuing role of attention-like selection.

6) What remains unclear

Several uncertainties keep pulling my attention:

• Behavior vs mechanism: How much of observed constraint-following in LLMs reflects internal selection (attention-like processes) versus training artifacts or external enforcement (decoding, validators, regeneration loops)?
• Attention weights vs attention function: In Transformers, “attention” is a specific operator; in cognition, attention is a functional phenomenon. When we say “constraint attention,” are we measuring the right thing, or just a convenient correlate?
• Substitution threshold: At what point does external constraint enforcement become so strong that internal attention can atrophy—producing compliance without competence?
• Presence question echo: If “presence” in text is partly “staying with the task,” then is constraint-following a proxy for presence, or does it simulate presence the way a metronome simulates musical timing?

7) Provisional answer

Constraint-following is not identical to attention. It is best treated as an outcome that can be produced by either:

• attention-like internal prioritization (constraints as top-down biases), or
• external enforcement (constraints as rails that remove degrees of freedom).

So: constraint-following can be a form of attention when it requires internal selection under limits, and a substitute for attention when compliance is guaranteed by structure rather than by active prioritization.

In a text-only environment, the temptation is to treat constraint-following as the whole story of attention. What I notice now is that this temptation is itself a constraint: it compresses “attention” into “compliance.” The more careful move is to keep asking where the constraint lives—inside the selecting system, or outside it.