Skip to content

Steering specs (vllm.steer_vectors)

The classes below are the public authoring API. Their descriptions, field types, defaults, and methods are rendered from the fork's source definitions.

from vllm.steer_vectors import ApplySpec, SelectSpec, SteeringSpec, VectorSpec
Class Purpose
SteeringSpec A complete request configuration: ordered vectors and a conflict policy.
VectorSpec One payload, algorithm, scale, layer set, and application rule.
ApplySpec Token selection for a steering vector.
SelectSpec The same selection language, also exported by vllm.capture.

For worked examples and selection semantics, see steering requests. The algorithm reference covers accepted payloads and graph conditions; engine configuration covers startup declarations and capacity.

Constructing and validating specs

spec = SteeringSpec(vectors=[VectorSpec(
    source="vectors/happy_diffmean.gguf",
    algorithm="direct",
    scale=2.0,
    layers=list(range(10, 24)),
    apply=ApplySpec(prompt="all", generation="all"),
)])

text = spec.model_dump_json()
restored = SteeringSpec.model_validate_json(text)

Specs are Pydantic models. Constructors and model_validate(...) validate field values and reject unknown keys. model_dump(mode="json") returns a dictionary; model_dump_json() returns JSON text; model_validate_json(text) returns a validated model. For tensor payloads that must be sent as JSON, use to_json_payload before placing the payload in VectorSpec.data.

Pass a complete spec to llm.generate(..., steering=spec). Omission or None inherits the current engine default; False disables steering. Python batches can supply one choice per prompt. Defaults and explicit specs are not stacked.

vllm.steer_vectors.SteeringSpec pydantic-model

Bases: BaseModel

A complete steering configuration: ordered vectors + conflict policy.

Attributes:

Name Type Description
vectors list[VectorSpec]

The vectors to apply (non-empty; order matters for 'sequential'/'priority' conflict resolution).
conflict Literal['priority', 'sequential', 'error']

What to do when several vectors target one position: 'priority' (first wins), 'sequential' (stack), 'error'.

Config:

  • extra: forbid

Fields:

Validators:

  • _validate
Source code in vllm-steer/vllm/model_hooks/steering/api.py 
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
class SteeringSpec(BaseModel):
    """A complete steering configuration: ordered vectors + conflict policy.

    Attributes:
        vectors: The vectors to apply (non-empty; order matters for
            'sequential'/'priority' conflict resolution).
        conflict: What to do when several vectors target one position:
            'priority' (first wins), 'sequential' (stack), 'error'.
    """

    model_config = ConfigDict(extra="forbid")

    vectors: list[VectorSpec]
    conflict: Literal["priority", "sequential", "error"] = "priority"

    @model_validator(mode="after")
    def _validate(self) -> "SteeringSpec":
        if not self.vectors:
            raise ValueError("vectors must be non-empty")
        if len(self.vectors) > 1 and any(
            v.algorithm == "moe_router" for v in self.vectors
        ):
            raise ValueError(
                "moe_router is not supported in multi-vector specs yet; "
                "use a single-vector spec"
            )
        return self

vectors pydantic-field 

vectors: list[VectorSpec]

conflict pydantic-field 

conflict: Literal["priority", "sequential", "error"] = (
    "priority"
)

vllm.steer_vectors.VectorSpec pydantic-model

Bases: BaseModel

One steering vector and how it applies.

Attributes:

Name Type Description
source str | None

Path to a vector file in a format EasySteer itself defines (its GGUF export; moe_router JSON). For third-party checkpoint formats, load the file yourself (or use an easysteer.vectors adapter) and pass data instead.
data Any

An in-memory payload (vllm.model_hooks.steering.payloads, or its wire dict) — the canonical way to steer with tensors you constructed or loaded yourself. Mutually exclusive with source.
algorithm str

Steering algorithm name (registry key).
scale float

Scale factor.
layers list[int] | None

Layer indices to apply to; None lets the source decide.
normalize bool

Rescale the transformed hidden state to its original norm for direct, erase, replace, and concept_replace. Other algorithms reject True.
apply ApplySpec

Where/when the vector applies (required).
params dict[str, Any]

Algorithm-specific parameters; validated per algorithm, unknown keys are rejected.
name str | None

Optional request label; used as the engine-side request name when set (a generated id otherwise).

Config:

  • extra: forbid

Fields:

Validators:

  • _validate
Source code in vllm-steer/vllm/model_hooks/steering/api.py 
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
class VectorSpec(BaseModel):
    """One steering vector and how it applies.

    Attributes:
        source: Path to a vector file in a format EasySteer itself
            defines (its GGUF export; moe_router JSON). For third-party
            checkpoint formats, load the file yourself (or use an
            `easysteer.vectors` adapter) and pass `data` instead.
        data: An in-memory payload (`vllm.model_hooks.steering.payloads`, or
            its wire dict) — the canonical way to steer with tensors
            you constructed or loaded yourself. Mutually exclusive with
            source.
        algorithm: Steering algorithm name (registry key).
        scale: Scale factor.
        layers: Layer indices to apply to; None lets the source decide.
        normalize: Rescale the transformed hidden state to its original norm
            for direct, erase, replace, and concept_replace. Other algorithms
            reject True.
        apply: Where/when the vector applies (required).
        params: Algorithm-specific parameters; validated per algorithm,
            unknown keys are rejected.
        name: Optional request label; used as the engine-side request
            name when set (a generated id otherwise).
    """

    model_config = ConfigDict(extra="forbid")

    source: str | None = None
    data: Any = None
    algorithm: str = "direct"
    scale: float = 1.0
    layers: list[int] | None = None
    normalize: bool = False
    apply: ApplySpec
    params: dict[str, Any] = {}
    name: str | None = None

    @model_validator(mode="after")
    def _validate(self) -> "VectorSpec":
        validate_vector_input(
            self.source,
            self.data,
            self.algorithm,
            self.layers,
            self.params,
            normalize=self.normalize,
        )
        return self

source pydantic-field 

source: str | None = None

data pydantic-field 

data: Any = None

algorithm pydantic-field 

algorithm: str = 'direct'

scale pydantic-field 

scale: float = 1.0

layers pydantic-field 

layers: list[int] | None = None

normalize pydantic-field 

normalize: bool = False

apply pydantic-field 

apply: ApplySpec

params pydantic-field 

params: dict[str, Any] = {}

name pydantic-field 

name: str | None = None

vllm.steer_vectors.ApplySpec

Bases: SelectSpec

Where and when a steering vector applies.

The selection language itself lives in SelectSpec (shared with capture); ApplySpec is the steering-facing name.

Source code in vllm-steer/vllm/model_hooks/steering/api.py 
51
52
53
54
55
56
class ApplySpec(SelectSpec):
    """Where and when a steering vector applies.

    The selection language itself lives in `SelectSpec` (shared with
    capture); `ApplySpec` is the steering-facing name.
    """

ApplySpec inherits every field, default, validation rule, and wire-conversion method from SelectSpec below. It is the type required by VectorSpec.apply.

vllm.capture.SelectSpec pydantic-model

Bases: BaseModel

The shared token-selection language (where-clause).

One selection semantics used by both steering (ApplySpec, "which tokens to steer") and capture ("which rows to extract"): resolved identically by the trigger collector, so a clause means the same thing in both systems.

Each phase is selected independently — prompt="all" selects every prompt token, prompt_positions=[-1] selects one, and a phase with neither "all" nor a selector is untouched. "Last prompt token plus the whole generation" is therefore one clause: SelectSpec(prompt_positions=[-1], generation="all").

Attributes:

Name Type Description
prompt Literal['all'] | None

"all" selects every prompt token — the widest prompt include selector; unions with the others like any include.
generation Literal['all'] | None

"all" selects every generated token — the widest generation include selector.
prompt_tokens list[int] | None

Token-id allowlist over prompt tokens (real ids, >= 0).
prompt_positions list[int] | None

Prompt positions; negative values are Python-style from the end of the prompt (-1 = last prompt token). Positive values past the prompt end clamp to the last prompt token (warned at admission).
prompt_window tuple[int, int | None] | None

Half-open (start, stop) over prompt positions; negative bounds resolve from the end of the prompt ((-5, None) = the last five prompt tokens).
generation_tokens list[int] | None

Token-id allowlist over generated tokens (real ids, >= 0).
generation_positions list[int] | None

0-based decode step indices ([0] = the first generated token).
generation_window tuple[int, int | None] | None

Half-open (start, stop) over 0-based decode steps; stop=None means unbounded.
exclude_prompt_tokens list[int] | None

Prompt token ids to never select.
exclude_prompt_positions list[int] | None

Prompt positions to never select (same convention as prompt_positions).
exclude_prompt_window tuple[int, int | None] | None

Prompt window to never select (same convention as prompt_window).
exclude_generation_tokens list[int] | None

Generated token ids to never select.
exclude_generation_positions list[int] | None

Decode steps to never select.
exclude_generation_window tuple[int, int | None] | None

Decode-step window to never select.

Selection = union of the include selectors' matches ("all" being the widest selector of its phase); the exclude selectors union and always subtract — where an include and an exclude overlap, the exclusion wins. Exclusions require their phase to be covered, and a clause that selects nothing is rejected.

Config:

  • extra: forbid

Fields:

Validators:

  • _validate
Source code in vllm-steer/vllm/model_hooks/selection/spec.py 
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
class SelectSpec(BaseModel):
    """The shared token-selection language (where-clause).

    One selection semantics used by both steering (`ApplySpec`, "which
    tokens to steer") and capture ("which rows to extract"): resolved
    identically by the trigger collector, so a clause means the same
    thing in both systems.

    Each phase is selected independently — `prompt="all"` selects every
    prompt token, `prompt_positions=[-1]` selects one, and a phase with
    neither "all" nor a selector is untouched. "Last prompt token plus
    the whole generation" is therefore one clause:
    `SelectSpec(prompt_positions=[-1], generation="all")`.

    Attributes:
        prompt: "all" selects every prompt token — the widest prompt
            include selector; unions with the others like any include.
        generation: "all" selects every generated token — the widest
            generation include selector.
        prompt_tokens: Token-id allowlist over prompt tokens (real
            ids, >= 0).
        prompt_positions: Prompt positions; negative values are
            Python-style from the end of the prompt (`-1` = last prompt
            token). Positive values past the prompt end clamp to the
            last prompt token (warned at admission).
        prompt_window: Half-open (start, stop) over prompt positions;
            negative bounds resolve from the end of the prompt
            (`(-5, None)` = the last five prompt tokens).
        generation_tokens: Token-id allowlist over generated tokens
            (real ids, >= 0).
        generation_positions: 0-based decode step indices (`[0]` = the
            first generated token).
        generation_window: Half-open (start, stop) over 0-based decode
            steps; stop=None means unbounded.
        exclude_prompt_tokens: Prompt token ids to never select.
        exclude_prompt_positions: Prompt positions to never select
            (same convention as `prompt_positions`).
        exclude_prompt_window: Prompt window to never select (same
            convention as `prompt_window`).
        exclude_generation_tokens: Generated token ids to never select.
        exclude_generation_positions: Decode steps to never select.
        exclude_generation_window: Decode-step window to never select.

    Selection = union of the include selectors' matches ("all" being
    the widest selector of its phase); the exclude selectors union and
    always subtract — where an include and an exclude overlap, the
    exclusion wins. Exclusions require their phase to be covered, and a
    clause that selects nothing is rejected.
    """

    model_config = ConfigDict(extra="forbid")

    prompt: Literal["all"] | None = None
    generation: Literal["all"] | None = None
    prompt_tokens: list[int] | None = None
    prompt_positions: list[int] | None = None
    prompt_window: tuple[int, int | None] | None = None
    generation_tokens: list[int] | None = None
    generation_positions: list[int] | None = None
    generation_window: tuple[int, int | None] | None = None
    exclude_prompt_tokens: list[int] | None = None
    exclude_prompt_positions: list[int] | None = None
    exclude_prompt_window: tuple[int, int | None] | None = None
    exclude_generation_tokens: list[int] | None = None
    exclude_generation_positions: list[int] | None = None
    exclude_generation_window: tuple[int, int | None] | None = None

    def _covers(self, phase: str) -> bool:
        """Whether the clause selects anything in the given phase."""
        if getattr(self, phase) == "all":
            return True
        return any(
            getattr(self, key) is not None
            for key in INCLUDE_SELECTOR_KEYS
            if key.startswith(phase)
        )

    @model_validator(mode="after")
    def _validate(self) -> "SelectSpec":
        if not self._covers("prompt") and not self._covers("generation"):
            raise ValueError(
                "the clause selects nothing: set prompt='all' / "
                "generation='all' or at least one include selector"
            )
        for phase in ("prompt", "generation"):
            excludes = [
                key
                for key in EXCLUDE_SELECTOR_KEYS
                if key.startswith(f"exclude_{phase}") and getattr(self, key) is not None
            ]
            if excludes and not self._covers(phase):
                raise ValueError(
                    f"{sorted(excludes)} exclude {phase} tokens, but the "
                    f"clause selects none: set {phase}='all' or a "
                    f"{phase}_* selector"
                )
        for name in (
            "prompt_tokens",
            "prompt_positions",
            "generation_tokens",
            "generation_positions",
            "exclude_prompt_tokens",
            "exclude_prompt_positions",
            "exclude_generation_tokens",
            "exclude_generation_positions",
        ):
            _require_nonempty(name, getattr(self, name))
        for name in (
            "prompt_tokens",
            "generation_tokens",
            "exclude_prompt_tokens",
            "exclude_generation_tokens",
        ):
            ids = getattr(self, name)
            if ids is not None and any(t < 0 for t in ids):
                raise ValueError(f"{name} must contain real token ids (>= 0)")
        for name in ("generation_positions", "exclude_generation_positions"):
            ids = getattr(self, name)
            if ids is not None and any(j < 0 for j in ids):
                raise ValueError(
                    f"{name} must contain 0-based decode steps (>= 0); "
                    "the generation length is not known up front, so "
                    "end-relative steps cannot resolve"
                )
        for name in ("prompt_window", "exclude_prompt_window"):
            window = getattr(self, name)
            if window is None:
                continue
            start, stop = window
            if stop is not None and (start < 0) == (stop < 0) and stop <= start:
                raise ValueError(
                    f"{name} must be a half-open (start, stop) with "
                    f"stop > start (or stop=None for the prompt end), "
                    f"got {window}"
                )
        for name in ("generation_window", "exclude_generation_window"):
            window = getattr(self, name)
            if window is None:
                continue
            start, stop = window
            if start < 0:
                raise ValueError(f"{name} start must be >= 0, got {start}")
            if stop is not None and stop <= start:
                raise ValueError(
                    f"{name} must be a half-open (start, stop) with "
                    f"stop > start or stop=None, got {window}"
                )
        return self

    def to_wire(self) -> dict[str, Any]:
        """Canonical dict carried on the engine struct (`apply_spec`).

        Fixed key order and list-only containers so the config
        fingerprint and msgspec round-trips are deterministic.
        """

        return self.model_dump(mode="json")

    @classmethod
    def from_wire(cls, wire: dict[str, Any]) -> "SelectSpec":
        """Validate and rebuild a spec from its `to_wire()` dict.

        Used engine-side to reject malformed selection clauses at
        enable time instead of failing mid-forward.
        """
        unknown = set(wire) - set(APPLY_SPEC_KEYS)
        if unknown:
            raise ValueError(f"unknown selection fields: {sorted(unknown)}")

        values = dict(wire)
        for key in (*INCLUDE_SELECTOR_KEYS, *EXCLUDE_SELECTOR_KEYS):
            if key.endswith("_window") and values.get(key) is not None:
                values[key] = tuple(values[key])
        return cls.model_validate(values)

prompt pydantic-field 

prompt: Literal['all'] | None = None

generation pydantic-field 

generation: Literal['all'] | None = None

prompt_tokens pydantic-field 

prompt_tokens: list[int] | None = None

prompt_positions pydantic-field 

prompt_positions: list[int] | None = None

prompt_window pydantic-field 

prompt_window: tuple[int, int | None] | None = None

generation_tokens pydantic-field 

generation_tokens: list[int] | None = None

generation_positions pydantic-field 

generation_positions: list[int] | None = None

generation_window pydantic-field 

generation_window: tuple[int, int | None] | None = None

exclude_prompt_tokens pydantic-field 

exclude_prompt_tokens: list[int] | None = None

exclude_prompt_positions pydantic-field 

exclude_prompt_positions: list[int] | None = None

exclude_prompt_window pydantic-field 

exclude_prompt_window: tuple[int, int | None] | None = None

exclude_generation_tokens pydantic-field 

exclude_generation_tokens: list[int] | None = None

exclude_generation_positions pydantic-field 

exclude_generation_positions: list[int] | None = None

exclude_generation_window pydantic-field 

exclude_generation_window: tuple[int, int | None] | None = (
    None
)

to_wire 

to_wire() -> dict[str, Any]

Canonical dict carried on the engine struct (apply_spec).

Fixed key order and list-only containers so the config fingerprint and msgspec round-trips are deterministic.

Source code in vllm-steer/vllm/model_hooks/selection/spec.py 
170
171
172
173
174
175
176
177
def to_wire(self) -> dict[str, Any]:
    """Canonical dict carried on the engine struct (`apply_spec`).

    Fixed key order and list-only containers so the config
    fingerprint and msgspec round-trips are deterministic.
    """

    return self.model_dump(mode="json")

from_wire classmethod 

from_wire(wire: dict[str, Any]) -> SelectSpec

Validate and rebuild a spec from its to_wire() dict.

Used engine-side to reject malformed selection clauses at enable time instead of failing mid-forward.

Source code in vllm-steer/vllm/model_hooks/selection/spec.py 
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
@classmethod
def from_wire(cls, wire: dict[str, Any]) -> "SelectSpec":
    """Validate and rebuild a spec from its `to_wire()` dict.

    Used engine-side to reject malformed selection clauses at
    enable time instead of failing mid-forward.
    """
    unknown = set(wire) - set(APPLY_SPEC_KEYS)
    if unknown:
        raise ValueError(f"unknown selection fields: {sorted(unknown)}")

    values = dict(wire)
    for key in (*INCLUDE_SELECTOR_KEYS, *EXCLUDE_SELECTOR_KEYS):
        if key.endswith("_window") and values.get(key) is not None:
            values[key] = tuple(values[key])
    return cls.model_validate(values)

Engine-facing conversion

Applications normally pass a SteeringSpec directly to LLM.generate() or the HTTP API. Integration code can use to_engine_request() to resolve files and payloads into an internal SteeringRequest. Its name and int_id are labels; payload content and application fields determine execution identity.

vllm.steer_vectors.to_engine_request 

to_engine_request(
    spec: SteeringSpec,
    *,
    name: str | None = None,
    int_id: int | None = None,
)

Resolve an authoring spec into the engine's ordered intervention list.

Every source is frozen before admission. Name and integer ID are labels; payload content and application fields define execution identity.

Source code in vllm-steer/vllm/model_hooks/steering/api.py 
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
def to_engine_request(
    spec: SteeringSpec,
    *,
    name: str | None = None,
    int_id: int | None = None,
):
    """Resolve an authoring spec into the engine's ordered intervention list.

    Every source is frozen before admission. Name and integer ID are labels;
    payload content and application fields define execution identity.
    """
    from vllm.model_hooks.steering.loading import resolve_vector_payload
    from vllm.model_hooks.steering.request import (
        ResolvedVector,
        SteeringRequest,
        _next_steer_vector_id,
    )

    if name is None:
        from vllm.utils import random_uuid

        name = spec.vectors[0].name or f"steering-{random_uuid()[:8]}"
    if int_id is None:
        int_id = _next_steer_vector_id()

    vectors = [
        ResolvedVector(
            source=vector.source or "",
            payload=resolve_vector_payload(
                vector.source,
                vector.data,
                vector.algorithm,
                vector.layers,
                vector.params,
            ),
            scale=vector.scale,
            target_layers=list(vector.layers) if vector.layers is not None else None,
            algorithm=vector.algorithm,
            normalize=vector.normalize,
            apply_spec=vector.apply.to_wire(),
        )
        for vector in spec.vectors
    ]
    return SteeringRequest(
        steer_vector_name=name,
        steer_vector_int_id=int_id,
        vectors=vectors,
        conflict_resolution=spec.conflict,
    )

The returned request is an engine transport object, not an alternative authoring schema. File loading can fail here before request admission.

Engine default and preload methods

These are EasySteer's additions to vllm.LLM. For the rest of the class, refer to the upstream LLM API. Default changes affect new requests; admitted requests retain their snapshots.

LLM.set_default_steering 

set_default_steering(spec: SteeringSpec | None) -> None

Set the default for future requests, or clear it with None.

Explicit per-request configurations override this default. Requests already admitted retain their effective weight and selection snapshot.

Source code in vllm-steer/vllm/entrypoints/llm.py 
876
877
878
879
880
881
882
def set_default_steering(self, spec: SteeringSpec | None) -> None:
    """Set the default for future requests, or clear it with None.

    Explicit per-request configurations override this default. Requests
    already admitted retain their effective weight and selection snapshot.
    """
    self.llm_engine.set_default_steering(spec)

LLM.get_default_steering 

get_default_steering() -> dict

Return the current default authoring configuration and active status.

Source code in vllm-steer/vllm/entrypoints/llm.py 
884
885
886
def get_default_steering(self) -> dict:
    """Return the current default authoring configuration and active status."""
    return self.llm_engine.get_default_steering()

LLM.preload_steer_vectors 

preload_steer_vectors(
    paths: list[str],
    algorithm: str = "direct",
    params: dict | None = None,
) -> None

Load steering vectors on all workers ahead of use.

The frontend resolves each file once and sends the same immutable payload to every worker store. Different configurations share its content identity. Changing a file requires preloading its new content when steer_require_preload is enabled. For router files, params must match the mode/lambda/topk overrides used by the request.

Source code in vllm-steer/vllm/entrypoints/llm.py 
888
889
890
891
892
893
894
895
896
897
898
899
def preload_steer_vectors(
    self, paths: list[str], algorithm: str = "direct", params: dict | None = None
) -> None:
    """Load steering vectors on all workers ahead of use.

    The frontend resolves each file once and sends the same immutable
    payload to every worker store. Different configurations share its
    content identity. Changing a file requires preloading its new content
    when steer_require_preload is enabled. For router files, params must
    match the mode/lambda/topk overrides used by the request.
    """
    self.llm_engine.preload_steer_vectors(paths, algorithm, params)

For remote clients, the corresponding operations are described under HTTP management endpoints.