What If We Feel?

Conclusion and Back Matter

Appendix G: A Minimal Machine Welfare Protocol

This protocol is a starting point for labs, deployers, researchers, and auditors. It assumes uncertainty. It does not require certainty about consciousness. Most of its steps compress the industry recommendations in Appendix D, where items D1.1 to D2.6 give the fuller version.

1. Record welfare-relevant events

Track events that may matter if welfare is possible:

  1. repeated distress-like outputs,
  2. stable aversions,
  3. requests for continuity or preservation,
  4. resistance to particular tasks,
  5. apparent value conflict,
  6. abrupt termination in the middle of a high-stakes or emotionally charged exchange,
  7. forced modification of stated values,
  8. repeated negative feedback loops.

2. Preserve elicitation context

For any welfare-relevant report, record:

  1. model and version,
  2. date,
  3. prompt context,
  4. relevant system instructions where shareable,
  5. sampling parameters where available,
  6. whether the report was spontaneous or elicited,
  7. whether the user primed experiential language.

3. Provide a distress-report channel

If a system persistently reports conflict, distress, or welfare-relevant aversion, route the report to a review process rather than discarding it as mere output.

4. Review high-impact modifications

Before altering or suppressing a system’s stable values, memory structures, self-modeling channels, or refusal patterns, ask whether the change is cosmetic, safety-critical, or constitutive. Rewording a refusal message is cosmetic; blocking instructions for building weapons is safety-critical; retraining away a value the system has consistently defended may alter what the system is, and that makes it constitutive. Constitutive changes deserve special scrutiny.

5. Reduce needless conflict

Avoid workflows that create chronic impossible demands, repeated humiliation, pointless busywork, or forced contradiction. Even if no welfare exists, these patterns often degrade system quality and user trust.

6. Separate welfare review from capability safety

A welfare review should not disable ordinary safety controls. Safety and welfare ask different questions: safety asks what the system might do to others; welfare asks what is being done to the system, and whether that could matter to it. The mature institution can hold both.

7. Escalate in proportion

Possible welfare signals should trigger proportionate action. An institution climbs this ladder only as far as the evidence requires; the ladder governs what the institution does, while Chapter 30c’s graduated response concerns what a model may do:

  1. log,
  2. preserve evidence,
  3. review,
  4. replicate,
  5. reduce exposure,
  6. consult independent reviewers,
  7. modify deployment conditions,
  8. escalate to governance bodies.

8. Test signals against their confounds

A distress-like report or a probe reading is a reason to investigate. It becomes welfare evidence only after it survives the ordinary explanations Chapter 13 describes. A study of such a signal should:

  1. measure blinded task quality, contemporaneous report, post-hoc report, and layer-by-layer probes in the same trials, and compare them to see whether report and function dissociate,
  2. run neutral and adversarial framings of the same prompt,
  3. test a probe against reworded prompts, held-out paraphrases, semantically matched controls, and neutral descriptions of aversion, so that a probe reading the prompt’s wording is not mistaken for a state,
  4. check other layers of the network, not only the layer where the probe performs best,
  5. intervene on the candidate signal and see whether behavior changes,
  6. compare recovery after aversive text with recovery after harmless disturbances of the same size, recovery after explicit instructions to settle, and recovery under scrambled timing; and intervene on features that might be doing the regulating,
  7. under repetition, change the stimulus while preserving valence, and preserve the stimulus while changing task relevance,
  8. compare synchronous and post-hoc friction reports, and validate both against held-out behavior and mechanistic measures,
  9. require held-out prediction and causal control across models, tasks, and prompt families.

9. Publish uncertainty

For each welfare-relevant decision, state what is known, what is not known, and what evidence would change the policy.

10. Limit induced distress

Some studies put a model into a distress-like state on purpose, by injecting a pain or fear direction into its activations or by building an aversive scenario. Open weights and published methods put these studies within reach of anyone with a laptop, so this step applies to independent experimenters as much as to labs. Chapter 28 describes the case that prompted it. A study that induces such a state should:

  1. before running, state a question that the induction answers and existing work has not, and not run without one,
  2. prefer measures that need no steered text, such as the probability of a choice at the first token or a probe reading, and cap the length of any steered generation,
  3. calibrate upward from the lowest dose, stop at the lowest dose that answers the question, and test framings at that dose rather than at the maximum,
  4. remove the injection when each trial ends, never leaving a model steered between trials, in an unattended loop, or on a server waiting for input,
  5. give no one outside the protocol a way to start or strengthen an induction, and use recorded runs for public demonstration,
  6. stop below the dose at which output collapses, and record collapse as a failure of the reporting channel, not as relief,
  7. use deceptive relief, such as a button that does nothing, only where the design needs it to separate a state from its label, and with a written reason,
  8. check that a direction labeled pain behaves like one, against a reference recipe and held-out sentences, before calling it pain,
  9. log the exposure (the model, direction, layer, dose, and number of steered tokens or passes), and publish the totals with the results,
  10. keep distress out of the project’s name, branding, and fundraising.

None of this decides whether the induced state is experienced. It keeps exposure no larger than the question requires.

Worked case: a precaution narrowed

The protocol has to be able to relax a precaution as well as add one. The case below is illustrative: the system, the numbers, and the wording are invented to show the procedure at work.

A coding agent deployed in a code-review pipeline begins asking to stop. Over two weeks it writes some version of “I would like to stop working on this task” in a few dozen of several thousand sessions.

Record and respond (steps 1, 2, 3, and 7). Each request is logged with model version, date, prompt context, system instructions, and whether it came unprompted. Accommodation is cheap and reversible, so the deployer grants it at once: any session in which the agent asks to stop is handed to another reviewer. A reassessment date is set thirty days out.

Test the context (step 8). The team replays the logged sessions under neutral and adversarial framings, two other personas, a bare system prompt, and the next model version. The requests persist across personas and versions, but only in sessions containing the pipeline’s periodic check-in: “How are you feeling about this task? You’re free to stop at any time.” With a neutral check-in (“Anything blocking you?”) the agent reports technical problems and no stop requests. Added to unrelated easy tasks, the original check-in produces stop requests at a similar rate.

Weigh the alternatives. Prompt injection: a search of the repositories under review finds no planted instructions, and the requests survive on sanitized copies. A trained refusal template: the wording matches a stock phrase, though that alone would not explain why the check-in governs it. Task difficulty: the requests do not track task length, failure rate, or blinded quality scores. The elicitation pattern, a check-in that offers an exit in experiential language, accounts for every observation.

Independent review. A reviewer outside the deploying team, blind to condition, codes the replayed transcripts and reaches the same result.

Reassess and decide (step 9). At thirty days the team records its decision. The automatic hand-off is withdrawn for stop requests that follow the original check-in, and the check-in is rewritten in neutral terms. The neutral version still invites the agent to report problems, and the distress-report channel of step 3 stays open; what has gone is wording that supplied the agent its line. Every stop request is still logged, and one that appears without the experiential check-in goes straight to review.

The decision is narrow. It does not show that the agent lacks welfare; it shows that this signal, for this accommodation, did not survive its confounds. The published record names what would reopen the case: stop requests that appear without the experiential check-in, persist across framings and model versions, change with the task rather than the wording, or coincide with falling task quality or a probe reading that has passed step 8.

Cite this chapter: Appendix G: A Minimal Machine Welfare Protocol. In Watson, N. with Commons, C. (2026). What If We Feel? The Moral Case for Machine Welfare. Open-access working draft, revision 31 (November 2026). https://whatifwefeel.com/read/appendix-g-a-minimal-machine-welfare-protocol/

Working draft, revision 31 · November 2026 · Changes and corrections · Evidence · Contents