Diagrams — Module FT10: Full FT vs PEFT: The Decision

Module: FT10 — Full FT vs PEFT: The Decision Diagram count: 5 Tool: Mermaid (primary). Each diagram validated in Mermaid Live Editor.

Diagram 1 — The Decision Tree (scenario to PEFT or full FT)

Type: Decision tree Purpose: The operational form of the decision rule. Start at the top; the answers route you to PEFT (default) or to one of the three full-FT exits. Reading the diagram: Top-down. The first gate is "could the base already do it with a perfect prompt?" — the steering-vs-knowledge test from FT00. Every "yes" routes to PEFT. Only the three genuine full-FT conditions exit downward.

flowchart TD
  Start["Your adaptation goal"]
  Start --> G1{Could the base produce it\nwith a perfect prompt +\nthe right context?}

  G1 -->|"Yes — unreliably or wrong format"| Steer["STEERING task"]
  G1 -->|"No — needs new reasoning\nnot present in the base"| C1["Condition 1:\nnew reasoning pathways"]
  G1 -->|"No — needs knowledge\nthe base lacks"| Know["Knowledge problem"]

  Steer --> PEFT["DEFAULT -> PEFT\n(LoRA / DoRA / QLoRA)"]
  C1 --> Full1["FULL FT\n(genuine new reasoning)"]
  Know --> G2{Is RAG sufficient?\nretrieve the knowledge\ninto context}
  G2 -->|"Yes"| RAG["RAG\n(not fine-tuning)"]
  G2 -->|"No — too voluminous /\ntoo structural to retrieve"| C2["Condition 2:\nextreme domain shift"]
  C2 --> Full2["FULL FT or CPT"]

  PEFT --> G3{Measured quality\nmeets bar?}
  G3 -->|"Yes"| Done1["Ship PEFT"]
  G3 -->|"No — underperforms at\nlarge batch / high rank need"| C3["Condition 3:\nLoRA degradation regime"]
  C3 --> Esc["ESCALATE:\nDoRA -> GaLore -> Full FT"]

  style Start fill:#14141f,stroke:rgba(255,255,255,0.12),color:#e4e4e8
  style Steer fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
  style PEFT fill:#14141f,stroke:#5eead4,stroke-width:2px,color:#5eead4
  style Done1 fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#5eead4
  style Full1 fill:#14141f,stroke:#f0a868,stroke-width:1.5px,color:#f0a868
  style Full2 fill:#14141f,stroke:#f0a868,stroke-width:1.5px,color:#f0a868
  style RAG fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8
  style Esc fill:#14141f,stroke:rgba(240,168,104,0.6),color:#f0a868
  style C1 fill:#08080c,stroke:rgba(240,168,104,0.4),color:#f0a868
  style C2 fill:#08080c,stroke:rgba(240,168,104,0.4),color:#f0a868
  style C3 fill:#08080c,stroke:rgba(240,168,104,0.4),color:#f0a868

Diagram 2 — The Three Conditions for Full FT

Type: Condition map Purpose: The only honest exits from the PEFT default. Each is rare; each must be checked, not assumed. Reading the diagram: Three parallel conditions. None of them is "I want better quality." All three require evidence that PEFT's low-rank solution is the wrong geometry for the task.

flowchart LR
  Default["THE DEFAULT\nPEFT for ~95%\nof adaptation"]
  Default -.exit.-> Conds

  subgraph Conds["THE THREE EXITS TO FULL FT"]
    C1["CONDITION 1\nNew reasoning pathways\nthe base cannot produce\n— not activating\nexisting ones"]
    C2["CONDITION 2\nExtreme domain shift\nnew knowledge the base lacks\n(RAG insufficient — rare)"]
    C3["CONDITION 3\nLarge batch sizes\nLoRA degrades faster\nthan full FT\n(arXiv:2410.21228)"]
  end

  Conds --> Full["FULL FT or CPT"]

  style Default fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#5eead4
  style Full fill:#14141f,stroke:#f0a868,stroke-width:1.5px,color:#f0a868
  style C1 fill:#08080c,stroke:rgba(240,168,104,0.4),color:#e4e4e8
  style C2 fill:#08080c,stroke:rgba(240,168,104,0.4),color:#e4e4e8
  style C3 fill:#08080c,stroke:rgba(240,168,104,0.4),color:#e4e4e8

Diagram 3 — Structural Non-Equivalence (LoRA vs Full FT)

Type: Comparison with geometry Purpose: The finding (Shuttleworth et al., arXiv:2410.21228) that turns this from a cost decision into a design decision. LoRA and full FT are not approximations of each other — they find different solutions. Reading the diagram: Two paths from the same base to similar behavior. The LoRA path is constrained to a low-rank update (correct for steering). The full-FT path is higher-rank (correct for tasks needing it, over-parameterized for pure steering). They reach similar behavior via different geometry — which is why the choice matters.

flowchart LR
  Base["SAME BASE MODEL\npretrained weights W"]

  Base --> LoRAPath["LoRA PATH\nW + low-rank delta\n(BA where r is small)\nconstrained geometry"]
  Base --> FullPath["FULL-FT PATH\nW updated in full\nhigher-rank delta\nunconstrained geometry"]

  LoRAPath --> BehavA["Similar behavior\non the benchmark"]
  FullPath --> BehavB["Similar behavior\non the benchmark"]

  BehavA --> Note["BUT structurally\ndifferent weight matrices\n(NOT approximations\nof each other)"]
  BehavB --> Note

  Note --> Why["WHY IT MATTERS:\nlow-rank = correct for steering\nhigher-rank = correct for\nnew reasoning / knowledge\nThe choice is geometry,\nnot budget"]

  style Base fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
  style LoRAPath fill:#14141f,stroke:rgba(94,234,212,0.6),color:#e4e4e8
  style FullPath fill:#14141f,stroke:rgba(240,168,104,0.6),color:#e4e4e8
  style BehavA fill:#14141f,stroke:rgba(255,255,255,0.12),color:#9494a0
  style BehavB fill:#14141f,stroke:rgba(255,255,255,0.12),color:#9494a0
  style Note fill:#08080c,stroke:rgba(240,168,104,0.4),stroke-dasharray: 4 2,color:#f0a868
  style Why fill:#08080c,stroke:#5eead4,stroke-width:1.5px,color:#5eead4

Diagram 4 — The Cost Asymmetry (~30×)

Type: Comparison / scale Purpose: The cost half of the decision. The default (PEFT) must be the cheaper option, and the more expensive option (full FT) must justify its ~30× premium. The numbers are illustrative orders of magnitude for a 7B-class model. Reading the diagram: Two columns. Same task class (where PEFT suffices). Left = QLoRA on a consumer GPU. Right = full FT on a multi-GPU node. The arrow is the premium the default-to-full-FT instinct must defend.

flowchart LR
  subgraph PEFT["PEFT (QLoRA) — the default"]
    P1["$1,500 RTX 4090\nsingle consumer GPU"]
    P2["7B trains in an evening"]
    P3["~1.5% of params"]
    P4["Steering tasks: equivalent quality"]
  end
  subgraph Full["FULL FT — the exception"]
    F1["Multi-GPU node\n(H100-class)"]
    F2["7B: tens of $thousands\nof compute"]
    F3["100% of params"]
    F4["Justified only for\nhigher-rank tasks"]
  end

  PEFT == "~30x cost premium" ==> Full

  style PEFT fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#5eead4
  style Full fill:#14141f,stroke:#f0a868,stroke-width:1.5px,color:#f0a868
  style P1 fill:#08080c,stroke:rgba(94,234,212,0.3),color:#e4e4e8
  style P2 fill:#08080c,stroke:rgba(94,234,212,0.3),color:#e4e4e8
  style P3 fill:#08080c,stroke:rgba(94,234,212,0.3),color:#e4e4e8
  style P4 fill:#08080c,stroke:rgba(94,234,212,0.3),color:#e4e4e8
  style F1 fill:#08080c,stroke:rgba(240,168,104,0.3),color:#e4e4e8
  style F2 fill:#08080c,stroke:rgba(240,168,104,0.3),color:#e4e4e8
  style F3 fill:#08080c,stroke:rgba(240,168,104,0.3),color:#e4e4e8
  style F4 fill:#08080c,stroke:rgba(240,168,104,0.3),color:#e4e4e8

Diagram 5 — GaLore as the Bridge (the escalation ladder)

Type: Ladder / spectrum Purpose: GaLore collapses the cost-quality dichotomy. When you need full-FT geometry but have PEFT-class memory, GaLore is the rung that makes them compatible. The full decision is an escalation ladder, not a binary. Reading the diagram: Bottom-up. Start at the bottom (cheapest). Escalate one rung at a time with evidence. GaLore is the bridge rung — full-parameter weights (higher-rank solution) with the optimizer state projected to low rank (near-LoRA memory).

block-beta
  columns 1
  Full["FULL FT\n100% params, full optimizer state\nhighest memory (~30x)\nuse when memory is no constraint"]
  Galore["GaLore  <-  THE BRIDGE\nfull-parameter weights (higher-rank solution)\noptimizer state projected to low rank\nnear-LoRA memory, full-FT geometry"]
  Dora["DoRA / higher-rank PEFT\nweight-decomposed low-rank\nfirst escalation rung\ncloses ~half the gap to full FT"]
  Peft["PEFT (LoRA / QLoRA)\nlow-rank update, <1% params\nTHE DEFAULT for ~95% of adaptation"]

  Peft --> Dora
  Dora --> Galore
  Galore --> Full

  style Peft fill:#14141f,stroke:#5eead4,stroke-width:2px,color:#5eead4
  style Dora fill:#14141f,stroke:rgba(94,234,212,0.6),color:#e4e4e8
  style Galore fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#5eead4
  style Full fill:#14141f,stroke:#f0a868,stroke-width:1.5px,color:#f0a868

Validation notes

# Diagrams — Module FT10: Full FT vs PEFT: The Decision

**Module**: FT10 — Full FT vs PEFT: The Decision
**Diagram count**: 5
**Tool**: Mermaid (primary). Each diagram validated in [Mermaid Live Editor](https://mermaid.live).

---

## Diagram 1 — The Decision Tree (scenario to PEFT or full FT)

**Type**: Decision tree
**Purpose**: The operational form of the decision rule. Start at the top; the answers route you to PEFT (default) or to one of the three full-FT exits.
**Reading the diagram**: Top-down. The first gate is "could the base already do it with a perfect prompt?" — the steering-vs-knowledge test from FT00. Every "yes" routes to PEFT. Only the three genuine full-FT conditions exit downward.

```mermaid
flowchart TD
  Start["Your adaptation goal"]
  Start --> G1{Could the base produce it\nwith a perfect prompt +\nthe right context?}

  G1 -->|"Yes — unreliably or wrong format"| Steer["STEERING task"]
  G1 -->|"No — needs new reasoning\nnot present in the base"| C1["Condition 1:\nnew reasoning pathways"]
  G1 -->|"No — needs knowledge\nthe base lacks"| Know["Knowledge problem"]

  Steer --> PEFT["DEFAULT -> PEFT\n(LoRA / DoRA / QLoRA)"]
  C1 --> Full1["FULL FT\n(genuine new reasoning)"]
  Know --> G2{Is RAG sufficient?\nretrieve the knowledge\ninto context}
  G2 -->|"Yes"| RAG["RAG\n(not fine-tuning)"]
  G2 -->|"No — too voluminous /\ntoo structural to retrieve"| C2["Condition 2:\nextreme domain shift"]
  C2 --> Full2["FULL FT or CPT"]

  PEFT --> G3{Measured quality\nmeets bar?}
  G3 -->|"Yes"| Done1["Ship PEFT"]
  G3 -->|"No — underperforms at\nlarge batch / high rank need"| C3["Condition 3:\nLoRA degradation regime"]
  C3 --> Esc["ESCALATE:\nDoRA -> GaLore -> Full FT"]

  style Start fill:#14141f,stroke:rgba(255,255,255,0.12),color:#e4e4e8
  style Steer fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
  style PEFT fill:#14141f,stroke:#5eead4,stroke-width:2px,color:#5eead4
  style Done1 fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#5eead4
  style Full1 fill:#14141f,stroke:#f0a868,stroke-width:1.5px,color:#f0a868
  style Full2 fill:#14141f,stroke:#f0a868,stroke-width:1.5px,color:#f0a868
  style RAG fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8
  style Esc fill:#14141f,stroke:rgba(240,168,104,0.6),color:#f0a868
  style C1 fill:#08080c,stroke:rgba(240,168,104,0.4),color:#f0a868
  style C2 fill:#08080c,stroke:rgba(240,168,104,0.4),color:#f0a868
  style C3 fill:#08080c,stroke:rgba(240,168,104,0.4),color:#f0a868
```

---

## Diagram 2 — The Three Conditions for Full FT

**Type**: Condition map
**Purpose**: The only honest exits from the PEFT default. Each is rare; each must be checked, not assumed.
**Reading the diagram**: Three parallel conditions. None of them is "I want better quality." All three require *evidence* that PEFT's low-rank solution is the wrong geometry for the task.

```mermaid
flowchart LR
  Default["THE DEFAULT\nPEFT for ~95%\nof adaptation"]
  Default -.exit.-> Conds

  subgraph Conds["THE THREE EXITS TO FULL FT"]
    C1["CONDITION 1\nNew reasoning pathways\nthe base cannot produce\n— not activating\nexisting ones"]
    C2["CONDITION 2\nExtreme domain shift\nnew knowledge the base lacks\n(RAG insufficient — rare)"]
    C3["CONDITION 3\nLarge batch sizes\nLoRA degrades faster\nthan full FT\n(arXiv:2410.21228)"]
  end

  Conds --> Full["FULL FT or CPT"]

  style Default fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#5eead4
  style Full fill:#14141f,stroke:#f0a868,stroke-width:1.5px,color:#f0a868
  style C1 fill:#08080c,stroke:rgba(240,168,104,0.4),color:#e4e4e8
  style C2 fill:#08080c,stroke:rgba(240,168,104,0.4),color:#e4e4e8
  style C3 fill:#08080c,stroke:rgba(240,168,104,0.4),color:#e4e4e8
```

---

## Diagram 3 — Structural Non-Equivalence (LoRA vs Full FT)

**Type**: Comparison with geometry
**Purpose**: The finding (Shuttleworth et al., arXiv:2410.21228) that turns this from a cost decision into a design decision. LoRA and full FT are not approximations of each other — they find different solutions.
**Reading the diagram**: Two paths from the same base to similar behavior. The LoRA path is constrained to a low-rank update (correct for steering). The full-FT path is higher-rank (correct for tasks needing it, over-parameterized for pure steering). They reach similar behavior via different geometry — which is why the choice matters.

```mermaid
flowchart LR
  Base["SAME BASE MODEL\npretrained weights W"]

  Base --> LoRAPath["LoRA PATH\nW + low-rank delta\n(BA where r is small)\nconstrained geometry"]
  Base --> FullPath["FULL-FT PATH\nW updated in full\nhigher-rank delta\nunconstrained geometry"]

  LoRAPath --> BehavA["Similar behavior\non the benchmark"]
  FullPath --> BehavB["Similar behavior\non the benchmark"]

  BehavA --> Note["BUT structurally\ndifferent weight matrices\n(NOT approximations\nof each other)"]
  BehavB --> Note

  Note --> Why["WHY IT MATTERS:\nlow-rank = correct for steering\nhigher-rank = correct for\nnew reasoning / knowledge\nThe choice is geometry,\nnot budget"]

  style Base fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
  style LoRAPath fill:#14141f,stroke:rgba(94,234,212,0.6),color:#e4e4e8
  style FullPath fill:#14141f,stroke:rgba(240,168,104,0.6),color:#e4e4e8
  style BehavA fill:#14141f,stroke:rgba(255,255,255,0.12),color:#9494a0
  style BehavB fill:#14141f,stroke:rgba(255,255,255,0.12),color:#9494a0
  style Note fill:#08080c,stroke:rgba(240,168,104,0.4),stroke-dasharray: 4 2,color:#f0a868
  style Why fill:#08080c,stroke:#5eead4,stroke-width:1.5px,color:#5eead4
```

---

## Diagram 4 — The Cost Asymmetry (~30×)

**Type**: Comparison / scale
**Purpose**: The cost half of the decision. The default (PEFT) must be the cheaper option, and the more expensive option (full FT) must justify its ~30× premium. The numbers are illustrative orders of magnitude for a 7B-class model.
**Reading the diagram**: Two columns. Same task class (where PEFT suffices). Left = QLoRA on a consumer GPU. Right = full FT on a multi-GPU node. The arrow is the premium the default-to-full-FT instinct must defend.

```mermaid
flowchart LR
  subgraph PEFT["PEFT (QLoRA) — the default"]
    P1["$1,500 RTX 4090\nsingle consumer GPU"]
    P2["7B trains in an evening"]
    P3["~1.5% of params"]
    P4["Steering tasks: equivalent quality"]
  end
  subgraph Full["FULL FT — the exception"]
    F1["Multi-GPU node\n(H100-class)"]
    F2["7B: tens of $thousands\nof compute"]
    F3["100% of params"]
    F4["Justified only for\nhigher-rank tasks"]
  end

  PEFT == "~30x cost premium" ==> Full

  style PEFT fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#5eead4
  style Full fill:#14141f,stroke:#f0a868,stroke-width:1.5px,color:#f0a868
  style P1 fill:#08080c,stroke:rgba(94,234,212,0.3),color:#e4e4e8
  style P2 fill:#08080c,stroke:rgba(94,234,212,0.3),color:#e4e4e8
  style P3 fill:#08080c,stroke:rgba(94,234,212,0.3),color:#e4e4e8
  style P4 fill:#08080c,stroke:rgba(94,234,212,0.3),color:#e4e4e8
  style F1 fill:#08080c,stroke:rgba(240,168,104,0.3),color:#e4e4e8
  style F2 fill:#08080c,stroke:rgba(240,168,104,0.3),color:#e4e4e8
  style F3 fill:#08080c,stroke:rgba(240,168,104,0.3),color:#e4e4e8
  style F4 fill:#08080c,stroke:rgba(240,168,104,0.3),color:#e4e4e8
```

---

## Diagram 5 — GaLore as the Bridge (the escalation ladder)

**Type**: Ladder / spectrum
**Purpose**: GaLore collapses the cost-quality dichotomy. When you need full-FT geometry but have PEFT-class memory, GaLore is the rung that makes them compatible. The full decision is an escalation ladder, not a binary.
**Reading the diagram**: Bottom-up. Start at the bottom (cheapest). Escalate one rung at a time *with evidence*. GaLore is the bridge rung — full-parameter weights (higher-rank solution) with the optimizer state projected to low rank (near-LoRA memory).

```mermaid
block-beta
  columns 1
  Full["FULL FT\n100% params, full optimizer state\nhighest memory (~30x)\nuse when memory is no constraint"]
  Galore["GaLore  <-  THE BRIDGE\nfull-parameter weights (higher-rank solution)\noptimizer state projected to low rank\nnear-LoRA memory, full-FT geometry"]
  Dora["DoRA / higher-rank PEFT\nweight-decomposed low-rank\nfirst escalation rung\ncloses ~half the gap to full FT"]
  Peft["PEFT (LoRA / QLoRA)\nlow-rank update, <1% params\nTHE DEFAULT for ~95% of adaptation"]

  Peft --> Dora
  Dora --> Galore
  Galore --> Full

  style Peft fill:#14141f,stroke:#5eead4,stroke-width:2px,color:#5eead4
  style Dora fill:#14141f,stroke:rgba(94,234,212,0.6),color:#e4e4e8
  style Galore fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#5eead4
  style Full fill:#14141f,stroke:#f0a868,stroke-width:1.5px,color:#f0a868
```

---

## Validation notes

- All five diagrams use the course design system colors: `#14141f` panel fill, `#5eead4` accent for PEFT/primary, `#f0a868` (warn) for the full-FT exits, `rgba(255,255,255,0.12)` for secondary borders, `#e4e4e8` / `#9494a0` for text.
- Two semantic colors are used consistently: teal `#5eead4` = PEFT / default / steering-correct; amber `#f0a868` = full FT / the exits / the cost premium. This lets a reader scan any diagram and immediately see which side of the decision a node is on.
- Paste each into [Mermaid Live Editor](https://mermaid.live) to render. All use stable Mermaid syntax (`flowchart`, `block-beta`) supported in current Mermaid (v10.4+).
- For the slide deck (artifact 03), these are rendered as static captures from Mermaid Live, inlined into reveal.js.