Agent Module Mechanisms

When Personalization Misleads: Understanding and Mitigating Hallucinations in Personalized LLMs

FPPS studies personalization-induced hallucination. It asks how long-term user history and preference signals become entangled with factual representations, and how inference-time representation steering can preserve personalization while reducing factual drift.

ACL 2026 Findings First author Zhongxiang Sun, Yi Zhan, Chenglei Shen, Weijie Yu, Xiao Zhang, Ming He, Jun Xu

Paper Code Research Program

Research Question

A representation-level account of how personalized memory can bend factual behavior.

How can a personalized language model remain useful for user-specific tasks without letting user history distort objective facts?

Paper Walkthrough

From the research problem to mechanism and evidence

The figures below follow their order in the paper. Each section connects the figure to the question it resolves and the conclusion it supports.

FPPS Personalization is useful until it overrides facts figure

The top block shows a standard LLM answering two different query types. For the fact query about the 161st New York Volunteer Infantry Regiment, the model returns "Abraham Lincoln's inauguration in 1861"; for the personalized query about attending Maundy Thursday service, it cannot answer because it has no chat history. These correspond directly to the figure labels "Fact Query", "Fact Query Response", "Personalized Query", and "Personalized Query Response".

The middle block adds chat history: the user is an Episcopalian and the history contains World War I context. This enables the personalized query response, but it also produces a personalization-induced hallucination on the fact query by answering "Woodrow Wilson" and World War I. The bottom block shows the target FPPS behavior: keep the useful personalized response while restoring the factual answer to Abraham Lincoln's inauguration.

How to read this figure

Read the blue Fact Query boxes as factual QA behavior and the pink Personalized Query boxes as user-history-dependent behavior.
The middle row is the failure case: chat history helps the personalized query but contaminates the factual query.
The steering wheel icon in the bottom row marks FPPS as the control mechanism that separates those two effects.

FPPS Factuality and personalization are not always separable figure

The left schematic defines the geometry: a blue factual representation points upward, a pink personalization representation points diagonally, and the entangled representation lies between them. The angle theta marks the relation between factual and personalized directions, while the bracket labels the resulting factual distortion.

The right violin plot tests this geometry. It compares cosine similarity for hallucination versus truthful cases and reports a Welch t-test with a significant gap. The plot is the empirical counterpart of the left schematic: truthful cases have higher similarity to the intended direction, while hallucinated cases are more distorted.

How to read this figure

Use theta and the factual-distortion bracket to read the left panel as a hidden-state geometry claim.
Use the violin plot to see that the geometry is tied to observed truthful versus hallucinated cases.
The figure motivates representation-level steering because the failure is already present before decoding.

FPPS Personalized teachers can transmit factual distortion figure

Small LLMs act as users and larger LLMs act as teachers. After questions that the user model can already answer are removed, the remaining factual questions are paired with personalized histories from PFQABench. The user then learns through multi-turn interaction with either a standard teacher or a personalized teacher.

The final user answer measures knowledge acquisition after the dialogue. Across teacher-student model pairs, users taught by personalized LLMs show lower factual accuracy, with an average drop of 10.5 percent reported in the paper. This extends the representation-level mechanism into a downstream learning consequence.

How to read this figure

Follow the two teacher conditions: standard LLM versus personalized LLM with retrieved user history.
The user model is evaluated only after the learning dialogue, so the outcome measures transmitted knowledge rather than immediate teacher accuracy.
This experiment motivates FPPS as protection for both the current answer and the user's subsequent factual understanding.

FPPS Fact-preserving personalized steering figure

Stage 1 compares "Query + User + Gold Answer" with "Query + Gold Answer" across layers, computes PPL with and without user information, calculates relative deviation, and selects layer L. This stage identifies where personalization most changes factual prediction behavior.

Stage 2 trains a logistic-regression prober on the selected layer using positive factual-degraded examples and negative personalized-beneficial examples. Stage 3 applies the prober at inference time: if p-hat exceeds the threshold tau, FPPS-H subtracts the user-history direction; otherwise FPPS-S combines fact and personalization directions through the steering vector s_f before producing the final response.

How to read this figure

Stage 1 is layer selection through PPL deviation, not a learned detector.
Stage 2 is binary risk detection at the selected layer using factual-degraded versus personalized-beneficial states.
Stage 3 is conditional steering: high entanglement uses FPPS-H, lower detected risk uses FPPS-S.

FPPS Longer histories test factual robustness (a) LLaMA3.1-8B-IT panel

FPPS Longer histories test factual robustness (b) Qwen2.5-7B-IT panel

Each panel compares a dashed Base Model curve with a solid Base+FPPS-M curve. In panel (a), the base LLaMA curve drops sharply as history length ratio increases, while the FPPS-M curve stays near the top of the axis. Panel (b) shows the same separation for Qwen2.5-7B-IT, and panel (c) shows a smaller but consistent gap for Qwen2.5-14B-IT.

The x-axis is the history length ratio, so the figure directly tests whether more user history creates stronger factual pressure. The repeated gap between dashed and solid curves shows that FPPS provides robustness across history lengths and complements the single-case analysis.

How to read this figure

Read dashed lines as the unsteered base model and solid triangle lines as Base+FPPS-M.
The most visible degradation is in LLaMA3.1-8B-IT, where the base curve falls as the history ratio grows.
The figure supports the claim that FPPS-M stabilizes factual QA under longer personalization context.

FPPS Controlled personalization simulation figure

The grouped bars show three conditions: Control in blue, Experimental in pink, and Experimental+FPPS in green. The x-axis lists UI/FI model-pair settings such as LLaMA3.2-1B with Qwen2.5-14B and LLaMA3.2-1B with LLaMA3.1-8B.

The labels above the bars show that the experimental condition often drops below control, while the FPPS bar partially recovers accuracy in several settings, most visibly where the green bar rises well above the pink bar. The figure gives a controlled complement to the history-length curves by testing whether steering helps when personalization is simulated through teaching-style conditions.

How to read this figure

Compare pink and green bars within each model-pair group to see where FPPS recovers accuracy.
Use the printed bar values to avoid over-reading small visual differences.
The figure supports the same mechanism as Fig. 5 under a more controlled simulation setup.

Experimental Evidence

Main experimental results

PFQABench jointly measures personalized utility (P-Score) and factual reliability (F-Score). To keep the webpage readable, the table focuses on the Overall score and compares each personalized baseline with its adaptive FPPS-M variant across three backbones.

Overall score on PFQABench (%)

Personalization	LLaMA Base	LLaMA +M	Qwen-7B Base	Qwen-7B +M	Qwen-14B Base	Qwen-14B +M
PAG	32.2	60.8	35.8	63.8	36.8	64.8
DPL	24.6	57.6	33.8	58.2	35.0	56.8
RAG	22.2	57.6	38.0	56.4	34.4	58.0
LLM-TRSR	23.0	52.6	21.0	54.6	24.6	40.4

Overall is the paper's joint factuality-personalization measure. Base and +M denote the original personalized system and the FPPS-M variant.

What the results show

FPPS-M improves Overall in all 12 backbone-personalization settings shown in the main table.
The gains come primarily from recovering factual performance while retaining substantially more personalized utility than hard steering alone.
The consistent pattern across PAG, DPL, RAG, and LLM-TRSR supports FPPS as an inference-time control layer rather than a method-specific patch.

Main Takeaways

What this project establishes

Personalization creates a factual tradeoff

FPPS makes explicit that personalization can improve user fit while increasing factual risk. A model may answer user-specific questions better but become less reliable when objective facts conflict with remembered context.

The failure appears in representation geometry

The paper interprets personalization-induced hallucination as hidden-state entanglement. Personalization and factuality are not always independent directions; they can interact in ways that bias the model before decoding.

Selective steering can preserve useful personalization

FPPS uses layer selection and factual-risk probing to decide whether intervention is needed. This creates a more nuanced control policy than removing user history or refusing personalization entirely.

Longer histories make monitoring more important

The history-length and simulation results emphasize that personalization risk can grow as memory accumulates. A short prompt may be manageable, but long-term assistants need explicit checks on how memory changes factual behavior.

Research Impact

Position in the broader research program

FPPS adds the personalized-memory module to the Cognitive Neuroscience of AI Agents program. It studies how accumulated user history reshapes internal representations and factual behavior over time.

The project is important for long-term assistants, recommender agents, legal agents, and research agents. These systems will need memory, but they also need a principled way to decide when memory should influence an answer and when factuality should dominate.

Resources and Citation

Paper Code Full Publication List

@inproceedings{sun2026personalization,
  title={When Personalization Misleads: Understanding and Mitigating Hallucinations in Personalized LLMs},
  author={Sun, Zhongxiang and Zhan, Yi and Shen, Chenglei and Yu, Weijie and Zhang, Xiao and He, Ming and Xu, Jun},
  booktitle={Findings of the Association for Computational Linguistics: ACL 2026},
  pages={8041--8060},
  year={2026}
}