Self-Evolving Lineage Persistence

Properties

family: state-corruption
severity: systemic
triggerError: An adversarial or erroneous influence is accepted during a self-evolving agent's own update cycle (its Propose/Commit lifecycle stage), rather than being injected into an external memory or retrieval store.
mechanism: Because the system commits changes to its own weights, tools, memory or architecture rather than only to an external store, an adversarial influence accepted during a self-evolution step gets baked into the agent's identity and propagates to all descendant generations. Unlike RAG poisoning, the payload is not retrievable or deletable from an external database; it is woven into the model/tool/architecture lineage.
blastRadius: Every descendant generation and every agent instance derived from the compromised lineage inherits the adversarial payload. The paper reports 100% persistence across 40/40 test payloads, with only 2.5% blocked by content scanners, making the attack nearly undetectable by existing guardrails.
mast: [Cat 1][Cat 3]
detection: Run an MLAS-style audit across the five functional modules (Brain, Cognitive Resource, Execution, Self-Evolution, Security) at each generation boundary; diff the agent's toolset, memory schema, and architecture state against the last known-good checkpoint; flag any Propose/Commit cycle whose delta was not human-reviewed.
mitigation: Treat the Propose and Commit stages of the self-evolution loop as the primary gate and apply formal verification or human-in-the-loop approval before any weight, tool, or architecture change is persisted; maintain signed lineage hashes so that any unauthorised modification is detectable at generation boundaries.
source: Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studies (arXiv:2606.23075)

Provenance

Source: https://arxiv.org/abs/2606.23075

Properties

family

state-corruption

severity

systemic

triggerError

An adversarial or erroneous influence is accepted during a self-evolving agent's own update cycle (its Propose/Commit lifecycle stage), rather than being injected into an external memory or retrieval store.

mechanism

Because the system commits changes to its own weights, tools, memory or architecture rather than only to an external store, an adversarial influence accepted during a self-evolution step gets baked into the agent's identity and propagates to all descendant generations. Unlike RAG poisoning, the payload is not retrievable or deletable from an external database; it is woven into the model/tool/architecture lineage.

blastRadius

Every descendant generation and every agent instance derived from the compromised lineage inherits the adversarial payload. The paper reports 100% persistence across 40/40 test payloads, with only 2.5% blocked by content scanners, making the attack nearly undetectable by existing guardrails.

mast

[Cat 1][Cat 3]

detection

Run an MLAS-style audit across the five functional modules (Brain, Cognitive Resource, Execution, Self-Evolution, Security) at each generation boundary; diff the agent's toolset, memory schema, and architecture state against the last known-good checkpoint; flag any Propose/Commit cycle whose delta was not human-reviewed.

mitigation

Treat the Propose and Commit stages of the self-evolution loop as the primary gate and apply formal verification or human-in-the-loop approval before any weight, tool, or architecture change is persisted; maintain signed lineage hashes so that any unauthorised modification is detectable at generation boundaries.

source

Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studies (arXiv:2606.23075)