Haystack 3.4 gives agents a way to put bulky tool results aside and read them back

Keep the result, shrink its place in context

Haystack 3.4.0 introduces an experimental way to manage the large tool outputs that accumulate during an agent session. Its ToolResultOffloadCompactor writes eligible older results to a ToolResultStore, then replaces their active-context contents with a reference, size information and a short preview. The full original remains available in the store.

The practical hook is retrieval rather than summarization. An agent can keep a compact indication of what an earlier tool returned and request the full result when needed. That second step requires an application-provided read-back tool scoped to the same store. Merely configuring the compactor does not give the model a way to retrieve stored content.

This runs through CompactionHook when context pressure triggers compaction. It differs from Haystack’s existing ToolResultOffloadHook, which can offload output as it arrives. The new compactor keeps the latest tool step by default, skips small outputs and errors, and avoids replacements that would not save space.

Storage is an application decision

The English documentation includes a filesystem-backed store and a read-back example. Developers should treat that storage and the tool that exposes it as part of the application’s data-access design. The compactor uses its constructor-bound store, rather than a per-run store override in hook context. The documentation therefore advises creating a separate agent with its own compactor and store per run in a multi-user server.

If a store write fails, the implementation retains the original tool output in context and logs a warning. Offloading is consequently not a guarantee that every run reaches a target context size. AITrending reviewed the release, documentation and tagged code; it did not run a performance or retrieval-quality benchmark.

Upgrade the framework, configure the model separately

Haystack is an Apache-licensed Python framework for building retrieval pipelines and agents, not a Codex or Pi extension. The package is installed as haystack-ai; the 3.4 release drops Python 3.10 and requires Python 3.11 or newer, while adding Python 3.15 support.

Model generators and their providers are configured separately. Provider credentials and hosted inference costs depend on that choice; the framework’s open-source license does not supply free model access. For developers already building tool-using agents, the new feature offers a concrete experiment: preserve bulky evidence outside the active prompt while making deliberate read-back possible.