I found hermes to be really lightweight, though I am on a relatively older version and built a custom plugin to lazily load mcps (that's probably in hermes proper by now). Compared to kilo it seems to consume far fewer tokens.
I keep my max context really small for personal assistant agents; they don't need it. Especially since compaction keeps anything important around anyway. I use 60k with Pi.
TLDR: It keeps ~20k tokens of recent conversations, then hands the rest of the conversation to another model with a special system & user prompt. This then fills out a template with relevant information.
Compaction has been a pretty painful part of local llm usage. Scrapping the current context and parsing almosy 128k of context then generating something like 5-10k tokens - that can take quite a while when you’re working with 10t/s-45t/s (depending on the model).
I pretty much just start a new session whenever i fill the context.
In my opinion this is one of the areas where GPUs provide a qualitatively different experience than unified memory boxes.
For an EPYC with a 5090 (no layers on CPU) vs an M3 max 128GB, qwen 3.6 27B at 128k context / 7k generation:
Cold: prefill + decode Hot (KV cached)
5090 40s + 2-3m = 3-4 min 2-3 min
M3 Max 128GB 14m + 8-10m = 22-25 min 8-10 min
This is for dense qwen (which I wouldn't run day to day on the mac) - in reality the mac is quite usable with MoEs but you definitely notice a difference.
sure, it's the price you pay, but you have to considered that context poisoning is basically a statistical certainty approaching 1 regardless of model size. As you grow the context, the likelihood of vague details getting conflated increases, so if you're _not_ cutting down the context you're increasing the probabilities of just basic random jitter in your logic and code.
The pruning does evict cache but not from the start, and only up till the last time you pruned it, as it's just adding messages on top.
but yes, it takes some additional tests and docs to keep it from just becoming hollowed out on tasks; I'd say about 10-20% of the time is just horribly loses what it's doing.
The advantage of running local stack is that you can do the compaction at the time of inference, i.e. some tool call runs out of context, you can just pause inference, purge/replace old tool calls with their summaries or just logs by operating directly over tokens on a GPU, rebuilding KV cache (one time prefill hit) and resuming the inference, easily being able to e.g. read 1000 markdowns, each 50k long, in a single LLM call. That's not possible with current agentic harnesses using LLM calls.
Compaction is painful if you run just one local LLM, the best way to avoid it is to keep context as small as possible.
One trick I find useful is to have one model with two KV caches running and while first cache has produced tokens, second cache immediately summarizes them during input tokens are being generated (tools time), then harness switches to the second KV cache which takes newly produced input tokens while KV in first cache is getting replaced with compacted summary tokens. This is a kind of ping pong, so we trade more space for less time. Still experimenting but it looks it works, and nice bonus it improves GPU utilization. Btw I have my own harness and model serving code, but it can be easily implemented in any other harness and model server.
I tend to ask a line of questions to the LLM as first step to a point the context is enough for me. Then i ask for the work i need but then get back to previous context using /tree.
It's like asking the questions needed to get the answer you need from the LLM, can be either an answer for a plan/todo or a task to code/change something.
I expect Pi is mostly used with OpenAI plans, and OpenAI has a dedicated compaction endpoint you should probably be using with their models instead of a compaction prompt.
There is a compaction routing plugin, as well as a general OAI compatibility plugin that simulates some of their harness features (ie. code_mode/batching tools) that also includes their compaction.
That said, from reading the pi subreddit I don't see people referencing these much.
I don't like any of current solutions when it comes to compaction. I'd love to have a way to say what exactly should be summarized, because most of the time I just need to compact some noisy MCP tool calls, test runs and things like that. Just let me pick what should be summarized and keep the rest as is.
Just make an extension (or ask Pi to write an extension for itself) that intercepts compaction and leaves only what you want, or rewrites it in any other way. Should be just a few lines.
I mean, not to be flippant but can't you just prompt the agent to write a file as you're getting closer to the compaction limit? I tend to just go to roughly 50-70% context utilization and then tell the agent to summarize the conversation and save it to a file, manually /clear, then say let's continue that last conversation. You can inspect the summary first and make any changes.
The way I've done it has been when there's a longer task, I give it a markdown document that has a plan with numbered steps, and then for each step start a fresh context window and tell it to update that doc as it goes with any decisions taken or deviations from the plan, or other context needed for future steps. If it gets close to the end of the context I tell it to summarize the current state and anything a fresh context would need to know.
Say, what happens when chain of summaries grows so long, that it still overflows context window. Is summarization runned over the summaries in the context window?
What more depth is there to go to? Compaction is a single LLM call (practically) which can have some deterministic diffing/extraction baked in, or multiple LLM calls (generally wasteful). There's only 1 summary in the context window at one time. Every prompt goes [CONVERSATION_HISTORY] + input -> model turn. As soon as total context exceeds that it compacts, so there's no summarisation overflow (you can enable an agent to access past summarisations from past compactions, but the snake starts to eat it's own tail).
Instead of compaction, has anyone seen a successful implementation of pruning? That is, the agent looks at the conversation history and removes any low-value messages.
For example, sometimes context will be taken up by a side tangent, tool call outputs, or low-value codebase exploration.
Much of the time, I prefer to preserve the history of my conversation instead of summarizing it. I find summarized conversations lead to more frustrating future chats because the LLM misses intent and or context. (Or, the presence of paragraphs and paragraphs of LLM output makes the next token predictor dumber? Unsure.)
I think there are a lot of strategies that will open up when costs come down but right now you take a hit on cache rate and thus costs every time you do anything other than wait until the last minute to compact
You can do things like throwing away or summarizing information from the most recent calls so you fall back to a recent checkpoint that is still cached
I’m still playing with it, but it essentially has the model define a region, and mark it with begin_task, and end_task. End_task also requires a fairly hefty summery of the entire region.
It then pulls that entire region and replaces it with the summary. If the model wants it can actually look into the output that was pruned from the conversation, but in practice the summaries have been good enough it hasn’t needed to.
I’ve been using it with 5.6 sol, and even really long sessions have like 6% context used at the end.
I'm very interested in this too. I feel like when my Claude session compacts I immediately have to re-alert it to critical aspects of the task, but it should be possible even to have a secondary low-skill agent crawl over the whole context window semi-continuously and nominate pieces for removal or summarization. Or at a higher level, have something able to decompose, like hey I realised we're actually working on three distinct aspects of this problem, let's split the context into those three and I'll just retain a high level summary of what's in the other ones, enough to know if I should pass off control to one of the others for a particular ask.
I feel like this is kind of subagents, but it doesn't have the clear task -> work -> return result -> die model, rather it's more closely aligned to how I myself multitask, maintaining several long-lived threads of execution, and having to intentionally "forget" about the other ones when it's time to lock in and do real work on the most important one.
I have run in to the same thing. Most recently I told my pi-agent to fork a git repo since our work had morphed into two different tasks that shouldn't be packed into the same context and workflow. Git fork is what I will use as a tool for this division in the future.
yeah ive read so many takes on how people survive compaction amnesia ive wired up my own "protocols" that are okay, have used things like beads, am okay burning a buttload of context right at the beginning of a post-compaction just to get up to speed and oriented.. but i still end up with a decent amount of sprawl and lately the writing style of claude has just gotten so dense and nonsensical that my eyes are starting to glaze over.
this seems to be the fate with all the big grand 'god' projects people are doing with AI. super epic omega knowledge bases, super epic omega personal platforms, etc. I don't think enough people admit that whatever they're working on has evolved into something they don't fully understand.
I built a harness for my own use that allows the agent to fork its own history. So for example it can “compact from” a specific item, replacing a branch with a summary; or do an “excursion”, basically a temporary branch, which is like a subagent but inherits context.
Sounds cool and it does make sensible decisions optically but I haven’t been able to prove that it is meaningfully better than normal compaction. Building harnesses that do interesting things is a lot easier than building more effective harnesses, I guess.
OMP has this, it's called "/shake" where bloat from tool calls and such is turned into an artifact ref instead of remaining in the context.
There are other pruning or compaction strategies you can configure too.
You can also use "/btw" to have side tangent conversations with the current context but with the benefit that it will not bloat your current context after the tangent is done.
I wish other harnesses were prioritising this sort of flexibility with context management, but they're likely more incentivised on optimising their cache rates
> You can also use "/btw" to have side tangent conversations with the current context
Ah, something I needed, but did not know I needed. Comes up ALL the time. I want to know something that is well off the task, but it's the best time to ask due to the specifically loaded context.
Potentially: remove thinking blocks, and keep the rest. At least this would ensure that the entire context of the conversation is still there, and anything said isn't lost.
Having a second model also iterate the resulting messages and remove low-value tool calls could also be interesting. Especially failed calls which add no value.
I do this in my own harness, where a context only exports certain messages when you step out of it — eg, my request for an essay and its final output, while dropping everything from the first draft through various intermediary edits.
That naturally trims a lot of context while removing mistakes from the context to prevent poisoning (ie, every draft but the final negatively contributes in some way — that’s why we edited them).
OMP changed the default compaction to images! Kinda nuts to read about. Saves the generation cost of the traditional compaction step and writes the context as tiny text to an image, if I was following correctly.
This is one of the most ridiculously awesome findings this year, that changed my mental model of how foundation models work.
It sounds crazy, but it's way cheaper and faster for multimodal models to read text on images than the equivalent text represented as tokens. Text tokens take a lot more bytes to represent internally than you would think. Give it images with rasterized text, and there is no OCR-like intermediate step that turns images into text tokens. It just goes straight into the multimodal context where it's all just linea algebra underneath. The cost is more lossiness than if you gave it the same tokens as text. Which is why you don't just render text prompts and files to read as images.
But compaction is inherently lossy, a summary 1-10% of the original length is going to lose things, by definition. So you keep the things you really need to get exactly right in text token summary, but put a lot more of what you would have just discarded into a PNG. Keeping the past context as text in an image is way less lossy than keeping a text summary of that context.
They peeked inside the layers and found the same neurons were usually activating in the middle layers when fed either text tokens or images of that same text. It just takes the model more time and kv cache to do it via text tokens.
I think the way prompt caching works really discourages more creative compaction techniques. Like perhaps some kind of heuristic progressive compaction that replaces tool results and thinking traces after use with pointers could potentially keep the model smart for much longer, but that'd mean breaking cache every turn, and possibly even within a turn, seriously driving up cost.
I don't like that it throws away the whole KV cache when compacting. It costs a cache miss of the whole conversation length, and that's a waste of time and money.
LLMs are perfectly capable of summarising the conversation without a new system prompt.
This seems to be a very intuitive take, but people often miss on the technicalities.
The prompt is assembled as tool schemas, then system instructions, then message history, and caching works on prefixes of that. So the cost of an edit is not its size, it is the size of everything behind it. Edit a tool definition and you have invalidated the system prompt and the whole conversation with it. Edit the tail and you pay for the tail.
I measured this by accident in a multi agent ablation where the only variable was whether the supervisor sent a fixed tool array or a per-task subset to the worker. Runs paying cache creation, out of 120 each: fixed 0 cache creation, per-task subset 58 cache creation. Under a prompt load that was $0.0382 per run against $0.0230, and on a clean context it reversed because there was no prefix worth caching.
Which is the argument for what Pi does here. A pointer is a tail edit. Rewriting is a head edit.
Their approach is minimalism. If you need something pi doesn’t have, the recommendation is to ask pi about it and have it build an extension for you, or look for a plugin.
That said, try it. You may find you didn’t need everything in those ecosystems.
I also find the SDK really valuable. Being able embed a truly minimal agent wherever you can run node/typescript is very powerful. Any model, any provider, full control over what it’s doing.
In my experience, the best approach to compaction is to never get to the point where you need compaction and to generally stay below about 30% context window utilization. Even for long agentic workflows this can be accomplished for quite a while, much longer than most people might think.
Here's what I do for each of my sessions:
1. For asides, off-topic work, or repetitive work that has already been done in the session, branch backwards (with /tree) and summarize.
2. If I've exceeded 30% or the 'price-doubling' multi-tier pricing, prune (my custom extension).
3. If I've already pruned and I'm still close to 30%, 'prune all' (more extensive prune).
Definition:
'/prune': Removes ~50% context on a fresh session (not previously pruned)
- Keeps: User messages, normal assistant prose, commands/status markers, extension receipts, model settings, and a plain-text receipt for each tool call.
- Removes: Thinking, signatures, actual tool calls/results, tool output, images, compaction summaries, and other extensions’ state.
'/prune-extended': Removes ~80% context on a fresh session
- Keeps: User messages, normal assistant prose and conclusions, commands/status markers, extension receipts, and model settings.
- Removes: Thinking, signatures, all tool calls/results and output, images, compaction summaries, other extensions’ state, and any tool-activity receipts created by /prune.
Both create a new session and delete the old one after a successful switch.
Using these I can keep a session going for weeks (or longer), even with extensive use and almost all the important context is preserved while dumping the less important context. Neither command requires an LLM summarization so they execute quickly.
What is that 30% number based on? Surely that's a model specific limit, and is based more on the absolute token length, not percentage, right? I'm not sure it makes sense for e.g. Opus 0.2M and Opus 1M to both degrade at 30% of their respective context lengths.
https://www.producttalk.org/context-rot this article summarises several papers that have explored this, and it does seem to be related to both the absolute number of tokens and the % of the window
I am surprised at 'removes actual tool calls/results, tool output'. Your approach with /prune seems to be 'keep the WHAT, remove the HOW (we got here)'. I would have thought that the HOW contains some useful signal.
The regular /prune command leaves tool call 'receipts', which includes the command executed and whether it succeeded or failed, but not results. The extended prune removes both.
The how is important, but I've found all of the decisions, question, answers, and results are the most important and the tool calls themselves secondary. When necessary, the tool calls can be deleted without much being lost.
This is terrible. Models have been RLed on looking at the previous tool call chain, and reasoning. No chance this does not reduce performance. The point of compaction is that it also includes useful signal from the tool outputs itself so agent does not repeat it afterwards
Long context windows reduce performance and exceeding your context window is impossible. It should be a given that deleting context.. is just that. You do it when you must to preserve your session without compaction.
Ampcode used a handoff feature for a while that I found genuinely useful [1] and then they removed it. Anecdotally, I felt it worked better than compaction.
My summarization creation functions over batches of 50 messages, and I don’t often lose important context any more. The loss comes from trying to stick a whole conversation in a single compaction request (at least in my case)
I want to like Pi but compaction is why I had to go back to Open Code…
My problem is that when in a loop and it’s calling tools, it won’t check how close it is to the compacting limit until the whole loop returns to you for the next prompt. And so if you have a run that could go for hours, it’s a gamble if you’ll OOM or an interrupting compaction breaks context and stops the loop without continuing.
There’s a few extensions that all try to solve this problem, but I’ve found none actually work :(
the way compaction works is pretty simple and highly relies on another model to do it, the only part the user has control of is when to do the compaction, which actually means when doing large amount of work in one shot, it should be planned from the beginning to be separated into works that can be reviewed, afterwards the model can compact
Great thread, I was just thinking about compaction. My current line of thought is that compaction/pruning/ctx management in general should be something ongoing and maybe recursive. For example:
User:'How is auth implemented?'
->
[thinking]
[codebase exploration with [thinking] in between, 10 file reads, 3 of which were "wrong"]
[thinking]
->
agent_response
This little exchange contains a WHAT (how auth actually is implemented) and a HOW (where that info is and how to retrieve it). Maybe this question was part of a larger task. I think that whole exchange could be summarised before it enters context, kind of like what happens with subagents. The main thread would then consist mostly of [summaries]. Eventually the context will fill up anyway and we would summarise those summaries again. Alternatively one could maintain a [master_summary], kind of like an internal state. So new [summaries] get integrated directly and the [master_summary] gets updated.
Because juggler has a nested-thread architecture, I had a great ah-ha! moment when I realised that with sub-threads, compaction becomes almost free: you just take the list of items in the conversation, move them all into a new sub-thread, then allow that sub-thread to summarise itself (this summary is what the parent thread sees). Hey-presto, your parent thread is now compacted, and the sub-thread contains all the old messages, so you can browse them if you need to, or just undo the whole operation if you change you mind. This struck me as super-elegant, and I was chuffed that it works.
However.. What I find myself actually doing now is just using the "new conversation" tool to hand over to a new conversation - I just tell the LLM "create a new conversation, and tell it everything it needs to continue this task". It gets you to the same place, but generally I'm not interested in the history, and having a fresh conversation with a new name usually feels neater.
Does anyone find compaction useful? I have always been a big believer of many small chats instead of large ones. There is certainly a time and a place where maybe I need to load in a lot to initial context BUT I often find that quality goes does as we near context limits and of course costs go massively up. I would rather repeat small notarized chunks than have a massive context window.
enforcing markdown files is much better. In a perfect world these docs are curated by humans, but if you can't be arsed then at least have some ground rules so these do not get flooded with context. Agents are pretty good at finding what they need for a task, if they're instructed correctly.
I stopped compacting and started making new chats every time and it has saved me an immense amount of context. I'm using deepseek and their cache hit is cheap, so combining this I don't spend more than 4 bucks a week.
I really don't mind keeping these docs up to date if it means I can leave my desk to go do something more important
There are some very interesting latent compaction approaches like this[1] for when you can control the whole inference stack. i.e in on-device and datacenter inference.
Gecko4072 | 21 hours ago
skinfaxi | 20 hours ago
pkulak | 20 hours ago
aolsenjazz | 4 hours ago
randomblock1 | 21 hours ago
See: https://github.com/earendil-works/pi/blob/main/packages/codi...
alansaber | 19 hours ago
skeledrew | 18 hours ago
kennywinker | 20 hours ago
I pretty much just start a new session whenever i fill the context.
xyzzy123 | 19 hours ago
For an EPYC with a 5090 (no layers on CPU) vs an M3 max 128GB, qwen 3.6 27B at 128k context / 7k generation:
This is for dense qwen (which I wouldn't run day to day on the mac) - in reality the mac is quite usable with MoEs but you definitely notice a difference.cyanydeez | 20 hours ago
I get it into 1M+ routinely on local models with operations between 50k-85k
skeledrew | 18 hours ago
cyanydeez | 16 hours ago
The pruning does evict cache but not from the start, and only up till the last time you pruned it, as it's just adding messages on top.
but yes, it takes some additional tests and docs to keep it from just becoming hollowed out on tasks; I'd say about 10-20% of the time is just horribly loses what it's doing.
storus | 20 hours ago
novaRom | 20 hours ago
One trick I find useful is to have one model with two KV caches running and while first cache has produced tokens, second cache immediately summarizes them during input tokens are being generated (tools time), then harness switches to the second KV cache which takes newly produced input tokens while KV in first cache is getting replaced with compacted summary tokens. This is a kind of ping pong, so we trade more space for less time. Still experimenting but it looks it works, and nice bonus it improves GPU utilization. Btw I have my own harness and model serving code, but it can be easily implemented in any other harness and model server.
kristianpaul | 18 hours ago
It's like asking the questions needed to get the answer you need from the LLM, can be either an answer for a plan/todo or a task to code/change something.
searealist | 20 hours ago
brandall10 | 19 hours ago
That said, from reading the pi subreddit I don't see people referencing these much.
damsta | 19 hours ago
flexagoon | 19 hours ago
> Extensions can intercept and customize both compaction and branch summarization
https://pi.dev/docs/latest/compaction
Just make an extension (or ask Pi to write an extension for itself) that intercepts compaction and leaves only what you want, or rewrites it in any other way. Should be just a few lines.
alansaber | 19 hours ago
fermuch | 16 hours ago
boorang | 14 hours ago
zavec | 6 hours ago
zahrevsky | 19 hours ago
Say, what happens when chain of summaries grows so long, that it still overflows context window. Is summarization runned over the summaries in the context window?
alansaber | 19 hours ago
skeledrew | 18 hours ago
kierangill | 19 hours ago
For example, sometimes context will be taken up by a side tangent, tool call outputs, or low-value codebase exploration.
Much of the time, I prefer to preserve the history of my conversation instead of summarizing it. I find summarized conversations lead to more frustrating future chats because the LLM misses intent and or context. (Or, the presence of paragraphs and paragraphs of LLM output makes the next token predictor dumber? Unsure.)
pjm331 | 18 hours ago
charcircuit | 11 hours ago
spott | 17 hours ago
https://github.com/spott/pi-task-compaction
I’m still playing with it, but it essentially has the model define a region, and mark it with begin_task, and end_task. End_task also requires a fairly hefty summery of the entire region.
It then pulls that entire region and replaces it with the summary. If the model wants it can actually look into the output that was pruned from the conversation, but in practice the summaries have been good enough it hasn’t needed to.
I’ve been using it with 5.6 sol, and even really long sessions have like 6% context used at the end.
mikepurvis | 17 hours ago
I feel like this is kind of subagents, but it doesn't have the clear task -> work -> return result -> die model, rather it's more closely aligned to how I myself multitask, maintaining several long-lived threads of execution, and having to intentionally "forget" about the other ones when it's time to lock in and do real work on the most important one.
Schlagbohrer | 8 hours ago
trueno | 6 hours ago
this seems to be the fate with all the big grand 'god' projects people are doing with AI. super epic omega knowledge bases, super epic omega personal platforms, etc. I don't think enough people admit that whatever they're working on has evolved into something they don't fully understand.
d4rkp4ttern | 16 hours ago
https://pchalasani.github.io/claude-code-tools/tools/aichat/...
jsw97 | 16 hours ago
Sounds cool and it does make sensible decisions optically but I haven’t been able to prove that it is meaningfully better than normal compaction. Building harnesses that do interesting things is a lot easier than building more effective harnesses, I guess.
behindsight | 15 hours ago
There are other pruning or compaction strategies you can configure too.
You can also use "/btw" to have side tangent conversations with the current context but with the benefit that it will not bloat your current context after the tangent is done.
I wish other harnesses were prioritising this sort of flexibility with context management, but they're likely more incentivised on optimising their cache rates
https://github.com/can1357/oh-my-pi
tyingq | 4 hours ago
Ah, something I needed, but did not know I needed. Comes up ALL the time. I want to know something that is well off the task, but it's the best time to ask due to the specifically loaded context.
WhyNotHugo | 13 hours ago
Having a second model also iterate the resulting messages and remove low-value tool calls could also be interesting. Especially failed calls which add no value.
marton78 | 8 hours ago
https://pi.dev/packages/context-fold
pzo | 12 hours ago
Dayshine | 12 hours ago
zmgsabst | 11 hours ago
That naturally trims a lot of context while removing mistakes from the context to prevent poisoning (ie, every draft but the final negatively contributes in some way — that’s why we edited them).
[OP] tosh | 10 hours ago
toasty228 | 5 hours ago
ivanovm | 4 hours ago
bhaktatejas922 | 3 hours ago
0-_-0 | 3 hours ago
ATMLOTTOBEER | 11 minutes ago
jakswa | 19 hours ago
UncleOxidant | 18 hours ago
jakswa | 10 hours ago
bakies | 15 hours ago
jubilanti | 12 hours ago
It sounds crazy, but it's way cheaper and faster for multimodal models to read text on images than the equivalent text represented as tokens. Text tokens take a lot more bytes to represent internally than you would think. Give it images with rasterized text, and there is no OCR-like intermediate step that turns images into text tokens. It just goes straight into the multimodal context where it's all just linea algebra underneath. The cost is more lossiness than if you gave it the same tokens as text. Which is why you don't just render text prompts and files to read as images.
But compaction is inherently lossy, a summary 1-10% of the original length is going to lose things, by definition. So you keep the things you really need to get exactly right in text token summary, but put a lot more of what you would have just discarded into a PNG. Keeping the past context as text in an image is way less lossy than keeping a text summary of that context.
They peeked inside the layers and found the same neurons were usually activating in the middle layers when fed either text tokens or images of that same text. It just takes the model more time and kv cache to do it via text tokens.
Deepseek was the first to really jump on this: https://github.com/deepseek-ai/DeepSeek-OCR/blob/main/DeepSe...
skeledrew | 18 hours ago
pornel | 18 hours ago
LLMs are perfectly capable of summarising the conversation without a new system prompt.
kristianpaul | 18 hours ago
pornel | 13 hours ago
This way you have a prefix cache hit on the session and don't pay any more than for a regular prompt.
pranayVarma0512 | 9 hours ago
The prompt is assembled as tool schemas, then system instructions, then message history, and caching works on prefixes of that. So the cost of an edit is not its size, it is the size of everything behind it. Edit a tool definition and you have invalidated the system prompt and the whole conversation with it. Edit the tail and you pay for the tail.
I measured this by accident in a multi agent ablation where the only variable was whether the supervisor sent a fixed tool array or a per-task subset to the worker. Runs paying cache creation, out of 120 each: fixed 0 cache creation, per-task subset 58 cache creation. Under a prompt load that was $0.0382 per run against $0.0230, and on a clean context it reversed because there was no prefix worth caching.
Which is the argument for what Pi does here. A pointer is a tail edit. Rewriting is a head edit.
meatmanek | 17 hours ago
brcmthrowaway | 17 hours ago
josh_p | 17 hours ago
Their approach is minimalism. If you need something pi doesn’t have, the recommendation is to ask pi about it and have it build an extension for you, or look for a plugin.
That said, try it. You may find you didn’t need everything in those ecosystems.
josh_p | 17 hours ago
Not a shill.. I just really like pi.
ifoxhz | 15 hours ago
arolihas | 16 hours ago
https://github.com/can1357/oh-my-pi
errantmind | 14 hours ago
Here's what I do for each of my sessions:
1. For asides, off-topic work, or repetitive work that has already been done in the session, branch backwards (with /tree) and summarize.
2. If I've exceeded 30% or the 'price-doubling' multi-tier pricing, prune (my custom extension).
3. If I've already pruned and I'm still close to 30%, 'prune all' (more extensive prune).
Definition:
'/prune': Removes ~50% context on a fresh session (not previously pruned)
'/prune-extended': Removes ~80% context on a fresh session Both create a new session and delete the old one after a successful switch.Using these I can keep a session going for weeks (or longer), even with extensive use and almost all the important context is preserved while dumping the less important context. Neither command requires an LLM summarization so they execute quickly.
hatthew | 13 hours ago
aevv | 11 hours ago
bvrmn | 7 hours ago
errantmind | 22 minutes ago
Imanari | 7 hours ago
errantmind | 26 minutes ago
The how is important, but I've found all of the decisions, question, answers, and results are the most important and the tool calls themselves secondary. When necessary, the tool calls can be deleted without much being lost.
MikhailTal | 2 hours ago
errantmind | 23 minutes ago
navs | 14 hours ago
[1] https://ampcode.com/news/handoff
badlogic | 8 hours ago
Aeolun | 12 hours ago
rcarmo | 11 hours ago
jedisct1 | 9 hours ago
alfiedotwtf | 8 hours ago
My problem is that when in a loop and it’s calling tools, it won’t check how close it is to the compacting limit until the whole loop returns to you for the next prompt. And so if you have a run that could go for hours, it’s a gamble if you’ll OOM or an interrupting compaction breaks context and stops the loop without continuing.
There’s a few extensions that all try to solve this problem, but I’ve found none actually work :(
imgyuri | 7 hours ago
Imanari | 6 hours ago
User:'How is auth implemented?' -> [thinking] [codebase exploration with [thinking] in between, 10 file reads, 3 of which were "wrong"] [thinking] -> agent_response
This little exchange contains a WHAT (how auth actually is implemented) and a HOW (where that info is and how to retrieve it). Maybe this question was part of a larger task. I think that whole exchange could be summarised before it enters context, kind of like what happens with subagents. The main thread would then consist mostly of [summaries]. Eventually the context will fill up anyway and we would summarise those summaries again. Alternatively one could maintain a [master_summary], kind of like an internal state. So new [summaries] get integrated directly and the [master_summary] gets updated.
julesrms | 6 hours ago
Because juggler has a nested-thread architecture, I had a great ah-ha! moment when I realised that with sub-threads, compaction becomes almost free: you just take the list of items in the conversation, move them all into a new sub-thread, then allow that sub-thread to summarise itself (this summary is what the parent thread sees). Hey-presto, your parent thread is now compacted, and the sub-thread contains all the old messages, so you can browse them if you need to, or just undo the whole operation if you change you mind. This struck me as super-elegant, and I was chuffed that it works.
However.. What I find myself actually doing now is just using the "new conversation" tool to hand over to a new conversation - I just tell the LLM "create a new conversation, and tell it everything it needs to continue this task". It gets you to the same place, but generally I'm not interested in the history, and having a fresh conversation with a new name usually feels neater.
infecto | 4 hours ago
ramon156 | 4 hours ago
I stopped compacting and started making new chats every time and it has saved me an immense amount of context. I'm using deepseek and their cache hit is cheap, so combining this I don't spend more than 4 bucks a week.
I really don't mind keeping these docs up to date if it means I can leave my desk to go do something more important
infecto | 3 hours ago
woadwarrior01 | 3 hours ago
[1]: https://arxiv.org/abs/2602.16284