I have nearly a decade of zsh history. Reading that article I came to the conclusion that I may have been hit by that bug in the past but I haven’t noticed.
Then I kept reading and the author mentions accidentally exporting HISTFILE[1] and I screamed in terror and ran to my computer as I realized a mistake I’ve been making for…ever.
Same here, atuin is such a time-saver, especially when it comes "oh dang, I ran this command on another host but I need it here now". It just works as you'd expect!
Some things are worth an alias, or a script, some things aren't quite worth either but you still wanna run them again in the future. I think that's where history helps. Awkward niche one-liners. Also sometimes you do something for the first time, take a bit to figure it out, then 6-12 months later it's easier to check your shell history than relearn the process from scratch again. Also sometimes you may want to run something new that's very similar to an old thing you ran, like maybe you forget which rsync args you've been using lately and you wanna use the same ones but on different files. Search up your last rsync, edit paths.
I used to start my history over with each PC, but last year when I upgraded my primary machine I decided to copy over my browser history and shell history in addition to the usual dotfiles and stuff. It's honestly been really nice and convenient. It almost feels like cheating that I can see stuff older than the machine that I did.
>Some things are worth an alias, or a script, some things aren't quite worth either but you still wanna run them again in the future
I think the bigger problem is that sometimes it's just hard to know upfront exactly what is worth an alias or a script. It may at the time seem like something you'll not need to bother with again and then it turns out a year or two later you have to deal with it a fair amount. I guess one could be in the habit of just always documenting absolutely everything and doing aliases/scripts by default but as a practical matter that's a great deal of time given that many one-liners truly genuinely are one-liners that really won't ever get used again.
Automated history recording helps fill that gap. If you find you do need something again down the road that was a certain amount of effort to figure out the first time, the second (or third or whatever your heuristic is) time is when you can make an alias or script for it. Having the history on-hand simultaneously helps give a jumping off point and helps one decide that yes, actually I better formalize this because I had to do it before on dates X, Y & Z.
Why do you care about 99.99% garbage when it's useful exactly for those .01% gems?
You don't look back at history, you search for it or it pops automatically
For example, for a command you run once a year you've spent your time researching proper use and created this invokation you'll never remember:
app -x +e -lsadfjku --avoid-footgun old new # maybe valuable comment/search tag for future you
then next year you just type app and have the autocomplete / search for app. Boom, history value received! No need to repeat the research process all over again
You can't expect much from a system that has those defaults, I'm all for traditions, but if I were to depend on a system for command history, it wouldn't be anything that reads a bash_history, a revolution is needed, not incremental optimizations
P.s: what I do is set it to unlimited on both size and line count, but it's not a forensic grade audit trace, we'd need an OS, or ssh command logger, preferably one that writes to an external disk, but it can be to a root controlled file, definitely not to a user owned file.
Security protects not only against attackers, but against bugs, a user's command history should not be deletable by user. Anything else is a wrong architecture that we are just carrying by inertia and laziness.
Absolutely not. It felt so laggy and buggy ~1 year ago. I think resizing my screen broke it at the time? And now it's loaded with AI and shortcuts overrides.
I switched to my custom fork of stinkpot posted a few days ago here, and it's so nice. I did have to add current-for filtering, but that was easy.
For me, I thought this until I tried spinning up my nix setup on a new machine and realised that a lot of my work patterns were being stored subconciously in my shell history. After using atuin with nix, each new machine became a carbon copy and I could spin up and instance and continue working seemlessly. There are also sometimes examples of things I do in work that I then want to repeat at home or vice versa which atuin makes easier
I still use it for now, but it looks like they added some AI stuff that I don't want. I have disabled that for now, but I'm looking into migrating to mcfly.
That's a bug? Happens to me with bash all the time, all I have to do is open 2 terminals at the same time... whichever one I close last is the one that writes its history. /shrug
This is exactly why I stopped using Bash around 2012. I was soooo tired of losing random shell history.
If you haven't looked into ZSH, it has some really nice customizations. Notably, from my config[1]:
# Remove dups
setopt HIST_FIND_NO_DUPS
setopt HIST_IGNORE_DUPS
# Don't log commands starting with a space
setopt HIST_IGNORE_SPACE
# Replcae bang-command !34 with the actual command in the history
setopt BANG_HIST
# Remove extraneous spaces
setopt HIST_REDUCE_BLANKS
# Write the history file in the ":start:elapsed;command" format
setopt EXTENDED_HISTORY
# Record run-time of the command
setopt INC_APPEND_HISTORY_TIME
I ran into this a few years ago (or I ran into a very similar bug). I started backing up my history file regularly because it was so annoying to lose all that history. I'll keep backing it up, but this fix is appreciated!
this is an example of overcomplicated solutions that could be avoided with the tool we already have for years - file system - why do this "heroic" effort of "cleverly" putting everything in one file from many sessions if we could just write command history of every session into a new separate file in a directory and read all history files from the dir instead of reading one file.
>why do this "heroic" effort of "cleverly" putting everything in one file from many sessions if we could just write command history of every session into a new separate file in a directory
For "normal" use in the terminal, a unified single history file is easier to deal with. For example, yesterday I was doing some forensics on the existence of mystery files in my backups from years ago and it was nice just loading up a single history file in the editor ... page up and page down repeatedly to eventually reconstruct the sequence of commands that explained the mystery files. The alternate idea of separate history files is less ergonomic for that. I'd have to open each history file (potentially hundreds) one by one in the editor. Trying to treat separate history files as "virtual single file" by using grep on the whole batch wouldn't work because I didn't know ahead of time what string to grep for. It required manually eyeballing the commands to consider which ones were the culprit. (Yes, if history files were separate, I could also concatenate "cat" them all to a temp file and load that into an editor ... but a history file that's already unified means I don't have to bother with that step and delete that temp file after I'm done.)
>HISTFILE="$HOME/zsh/$(date '+%y:%m%d:%H%M')-$$"
However, for "not normal" usage of a shell in a coding editor, I do create separate history files with a timestamp like your suggestion. In dev environments, I tend to create lots of dirty throwaway shell commands and I don't want them polluting the "normal" shell history.
It's still inconvenient because the .bash_history and .zsh_history files I was studying were inside of .tar.xz and .zip backup files. I wouldn't want to crack open those archives just to concatenate a hypothetical set of separate history files to single temp files. Instead, the GUI archive browser I used just highlights a single history file and loads it into the editor.
Or replace fopen with a SQLite database :-) like IPython does [1], but now it becomes more complicated to read and edit it including from other programs, say a text editor [2].
I have lost commands in the Bash history as well and I don't know why. Fortunately, I store all commands executed (+ time, return value etc.) in an sqlite database so I run a program from time to time to import that into the Bash history. The Bash history is useful for Ctrl+R
aclindsa | 20 hours ago
mmh0000 | 20 hours ago
Then I kept reading and the author mentions accidentally exporting HISTFILE[1] and I screamed in terror and ran to my computer as I realized a mistake I’ve been making for…ever.
I am now both happy and sad I read this article.
[1] https://github.com/stapelberg/configfiles/commit/32dcda0f49a...
PyWoody | 18 hours ago
Have you ever looked back on your history?
TacticalCoder | 17 hours ago
Same
> Have you ever looked back on your history?
All the time? I search in my history, re-launch past commands, etc.
Now it's typically recent history, usually same day.
I know about the shortcuts etc. but I also need to at times pipe history into other commands so I've got a two letters alias:
Which greps into my history. I use that h? all the time.unsnap_biceps | 17 hours ago
fragmede | 17 hours ago
musicmatze | 11 hours ago
inkyoto | 15 hours ago
I also hold over a decade worth of zsh history and revisit it from time to time to reconstruct the full memory of what I was working on N years ago.
opan | 15 hours ago
I used to start my history over with each PC, but last year when I upgraded my primary machine I decided to copy over my browser history and shell history in addition to the usual dotfiles and stuff. It's honestly been really nice and convenient. It almost feels like cheating that I can see stuff older than the machine that I did.
xoa | 5 hours ago
I think the bigger problem is that sometimes it's just hard to know upfront exactly what is worth an alias or a script. It may at the time seem like something you'll not need to bother with again and then it turns out a year or two later you have to deal with it a fair amount. I guess one could be in the habit of just always documenting absolutely everything and doing aliases/scripts by default but as a practical matter that's a great deal of time given that many one-liners truly genuinely are one-liners that really won't ever get used again.
Automated history recording helps fill that gap. If you find you do need something again down the road that was a certain amount of effort to figure out the first time, the second (or third or whatever your heuristic is) time is when you can make an alias or script for it. Having the history on-hand simultaneously helps give a jumping off point and helps one decide that yes, actually I better formalize this because I had to do it before on dates X, Y & Z.
eviks | 6 hours ago
For example, for a command you run once a year you've spent your time researching proper use and created this invokation you'll never remember:
app -x +e -lsadfjku --avoid-footgun old new # maybe valuable comment/search tag for future you
then next year you just type app and have the autocomplete / search for app. Boom, history value received! No need to repeat the research process all over again
javier2 | 8 hours ago
Does it put `export` statements from the histfile into the current env?
inigyou | 8 hours ago
TZubiri | 20 hours ago
You can't expect much from a system that has those defaults, I'm all for traditions, but if I were to depend on a system for command history, it wouldn't be anything that reads a bash_history, a revolution is needed, not incremental optimizations
P.s: what I do is set it to unlimited on both size and line count, but it's not a forensic grade audit trace, we'd need an OS, or ssh command logger, preferably one that writes to an external disk, but it can be to a root controlled file, definitely not to a user owned file.
Security protects not only against attackers, but against bugs, a user's command history should not be deletable by user. Anything else is a wrong architecture that we are just carrying by inertia and laziness.
fragmede | 18 hours ago
a2ff6eeb0 | 18 hours ago
hn92726819 | 16 hours ago
I switched to my custom fork of stinkpot posted a few days ago here, and it's so nice. I did have to add current-for filtering, but that was easy.
NSPG911 | 15 hours ago
marliechiller | 9 hours ago
lap007 | 13 hours ago
[[ -n `find . -maxdepth 1 -name .zsh_history -size -10k` ]] && echo "ZSH history is empty!!!"
Maledictus | 6 hours ago
nubinetwork | 16 hours ago
valleyer | 16 hours ago
shock | 9 hours ago
mmh0000 | 16 hours ago
If you haven't looked into ZSH, it has some really nice customizations. Notably, from my config[1]:
[1] https://doc.xn0.org/.zshrcpraash | 12 hours ago
chillpenguin | 16 hours ago
pratyahava | 13 hours ago
pratyahava | 13 hours ago
jasode | 11 hours ago
For "normal" use in the terminal, a unified single history file is easier to deal with. For example, yesterday I was doing some forensics on the existence of mystery files in my backups from years ago and it was nice just loading up a single history file in the editor ... page up and page down repeatedly to eventually reconstruct the sequence of commands that explained the mystery files. The alternate idea of separate history files is less ergonomic for that. I'd have to open each history file (potentially hundreds) one by one in the editor. Trying to treat separate history files as "virtual single file" by using grep on the whole batch wouldn't work because I didn't know ahead of time what string to grep for. It required manually eyeballing the commands to consider which ones were the culprit. (Yes, if history files were separate, I could also concatenate "cat" them all to a temp file and load that into an editor ... but a history file that's already unified means I don't have to bother with that step and delete that temp file after I'm done.)
>HISTFILE="$HOME/zsh/$(date '+%y:%m%d:%H%M')-$$"
However, for "not normal" usage of a shell in a coding editor, I do create separate history files with a timestamp like your suggestion. In dev environments, I tend to create lots of dirty throwaway shell commands and I don't want them polluting the "normal" shell history.
pratyahava | 11 hours ago
i mean, the problem of viewing all history files at once is much easier than writing to the same one file from multiple sessions.
jasode | 10 hours ago
It's still inconvenient because the .bash_history and .zsh_history files I was studying were inside of .tar.xz and .zip backup files. I wouldn't want to crack open those archives just to concatenate a hypothetical set of separate history files to single temp files. Instead, the GUI archive browser I used just highlights a single history file and loads it into the editor.
ciupicri | 7 hours ago
[1]: https://ipython.readthedocs.io/en/stable/interactive/referen...
[2]: https://stackoverflow.com/a/39415188
conferza | 10 hours ago
inigyou | 7 hours ago
Valodim | 2 hours ago
petters | 7 hours ago
croemer | 7 hours ago