mcfly: Really slow startup with large history file, MCFLY_HISTORY_LIMIT doesn't help

I have this in my .bashrc:

export MCFLY_HISTORY_LIMIT=10000
eval "$(mcfly init bash)"

Doing . ~/.bashrc takes ~forever. I have no idea how long, because I did ctrl+c after a minute or so. This is a new install. I’ve tried changing MCFLY_HISTORY_LIMIT=1000, doing rm ~/.mcfly/history.db, etc… still takes ages/doesn’t complete. Lots of disk i/o too.

I assume this is because I have a large history file. These are my current bash history related settings:

#
# History settings
#
shopt -s histappend # Append to the history file, don't overwrite it
shopt -s cmdhist # Save multi-line commands as one command
export HISTSIZE=-1 # Keeps all history
export HISTFILESIZE=-1 # Infinite file size
export HISTTIMEFORMAT='%F %T ' # Useful timestamp format
export HISTIGNORE="&:[ ]*:exit:clear"
export HISTFILE=~/.bash_eternal_history
# Record each line as it gets issued
[[ "$PROMPT_COMMAND" == *'history -a'* ]] || export PROMPT_COMMAND="history -a;$PROMPT_COMMAND"

My ~/.bash_eternal_history file is currently 497.5 KiB, 26,339 lines. It also includes timestamps, so the last bit of it looks like this:

#1615575032
. ~/.bashrc
#1615575046
rm ~/.mcfly/history.db
#1615575049
micro ~/bash/mcfly.bash
#1615575057
micro ~/.bashrc
#1615575459
micro ~/bash/env.bash

If I do this, mcfly starts working:

$ mv ~/.bash_eternal_history ~/_.bash_eternal_history
$ rm ~/.mcfly/history.db
$ . ~/.bashrc

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Comments: 17 (9 by maintainers)

Most upvoted comments

I installed today and the initial import took 12s on a 13 Mb history file.

🕙 2021-08-02 11:10:01  ❯ source ~/.bashrc 
McFly: Importing shell history for the first time. This may take a minute or two...done.

✦2 🕙 2021-08-02 11:10:13  ❯  ls -al ~/.mcfly/history.db ~/.dotfiles/bash/.bash_eternal_history
.rw-r--r-- michaeljoseph staff 13 MB Mon Aug  2 11:15:34 2021  /Users/michaeljoseph/.dotfiles/bash/.bash_eternal_history
.rw-r--r-- michaeljoseph staff 44 MB Mon Aug  2 11:15:34 2021  /Users/michaeljoseph/.mcfly/history.db