Expanding Eshell

Table of Contents

A literate programming file for configuring the Emacs Shell.

Introduction

While I like vterm for logging into remote systems, I find Emacs’ shell, eshell, an interesting alternative. If you find the documentation lacking, I documented most features, and you might find the following helpful.

Tell straight to use the built-in eshell: 

(use-package eshell
  :straight (:type built-in)
  :hook (eshell-mode . 'ha-eshell-setup))

After reading this essay, I decided to try hiding the mode line in eshell windows … at least, until I get the mode line to display more important information. Note that hiding the mode line is fairly straight-forward, but others might want to use the hide-mode-line package that turns that mode-line definition into a minor mode that can be toggled.

I like debug-on-error, but not in Eshell, so I set this up when entering Eshell: 

(defun ha-eshell-setup ()
  (make-local-variable 'debug-on-error)
  (setq mode-line-format nil
        debug-on-error nil))

Navigation and Keys

Along with the regular Emacs keybindings, Eshell comes with some interesting features:

  • M-RET gives you a prompt, even when you are running another command. Since eshell passes all input to subprocesses, there is no automatic input queueing as there is with other shells.
  • C-c C-p / C-c C-n jumps between command inputs using Eat.
  • C-c C-t truncates the buffer if it grows too large.
  • C-c C-r will move point to the beginning of the output of the last command. With a prefix argument, eshell narrows to view its output.
  • C-c C-o will delete the output from the last command.
  • C-c C-f will move forward a complete shell argument, but I usually either type Escape to enter Evil mode, or just issue M-f / M-b.
  • C-c C-b will move backward a complete shell argument.
  • M-S-r shows/selects a better history of commands (see below). Yes, the up/down arrows keys (as well as M-p / M-n) scroll through this, but that is slow.

Control-D Double Duty

Used to C-d exiting from a shell? Want it to keep working, but still allow deleting a character? We can have it both (thanks to wasamasa): 

(defun ha-eshell-quit-or-delete-char (arg)
  "The `C-d' sequence closes window or deletes a character."
  (interactive "p")
  (if (and (eolp) (looking-back eshell-prompt-regexp))
      (progn
        (eshell-life-is-too-much) ; Why not? (eshell/exit)
        (ignore-errors
          (delete-window)))
    (delete-forward-char arg)))

Pager Setup

If any program wants to pause the output through the $PAGER variable, well, we don’t need that: 

(setenv "PAGER" "cat")

Argument Completion

Shell completion uses the flexible pcomplete mechanism internally, which allows you to program the completions per shell command. To know more, check out this blog post, about how to configure pcomplete for git commands. The pcmpl-args package extends pcomplete with completion support for more commands, like the Fish and other modern shells. I love how a package can gives benefits without requiring learning anything. 

(use-package pcmpl-args)

Note that this will work with shell-command as well.

Better Command Line History

On this discussion a little gem for using IDO to search back through the history, instead of M-R to prompt for the history. 

(defun eshell-insert-history ()
  "Displays the eshell history to select and insert back into your eshell."
  (interactive)
  (insert (completing-read "Eshell history: "
                               (delete-dups
                                (ring-elements eshell-history-ring)))))

Predicate Filters and Modifiers

This is a cool and under-rated feature of Eshell. Type the following commands into an Eshell buffer for the details:

  • eshell-display-modifier-help
  • eshell-display-predicate-help

In short, use parenthesis to limit the files, for instance: 

ls *.sh(*)   # List shell scripts that are executable
ls *(/)      # List directories (recursively)

And parens with a colon character transform the filename, useful with for: 

for F in *.org(:r) { mv $F.org $F.el }

Keep in mind that the predicates are somewhat finicky. For instance, the following doesn’t work: 

mv $f $f(:r).txt

But you could call this: 

mv $f $f(:s/org$/txt/)

The T predicate filter allows me to limit file results that have internal org-mode tags. 

$ ls *.org(T'org')
ha-agendas.org          ha-org-journaling.org      ha-org.org
ha-capturing-notes.org  ha-org-publishing.org
ha-org-clipboard.org    ha-org-word-processor.org

For instance, eshell will send files that have a #+TAGS: header with a macos label to the grep function: 

$ grep brew *.org(T'macos')

As described in this essay, to extend Eshell, we need a two-part function:

  1. Parse the Eshell buffer to look for the parameter (and move the point past the parameter).
  2. A predicate function that takes a file as a parameter.

For the first step, we have our function called as it helps parse the text. Based on what it sees, it returns the predicate function used to filter the files: 

(defun eshell-org-file-tags ()
  "Parse the eshell text at point.
Looks for parameters surrounded in single quotes. Returns a
function that takes a FILE and returns nil if the file given to
it doesn't contain the org-mode #+TAGS: entry specified."

  ;; Step 1. Parse the eshell buffer for our tag between quotes
  ;;         Make sure to move point to the end of the match:
  (if (looking-at (rx "'" (group (one-or-more (not (or ")" "'"))))"'"))
      (let* ((tag (match-string 1))
             (reg (rx line-start
                      "#+" (optional "file") "tags:"
                      (one-or-more space)
                      (zero-or-more any)
                      (literal tag) word-end)))
        (goto-char (match-end 0))

        ;; Step 2. Return the predicate function:
        ;;         Careful when accessing the `reg' variable.
        `(lambda (file)
           (with-temp-buffer
             (insert-file-contents file)
             (re-search-forward ,reg nil t 1))))
    (error "The `T' predicate takes an org-mode tag value in single quotes.")))

Then we need add that function to the eshell-predicate-alist as the T tag: 

(defun ha-eshell-add-predicates ()
  "A hook to add a `eshell-org-file-tags' predicate filter to eshell."
  (add-to-list 'eshell-predicate-alist '(?T . (eshell-org-file-tags))))

Note: We can’t add it to the list until after we start our first eshell session, so we add it to the eshell-pred-load-hook.

Aliases

Gotta have some shell aliases, right? We have three ways of doing that. First, enter them into an eshell session: 

alias ll 'ls -AlohG --color=always'

Note that you need single quotes (not double quotes). Also note that more than one parameter doesn’t work with aliases (to resolve that, we need to write a function).

Second, you can create/populate the alias file, ~/.emacs.d/eshell/alias … as long as you don’t use those single quotes: 

alias ll ls -AlohG --color=always
alias cls clear 1
alias d dired $1
alias find echo 'Please use fd <pattern> <paths> instead.'

Yeah, the variable $* doesn’t work as you’d expect, so use $1 when calling Emacs functions that take one parameter). For instance, while I would like to have the following, the real solution is to make functions (see below for details). 

alias less view-file

Third, you want more control, you can use the eshell/alias function, but it doesn’t honor $1 and other parameters, so we could create conditionally create function that we add to the eshell-mode-hook, for instance: 

(defun ha-eshell-add-aliases ()
  "Call `eshell/alias' to define my aliases."
  ;; The 'ls' executable requires the Gnu version on the Mac
  (let ((ls (if (file-exists-p "/opt/homebrew/bin/gls")
                "/opt/homebrew/bin/gls" ; mac issue only
              "/bin/ls")))
    (eshell/alias "ll" (concat ls " -AlohG --color=always"))))

I have also had a lot of trouble getting aliases to work, for instance dired works, but less does not: 

alias less view-file $1
alias d dired $1

To work around this, I create functions instead. For instance …

The basename shell command strips off a parent, and can strip off the extension. However, you have to specify the extension. Why not take advantage of Emacs’ file-name-base to strip it off without needing to specify it. 

(defalias 'eshell/base 'file-name-base)

However, the following doesn’t work as expected: 

$ set f some-file.org # usually this is from a 'for' loop
$ mv $f {base $f}.txt

While the sequence {base $f} works, the .txt without a space screws it up, creating a list. So I create this simple function: 

(defun eshell/newbase (file newbase)
  (unless (string-match (rx bos ".") newbase)
    (setq newbase (concat "." newbase)))
  (concat (file-name-base file) newbase))

Which works as expected: 

$ echo {newbase $f txt}
some-file.txt

A better approach, however, is to take advantage of modifiers.

Eshell Functions

Any function that begins with eshell/ is available as a command (with the remaining letters). For instance: 

(defun eshell/greet (&rest params)
  (let ((greeting (seq-random-elt
                   '(Hello Greeting Howdy "How's it going"))))
    (if params
        (format "%s, %s!" greeting
                (propertize (car params) 'face
                            '(:weight bold :foreground "lightblue")))
      "Hello World.")))

Not everything has to be an Emacs function, as the normal executables are available. Once I had a function eshell/f as a replacement for find, but the fd project is better.

Since eshell is an Emacs shell, I try to think how to use Emacs buffers in a shell-focused workflow. For instance, use view-file instead of less, as it will show a file with syntax coloring, and typing q returns to your shell session.

This helper function can tell me if an executable program is available, and return its location: 

(defun ha-find-executable (program)
  "Return full path to executable PROGRAM on the `exec-path'."
  (first
   (-filter 'file-executable-p
            (--map (expand-file-name program it) (exec-path)))))

Calling Emacs functions that take a single argument from eshell that could accept zero or more, can result in an error. This helper function can open each argument in a different window. It takes two functions, and calls the first function on the first argument, and calls the second function on each of the rest: 

(defun eshell-fn-on-files (fun1 fun2 args)
  "Call FUN1 on the first element in list, ARGS.
Call FUN2 on all the rest of the elements in ARGS."
  (unless (null args)
    (let ((filenames (flatten-list args)))
      (funcall fun1 (car filenames))
      (when (cdr filenames)
        (mapcar fun2 (cdr filenames))))
    ;; Return an empty string, as the return value from `fun1'
    ;; probably isn't helpful to display in the `eshell' window.
    ""))

The eshell-command is supposed to be an interactive command for prompting for a shell command in the mini-buffer. However, I have some functions that run a command and gather the output. For that, we call eshell-command but a t for the second argument: 

(defun eshell-command-to-string (command)
  "Return results of executing COMMAND in an eshell environtment.
The COMMAND can either be a string or a list."
  (when (listp command)
    ;; Since `eshell-command' accepts a string (and we want all its
    ;; other goodies), we synthesize a string, but since `command'
    ;; could be a parsed list, we quote all of the arguments.
    ;;
    ;; Hacky. Until I figure out a better way to call eshell,
    ;; as `eshell-named-command' doesn't work reliably:
    (setq command (s-join " " (cons (first command)
                                    (--map (format "\"%s\"" it) (rest command))))))
  (with-temp-buffer
    (eshell-command command t)
    (buffer-string)))

Getopts

I need a function to analyze command line options. I’ve tried to use eshell-eval-using-options, but it lacks the ability to have both dashed parameter arguments and non-parameter arguments. For instance, I want to type: 

flow --lines some-buffer another-buffer

To have both a —lines parameter, as well as a list of buffers, so I’ll need to roll my own. While the shell-getopts function works, it doesn’t do the following:

  • Separates more than one single letter options, like -la … it accepts the -l but would ignore the implied -a.
  • Requires that all options go before the rest of the parameters.
  • Doesn’t allow default values for a parameter.

This wee beastie takes a list of arguments given to the function, along with a argument definition, and returns a hash-table of results. 

(defun eshell-getopts (defargs args)
  "Return hash table of ARGS parsed against DEFARGS.
Where DEFARGS is an argument definition, a list of plists.
For instance:
   '((:name number :short \"n\"                 :parameter integer :default 0)
     (:name title  :short \"t\" :long \"title\" :parameter string)
     (:name debug  :short \"d\" :long \"debug\"))

If ARGS, a list of _command line parameters_ is something like:

    '(\"-d\" \"-n\" \"4\" \"--title\" \"How are that\" \"this\" \"is\" \"extra\")

The hashtable return would contain these entries:

    debug t
    number 4  ; as a number
    title \"How are that\" ; as a string
    parameters (\"this\" \"is\" \"extra\") ; as a list of strings "
  (let ((retmap    (make-hash-table))
        (short-arg (rx string-start "-" (group alnum)))
        (long-arg  (rx string-start "--" (group (1+ any)))))

    ;; Let's not pollute the Emacs name space with tiny functions, as
    ;; well as we want these functions to have access to the "somewhat
    ;; global variables", `retmap' and `defargs', we use the magical
    ;; `cl-labels' macro to define small functions:

    (cl-labels ((match-short (str defarg)
                  ;; Return t if STR matches against DEFARG's short label:
                  (and (string-match short-arg str)
                       (string= (match-string 1 str)
                                (plist-get defarg :short))))

                (match-long (str defarg)
                  ;; Return t if STR matches against DEFARG's long label:
                  (and (string-match long-arg str)
                       (string= (match-string 1 str)
                                (plist-get defarg :long))))

                (match-arg (str defarg)
                  ;; Return DEFARG if STR matches its definition (and it's a string):
                  (when (and (stringp str)
                             (or (match-short str defarg)
                                 (match-long str defarg)))
                    defarg))

                (find-argdef (str)
                  ;; Return entry in DEFARGS that matches STR:
                  (first (--filter (match-arg str it) defargs)))

                (process-args (arg parm rest)
                  (when arg
                    (let* ((defarg (find-argdef arg))
                           (key    (plist-get defarg :name)))
                      (cond
                       ;; If ARG doesn't match any definition, add
                       ;; everything else to PARAMETERS key:
                       ((null defarg)
                        (puthash 'parameters (cons arg rest) retmap))

                       ((plist-get defarg :help)
                        (error (documentation (plist-get defarg :help))))

                       ;; If argument definition has a integer parameter,
                       ;; convert next entry as a number and process rest:
                       ((eq (plist-get defarg :parameter) 'integer)
                        (puthash key (string-to-number parm) retmap)
                        (process-args (cadr rest) (caddr rest) (cddr rest)))

                       ;; If argument definition has a parameter, use
                       ;; the next entry as the value and process rest:
                       ((plist-get defarg :parameter)
                        (puthash key parm retmap)
                        (process-args (cadr rest) (caddr rest) (cddr rest)))

                       ;; No parameter? Store true for its key:
                       (t
                        (puthash key t retmap)
                        (process-args (first rest) (second rest) (cdr rest))))))))

      (process-args (first args) (second args) (cdr args))
      retmap)))

Let’s make some test examples: 

(ert-deftest eshell-getopts-test ()
  (let* ((defargs
          '((:name number :short "n"                :parameter integer :default 0)
            (:name title  :short "t" :long "title"  :parameter string)
            (:name debug  :short "d" :long "debug")))
         (no-options   '())
         (just-params  '("apple" "banana" "carrot"))
         (just-options '("-d" "-t" "this is a title"))
         (all-options  '("-d" "-n" "4" "--title" "My title" "apple" "banana" "carrot"))
         (odd-params   `("ha-eshell.org" ,(get-buffer "ha-eshell.org"))))

    ;; No options ...
    (should (= (hash-table-count (eshell-getopts defargs no-options)) 0))

    ;; Just parameters, no options
    (let ((opts (eshell-getopts defargs just-params)))
      (should (= (hash-table-count opts) 1))
      (should (= (length (gethash 'parameters opts)) 3)))

    ;; No parameters, few options
    (let ((opts (eshell-getopts defargs just-options)))
      (should (= (hash-table-count opts) 2))
      (should (= (length (gethash 'parameters opts)) 0))
      (should (gethash 'debug opts))
      (should (string= (gethash 'title opts) "this is a title")))

    ;; All options
    (let ((opts (eshell-getopts defargs all-options)))
      (should (= (hash-table-count opts) 4))
      (should (gethash 'debug opts))
      (should (= (gethash 'number opts) 4))
      (should (string= (gethash 'title opts) "My title"))
      (should (= (length (gethash 'parameters opts)) 3)))

    (let* ((opts  (eshell-getopts defargs odd-params))
           (parms (gethash 'parameters opts)))

      (should (= (hash-table-count opts) 1))
      (should (= (length parms) 2))
      (should (stringp (first parms)))
      (should (bufferp (second parms))))))

Setting Variables

To set a variable in Eshell, you use good ol’ setq, but that would create global variables. We can make a version for Eshell, that makes buffer-local variables. While we are at it, let’s set it as environment variables too. Then everyone can pick it up: 

(defun eshell/set (&rest args)
  "Creates a buffer local variable.
The first parameters of ARGS is the name of the variable.
The other parameters are the values. If not given, the
variable is deleted."
  (let* ((var (car args))
         (var-sym (make-symbol var))
         ;; Convert value to a string
         (val (pcase (seq-length (cdr args))
                (0 nil)
                (1 (format "%s" (cadr args)))
                (_ (thread-last (cdr args)
                                (seq-map 'eshell-stringify)
                                (s-join " "))))))
    (if val
        (progn
          (set (make-local-variable var-sym) val)
          (setenv var val))

      ;; If we don't get both a variable and a value, let's try to
      ;; delete the variable:
      (makunbound var-sym)
      (setenv var))))

Something like: 

$ set a 42 b 7

Would create a single variable, a, set to the value of the string, 42 b 7 which is similar to how the Fish shell works.

Less and More

While I can type find-file, I often use e as an alias for emacsclient in Terminals, so let’s do something similar for eshell: Also note that we can take advantage of the eshell-fn-on-files function to expand the find-file (which takes one argument), to open more than one file at one time. 

(defun eshell/e (&rest files)
  "Essentially an alias to the `find-file' function."
  (eshell-fn-on-files 'find-file 'find-file-other-window files))

(defun eshell/ee (&rest files)
  "Edit one or more files in another window."
  (eshell-fn-on-files 'find-file-other-window 'find-file-other-window files))

No way would I accidentally type any of the following commands: 

(defalias 'eshell/emacs 'eshell/e)
(defalias 'eshell/vi 'eshell/e)
(defalias 'eshell/vim 'eshell/e)

Both less and more are the same to me. as I want to scroll through a file. Sure the bat program is cool, but from eshell, we could call view-file, and hit q to quit and return to the shell. 

(defun eshell/less (&rest files)
  "Essentially an alias to the `view-file' function."
  (eshell-fn-on-files 'view-file 'view-file-other-window files))

Do I type more any more than less? 

(defalias 'eshell/more 'eshell/less)
(defalias 'eshell/view 'eshell/less)

Ebb and Flow output to Emacs Buffers

This is an interesting experiment.

Typing a command, but the output isn’t right. So you punch the up arrow, and re-run the command, but this time pass the output through executables like tr, grep, and even awk. Still not right? Rinse and repeat. Tedious. Since using Emacs to edit text is what we do best, what if we took the output of a command from Eshell, edit that output in a buffer, and then use that edited output in further commands?

I call this workflow of sending command output back and forth into an Emacs buffer, an ebb and flow approach, where the ebb function (for Edit a Bumped Buffer … or something like that), takes some command output, and opens it in a buffer (with an ebbflow minor mode), allowing us to edit or alter the data. Pull that data back to the Eshell session with the flow function (for Fetch buffer data by Lines or Words … naming is hard).

The ebbflow Buffer

If I don’t specify a specific buffer name, we use this default value: 

(defvar ha-eshell-ebbflow-buffername "*eshell-edit*"
  "The name of the buffer that eshell can use to store temporary input/output.")

This buffer has a minor-mode that binds C-c C-q to close the window and return to the Eshell that spawned it: 

(defun ha-eshell-ebbflow-return ()
  "Close the ebb-flow window and return to Eshell session."
  (interactive)
  (if (and (boundp 'ha-eshell-ebbflow-return-buffer)
           (bufferp 'ha-eshell-ebbflow-return-buffer))
      (pop-to-buffer ha-eshell-ebbflow-return-buffer)
    (bury-buffer)))

(define-minor-mode ebbflow-mode
  "Editing a flow from the Eshell ebb command, so flow can pull it back."
  :lighter " ebb"
  :keymap (let ((map (make-sparse-keymap)))
            (define-key map (kbd "C-c C-q") 'ha-eshell-ebbflow-return)
            map))

Since I use Evil, I also add Q to call this function: 

(when (fboundp 'evil-define-key)
  (evil-define-key 'normal ebbflow-mode-map "Q" 'ha-eshell-ebbflow-return))

flow (or Buffer Cat)

Eshell can send the output of a command sequence to a buffer: 

rg -i red > #<scratch>

But I can’t find a way to use the contents of buffers to use as part of the standard input to another as the start of a pipeline. Let’s create a function to fetch buffer contents.

I’m calling the ability to get a buffer contents, flow (Fetch contents as Lines Or Words). While this function will fetch the contents of any buffer, if one is not given, it will fetch the default, ha-eshell-ebbflow-buffername. Once the content is fetched, given the correct argument, it may convert the data:

as lines
separating the data on newlines. Useful for passing to for loops
as words
separating on spaces. Useful if the data is filenames
as a string
no conversion
(defun eshell/flow (&rest args)
  "Output the contents of one or more buffers as a string.
Usage: flow [OPTION] [BUFFER ...]
    -h, --help           show this usage screen
    -l, --lines          output contents as a list of lines
    -w, --words          output contents as a list of space-separated elements "
  (let* ((options (eshell-getopts '((:name words  :short "w" :long "words")
                                    (:name lines  :short "l" :long "lines")
                                    (:name string :short "s" :long "string")
                                    (:name help   :short "h" :long "help"
                                           :help eshell/flow))
                                  args))
         (buffers (gethash 'parameters options))
         (content (thread-last buffers
                               (-map 'eshell-flow-buffer-contents)
                               (s-join "\n"))))
    (if (gethash 'help options)
        (error (documentation 'eshell/flow))

      ;; No buffer specified? Use the default buffer's contents:
      (unless buffers
        (setq content
              (eshell-flow-buffer-contents ha-eshell-ebbflow-buffername)))

      ;; Do we need to convert the output to lines or split on words?
      (cond
       ((gethash 'words options) (split-string content))
       ((gethash 'lines options) (split-string content "\n"))
       (t                        content)))))

Straight-forward to acquire the contents of a buffer : 

(defun eshell-flow-buffer-contents (buffer-name)
  "Return the contents of BUFFER as a string."
  (when buffer-name
    (save-window-excursion
      (switch-to-buffer (get-buffer buffer-name))
      (buffer-substring-no-properties (point-min) (point-max)))))

Specify the buffers with either the Eshell approach, e.g. #<buffer buffer-name>, or a string, ’*scratch*’, and if I don’t specify any buffer, we’ll use the default buffer: 

(defun eshell-flow-buffers (buffers)
  "Convert the list, BUFFERS, to actual buffers if given buffer names."
  (if buffers
      (--map (cond
              ((bufferp it) it)
              ((stringp it) (get-buffer it))
              (t            (error (format "Illegal argument of type %s: %s\n%s"
                                           (type-of arg) it
                                           (documentation 'eshell/flow)))))
             buffers)
    ;; No buffers given? Use the default buffer:
    (list (get-buffer ha-eshell-ebbflow-buffername))))

I used to call this function, bcat (for buffer cat), and I sometimes type this: 

(defalias 'eshell/bcat 'eshell/flow)

ebb: Bump Data to a Buffer

The ebb function puts content into the ebbflow buffer. Any content given to it on the command line is placed into the buffer, for instance:

ebb foobar
replaces the contents of the buffer with the text, foobar
ebb -p foobar
adds the text at the beginning

We have three separate use-cases:

  1. Execute a command, inserting the output into the buffer (good if we know the output will be long, complicated, or needing manipulation)
  2. Insert one or more files into the buffer (this assumes the files are data)
  3. Grab the output from the last executed Eshell command (what happens when we don’t give it a command string or files to read)
(defun eshell/ebb (&rest args)
  "Insert text content into *eshell-edit* buffer, or if not text is given, the output of last command.
Usage: ebb [OPTION] [text content]
    -h, --help    show this usage screen
    -m, --mode    specify the major-mode for the *eshell-edit* buffer, e.g. json
    -n, --newline separate the text contents by newlines (this is default)
    -s, --spaces  separate the text contents by spaces, instead of newlines
    -b, --begin   add text content to the beginning of the *eshell-edit* buffer
    -e, --end     add text content to the end of *eshell-edit* buffer
    -i, --insert  add text content to *eshell-edit* at point"
  (let* ((options  (eshell-getopts '((:name insert      :short "i" :long "insert")
                                     (:name append      :short "e" :long "end")
                                     (:name prepend     :short "b" :long "begin")
                                     (:name newline     :short "n" :long "newline")
                                     (:name spaces      :short "s" :long "spaces")
                                     (:name mode-option :short "m" :long "mode" :parameter string)
                                     (:name help        :short "h" :long "help"
                                            :help eshell/ebb))
                                   args))
         (location (cond
                    ((gethash 'insert  options) :insert)
                    ((gethash 'append  options) :append)
                    ((gethash 'prepend options) :prepend)
                    (t                          :replace)))
         (params   (gethash 'parameters options)))

    (if (seq-empty-p params)
        ((ha-eshell-ebb-output location))
      (ha-eshell-ebb-string location (gethash 'spaces options) params))

    ;; At this point, we are in the `ha-eshell-ebbflow-buffername', and
    ;; the buffer contains the inserted data. Did we specify a major-mode?
    (when-let ((mode-option (gethash 'mode-option options)))
      (if (s-starts-with? "js" mode-option)
          (js-json-mode)  ; Or should we just go to json-ts-mode?
        (funcall (intern (concat mode-option "-mode")))))

    ;; Flip on the minor mode-option so we can close the window later on:
    (ebbflow-mode +1)
    (goto-char (point-min)))

  nil) ; Return `nil' so that it doesn't print anything in `eshell'.

Each of the use-case functions described needs to switch to the *eshell-edit* buffer, and either clear it out or position the cursor. 

(defun ha-eshell-ebb-switch-to-buffer (insert-location)
  "Switch to `ha-eshell-ebbflow-buffername' and get the buffer ready for new data."
  (let ((return-buffer (current-buffer)))

    (if-let ((ebbwindow (get-buffer-window ha-eshell-ebbflow-buffername)))
        (select-window ebbwindow)
      (switch-to-buffer ha-eshell-ebbflow-buffername)
      (setq-local ha-eshell-ebbflow-close-window t))

    (setq-local ha-eshell-ebbflow-return-buffer return-buffer)
    (ebbflow-mode)

    (cl-case insert-location
      (:append  (goto-char (point-max)))
      (:prepend (goto-char (point-min)))
      (:insert   nil)
      (:replace (delete-region (point-min) (point-max))))))

One way to call ebb is with a command wrapped in braces, e.g. ebb { ls -1 }, which calls this function, as the output from the { … } sub-shell is passed as arguments to the ebb command, and appears as command-results: 

(defun ha-eshell-ebb-string (insert-location space-separator-p command-results)
  "Insert the COMMAND-RESULTS into the `ha-eshell-ebbflow-buffername`.
Contents are placed based on INSERT-LOCATION and, if given, separated
by SEPARATOR (which defaults to a space)."
  (let* ((sep (if space-separator-p " " "\n"))
         (str (string-join (-flatten command-results) sep)))
    (ha-eshell-ebb-switch-to-buffer insert-location)
    (insert str)))

Command string passed to eshell-command: 

(defun ha-eshell-ebb-command (insert-location command-parts)
  "Call `eshell-command' with the COMMAND-PARTS.
Inserts the output into `ha-eshell-ebbflow-buffername'"
  (let ((command-string (string-join command-parts " ")))
    (ha-eshell-ebb-switch-to-buffer insert-location)
    (eshell-command command-string t)))

Given one or more filenames to the ebb command, concatenates each into the buffer. 

(defun ha-eshell-ebb-files (insert-location files)
  "Insert the FILES at the INSERT-LOCATION tin `ha-eshell-ebbflow-buffername'."
  (ha-eshell-ebb-switch-to-buffer insert-location)
  (dolist (file files)
    (insert-file file)
    (insert "\n")))

If we were not given a command to execute or a list of files to insert, we want to grab the output from the last executed command in the eshell buffer. To do this, we need to move to the start of the output, and then search for the prompt. Luckily Eshell assumes we have set up the eshell-prompt-regexp variable: 

(defun ha-eshell-last-output ()
  "Return contents of the last command execusion in an Eshell buffer."
  (let ((start  (save-excursion
                   (goto-char eshell-last-output-start)
                   (re-search-backward eshell-prompt-regexp)
                   (next-line)
                   (line-beginning-position)))
        (end    eshell-last-output-start))
    (buffer-substring-no-properties start end)))

(defun ha-eshell-ebb-output (insert-location)
  "Grab output from previous eshell command, inserting it into our buffer.
Gives the INSERT-LOCATION to `ha-eshell-ebb-switch-to-buffer'."
  (let ((contents (ha-eshell-last-output)))
    (ha-eshell-ebb-switch-to-buffer insert-location)
    (insert contents)))

Using the Ebb and Flow Functions

In Summary, to place the output of a command in an editable buffer, either begin command with ebb, like: 

ebb make image-status

Or run the command, as normal, and then call ebb without any parameters to grab the output of the last command.

Note, you can run additional commands to add to the *eshell-edit* buffer, calling ebb with one of these parameters:

-a
append the command output to the buffer
-p
prepend the output to the buffer
-i
insert at the current point position in the buffer

After altering the *eshell-edit* buffer, use flow to pull it back, as in: 













X, Copies the Spot





The x command extracts a piece of information from the output of the previous command in an Eshell buffer.
Perhaps an example is in order:



$ make image-id
Job ID is 16a6df20-7a9e-491a-8e87-39964c8ead4e
Image ID is 4c4d8e93-dac5-4e39-95f7-a568689102e2
make: 'image-id' is up to date.

$ image details { x 2 4 }




In this case, it took the 4th column in the 2nd row of output, and returns the UUID, 4c4d8e93-dac5-4e39-95f7-a568689102e2.






(defun eshell/x (&rest args)
  "Return a cell of information from the previous command in an Eshell buffer.
The first ARGS is the line number (one-based), and the second
ARGS, if given, is the column where the fields are separated by
whitespace.

This allows a sequence of commands like, where you don't have to
copy/paste the output (if it is simple), for instance:

    $ ls
    ...
    $ ls -l { x 2 3 }

If the initial argument is a string instead of a number, then it
returns the first word that starts with that it."
  (defun x-cells (table row col)
    (let* ((newlines (rx (one-or-more (any "\n" "\r"))))
           (fields   (rx (one-or-more (any "\t" " "))))
           (rows     (split-string table newlines t))
           (line     (nth row rows)))
      (if col
        (nth col (split-string line fields t))
      line)))

  (defun x-match (text starter)
    (let ((words (split-string text nil t)))
      (--first (s-starts-with? starter it) words)))

  (let* ((arg1     (first args))
         (arg2     (second args))
         (contents (ha-eshell-last-output)))
    (cond
     ((numberp arg1) (x-cells contents arg1 arg2))
     ((stringp arg1) (x-match contents arg1))
     (t              contents))))











Git





I used to have a number g-prefixed aliases to call git-related commands, but now, I call instead. My gst command is an alias to magit-status, but using the alias doesn’t pull in the current working directory, so I make it a function, instead:





(defun eshell/gst (&rest args)
    (magit-status (pop args) nil)
    (eshell/echo))   ;; The echo command suppresses output











Replace ls





I like the output of the lsd program, and want ls to call it, if available.





(defvar ha-lsd (ha-find-executable "lsd")
  "Location of the `lsd' program, if installed.")







The problem I have with lsd is that it does not display in columns or colorize its output in eshell (even when changing the TERM variable). Since I already wrote this code, I’m re-purposing it and expanding it. Step one is to have a function that gives a list of files for a directory (notice it doesn’t take options, for if I am going for special output, I’ll be calling ls directly).





(defun ha-eshell-ls-files (&optional directory)
  "Return a list of directories in DIRECTORY or `default-directory' if null."
  (let ((default-directory (or directory default-directory)))
    (if ha-lsd
        (shell-command-to-list (format "%s --icon always" ha-lsd))

      (directory-files default-directory nil
                       (rx string-start
                           (not (any "." "#"))
                           (one-or-more any)
                           (not "~")
                           string-end)))))







Given a filename, let’s pad and colorize it based on file attributes:





(defun ha-eshell-ls-filename (filename padded-fmt &optional directory)
  "Return a prettized version of FILE based on its attributes.
Formats the string with PADDED-FMT."
  (let ((file (expand-file-name (if (string-match (rx (group alpha (zero-or-more any))) filename)
                                    (match-string 1 filename)
                                  filename)
                                directory))
        (import-rx  (rx "README"))
        (image-rx   (rx "." (or "png" "jpg" "jpeg" "tif" "wav") string-end))
        (code-rx    (rx "." (or "el" "py" "rb") string-end))
        (docs-rx    (rx "." (or "org" "md") string-end)))
    (format padded-fmt
            (cond
             ((file-directory-p file)
              (propertize filename 'face 'eshell-ls-directory))
             ((file-executable-p file)
              (propertize filename 'face 'eshell-ls-executable))
             ((string-match import-rx file)
              (propertize filename 'face '(:foreground "orange")))
             ((string-match image-rx file)
              (propertize filename 'face 'eshell-ls-special))
             ((file-symlink-p file)
              (propertize filename 'face 'eshell-ls-symlink))
             ((not (file-readable-p file))
              (propertize filename 'face 'eshell-ls-unreadable))
             (t
              filename)))))







This function pulls all the calls to ha-eshell-ls-file to create columns to make a multi-line string:





(defun ha-eshell-ls (&optional directory)
  "Return a formatted string of files for a directory.
The string is a pretty version with columns and whatnot."
  (let* ((files   (ha-eshell-ls-files (or directory default-directory)))
         (longest (--reduce-from (max acc (length it)) 1 files))
         (width   (window-total-width))
         (columns (/ width (+ longest 3)))
         (padded  (if ha-lsd
                      (format "%%-%ds " longest)
                    (format "• %%-%ds " longest))))
    (cl-flet* ((process-lines (files)
                              (s-join "" (--map (ha-eshell-ls-filename it padded directory) files)))
               (process-files (table)
                              (s-join "\n" (--map (process-lines it) table))))

      (concat (process-files (seq-partition files columns)) "\n\n"))))







While the ha-eshell-ls takes a directory, this version puts the canonical directory as a label before the listing, and this calls it directly specifying the directory name(s):





(defun ha-eshell-ls-directory (directory)
  "Print the DIRECTORY name and its contents."
  (let ((dir (file-truename directory)))
    (concat
     (propertize dir 'face '(:foreground "gold" :underline t))
     ":\n"
     (ha-eshell-ls dir))))






I have the interface program to work with eshell.





(defun eshell/lsd (&rest args)
  (let ((lsd (ha-find-executable "lsd")))
    (cond
     ;; I expect to call this function without any arguments most of the time:
     ((and lsd (null args))
      (ha-eshell-ls))
     ;; Called with other directories? Print them all, one at a time:
     ((and lsd (--none? (string-match (rx string-start "-") it) args))
      (mapconcat 'ha-eshell-ls-directory args ""))
     ;; Calling the function with -l or other arguments, don't bother. Call ls:
     (t (eshell/ls args)))))







Which needs an ls alias:





;; (eshell/alias "lss" "echo $@")











Regular Expressions





I think using the rx macro with applications like grep is great reason why eshell rocks. Assuming we can’t remember cryptic regular expression syntax, we could look for a GUID-like strings using ripgrep with:





$ rg (rx (one-or-more hex) "-" (one-or-more hex))






The problem with this trick is that rx outputs an Emacs-compatible regular expression, which doesn’t always match regular expressions accepted by most applications.





The pcre2el project can convert from a Lisp regular expression to a PCRE (Perl Compatible Regular Expression), acceptable by ripgrep.





(use-package pcre2el
  :straight (:host github :repo "joddie/pcre2el")
  :config
  (defmacro prx (&rest expressions)
    "Convert the rx-compatible regular EXPRESSIONS to PCRE.
  Most shell applications accept Perl Compatible Regular Expressions."
    `(rx-let ((integer (1+ digit))
              (float   (seq integer "." integer))
              (b256    (seq (optional (or "1" "2"))
                            (regexp "[0-9]\\{1,2\\}")))
              (ipaddr  (seq b256 "." b256 "." b256 "." b256))
              (time    (seq digit (optional digit) ":" (= 2 digit) (optional ":" (= 2 digit))))
              (email   (seq (1+ (regexp "[^,< ]")) "@" (1+ (seq (1+ (any alnum "-"))) ".") (1+ alnum)))
              (date    (seq (= 2 digit) (or "/" "-") (= 2 digit) (or "/" "-") (= 4 digit)))
              (ymd     (seq (= 4 digit) (or "/" "-") (= 2 digit) (or "/" "-") (= 2 digit)))
              (uuid    (seq (= 8 hex) "-" (= 3 (seq (= 4 hex) "-")) (= 12 hex)))
              (guid    (seq uuid)))
       (rxt-elisp-to-pcre (rx ,@expressions)))))











Map over Files





While I like eshell’s for loop well enough (if I can remember the syntax), as in:





for file in *.org {
  chmod a+x $file
}






I like the idea of using a map structure, for instance, wouldn’t it be cool to type something like:





do chmod a+x *.org






How would this work without special syntax? Well, eshell sends the *.org as a list of files, which we could use as the delimiter. The downside is that we want to list the files, we need to actually list the files, as in:





do chmod a+x (list "a.org" "c.org")






Pretty ugly, but what about using :: as a separator of the lambda from the list, like:





do chmod a+x :: *.org b.txt







Here is my initial function. After separating the arguments into two groups (split on the :: string), we iterate over the file elements, creating a form that includes the filename.





(defun eshell/do (&rest args)
  "Execute a command sequence over a collection of file elements.
Separate the sequence and the elements with a `::' string.
For instance:

    do chown _ angela :: *.org(u'oscar')

The function substitutes the `_' sequence to a single filename
element, and if not specified, it appends the file name to the
command. So the following works as expected:

    do chmod a+x :: *.org"
  (seq-let (forms elements) (-split-on "::" args)
    (dolist (element (-flatten (-concat elements)))
      (message "Working on %s ... %s" element forms)
      (let* ((form (if (-contains? forms "_")
                       (-replace "_" element forms)
                     (-snoc forms element)))
             (cmd  (car form))
             (args (cdr form)))
        (eshell-named-command cmd args)))))






The eshell-named-command takes the command separately from the arguments, so we use car and cdr on the form.









Last Results





The shell-underscore project looks pretty cool, where the _ character represents a filename with the contents of the previous command (you know, like if you were planning on it, you’d tee at the end of every command). An interesting idea that I could duplicate.





While diving into the eshell source code, I noticed the special variables, $$ and $_ sometimes contains the output of the last command. For instance:



$ echo "hello world"
hello world
$ echo $$
hello world




What I would like is something like this to work:



$ ls *.org(U)
a.org b.org f.org
$ rg "foobar" $$





The problem may be between calling Emacs functions versus external commands, as the echo works, but the call to ls doesn’t:



$ ls *.org(U) b.txt
a.org b.org f.org b.txt

$ echo Nam $$
("Nam" nil)





I over-write that special variables to behave as expected:



    

  • A hook runs after every command
    • 

  • It copies the previous command’s output to a ring (so that I can get the last as well as the fifth one)
    • 

  • Create a replacement function for $$ to read from my history ring
    • 





Let’s first make a ring that stores the output:





(defvar ha-eshell-output (make-ring 10)
  "A ring (looped list) storing history of eshell command output.")







The following function does the work of saving the output of the last command. We can get this because after every command, eshell updates two variables, eshell-last-input-end (the start of the output), and eshell-last-output-start (the end of the output):





(defun ha-eshell-store-last-output ()
  "Store the output from the last eshell command.
Called after every command by connecting to the `eshell-post-command-hook'."
  (let ((output
         (buffer-substring-no-properties eshell-last-input-end eshell-last-output-start)))
    (ring-insert ha-eshell-output output)))







Now we save this output after every command by adding it to the eshell-post-command-hook:





(add-hook 'eshell-post-command-hook 'ha-eshell-store-last-output)







Next, this function returns values from the history ring. I feel the need to have different ways of returning the output data.
Unlike the behavior of the original shell (and most of its descendents, like bash), eshell doesn’t automatically split on whitespace. For instance, echo called this way:



$ echo a b *.txt
("a" "b"
 ("b.txt" "date today.txt"))




Given a list of three elements: a, b, and a list of all files in the current directory with an .org extension. An interesting side-effect is that spaces in filenames are often okay. If I specify and argument of text, it should return the command’s output as a string, but if I give it, list, it should contain the same information, but separated by spaces, into a list. For instance, if we are passing the output from ls to grep, we would use this format.





Like the shell-underscore project mentioned earlier, I can access the output stored from a file when given a file argument (the output will hold this temporary filename).





(defun eshell/output (&rest args)
  "Return an eshell command output from its history.

The first argument is the index into the historical past, where
`0' is the most recent, `1' is the next oldest, etc.

The second argument represents the returned output:
 * `text' :: as a string
 * `list' :: as a list of elements separated by whitespace
 * `file' :: as a filename that contains the output

If the first argument is not a number, it assumes the format
to be `:text'.
"
  (let (frmt element)
    (cond
     ((> (length args) 1)  (setq frmt (cadr args)
                                 element (car args)))
     ((= (length args) 0)  (setq frmt "text"
                                 element 0))
     ((numberp (car args)) (setq frmt "text"
                                 element (car args)))
     ((= (length args) 1)  (setq frmt (car args)
                                 element 0)))

    (if-let ((results (ring-ref ha-eshell-output (or element 0))))
        (cl-case (string-to-char frmt)
          (?l     (split-string results))
          (?f     (ha-eshell-store-file-output results))
          (otherwise (s-trim results)))
      "")))

(defun ha-eshell-store-file-output (results)
  "Writes the string, RESULTS, to a temporary file and returns that file name."
  (let ((filename (make-temp-file "ha-eshell-")))
    (with-temp-file filename
      (insert results))
    filename))







How would this function work in practice?



$ ls
a.org  b.txt  c.org  date today.txt  ever

$ output
a.org  b.txt  c.org  date today.txt  ever

$ echo { output list }
("a.org" "b.txt" "c.org" "date" "today.txt" "ever")




Notice how commands between { … } are eshell commands, otherwise, if I replace the braces with parens, I would have to write eshell/output. Let’s try the history feature:



$ echo "oldest"
oldest

$ echo "old"
old

$ echo "recent"
recent

$ echo "newest"
newest

$ echo { output 2 }
old





Eshell has a feature where special variables (stored in eshell-variable-aliases-list), can be a function.  The $$ holds text-formatted output, and $_ contains list-formatted output, and $OUTPUT can be the output stored in a file.





(with-eval-after-load "eshell"
  (defvar eshell-variable-aliases-list nil "Autoloading this eshell-defined variable")
  (add-to-list 'eshell-variable-aliases-list '("$"  ha-eshell-output-text))
  (add-to-list 'eshell-variable-aliases-list '("_"  ha-eshell-output-list))
  (add-to-list 'eshell-variable-aliases-list '("OUTPUT" ha-eshell-output-file)))






Without this change, the $$ variable calls eshell-last-command-result, where I believe my version (with history) may work more reliably. I define these helper functions:





(defun ha-eshell-output (format-type indices)
  "Wrapper around `eshell/output' for the `eshell-variable-aliases-list'."
  (if indices
      (eshell/output (string-to-number (caar indices)) format-type)
    (eshell/output 0 format-type)))

(defun ha-eshell-output-text (&optional indices &rest ignored)
  "A _text_ wrapper around `eshell/output' for the `eshell-variable-aliases-list'."
  (ha-eshell-output "text" indices))

(defun ha-eshell-output-list (&optional indices &rest ignored)
  "A _list_ wrapper around `eshell/output' for the `eshell-variable-aliases-list'."
  (ha-eshell-output "list" indices))

(defun ha-eshell-output-file (&optional indices &rest ignored)
  "A _file_ wrapper around `eshell/output' for the `eshell-variable-aliases-list'."
  (ha-eshell-output "file" indices))







How would this look? Something like:



$ echo a
a
$ echo b
b
$ echo c
c
$ echo $$
c
$ echo $$[2]
b





The final trick is being able to count backwards and remember they are always shifting. I guess if I wanted to remember the output for more than one command, I could do:



$ ls *.org(U) b.txt
a.org  b.txt

$ chmod o+w $_

$ rg Nam $_[1]
a.org
8:Nam vestibulum accumsan nisl.

b.txt
1:Nam euismod tellus id erat.
7:Name three animals that start with C




Wanna see something cool about Eshell? Let’s swirl Lisp and Shell commands:



$ rg (rx line-start "Nam ") $_[2]
b.txt
1:Nam euismod tellus id erat.

a.org
8:Nam vestibulum accumsan nisl.









Engineering Notebook





I want both the command and the output (as well as comments) to be able to go into an org-mode file, I call my engineering notebook. Where in that file? If I use en that goes in a “General Notes” section, and ec goes into the currently clocked in task in that file.





I use ex to refer to both en / ec. Use cases:





ex <command>
run the command given and send the output to the notebook


ex [-n #]
grab the output from a previously executed command (defaults to last one)


ex -c "<comment>" <command>
run command and write the comment to the current date in the notebook


ex <command> :: <comment>
run command and write comment to the notebook


<command> > ex
write output from command to the notebook. This won’t add the command that generated the output.






The -c option can be combined with the command, but I don’t want it to grab the last output, as I think I would just like to send text to the notebook as after thoughts. If the option to -c is blank, perhaps it just calls the capture template to allow me to enter voluminous content.





This requires capture templates that don’t do any formatting. I will reused c c from capturing-notes code, and create other templates under e prefix:





;; (setq org-capture-templates nil)
(add-to-list 'org-capture-templates
             '("e" "Engineering Notebook"))

(add-to-list 'org-capture-templates
             '("ee" "Notes and Commentary" plain
               (file+olp+datetree org-default-notes-file "General Notes")
               "%i" :empty-lines 1 :tree-type month :unnarrowed t))

(add-to-list 'org-capture-templates
             '("ef" "Piped-in Contents" plain
               (file+olp+datetree org-default-notes-file "General Notes")
               "%i" :immediate-finish t :empty-lines 1 :tree-type month))








(defun ha-eshell-engineering-notebook (capture-template args)
  "Capture commands and output from Eshell into an Engineering Notebook.

Usage:   ex [ options ] [ command string ] [ :: prefixed comments ]]

A _command string_ is an eshell-compatible shell comman to run,
and if not given, uses previous commands in the Eshell history.

Options:
  -c, --comment   A comment string displayed before the command
  -n, --history   The historical command to use, where `0' is the
                  previous command, and `1' is the command before that.
  -t, --template  The `keys' string to specify the capture template"
  (let* (output
         (options  (eshell-getopts
                    '((:name comment :short "c" :long "comment" :parameter string)
                      (:name history :short "n" :long "history" :parameter integer)
                      (:name captemp :short "t" :long "template" :parameter string)
                      (:name interact :short "i" :long "interactive")
                      (:name help    :short "h" :long "help"
                             :help ha-eshell-engineering-notebook))
                    args))
         (sh-call  (gethash 'parameters options))
         (sh-parts (-split-on "::" sh-call))
         (command  (s-join " " (first sh-parts)))
         ;; Combine the -c parameter with text following ::
         (comment  (s-join " " (cons (gethash 'comment options)
                                     (second sh-parts))))
         (history  (or (gethash 'history options) 0)))

    ;; Given a -t option? Override the default:
    (when (gethash 'captemp options)
      (setq capture-template (gethash 'captemp options)))

    (when (gethash 'interact options)
      (setq capture-template "ee"))

    (cond
     (sh-call   ; Gotta a command, run it!
      (ha-eshell-engineering-capture capture-template comment command
                                     (eshell-command-to-string (first sh-parts))))
     (t         ; Otherwise, get the history
      (ha-eshell-engineering-capture capture-template comment
                                     (ring-ref eshell-history-ring (1+ history))
                                     (eshell/output history))))))

(defun ha-eshell-engineering-capture (capture-template comment cmd out)
  "Capture formatted string in CAPTURE-TEMPLATE.
Base the string created on COMMENT, CMD, and OUT. Return OUTPUT."
  (let* ((command (when cmd (s-trim cmd)))
         (output  (when out (s-trim out)))
         (results (concat
                   (when comment (format "%s\n\n" comment))
                   (when command (format "#+begin_src shell\n  %s\n#+end_src\n\n" command))
                   (when (and command output) "#+results:\n")
                   (when output  (format "#+begin_example\n%s\n#+end_example\n" output)))))

    (message results)
    (org-capture-string results capture-template)

    ;; Return output from the command, or nothing if there wasn't anything:
    (or output "")))






And now we have a en and a ec version:





(defun eshell/en (&rest args)
  "Call `ha-eshell-engineering-notebook' to \"General Notes\"."
  (interactive)
  (ha-eshell-engineering-notebook "ef" args))

(defun eshell/ec (&rest args)
  "Call `ha-eshell-engineering-notebook' to current clocked-in task."
  (interactive)
  (ha-eshell-engineering-notebook "cc" args))







This function simply calls with a template:





(defun eshell/cap (&rest args)
  "Call `org-capture' with the `ee' template to enter text into the engineering notebook."
  (org-capture nil "ee"))








(defun ha-eshell-target-engineering-notebook (output)
  "Write OUTPUT into the engineering notebook via `org-capture'."
  (ha-eshell-engineering-capture "ef" nil nil output))

(defun ha-eshell-target-clocked-in-task (output)
  "Write OUTPUT into the current clocked in task via `org-capture'."
  (ha-eshell-engineering-capture "cc" nil nil output))






And finally, add our new functions to :





(with-eval-after-load "eshell"
  (add-to-list 'eshell-virtual-targets '("/dev/e" ha-eshell-target-engineering-notebook nil))
  (add-to-list 'eshell-virtual-targets '("/dev/c" ha-eshell-target-engineering-notebook nil)))













EAT and Eshell





The Emulate a Terminal project provides flicker-free, perfect display, of visual commands in Eshell, eliminating one of my primary issue with using Eshell all the time.  (Check out Akib Azmain Turja’s talk at EmacsConf2023).






(use-package eat
  :after eshell
  :straight (:repo "https://codeberg.org/akib/emacs-eat")
  :hook (eshell-load . #'eat-eshell-visual-command-mode))







Note that the eat-eshell-visual-command-mode also kicks off the global minor mode, eat-eshell-mode. The big advantage of Eat is the three input modes, however, in Eshell with Evil, I can just type Escape to go into Emacs Mode, and G A to return to typing Terminal commands.









Special Prompt





Following these instructions, we build a better prompt with the Git branch in it (Of course, it matches my Bash prompt). First, we need a function that returns a string with the Git branch in it, e.g. “:master”





(defun curr-dir-git-branch-string (pwd)
  "Returns current git branch as a string, or the empty string if
PWD is not in a git repo (or the git command is not found)."
  (interactive)
  (when (and (not (file-remote-p pwd))
             (eshell-search-path "git")
             (locate-dominating-file pwd ".git"))
    (let* ((git-url    (shell-command-to-string "git config --get remote.origin.url"))
           (git-repo   (file-name-base (s-trim git-url)))
           (git-output (shell-command-to-string (concat "git rev-parse --abbrev-ref HEAD")))
           (git-branch (s-trim git-output))
           (git-icon   "\xe0a0")
           (git-icon2  (propertize "\xf020" 'face `(:family "octicons"))))
      (concat git-repo " " git-icon2 " " git-branch))))







The function takes the current directory passed in via pwd and replaces the $HOME part with a tilde. I’m sure this function already exists in the eshell source, but I didn’t find it…





(defun pwd-replace-home (pwd)
  "Replace home in PWD with tilde (~) character."
  (interactive)
  (let* ((home (expand-file-name (getenv "HOME")))
         (home-len (length home)))
    (if (and
         (>= (length pwd) home-len)
         (equal home (substring pwd 0 home-len)))
        (concat "~" (substring pwd home-len))
      pwd)))







Make the directory name be shorter… by replacing all directory names with its first names. We leave the last two to be the full names. Why yes, I did steal this.





(defun pwd-shorten-dirs (pwd)
  "Shorten all directory names in PWD except the last two."
  (let ((p-lst (split-string pwd "/")))
    (if (> (length p-lst) 2)
        (concat
         (mapconcat (lambda (elm) (if (zerop (length elm)) ""
                               (substring elm 0 1)))
                    (butlast p-lst 2)
                    "/")
         "/"
         (mapconcat (lambda (elm) elm)
                    (last p-lst 2)
                    "/"))
      pwd)))  ;; Otherwise, we return the PWD







Break up the directory into a “parent” and a “base”:





(defun split-directory-prompt (directory)
  (if (string-match-p ".*/.*" directory)
      (list (file-name-directory directory) (file-name-base directory))
    (list "" directory)))







Using virtual environments for certain languages is helpful to know, since I change them based on the directory.





(defun ruby-prompt ()
  "Returns a string (may be empty) based on the current Ruby Virtual Environment."
  (let* ((executable "~/.rvm/bin/rvm-prompt")
         (command    (concat executable "v g")))
    (when (file-exists-p executable)
      (let* ((results (shell-command-to-string executable))
             (cleaned (string-trim results))
             (gem     (propertize "\xe92b" 'face `(:family "alltheicons"))))
        (when (and cleaned (not (equal cleaned "")))
          (s-replace "ruby-" gem cleaned))))))

(defun python-prompt ()
  "Returns a string (may be empty) based on the current Python
   Virtual Environment. Assuming I've called the M-x command:
   `pyenv-mode-set'."
  (when (fboundp #'pyenv-mode-version)
    (let ((venv (pyenv-mode-version)))
      (when venv
        (concat
         (propertize "\xe928" 'face `(:family "alltheicons"))
         (pyenv-mode-version))))))







Now tie it all together with a prompt function can color each of the prompts components.





(defun eshell/eshell-local-prompt-function ()
  "A prompt for eshell that works locally (in that it assumes it
could run certain commands) to make a prettier, more-helpful
local prompt."
  (interactive)
  (let* ((pwd        (eshell/pwd))
         (directory (split-directory-prompt
                     (pwd-shorten-dirs
                      (pwd-replace-home pwd))))
         (parent (car directory))
         (name   (cadr directory))
         (branch (curr-dir-git-branch-string pwd))
         (ruby   (when (not (file-remote-p pwd)) (ruby-prompt)))
         (python (when (not (file-remote-p pwd)) (python-prompt)))

         (dark-env (eq 'dark (frame-parameter nil 'background-mode)))
         (for-bars                 `(:weight bold))
         (for-parent  (if dark-env `(:foreground "dark orange") `(:foreground "blue")))
         (for-dir     (if dark-env `(:foreground "orange" :weight bold)
                        `(:foreground "blue" :weight bold)))
         (for-git                  `(:foreground "green"))
         (for-ruby                 `(:foreground "red"))
         (for-python               `(:foreground "#5555FF")))

    (concat
     (propertize "⟣─ "    'face for-bars)
     (propertize parent   'face for-parent)
     (propertize name     'face for-dir)
     (when branch
       (concat (propertize " ── "    'face for-bars)
               (propertize branch   'face for-git)))
     ;; (when ruby
     ;;   (concat (propertize " ── " 'face for-bars)
     ;;           (propertize ruby   'face for-ruby)))
     ;; (when python
     ;;   (concat (propertize " ── " 'face for-bars)
     ;;           (propertize python 'face for-python)))
     (propertize "\n"     'face for-bars)
     (propertize (if (= (user-uid) 0) " #" " $") 'face `(:weight ultra-bold))
     ;; (propertize " └→" 'face (if (= (user-uid) 0) `(:weight ultra-bold :foreground "red") `(:weight ultra-bold)))
     (propertize " "    'face `(:weight bold)))))

(setq-default eshell-prompt-function #'eshell/eshell-local-prompt-function)






Here is the result:
nkpwII0.png







Simple Prompt with Mode Line





To achieve more screen estate, leave your prompt simple:





(setq eshell-prompt-function (lambda () "$ ")
      eshell-prompt-regexp (rx line-start (or "$" "#") (1+ space)))







Display detailed information, like the current working directory, in the mode line using which-function-mode.





The eshell/pwd function returns the current working directory, but we need to have a function that returns that only in eshell-mode, otherwise, we will have the current working directory in every buffer:





(defun ha-eshell-mode-line ()
  "Return the current working directory if in eshell-mode."
  (when (eq major-mode 'eshell-mode)
    (thread-last default-directory
                 (s-replace-regexp (rx (eval (getenv "HOME"))) "~")
                 (s-replace-regexp (rx "/" line-end) ""))))







Add this function to the which-func-functions list:





(add-to-list 'which-func-functions 'ha-eshell-mode-line)







Turn on the global minor mode to display this. See leader for that.









Fringe Status





The eshell-fringe-status project shows a color-coded icon of the previous command run (green for success, red for error). Doesn’t work reliably, but the fringe is inconspicuous. Seems to me, that if would be useful to rejuggle those fringe markers so that the marker matched the command entered (instead of seeing a red mark, and needing to scroll back to seethe  command that made the error). Still…





(use-package eshell-fringe-status
  :hook (eshell-mode . eshell-fringe-status-mode))











Opening Banner





Whenever I open a shell, I instinctively type ls … so why not do that automatically? The eshell-banner-message variable, while defaults to a string, this variable can be a form (an s-expression) that calls a function, so I made a customized ls that can be attractive:





(defun ha-eshell-banner ()
  "Return a string containing the files in the current directory."
  (let ((fg (face-attribute 'default :background))
        (bg (face-attribute 'default :foreground))
        (bg "#c09644")
        (dd (replace-regexp-in-string (getenv "HOME") "~" default-directory))
        (gs (or (ha-eshell-banner-git-branch) "")))
    (condition-case err
        (concat
         ;; Line 1
         (propertize
          (format "   %s  •  ⑆ %s   " dd gs)
          'face `(:background ,bg :foreground ,fg))
         "\n"
         ;; Line 2
         (ha-dad-joke)
         "\n\n")
      (error "🐚 Welcome to Eshell\n\n"))))

(defun ha-eshell-banner-git-branch ()
  "Return simplified Git branch for current directory."
  (ignore-errors
    (thread-last "git status --short --branch --ahead-behind 2>/dev/null"
                 (shell-command-to-list)
                 (first)
                 (replace-regexp-in-string
                  (rx "## "
                      (group (zero-or-more not-newline))
                      (zero-or-more anychar))
                  "\\1")
                 (replace-regexp-in-string
                  (rx "...") " → "))))













Shell Windows





Now that I often need to pop into remote systems to run a shell or commands, I create helper functions to create those buffer windows. Each buffer begins with eshell: allowing me to have more than one eshells, typically, one per project.







Shell There





The basis for distinguishing a shell is its parent location. Before starting eshell, we make a small window, set the buffer name (using the eshell-buffer-name):






(defun eshell--buffer-from-dir (dir)
  "Return buffer name of an Eshell based on DIR."
  (format "*eshell: %s*"
          (thread-first dir
                        (split-string "/" t)
                        (last)
                        (car))))

(defun eshell-there (parent)
  "Open an eshell session in a PARENT directory.
The window is smaller and named after this directory.
If an Eshell is already present that has been named
after PARENT, pop to that buffer instead."
  (if-let* ((term-name (eshell--buffer-from-dir parent))
            (buf-name  (seq-contains (buffer-list) term-name
                                     (lambda (a b) (string-equal (buffer-name b) a)))))
      (pop-to-buffer buf-name)

    (let* ((default-directory parent)
           (height (/ (window-total-height) 3)))
      (split-window-vertically (- height))
      (other-window 1)
      (setq eshell-buffer-name term-name)
      (eshell))))







And we can run a command in an opened Eshell buffer:





(defun ha-eshell-send (command &optional dir)
  "Send COMMAND to the Eshell buffer named with DIR.
  The Eshell may have moved away from the directory originally
  opened with DIR, but it should have the name of the buffer.
  See `eshell--buffer-from-dir'."
  (interactive "sCommand to Send: ")
  (unless dir
    (setq dir (project-root (project-current))))
  (save-window-excursion
    (eshell-there dir)
    (goto-char (point-max))
    (insert command)
    (eshell-send-input)))











Shell Here





This version of the eshell bases the location on the current buffer’s parent directory:





(defun eshell-here ()
  "Opens a new shell in the directory of the current buffer.
Renames the eshell buffer to match that directory to allow more
than one eshell window."
  (interactive)
  (eshell-there (if (buffer-file-name)
                    (file-name-directory (buffer-file-name))
                  default-directory)))






And let’s bind it:





(bind-key "C-!" 'eshell-here)











Shell Over There





Would be nice to be able to run an eshell session and use Tramp to connect to the remote host in one fell swoop:





(defun eshell-remote (host)
  "Creates an eshell session that uses Tramp to automatically
connect to a remote system, HOST.  The hostname can be either the
IP address, or FQDN, and can specify the user account, as in
root@blah.com. HOST can also be a complete Tramp reference."
  (interactive "sHost: ")

  (let ((destination-path
         (cond
          ;; Is the HOST already an absolute tramp reference?
          ((string-match-p (rx line-start "/") host) host)

          ;; Does it match any acceptable reference? Get the parts:
          ((string-match-p (ha-eshell-host-regexp 'full) host)
           (string-match (ha-eshell-host-regexp 'full) host) ;; Why!?
           (let* ((user1 (match-string 2 host))
                  (host1 (match-string 3 host))
                  (user2 (match-string 6 host))
                  (host2 (match-string 7 host)))
             (if host1
                 (ha-eshell-host->tramp user1 host1)
               (ha-eshell-host->tramp user2 host2))))

          ;; Otherwise, we assume we have a hostname from a string?
          ;; Convert to a simple 'default' tramp URL:
          (t (format "/%s:" host)))))
    (eshell-there destination-path)))











Shell Here to There





Since I have Org files that contains tables of system to remotely connect to, I figured I should have a little function that can jump to a host found listed anywhere on the line.





The regular expression associated with IP addresses, hostnames, user accounts (of the form, jenkins@my.build.server, or even full Tramp references, is a bit…uhm, hairy. And since I want to reuse these, I will hide them in a function:





(defun ha-eshell-host-regexp (regexp)
  "Returns a particular regular expression based on symbol, REGEXP"
  (let* ((user-regexp      "\\(\\([[:alnum:]._-]+\\)@\\)?")
         (tramp-regexp     "\\b/ssh:[:graph:]+")
         (ip-char          "[[:digit:]]")
         (ip-plus-period   (concat ip-char "+" "\\."))
         (ip-regexp        (concat "\\(\\(" ip-plus-period "\\)\\{3\\}" ip-char "+\\)"))
         (host-char        "[[:alpha:][:digit:]-]")
         (host-plus-period (concat host-char "+" "\\."))
         (host-regexp      (concat "\\(\\(" host-plus-period "\\)+" host-char "+\\)"))
         (horrific-regexp  (concat "\\b"
                                   user-regexp ip-regexp
                                   "\\|"
                                   user-regexp host-regexp
                                   "\\b")))
    (cond
     ((eq regexp 'tramp) tramp-regexp)
     ((eq regexp 'host)  host-regexp)
     ((eq regexp 'full)  horrific-regexp))))







The function to scan a line for hostname patterns uses different function calls that what I could use for eshell-there, so let’s save-excursion and hunt around:





(defun ha-eshell-scan-for-hostnames ()
  "Helper function to scan the current line for any hostnames, IP
or Tramp references.  This returns a tuple of the username (if
found) and the hostname.

If found a Tramp reference, the username part of the tuple is `nil'."
  (save-excursion
    (goto-char (line-beginning-position))
    (if (search-forward-regexp (ha-eshell-host-regexp 'tramp) (line-end-position) t)
        (cons nil (buffer-substring-no-properties (match-beginning 0) (match-end 0)))

      ;; Returns the text associated with match expression, NUM or `nil' if found no match
      (cl-flet ((ha-eshell-get-expression (num) (if-let ((first (match-beginning num))
                                                         (end   (match-end num)))
                                                    (buffer-substring-no-properties first end))))

        (search-forward-regexp (ha-eshell-host-regexp 'full) (line-end-position))

        ;; Until robust, let's keep this debugging code here:
        ;; (message (mapconcat (lambda (tup) (if-let ((s (car tup))
        ;;                                       (e (cadr tup)))
        ;;                                  (buffer-substring-no-properties s e)
        ;;                                "null"))
        ;;             (-partition 2 (match-data t)) " -- "))

        (let ((user1 (ha-eshell-get-expression 2))
              (host1 (ha-eshell-get-expression 3))
              (user2 (ha-eshell-get-expression 6))
              (host2 (ha-eshell-get-expression 7)))
          (if host1
              (list user1 host1)
            (list user2 host2)))))))







Tramp reference can be long when attempting to connect as another user account using the pipe symbol.





(defun ha-eshell-host->tramp (username hostname &optional prefer-root)
  "Return a TRAMP reference based on a USERNAME and HOSTNAME
that refers to any host or IP address."
  (cond ((string-match-p "^/" hostname)
           hostname)
        ((or (and prefer-root (not username)) (equal username "root"))
           (format "/ssh:%s|sudo:%s:" hostname hostname))
        ((or (null username) (equal username user-login-name))
           (format "/ssh:%s:" hostname))
        (t
         (format "/ssh:%s@%s:" username hostname))))







This function pulls it all together:





(defun eshell-here-on-line (p)
  "Search the current line for an IP address or hostname, and call the `eshell-here' function.

Call with PREFIX to connect with the `root' useraccount, via
`sudo'."
  (interactive "p")
  (seq-let (user host) (ha-eshell-scan-for-hostnames)
    (let ((destination (ha-eshell-host->tramp user host (> p 1))))
      (message "Connecting to: %s" destination)
      (eshell-there destination))))













Command on the File Buffer





Sometimes you need to change something about the current file you are editing…like the permissions or even execute it. Hitting Command-1 will prompt for a shell command string and then append the current file to it and execute it.





(defun execute-command-on-file-buffer (cmd)
  "Executes a shell command, CMD, on the current buffer's file.
Appends the filename to the command if not specified, so:

    chmod a+x

Works as expected. We replace the special variable `$$' with the
filename of the buffer. So:

    mv $$ `basename $$`.txt"
  (interactive (list (read-shell-command "Execute command on File Buffer: ")))
  (let* ((file-name (buffer-file-name))
         (full-cmd (cond ((string-match (rx "$$") cmd)
                          (replace-regexp-in-string (rx "$$") file-name cmd))
                         ((and file-name (string-match (rx (literal file-name)) cmd))
                          cmd)
                         (t
                          (concat cmd " " file-name)))))
    (message "Executing: %s" full-cmd)
    (shell-command full-cmd)))











Configuration





Here is where we associate all the functions and their hooks with eshell, through the magic of use-package.





(use-package eshell
  :straight (:type built-in)
  :custom (eshell-banner-message '(ha-eshell-banner))

  :init
  (setq eshell-error-if-no-glob t
        ;; This jumps back to the prompt:
        eshell-scroll-to-bottom-on-input 'all
        eshell-hist-ignoredups t
        eshell-save-history-on-exit t

        ;; Since eshell starts fast, let's dismiss it on exit:
        eshell-kill-on-exit t
        eshell-destroy-buffer-when-process-dies t

        ;; Can you remember the parameter differences between the
        ;; executables `chmod' and `find' and their Emacs counterpart?
        ;; Me neither, so this makes it act a bit more shell-like:
        eshell-prefer-lisp-functions nil)

  :hook ((eshell-pred-load . ha-eshell-add-predicates))

  :bind (("M-!" . eshell-command)
         :map eshell-mode-map
         ("M-R"   . eshell-insert-history)
         ("C-d"   . ha-eshell-quit-or-delete-char)))






Note that the default list to eshell-visual-commands is good enough, but some of my newer Rust-based apps need to be added:





(use-package eshell
  :config
  (add-to-list 'eshell-visual-commands "ssh"))






Calling use-package with :config seems to be just as effective as calling with-eval-after-load.





Add leader commands to call my defined functions:





(ha-leader
  "!" '("eshell cmd" . execute-command-on-file-buffer)
  "a e"   '(:ignore t :which-key "eshell")
  "a e e" '("new eshell"          . eshell-here)
  "a e r" '("remote"              . eshell-remote)
  "a e p" '("project"             . eshell-project)
  "a e g" '("at point"            . eshell-here-on-line)
  "a e b" '("exec on file-buffer" . execute-command-on-file-buffer))






No, i’m not sure why use-package has an issue with both :hook, :bind and :config directives in sequence.