hamacs/ha-programming.org

General Programming Configuration

A literate programming file for helping me program.

Introduction

Seems that all programming interfaces and workflows behave similarly. One other helper routine is a general macro for org-mode files:

  (general-create-definer ha-prog-leader
      :states '(normal visual motion)
      :keymaps 'prog-mode-map
      :prefix "SPC m"
      :global-prefix "<f17>"
      :non-normal-prefix "S-SPC")

General

The following work for all programming languages.

direnv

Farm off commands into virtual environments:

  (use-package direnv
    :init
    (setq direnv--executable "/usr/local/bin/direnv"
          direnv-always-show-summary t
          direnv-show-paths-in-summary t)
    :config
    (direnv-mode))

Spell Checking Comments

The flyspell-prog-mode checks for misspellings in comments.

  (use-package flyspell
    :hook (prog-mode . flyspell-prog-mode))

Flycheck

Why use flycheck over the built-in flymake? Speed used to be the advantage, but I’m now pushing much of this to LSP, so speed is less of an issue. What about when I am not using LSP? Also, since I’ve hooked grammar checkers, I need this with global keybindings.

  (use-package flycheck
    :init
    (setq next-error-message-highlight t)
    :bind (:map flycheck-error-list-mode-map
                ("C-n" . 'flycheck-error-list-next-error)
                ("C-p" . 'flycheck-error-list-previous-error)
                ("j"   . 'flycheck-error-list-next-error)
                ("k"   . 'flycheck-error-list-previous-error))
    :config
    (defun flycheck-enable-checker ()
      "Not sure why flycheck disables working checkers."
      (interactive)
      (let (( current-prefix-arg '(4))) ; C-u
        (call-interactively 'flycheck-disable-checker)))

    (flymake-mode -1)
    (global-flycheck-mode)
    (ha-leader "t c" 'flycheck-mode)

    (ha-leader
      ">" '("next problem" . flycheck-next-error)
      "<" '("previous problem" . flycheck-previous-error)

      "e" '(:ignore t :which-key "errors")
      "e n" '(flycheck-next-error     :repeat t :wk "next")
      "e N" '(flycheck-next-error     :repeat t :wk "next")
      "e p" '(flycheck-previous-error :repeat t :wk "previous")
      "e P" '(flycheck-previous-error :repeat t :wk "previous")

      "e b" '("error buffer"     . flycheck-buffer)
      "e c" '("clear"            . flycheck-clear)
      "e l" '("list all"         . flycheck-list-errors)
      "e g" '("goto error"       . counsel-flycheck)
      "e y" '("copy errors"      . flycheck-copy-errors-as-kill)
      "e s" '("select checker"   . flycheck-select-checker)
      "e ?" '("describe checker" . flycheck-describe-checker)
      "e h" '("display error"    . flycheck-display-error-at-point)
      "e e" '("explain error"    . flycheck-explain-error-at-point)
      "e H" '("help"             . flycheck-info)
      "e i" '("manual"           . flycheck-manual)
      "e V" '("verify-setup"     . flycheck-verify-setup)
      "e v" '("version"          . flycheck-verify-checker)
      "e E" '("enable checker"   . flycheck-enable-checker)
      "e x" '("disable checker"  . flycheck-disable-checker)
      "e t" '("toggle flycheck"  . flycheck-mode)))

Documentation

I’m interested in using devdocs instead, which is similar, but keeps it all inside Emacs (and works on my Linux system). Two Emacs projects compete for this position. The Emacs devdocs project is active, and seems to work well. Its advantage is a special mode for moving around the documentation.

  (use-package devdocs
    :general (:states 'normal "gD" 'devdocs-lookup)

    :config
    (ha-prog-leader
      "d"  '(:ignore t :which-key "docs")
      "d e" '("eldoc" . eldoc)
      "d d" '("open" . devdocs-lookup)
      "d p" '("peruse" . devdocs-peruse)
      "d i" '("install" . devdocs-install)
      "d u" '("update" . devdocs-update-all)
      "d x" '("uninstall" . devdocs-delete)
      "d s" '("search" . devdocs-search)))

The devdocs-browser project acts similar, but with slightly different command names. Its advantage is that it allows for downloading docs and having it available offline, in fact, you can’t search for a function, until you download its pack. This is slightly faster because of this.

  (use-package devdocs-browser
    :general (:states 'normal "gD" 'devdocs-browser-open)

    :config
    (ha-prog-leader
      "d"  '(:ignore t :which-key "docs")
      "d d" '("open" . devdocs-browser-open)
      "d D" '("open in" . devdocs-browser-open-in)
      "d l" '("list" . devdocs-browser-list-docs)
      "d u" '("update" . devdocs-browser-update-docs)
      "d i" '("install" . devdocs-browser-install-doc)
      "d x" '("uninstall" . devdocs-browser-uninstall-doc)
      "d U" '("upgrade" . devdocs-browser-upgrade-doc)
      "d o" '("download" . devdocs-browser-download-offline-data)
      "d O" '("remove download" . devdocs-browser-remove-offline-data)))

Code Folding

While Emacs has options for viewing and moving around code, sometimes, we could collapse all functions, and then start to expand them one at a time. For this, we could enable the built-in hide-show feature:

  (use-package hide-show
    :straight (:type built-in)
    :init
    (setq hs-hide-comments t
          hs-hide-initial-comment-block t
          hs-isearch-open t)
    :hook (prog-mode . hs-minor-mode))

Note that hide-show doesn’t work with complex YAML files. The origami mode works better out-of-the-box, as it works with Python and Lisp, but falls back to indents as the format, which works well.

  (use-package origami
    :init
    (setq origami-fold-replacement "⤵")
    :hook (prog-mode . origami-mode))

To take advantage of this, type:

z m

To collapse everything

z r

To open everything

z o

To open a particular section

z c

To collapse a section (like a function)

z a

Toggles open to close

Note: Yes, we could use vimish-fold (and its cousin, evil-vimish-fold) and we’ll see if I need those.

Navigation with dumb-jump

Once upon a time, we use to create a TAGS file that contained the database for navigating code bases, but with new faster versions of grep, e.g. ack, ag (aka, the Silver Searcher), ugrep and ripgrep, we should be able to use them. but I want to:

• Be in a function, and see its callers. For this, the rg-dwim function is my bread-and-butter.
• Be on a function, and jump to the definition. For this, I use dumb-jump, which uses the above utilities.

  (use-package dumb-jump
    :init
    (setq dumb-jump-prefer-searcher 'rg)

    :config
    (setq xref-show-definitions-function #'xref-show-definitions-completing-read)

    (add-hook 'xref-backend-functions #'dumb-jump-xref-activate)
    ;; (add-to-list 'evil-goto-definition-functions #'dumb-jump)

    ;; Remove this now that https://github.com/jacktasia/dumb-jump/issues/338
    ;; (defun evil-set-jump-args (&rest ns) (evil-set-jump))
    ;; (advice-add 'dumb-jump-goto-file-line :before #'evil-set-jump-args)

    (ha-prog-leader
      "s"  '(:ignore t :which-key "search")
      "s s" '("search"       . xref-find-apropos)
      "s d" '("definitions"  . xref-find-definitions)
      "s o" '("other window" . xref-find-definitions-other-window)
      "s r" '("references"   . xref-find-references)
      "s b" '("back"         . xref-go-back)
      "s f" '("forward"      . xref-go-forward))

    :general (:states 'normal
                      "g." 'xref-find-definitions
                      "g>" 'xref-find-definitions-other-window
                      "g," 'xref-go-back
                      "g<" 'xref-go-forward
                      "g/" 'xref-find-references
                      "g?" 'xref-find-references-and-replace
                      "gh" 'xref-find-apropos
                      "gb" 'xref-go-back))

I have two different jumping systems, the Xref interface and Evil’s. While comparable goals, they are behave different. Let’s compare evil keybindings:

M-.	g .	xref-find-definitions (also g d for evil-goto-definition)†
	g >	xref-find-definitions-other-window
M-,	g ,	xref-go-back (see xref-pop-marker-stack)
C-M-,	g <	xref-go-forward (kinda like xref-find-definitions)
M-?	g /	xref-find-references to go from definition to code calls‡
	g ?	xref-find-references-and-replace could be more accurate than iEdit.
C-M-.	g h	xref-find-apropos … doesn’t work well without LSP
C-TAB		perform completion around point (also M-TAB), see Auto Completion.

† Prefix to prompt for the term \ ‡ If it finds more than one definition, Emacs displays the xref buffer, allowing you to select the definition.

Language Server Protocol (LSP) Integration

The LSP is a way to connect editors (like Emacs) to languages (like Lisp)… wait, no. While originally designed for VS Code and probably Python, we can abstract away Jedi and the Emacs integration to Jedi (and duplicate everything for Ruby, and Clojure, and…).

Emacs has two LSP projects, and while I have used /git/howard/hamacs/src/commit/3e1494663610b2569fd6ec7e0e0e6d7d6ed539b0/LSP%20Mode, but since I don’t have heavy IDE requirements, I am finding that /git/howard/hamacs/src/commit/3e1494663610b2569fd6ec7e0e0e6d7d6ed539b0/eglot to be simpler.

eglot

The eglot package connects to Emacs’ standard command interface, so the eglot-specific code is connects the xref interface to controlling backend servers. That said, it has a couple of eglot- commands that I want easy access to:

  (use-package eglot
    :init
    (setq eglot-connect-timeout 10
          eglot-autoshutdown t)

    :config
    (ha-prog-leader
      "w"  '(:ignore t :which-key "eglot")
      "ws" '("start" . eglot))

    ;; The following leader-like keys, are only available when I have started LSP:
    :general
    (:states 'normal :keymaps 'eglot-mode-map
             "SPC m w r" '("restart"  . eglot-reconnect)
             "SPC m w b" '("events"   . eglot-events-buffer)
             "SPC m w e" '("errors"   . eglot-stderr-buffer)
             "SPC m w q" '("quit"     . eglot-shutdown)
             "SPC m w Q" '("quit all" . eglot-shutdown-all)

             "SPC m l"  '(:ignore t :which-key "lsp")
             "SPC m l r" '("rename"   . eglot-rename)
             "SPC m l f" '("format"   . eglot-format)
             "SPC m l a" '("actions"  . eglot-code-actions)
             "SPC m l i" '("imports"  . eglot-code-action-organize-imports)
             "SPC m l d" '("doc"      . eglot-lookup-documentation)))

The following was stolen from Doom’s configuration:

  (defvar eglot--help-buffer nil)

  (defun eglot-lookup-documentation ()
    "Request documentation for the thing at point."
    (interactive)
    (eglot--dbind ((Hover) contents range)
        (jsonrpc-request (eglot--current-server-or-lose) :textDocument/hover
                         (eglot--TextDocumentPositionParams))
      (let ((blurb (and (not (seq-empty-p contents))
                        (eglot--hover-info contents range)))
            (hint (thing-at-point 'symbol)))
        (if blurb
            (with-current-buffer
                (or (and (buffer-live-p eglot--help-buffer)
                         eglot--help-buffer)
                    (setq eglot--help-buffer (generate-new-buffer "*eglot-help*")))
              (with-help-window (current-buffer)
                (rename-buffer (format "*eglot-help for %s*" hint))
                (with-current-buffer standard-output (insert blurb))
                (setq-local nobreak-char-display nil)))
          (display-local-help))))
    'deferred)

eglot with Consult

The consult-eglot project adds a Consult interface to lookup symbols from the LSP server.

  (use-package consult-eglot
    :general
    (:states 'normal :keymaps 'eglot-mode-map
             "g h" '("find apropos"      . consult-eglot-symbols)
             "SPC m l s" '("find symbol" . consult-eglot-symbols)))

Display Configuration

Using the Doom Modeline to add notifications:

  (use-package doom-modeline
    :config
    (setq doom-modeline-lsp t
          doom-modeline-env-version t))

Function Call Notifications

As I've mentioned on my website, I've created a beep function that notifies when long running processes complete.

  (use-package alert
    :init
    (setq alert-default-style
          (if (ha-running-on-macos?)
              'osx-notifier
            'libnotify)))

  (use-package beep
    :straight nil   ; Already in the load-path
    :hook (after-init . (lambda () (beep--when-finished "Emacs has started")))
    :config
    (dolist (func '(org-publish
                    org-publish-all
                    org-publish-project
                    compile
                    shell-command))
      (advice-add func :around #'beep-when-runs-too-long)))

While that code advices the publishing and compile commands, I may want to add more.

iEdit

While there are language-specific ways to rename variables and functions, iedit is often sufficient.

  (use-package iedit
    :config
    (ha-leader "s e" '("iedit" . iedit-mode)))

Commenting

I like comment-dwim (M-;), and I like comment-box, but I have an odd personal style that I like to codify:

  (defun ha-comment-line (&optional start end)
    "Comment a line or region with a block-level format.
  Calls `comment-region' with START and END set to the region or
  the start and end of the line."
    (interactive)
    (when (or (null start) (not (region-active-p)))
      (setq start (line-beginning-position))
      (setq end   (line-end-position)))
    (save-excursion
      (narrow-to-region start end)
      (upcase-region start end)
      (goto-char (point-min))
      (insert "------------------------------------------------------------------------\n")
      (goto-char (point-max))
      (insert "\n------------------------------------------------------------------------")
      (comment-region (point-min) (point-max))
      (widen)))

And a keybinding:

  (ha-prog-leader "c" '("comment line" . ha-comment-line))

Evaluation

While I like eval-print-last-sexp, I would like a bit of formatting in order to keep the results in the file.

  (defun ha-eval-print-last-sexp (&optional internal-arg)
    "Evaluate the expression located before the point.
  Insert results back into the buffer at the end of the line after
  a comment."
    (interactive)
    (save-excursion
      (eval-print-last-sexp internal-arg))
    (end-of-line)
    (insert "  ")
    (insert comment-start)
    (insert "⟹ ")
    (dotimes (i 2)
      (next-line)
      (join-line)))

Typical keybindings for all programming modes:

  (ha-prog-leader
     "e"  '(:ignore t :which-key "eval")
     "e ;" '("expression" . eval-expression)
     "e b" '("buffer" . eval-buffer)
     "e f" '("function" . eval-defun)
     "e r" '("region" . eval-region)
     "e e" '("last s-exp" . eval-last-sexp)
     "e p" '("print s-exp" . ha-eval-print-last-sexp))

Ligatures

The idea of using math symbols for a programming languages keywords is cute, but can be confusing, so I use it sparingly:

  (defun ha-prettify-prog ()
    "Extends the `prettify-symbols-alist' for programming."
    (mapc (lambda (pair) (push pair prettify-symbols-alist))
          '(("lambda" . "𝝀")
            (">=" . "≥")
            ("<=" . "≤")
            ("!=" . "≠")))
    (prettify-symbols-mode))

  (add-hook 'prog-mode-hook 'ha-prettify-prog)

Hopefully I can follow Mickey Petersen's essay on getting full ligatures working, but right now, they don’t work on the Mac, and that is my current workhorse.

Compiling

The compile function lets me enter a command to run, or I can search the history for a previous run. What it doesn’t give me, is a project-specific list of commands. Perhaps, for each project, I define in .dir-locals.el a variable, compile-command-list, like:

  ((nil . ((compile-command . "make -k ")
           (compile-command-list . ("ansible-playbook playbooks/confluence_test.yml"
                                "ansible-playbook playbooks/refresh_inventory.yml")))))

To make the compile-command-list variable less risky, we need to declare it:

  (defvar compile-command-list nil "A list of potential commands to give to `ha-project-compile'.")

  (defun ha-make-compile-command-list-safe ()
    "Add the current value of `compile-command-list' safe."
    (interactive)
    (add-to-list 'safe-local-variable-values `(compile-command-list . ,compile-command-list)))

What compile commands should I have on offer? Along with the values in compile-command-list (if set), I could look at files in the project’s root and get targets from a Makefile, etc. We’ll use helper functions I define later:

  (defun ha--compile-command-list ()
    "Return list of potential commands for a project."
    (let ((default-directory (projectile-project-root)))
      ;; Make a list of ALL the things.
      ;; Note that `concat' returns an empty string if you give it null,
      ;; so we use `-concat' the dash library:
      (-concat
       compile-history
       (ha--makefile-completions)
       (ha--toxfile-completions)
       (when (and (boundp 'compile-command-list) (listp compile-command-list))
         compile-command-list))))

My replacement to compile uses my new completing-read function:

  (defun ha-project-compile (command)
    "Run `compile' from a list of directory-specific commands."
    (interactive (list (completing-read "Compile command: "
                                        (ha--compile-command-list)
                                        nil nil "" 'compile-history)))
    (let ((default-directory (projectile-project-root)))
      (cond
       ((string-match rx-compile-to-vterm command)  (ha-compile-vterm command))
       ((string-match rx-compile-to-eshell command) (ha-compile-eshell command))
       (t                                           (compile command)))))

If I end a command with a |v, it sends the compile command to a vterm session for the project, allowing me to continue the commands:

  (defvar rx-compile-to-vterm  (rx "|" (0+ space) "v" (0+ space) line-end))

  (defun ha-compile-vterm (full-command &optional project-dir)
    (unless project-dir
      (setq project-dir (projectile-project-name)))

    ;; (add-to-list 'compile-history full-command)
    (let ((command (replace-regexp-in-string rx-compile-to-vterm "" full-command)))
      (ha-shell-send command project-dir)))

And what about sending stuff to Eshell as well?

  (defvar rx-compile-to-eshell (rx "|" (0+ space) "s" (0+ space) line-end))

And let’s add it to the Project leader:

  (ha-leader "p C" 'ha-project-compile)

Note that p c (to call recompile) should still work.

Other people’s projects:

makefile-executor.el

works only with Makefiles

imake

works only with Makefiles that are formatted with a help: target

Taskrunner project

requires ivy or helm, but perhaps I could use the underlying infrastructure to good ol’ completing-read

Note: Someday I may want to convert my Makefile projects to Taskfile.

Makefile Completion

This magic script is what Bash uses for completion when you type make and hit the TAB:

make -qRrp : 2> /dev/null | awk -F':' '/^[a-zA-Z0-9][^$#\\/\\t=]*:([^=]|$)/ {split($1,A,/ /);for(i in A)print A[i]}'

Which makes it easy to get a list of completions for my compile function:

  (defun ha--makefile-completions ()
    "Returns a list of targets from the Makefile in the current directory."
    (when (file-exists-p "Makefile")
      (--map (format "make -k %s" it)
             (shell-command-to-list "<<make-targets>>"))))

Python Tox Completion

Let’s just grab the environments to run:

  (defun ha--toxfile-completions ()
    "Returns a list of targets from the tox.ini in the current directory."
    (when (file-exists-p "tox.ini")
      (--map (format "tox -e %s" it)
             (shell-command-to-list "tox -a"))))

Languages

Simple to configure languages go here. More advanced languages go into their own files… eventually.

Configuration Files

So many configuration files to track:

  (use-package conf-mode
    :mode (("\\.conf\\'"     . conf-space-mode)
           ("\\.repo\\'"     . conf-unix-mode)
           ("\\.setup.*\\'"  . conf-space-mode)))

JSON

While interested in the tree-sitter extensions for JSON, e.g. json-ts-mode, that comes with Emacs 29, I’ll deal with what is bundled now.

Markdown

All the READMEs and other documentation use markdown-mode.

  (use-package markdown-mode
    :straight (:host github :repo "jrblevin/markdown-mode")
    :mode ((rx ".md" string-end) . gfm-mode)
    :init (setq markdown-command "multimarkdown")
    :general
    (:states 'normal :no-autoload t :keymaps 'markdown-mode-map
             "SPC m l" '("insert link" . markdown-insert-link)
             ;; SPC u 3 SPC m h for a third-level header:
             "SPC m h" '("insert header" . markdown-insert-header-dwim)
             "SPC m e" '("export" . markdown-export)
             "SPC m p" '("preview" . markdown-export-and-preview)))

Note that the markdown-specific commands use the C-c C-c and C-c C-s prefixes.

Ansible

Doing a lot of YAML work, but this project needs a new maintainer.

  (use-package yaml-mode
    :mode (rx ".y" (optional "a") "ml" string-end)
          (rx (optional ".") "yamllint")
    :hook (yaml-mode . display-line-numbers-mode)

    :config
    (use-package flycheck-yamllint
      :after (flycheck)
      :config (flycheck-yamllint-setup)
      :hook   (yaml-mode . flycheck-mode)))

Note this needs the following to run properly:

  pip install yamllint

Ansible uses Jinja, so we install the jinja2-mode:

  (use-package jinja2-mode
    :mode (rx ".j2" string-end))

Do I consider all YAML files an Ansible file needing ansible-mode?

  (use-package ansible
    :init
    (setq ansible-vault-password-file "~/.ansible-vault-passfile")
    ;; :hook (yaml-mode . ansible-mode)
    :config
    (ha-leader "t y" 'ansible))

The ansible-vault-password-file variable needs to change per project, so let’s use the .dir-locals.el file, for instance:

  ((nil . ((ansible-vault-password-file . "playbooks/.vault-password"))))

The YAML files get access Ansible’s documentation using the ansible-doc project:

  (use-package ansible-doc
    :hook (yaml-mode . ansible-doc-mode)
    :config
    (ha-local-leader :keymaps 'yaml-mode-map
      "d"  '(:ignore t :which-key "docs")
      "d d" 'ansible-doc))

The poly-ansible project uses polymode, gluing jinja2-mode into yaml-mode.

  (use-package polymode)

  (use-package poly-ansible
    :after polymode
    :straight (:host github :repo "emacsmirror/poly-ansible")
    :hook ((yaml-mode . poly-ansible-mode)
           (poly-ansible-mode . font-lock-update)))

Can we integrate Ansible with LSP using ansible-language-server project (see this documentation)?

First, use npm to install the program:

  npm installl -g @ansible/ansible-language-server

Let’s assume that all YAML files can have access to this:

  (use-package eglot
    :config
    (add-to-list 'eglot-server-programs '(yaml-mode "ansible-language-server" "--stdio")))

Shell Scripts

While I don't like writing them, I can't get away from them. Check out the goodies in this video.

While filename extensions work fine most of the time, I don't like to pre-pend .sh to the shell scripts I write, and instead, would like to associate shell-mode with all files in a bin directory:

  (use-package sh-mode
    :straight (:type built-in)
    :mode (rx (or (seq ".sh" eol)
                  "/bin/"))
    :init
    (setq sh-basic-offset 2
          sh-indentation 2)
    :config
    (ha-auto-insert-file (rx (or (seq ".sh" eol)
                                 "/bin/"))
                         "sh-mode.sh")
    :hook
    (after-save . executable-make-buffer-file-executable-if-script-p))

Note: we make the script executable by default. See this essay for details, but it turns on the executable bit if the script has a shebang at the top of the file.

The shellcheck project integrates with /git/howard/hamacs/src/commit/3e1494663610b2569fd6ec7e0e0e6d7d6ed539b0/Flycheck. First, install the executable into the system, for instance, on a Mac:

  brew install shellcheck

And we can enable it:

  (flycheck-may-enable-checker 'sh-shellcheck)

Place the following on a line before a shell script warning to ignore it:

# shellcheck disable=SC2116,SC2086

See this page for details.

Integration with the Bash LSP implementation. First, install that too:

  brew install bash-language-server

Fish Shell

I think the fish shell is an interesting experiment (and I appreciate the basics that come with fish-mode).

  (use-package fish-mode
    :mode (rx ".fish" eol)
    :config
    (ha-auto-insert-file (rx ".fish") "fish-mode.sh")
    :hook
    (fish-mode . (lambda () (add-hook 'before-save-hook 'fish_indent-before-save))))