8eaf1d5600
I can not jump to any Org file in my project whether or not that file has been committed (as `rg` limits the searches to git repository files).
26 KiB
My Emacs Bootstrap
A literate programming file for bootstraping my Emacs Configuration.
Introduction
This file contains all the variable definitions and library loading for the other files in my project.
I'm installing everything using the straight.el for package installation and management. This is initialization code configured in initialize, and calls to use-package now accepts a :straight parameter that allows me to retrieve special versions of some packages.
See the details in this essay.
Initial Settings
OS Path and Native Compilation
Helper functions to allow code for specific operating systems:
(defun ha-running-on-macos? ()
"Return non-nil if running on Mac OS systems."
(equal system-type 'darwin))
(defun ha-running-on-linux? ()
"Return non-nil if running on Linux systems."
(equal system-type 'gnu/linux))
With the way I start Emacs, I may not have the PATH I actually use (from the shell) available, so we'll force it (code taken from here):
(defun set-exec-path-from-shell ()
"Set up Emacs' `exec-path' and PATH environment variable to match
that used by the user's shell.
The MacOS, where GUI apps are not started from a shell, requires this."
(interactive)
(let* ((path-from-shell (shell-command-to-string "echo $PATH"))
(trimmed-path (replace-regexp-in-string (rx (zero-or-more space) eol)
"" path-from-shell))
(in-fish? (string-match (rx "fish" eol)
(shell-command-to-string "echo $SHELL")))
(separator (if in-fish? " " ":"))
(env-path (if in-fish? (replace-regexp-in-string " " ":" trimmed-path) trimmed-path)))
(message "PATH=%s" path-from-shell)
(setenv "PATH" env-path)
(setq exec-path (split-string trimmed-path separator))))
Clear up a Mac-specific issue that sometimes arises since I'm switching to native compilation project, as the Emacs.app that I use doesn't have its bin directory, e.g. Emacs.app/Contents/MacOS/bin:
(when (ha-running-on-macos?)
(add-to-list 'exec-path "/usr/local/bin")
(add-to-list 'exec-path "/opt/homebrew/bin")
(add-to-list 'exec-path (concat invocation-directory "bin") t))Getting tired off all the packages that I load spewing a bunch of warnings that I can't do anything about:
(when (and (fboundp 'native-comp-available-p)
(native-comp-available-p))
(setq native-comp-async-report-warnings-errors nil
native-comp-deferred-compilation t))Basic Libraries
The following packages come with Emacs, but seems like they still need loading:
(use-package cl-lib
:straight (:type built-in)
:init (defun first (elt) (car elt))
:commands (first))
(require 'subr-x)Ugh. Why am I getting a missing first function error? I define a simple implementation, that the CL library will overwrite … at some point.
While most libraries will take care of their dependencies, I want to install my dependent libraries, e.g, Magnar Sveen's Clojure-inspired dash.el project:
(use-package dash)Sure this package is essentially syntactic sugar, and to help share my configuration, I attempt to use thread-last instead of ->>, but, I still like it.
The s.el project is a simpler string manipulation library that I (and other projects) use:
(use-package s)Manipulate file paths with the f.el project:
(use-package f)The shell-command function is useful, but having it split the output into a list is a helpful abstraction:
(defun shell-command-to-list (command)
"Return list of lines from running COMMAND in shell."
(thread-last command
shell-command-to-string
s-lines
(-map 's-trim)
(-remove 's-blank-str?)))And let’s see the results:
(ert-deftest shell-command-to-list-test ()
(should (equal '("hello world")
(shell-command-to-list "echo hello world")))
;; We don't need blank lines:
(should (equal '("hello world" "goodbye for now")
(shell-command-to-list "echo '\n\nhello world\n\ngoodbye for now\n\n'"))
;; No output? Return null:
(should (null (shell-command-to-list "echo")))
;; No line should contain carriage returns:
(should (null (seq-filter
(lambda (line) (s-contains? "\n" line))
(shell-command-to-list "ls")))))My Code Location
Much of my more complicated code comes from my website essays and other projects. The destination shows up here:
(add-to-list 'load-path (f-expand "~/.emacs.d/elisp"))Hopefully, this will tie me over while I transition.
Emacs Server Control
I actually run two instances of Emacs on some systems, where one instance has all my work-related projects, perspectives, and packages installed (like LSP), and my personal instance has other packages running (like IRC and Mail). I need a function that can make that distinction, and based on that, it will set server-start appropriately, so that emacsclient can call into the correct one.
(defun ha-emacs-for-work? ()
"Return non-nil when the Emacs instance is for work.
Matches based on a `FOR_WORK' environment variable."
(and (f-dir? "~/work")
(getenv "FOR_WORK")))And now start the server with an appropriate tag name:
(if (not (ha-emacs-for-work?))
(setq server-name "personal")
(setq server-name "work")
(when (ha-running-on-macos?)
(set-exec-path-from-shell)))
(server-start)Load the Rest
The following defines the rest of my org-mode literate files, that I load later with the ha-hamacs-load function:
(defvar ha-hamacs-files (flatten-list
`("ha-private.org"
"ha-config.org"
;; "ha-leader.org"
"ha-evil.org"
;; "ha-meow.org"
"ha-applications.org"
,(when (display-graphic-p)
"ha-display.org")
"ha-org.org"
,(when (display-graphic-p)
"ha-org-word-processor.org")
"ha-org-clipboard.org"
"ha-capturing-notes.org"
"ha-agendas.org"
"ha-data.org"
"ha-passwords.org"
"ha-eshell.org"
"ha-remoting.org"
"ha-programming.org"
"ha-programming-elisp.org"
"ha-programming-python.org"
,(if (ha-emacs-for-work?)
'("ha-org-sprint.org"
"ha-programming-ansible.org"
;; "ha-programming-ruby.org"
"ha-work.org")
;; Personal Editor
'("ha-org-journaling.org"
;; "ha-irc.org"
"ha-org-publishing.org"
"ha-email.org"
"ha-aux-apps.org"
"ha-feed-reader.org"))
"ha-dashboard.org"))
"List of org files that complete the hamacs project.")
The list of hamacs org-formatted files stored in ha-hamacs-files is selectively short, and doesn’t include all files, for instance, certain languages that I’m learning aren’t automatically included. The function, ha-hamacs-files will return the list loaded at startup, as well as with an optional parameter, return them all.
(defun ha-hamacs-files (&optional all)
"Return a list of my org files in my `hamacs' directory."
(if (not all)
ha-hamacs-files
(thread-last (rx ".org" string-end)
(directory-files hamacs-source-dir nil)
(append ha-hamacs-files)
(--filter (not (string-match (rx "README") it)))
(-uniq))))With this function, we can test/debug/reload any individual file, via:
(defun ha-hamacs-load (file)
"Load or reload an org-mode FILE containing literate
Emacs configuration code."
(interactive (list (completing-read "Org file: "
(ha-hamacs-files :all))))
(let ((full-file (f-join hamacs-source-dir file)))
(when (f-exists? full-file)
(ignore-errors
(org-babel-load-file full-file)))))Tangling the Hamacs
And this similar function, will tangle one of my files. Notice that in order to increase the speed of the tangling process (and not wanting to pollute a project perspective), I use a temporary buffer instead of find-file.
(defun ha-hamacs-tangle (file)
"Tangle an org-mode FILE containing literate Emacs
configuration code."
(interactive (list (completing-read "Org file: "
(ha-hamacs-files :all))))
(let ((full-file (f-join hamacs-source-dir file))
(target (file-name-concat "~/emacs.d/elisp"
(concat (file-name-sans-extension file)
".el"))))
(when (f-exists? full-file)
(ignore-errors
(with-temp-buffer
(insert-file-contents full-file)
(with-current-buffer (concat temporary-file-directory file)
(org-babel-tangle nil target (rx "emacs-lisp"))))))))And we can now reload all startup files:
(defun ha-hamacs-reload-all ()
"Reload our entire ecosystem of configuration files."
(interactive)
(dolist (file (ha-hamacs-files))
(unless (equal file "bootstrap.org")
(ha-hamacs-load file))))And we can tangle all the files:
(defun ha-hamacs-tangle-all ()
"Tangle all my Org initialization/configuration files."
(interactive)
(dolist (file (ha-hamacs-files))
(unless (equal file "bootstrap.org")
(ha-hamacs-tangle file))))Edit my Files
Changing my Emacs configuration is as simple as editing an Org file containing the code, and evaluating that block or expression. Or even re-loading the entire file as described above. Calling find-file (or more often project-find-file) is sufficient but quicker if I supply a focused list of just the files in my project:
(defun ha-hamacs-find-file (file)
"Call `find-file' FILE.
When called interactively, present org files containing
my literate Emacs configuration code."
(interactive (list (completing-read "Org file: "
(ha-hamacs-files :all))))
(let ((full-file (f-join hamacs-source-dir file)))
(find-file full-file)))As I refine my project and re-organize the content, I don’t always remember where I put the configuration for something like eww, and some files, like my default config has grown cumbersome. Currently, after loading the file, I issue a call to consult-imenu to get to the right location.
The following section shows some code I wrote one evening, to use the fuzzy matching features of Orderless, to choose a headline in any of my Org configuration files, and then load that file to jump to that headline. The interface is ha-hamacs-edit-file-heading, and the supporting functions begin with ha-hamacs-edit-:
(defun ha-hamacs-edit-file-heading ()
"Edit a file based on a particular heading.
After presenting list of headings from all Org files,
it loads the file, and jumps to the line number where
the heading is located."
(interactive)
(let* ((file-headings (ha-hamacs-edit--file-heading-list))
(file-choice (completing-read "Edit Heading: " file-headings))
(file-tuple (alist-get file-choice file-headings
nil nil 'string-equal)))
(find-file (first file-tuple))
(goto-line (second file-tuple))))
This function collects all possible headers by issuing a call to ripgrep, which returns something like:
ha-applications.org:29:* Git and Magit ha-applications.org:85:** Git Gutter ha-applications.org:110:** Git Delta ha-applications.org:136:** Git with Difftastic ... "ha-applications.org:385:* Web Browsing ha-applications.org:386:** EWW ...
We then filtering out non-useful headers (with ha-hamcs-edit—filter-heading), and convert the headlines with ha-hamcs-edit—process-entry to be more presentable:
(defun ha-hamacs-edit--file-heading-list ()
"Return list of lists of headlines and file locations.
Using the output from the shell command, `ha-hamacs-edit-ripgrep-headers',
it parses and returns something like:
'((\"Applications∷ Git and Magit\" \"ha-applications.org\" 29)
(\"Applications∷ Git and Magit ﹥ Git Gutter\" \"ha-applications.org\" 85)
(\"Applications∷ Git and Magit ﹥ Git Delta\" \"ha-applications.org\" 110)
(\"Applications∷ Git and Magit ﹥ Time Machine\" \"ha-applications.org\" 265)
...)"
(let ((default-directory hamacs-source-dir))
(thread-last ha-hamacs-edit-ripgrep-headers
(shell-command-to-list)
(seq-remove 'ha-hamacs-edit--filter-heading)
(seq-map 'ha-hamacs-edit--process-entry))))
As the function’s documentation string claims, I create file-head-list that contains the data structure necessary for completing-read as well as the information I need to load/jump to a position in the file. This is a three-element list of the headline, filename and line number for each entry:
'(("Applications∷ Git and Magit" "ha-applications.org" 29)
("Applications∷ Git and Magit ﹥ Git Gutter" "ha-applications.org" 85)
("Applications∷ Git and Magit ﹥ Git Delta" "ha-applications.org" 110)
("Applications∷ Git and Magit ﹥ Time Machine" "ha-applications.org" 265)
("Applications∷ Git and Magit ﹥ Gist" "ha-applications.org" 272)
("Applications∷ Git and Magit ﹥ Forge" "ha-applications.org" 296)
("Applications∷ Git and Magit ﹥ Pushing is Bad" "ha-applications.org" 334)
("Applications∷ Git and Magit ﹥ Github Search?" "ha-applications.org" 347)
("Applications∷ ediff" "ha-applications.org" 360)
("Applications∷ Web Browsing" "ha-applications.org" 385)
("Applications∷ Web Browsing ﹥ EWW" "ha-applications.org" 386)
;; ...
)
We’ll use a shell command to call ripgrep to search my collection of org files:
(defvar ha-hamacs-edit-ripgrep-headers
(concat "rg"
" --no-heading"
" --line-number"
" --max-depth 1"
" -e '^\\*+ '"
" *.org")
"A ripgrep shell call to search my headers.")Not every header should be a destination, as many of my org files have duplicate headlines, like Introduction and Technical Artifacts, so I can create a regular expression to remove or flush entries:
(defvar ha-hamacs-edit-flush-headers
(rx "*" (one-or-more space)
(or "Introduction"
"Install"
"Overview"
"Summary"
"Technical Artifacts"))
"Regular expression matching headers to purge.")
And this function, callable by the filter function, uses the regular expression and returns true (well, non-nil) if the line entry given, rg-input, should be removed:
(defun ha-hamacs-edit--filter-heading (rg-input)
"Return non-nil if we should remove RG-INPUT.
These are headings with typical, non-unique entries,
like Introduction and Summary."
(string-match ha-hamacs-edit-flush-headers rg-input))
The seq-map needs to take each line from the ripgrep call and convert it to a list that I can use for the completing-read prompt. I love the combination of seq-let and s-match from Magnar’s String library. The built-in function, string-match returns the index in the string where the match occurs, and this is useful for positioning a prompt, in this case, I want the contents of the matches, and s-match returns each grouping.
(defun ha-hamacs-edit--process-entry (rg-input)
"Return list of heading, file and line number.
Parses the line entry, RG-INPUT, from a call to `rg',
using the regular expression, `ha-hamacs-edit-rx-ripgrep'.
Returns something like:
(\"Some Heading\" \"some-file.org\" 42)"
(seq-let (_ file line level head)
(s-match ha-hamacs-edit-rx-ripgrep rg-input)
(list (ha-hamacs-edit--new-heading file head (length level))
file
(string-to-number line))))Before we dive into the implementation of this function, let’s write a test to validate (and explain) what we expect to return:
(ert-deftest ha-hamacs-edit--process-entry-test ()
(setq ha-hamacs-edit-prev-head-list '())
(should (equal
(ha-hamacs-edit--process-entry
"ha-somefile.org:42:* A Nice Headline :ignored:")
'("Somefile∷ A Nice Headline " "ha-somefile.org" 42)))
;; For second-level headlines, we need to keep track of its parent,
;; and for this, we use a global variable, which we can set for the
;; purposes of this test:
(setq ha-hamacs-edit-prev-head-list '("Parent"))
(should (equal
(ha-hamacs-edit--process-entry
"ha-somefile.org:73:** Another Headline")
'("Somefile∷ Parent﹥ Another Headline"
"ha-somefile.org" 73)))
(setq ha-hamacs-edit-prev-head-list '("Parent" "Subparent"))
(should (equal
(ha-hamacs-edit--process-entry
"ha-somefile.org:73:*** Deep Heading")
'("Somefile∷ Parent﹥ Subparent﹥ Deep Heading"
"ha-somefile.org" 73)))
(setq ha-hamacs-edit-prev-head-list '("Parent" "Subparent"
"Subby" "Deepsubby"))
(should (equal
(ha-hamacs-edit--process-entry
"ha-somefile.org:73:***** Deepest Heading")
'("Somefile∷ ... Deepest Heading"
"ha-somefile.org" 73))))
We next need a regular expression to pass to s-match to parse the output:
(defvar ha-hamacs-edit-rx-ripgrep
(rx (group (one-or-more (not ":"))) ":" ; filename
(group (one-or-more digit)) ":" ; line number
(group (one-or-more "*")) ; header asterisks
(one-or-more space)
(group (one-or-more (not ":")))) ; headline without tags
"Regular expression of ripgrep default output with groups.")
The —new-heading function will prepend the name of the file and its parent headlines (if any) to the headline to be more useful in both understanding the relative context of the headline, as well as better to search using fuzzy matching.
This context is especially important as completing-read will place the most recent choices at the top.
I found the use of setf to be quite helpful in manipulating the list of parents. Remember a list in a Lisp, is a linked list, and we can easily replace one or more parts, by pointing to an new list. This is my first iteration of this function, and I might come back and simplify it.
Essentially, if we get to a top-level headline, we set the list of parents to a list containing that new headline. If we get a second-level headine, B, and our parent list is A, we create a list ’(A B) by setting the cdr of ’(A) to the list ’(B). The advantage of this approach is that if the parent list is ’(A C D), the setf works the same, and the dangled sublist, ’(C D) gets garbage collected.
(defun ha-hamacs-edit--new-heading (file head level)
"Return readable entry from FILE and org headline, HEAD.
The HEAD headline is, when LEVEL is greater than 1,
to include parent headlines. This is done by storing
the list of parents in `ha-hamacs-edit-prev-head-list'."
;; Reset the parent list to include the new HEAD:
(cond
((= level 1)
(setq ha-hamacs-edit-prev-head-list (list head)))
((= level 2)
(setf (cdr ha-hamacs-edit-prev-head-list) (list head)))
((= level 3)
(setf (cddr ha-hamacs-edit-prev-head-list) (list head)))
((= level 4)
(setf (cdddr ha-hamacs-edit-prev-head-list) (list head)))
((= level 5)
(setf (cddddr ha-hamacs-edit-prev-head-list) (list head))))
;; Let's never go any deeper than this...
(format "%s∷ %s"
(ha-hamacs-edit--file-title file)
(s-join "﹥ " ha-hamacs-edit-prev-head-list)))The following test should pass some mustard and explain how this function works:
(ert-deftest ha-hamacs-edit--new-heading-test ()
(should (equal
(ha-hamacs-edit--new-heading "ha-foobar.org" "Apples" 1)
"Foobar∷ Apples"))
(setq ha-hamacs-edit-prev-head-list '("Apples"))
(should (equal
(ha-hamacs-edit--new-heading "ha-foobar.org" "Oranges" 2)
"Foobar∷ Apples﹥ Oranges"))
(setq ha-hamacs-edit-prev-head-list '("Apples" "Oranges"))
(should (equal
(ha-hamacs-edit--new-heading "ha-foobar.org" "Bananas" 3)
"Foobar∷ Apples﹥ Oranges﹥ Bananas"))
(setq ha-hamacs-edit-prev-head-list '("Apples" "Oranges" "Bananas"))
(should (equal
(ha-hamacs-edit--new-heading "ha-foobar.org" "Cantaloupe" 4)
"Foobar∷ Apples﹥ Oranges﹥ Bananas﹥ Cantaloupe")))I store the current list of parents, in the following global variable, gasp:
(defvar ha-hamacs-edit-prev-head-list '("" "")
"The current parents of headlines as a list.")
I would like to make the filename more readable, I use the s-match again, to get the groups of a regular expression, remove all the dashes, and use s-titleize to capitalize each word:
(defun ha-hamacs-edit--file-title (file)
"Return a more readable string from FILE."
(s-with file
(s-match ha-hamacs-edit-file-to-title)
(second)
(s-replace "-" " ")
(s-titleize)))
(defvar ha-hamacs-edit-file-to-title
(rx (optional (or "README-" "ha-"))
(group (one-or-more any)) ".org")
"Regular expression for extracting the interesting part of a
file to use as a title.")So the following tests should pass:
(ert-deftest ha-hamacs-edit-file-title-test ()
(should (equal (ha-hamacs-edit-file-title "ha-apples.org") "Apples"))
(should (equal (ha-hamacs-edit-file-title "apples.org") "Apples"))
(should (equal (ha-hamacs-edit-file-title "README-apples.org") "Apples"))
(should (equal (ha-hamacs-edit-file-title "README.org") "Readme")))Whew … and do it all:
(ha-hamacs-reload-all)