746 lines
36 KiB
Org Mode
746 lines
36 KiB
Org Mode
#+title: Literate Programming with Org
|
||
#+author: Howard Abrams
|
||
#+date: 2024-07-07
|
||
#+filetags: emacs hamacs
|
||
#+lastmod: [2024-07-26 Fri]
|
||
|
||
A literate programming file for literate programming in Emacs Org Files.
|
||
|
||
#+begin_src emacs-lisp :exports none
|
||
;;; ha-org-literate --- literate programming helpers -*- lexical-binding: t; -*-
|
||
;;
|
||
;; © 2024 Howard Abrams
|
||
;; Licensed under a Creative Commons Attribution 4.0 International License.
|
||
;; See http://creativecommons.org/licenses/by/4.0/
|
||
;;
|
||
;; Author: Howard Abrams <http://gitlab.com/howardabrams>
|
||
;; Maintainer: Howard Abrams
|
||
;; Created: July 7, 2024
|
||
;;
|
||
;; While obvious, GNU Emacs does not include this file or project.
|
||
;;
|
||
;;; Commentary:
|
||
;;
|
||
;; This file contains a collection of functions to easy some of the
|
||
;; sharp edges when doing literate programming in Org files.
|
||
;;
|
||
;; *NB:* Do not edit this file. Instead, edit the original
|
||
;; literate file at:
|
||
;; /home/howard/other/hamacs/ha-org-literate.org
|
||
;; And tangle the file to recreate this one.
|
||
;;
|
||
;;; Code:
|
||
#+end_src
|
||
* Introduction
|
||
I do a lot of /literate programming/ using capabilities found in the Org project. Over the years, I’ve smoothed some of the rough edges by writing supporting functions, collected below.
|
||
|
||
What are the advantage of [[https://en.wikipedia.org/wiki/Literate_programming][literate programming]]? I listed some in my essay and video about my [[https://howardism.org/Technical/Emacs/literate-devops.html][literate devops]] ideas, but a brief recap:
|
||
|
||
- ambiguous and/or complicated ideas can be reasoned about in prose before code
|
||
- links to ideas, code snippets and supporting projects are more accessible than in comments
|
||
- diagrams written in [[file:ha-org.org::*PlantUML][PlantUML]], [[file:ha-org.org::*Graphviz][Graphviz]], or [[file:ha-org.org::*Pikchr][Pikchr]] maintained and displayed with code
|
||
- full access to org features, like clocks, task lists and agendas
|
||
- mixing languages in the same file, e.g. shell instructions for installing supporting modules before the code that uses that module
|
||
|
||
If literate programming is so great, why doesn’t everyone do it? Most of the /advantages/ listed above are advantages only because we are using Emacs and Org Mode, and getting teams to adopt this requires changing editors and workflows. Therefore, literate programming is a solo endeavor.
|
||
|
||
Since we are using Emacs, the downsides of using a literate approach (even for personal projects) can be minimized.
|
||
|
||
*Note:* What follows is advanced LP usage. I would recommend checking out my [[https://www.howardism.org/Technical/Emacs/literate-programming-tutorial.html][Introduction to LP in Org]] before venturing down this essay.
|
||
* Navigating Code Blocks
|
||
:PROPERTIES:
|
||
:ID: 3230b1f4-0d2d-47c7-9f3d-fa53083f8c8d
|
||
:END:
|
||
I’ve been using Oleh Krehel’s (abo-abo) [[https://github.com/abo-abo/avy][Avy project]] to jump around the screen for years, and I just learned that I can wrap the =avy-jump= function to provide either/or regular expression and action to perform.
|
||
|
||
For instance, the following function can be used to quickly select a source code block, and jump to it:
|
||
|
||
#+begin_src emacs-lisp
|
||
(defun avy-jump-org-block ()
|
||
"Jump to org block using Avy subsystem."
|
||
(interactive)
|
||
(avy-jump (rx line-start (zero-or-more blank) "#+begin_src")
|
||
:action 'goto-char)
|
||
;; Jump _into_ the block:
|
||
(next-line))
|
||
#+end_src
|
||
|
||
I need to take advantage of this feature more.
|
||
* Evaluating Code
|
||
Hitting ~C-c C-c~ in a source code block /evaluates/ the code. Simple, sure, but the following enhancements make this more accessible.
|
||
** Evaluating a Block
|
||
:PROPERTIES:
|
||
:ID: 93a9695c-67be-448a-b068-9727cd0aa9b0
|
||
:END:
|
||
At times I would like to jump to a particular block, evaluate the code, and jump back. This seems like a great job for the [[https://github.com/abo-abo/avy][avy project]]. The =avy-jump= function takes a regular expression of text /in the frame/ (which means you can specify text in other windows), and highlights each match. Normally, selecting a match moves the cursor to that match, the =avy-jump= accepts a function to execute instead:
|
||
|
||
#+begin_src emacs-lisp
|
||
(defun org-babel-execute-src-block-at-point (&optional point)
|
||
"Call `org-babel-execute-src-block' at POINT."
|
||
(save-excursion
|
||
(goto-char point)
|
||
(org-babel-execute-src-block)))
|
||
|
||
(defun avy-org-babel-execute-src-block ()
|
||
"Call `org-babel-execute-src-block' on block given by Avy.
|
||
Use Avy subsystem to select a visible Org source code block,
|
||
e.g. `#+begin_src', and then executes the code without moving
|
||
the point."
|
||
(interactive)
|
||
(avy-jump (rx line-start (zero-or-more blank) "#+begin_src")
|
||
:action 'org-babel-execute-src-block-at-point))
|
||
#+end_src
|
||
|
||
In this case, =avy-org-babel-execute-src-block= highlights all /visible blocks/ on the frame, with a letter on each. Selecting the letter, evaluates that block without moving the cursor.
|
||
|
||
TODO Screenshot of multiple highlighted blocks.
|
||
** Evaluating a Section
|
||
:PROPERTIES:
|
||
:ID: 188e378c-bed4-463c-98d4-d22be1845bc2
|
||
:END:
|
||
A trick to =org-babel-tangle=, is that it tangles /what is shown/, that is, it will only tangle code blocks that are visible after narrowing to the current org section. This means, we can call =org-narrow-to-subtree= to temporary hide everything in the org file except the current heading, evaluate all blocks in the “now visible” buffer, and then widen:
|
||
|
||
#+begin_src emacs-lisp :results silent
|
||
(defun org-babel-execute-subtree ()
|
||
"Execute all Org source blocks in current subtree."
|
||
(interactive "P")
|
||
(save-excursion
|
||
(org-narrow-to-subtree)
|
||
(org-babel-execute-buffer)
|
||
(widen)))
|
||
#+end_src
|
||
** Editing a Block
|
||
:PROPERTIES:
|
||
:ID: f143bbd6-fb4d-45b8-bcfa-196c7a26ed34
|
||
:END:
|
||
Why navigate to a block, just to focus on that block in a dedicated buffer, when we can take advantage of the =avy-jump= and edit any visible block?
|
||
|
||
#+begin_src emacs-lisp
|
||
(defun org-babel-edit-src-block-at-point (&optional point)
|
||
"Call `org-babel-execute-src-block' at POINT."
|
||
(save-excursion
|
||
(goto-char point)
|
||
(org-edit-src-code)))
|
||
|
||
(defun avy-org-babel-edit-src-block ()
|
||
"Call `org-edit-src-code' on block given by Avy.
|
||
Use Avy subsystem to select a visible Org source code block,
|
||
e.g. `#+begin_src', and then executes the code without moving
|
||
the point."
|
||
(interactive)
|
||
(avy-jump (rx line-start (zero-or-more blank) "#+begin_src")
|
||
:action
|
||
'org-babel-edit-src-block-at-point))
|
||
#+end_src
|
||
|
||
* Finding Code
|
||
One of the issues with literate programming is not being able to use the same interface for moving around code when the source code is in org files.
|
||
|
||
** XRef Interface
|
||
:PROPERTIES:
|
||
:ID: 4dc771a4-b974-4b0d-9cc3-a943108c9d3a
|
||
:END:
|
||
The Emacs interface for jumping to function definitions and variable declarations is called xref (see [[https://www.ackerleytng.com/posts/emacs-xref/][this great article]] for an overview of the interface). I think it would be great to be able, even within the prose of an org file, to jump to the definition of a function that is defined in an org file.
|
||
|
||
- [[*Definitions][Definitions]] :: To jump to the line where a macro, function or variable is defined.
|
||
- [[*References][References]] :: To get a list of all /calls/ or usage of a symbol, but only within code blocks.
|
||
- [[*Apropos][Apropos]] :: To get a list of all references, even within org-mode prose.
|
||
|
||
In a normal source code file, you know the language, so you have way of figuring out what a symbol is and how it could be defined in that language. In org files, however, one can use multiple languages, even in the same file.
|
||
|
||
In the code that follows, I’ve made an assumption that I will primarily use this xref interface for Emacs Lisp code, however, it wouldn’t take much (a single regular expression) to convert to another language.
|
||
|
||
Taking a cue from [[https://github.com/jacktasia/dumb-jump][dumb-jump]], I’ve decided to not attempt to build any sort of [[https://github.com/dedi/gxref/][tag interaction]], but instead, call [[https://github.com/BurntSushi/ripgrep/blob/master/GUIDE.md][ripgrep]]. I love that its =–-json= option outputs much more parseable text.
|
||
*** Symbols
|
||
I wrote the =ha-literate-symbol-at-point= function as an attempt at being clever with figuring out what sort of symbol references we would want from an org file. I assume that a symbol may be written surrounded by =~= or ~=~ characters (for code and verbatim text), as well as in quotes or braces, etc.
|
||
|
||
While the goal is Emacs Lisp (and it mostly works for that), it will probably work for other languages as well.
|
||
|
||
#+begin_src emacs-lisp
|
||
(defun ha-literate-symbol-at-point ()
|
||
"Return an alphanumeric sequence at point.
|
||
Assuming the sequence can be surrounded by typical
|
||
punctuation found in org-mode and markdown files."
|
||
(save-excursion
|
||
;; Position point at the first alnum character of the symbol:
|
||
(cond ((looking-at (rx (any "=~({<\"'“`") alnum))
|
||
(forward-char))
|
||
;; Otherwise go back to get "inside" a symbol:
|
||
((not (looking-at (rx alnum)))
|
||
(re-search-backward (rx alnum))))
|
||
|
||
;; Move point to start and end of the symbol:
|
||
(let ((start (progn (skip-chars-backward "a-zA-Z0-9_-") (point)))
|
||
(end (progn (skip-chars-forward "?a-zA-Z0-9_-") (point))))
|
||
(buffer-substring-no-properties start end))))
|
||
#+end_src
|
||
|
||
Examples of references in an Org file that should work:
|
||
- =ha-literate-symbol-at-point=
|
||
- “ha-literate-symbol-at-point”
|
||
- `ha-literate-symbol-at-point`
|
||
|
||
This magical incantation connects our function to Xref with an =org-babel= backend:
|
||
|
||
#+begin_src emacs-lisp
|
||
(cl-defmethod xref-backend-identifier-at-point ((_backend (eql org-babel)))
|
||
(ha-literate-symbol-at-point))
|
||
#+end_src
|
||
*** Calling ripgrep
|
||
This helper function does the work of calling =ripgrep=, parsing its output, and filtering only the /matches/ line. Yes, an interesting feature of =rg= is that it spits out a /sequence/ of JSON-formatted text, so we can use =seq-filter= to grab lines that represent a match, and =seq-map= to “do the work”. Since we have a couple of ways of /doing the work/, we pass in a function, =processor=, which, along with transforming the results, could spit out =nulls=, so the =seq-filter= with the =identity= function eliminates that.
|
||
|
||
#+begin_src emacs-lisp
|
||
(defun ha-literate--ripgrep-matches (processor regex)
|
||
"Return list of running PROCESSOR of `rg' matches from REGEXP.
|
||
PROCESSOR is called with an assoc-list of the JSON output from
|
||
the call to ripgrep."
|
||
(let* ((default-directory (if (project-current)
|
||
(project-root (project-current))
|
||
default-directory))
|
||
(search-str (rxt-elisp-to-pcre regex))
|
||
(command (format "rg --json '%s' *.org" search-str)))
|
||
|
||
(message "Calling %s" command)
|
||
(thread-last command
|
||
(shell-command-to-list)
|
||
(seq-map 'ha-literate--parse-rg-line)
|
||
(seq-filter 'ha-literate--only-matches)
|
||
(seq-map processor)
|
||
;; Remove any nulls from the list:
|
||
(seq-filter 'identity))))
|
||
#+end_src
|
||
|
||
Note: the =processor= function creates an =xref= object, described below. See =ha-literate--process-rg-line=.
|
||
|
||
The output from =ripgrep= goes through a couple of transformation functions listed here:
|
||
|
||
#+begin_src emacs-lisp
|
||
(defun ha-literate--parse-rg-line (line)
|
||
"Process LINE as a JSON object with `json-parse-string'."
|
||
(json-parse-string line :object-type 'alist
|
||
:array-type 'list))
|
||
|
||
(defun ha-literate--only-matches (json-data)
|
||
"Return non-nil if JSON-DATA is an alist with key `type' and value `match'."
|
||
(string-equal "match" (alist-get 'type json-data)))
|
||
#+end_src
|
||
*** Definitions
|
||
As mentioned above, let’s assume we can use =ripgrep= to search for /definitions/ in Lisp. I choose that because most of my literate programming is in Emacs Lisp. This regular expression should work with things like =defun= and =defvar=, etc. as well as =use-package=, allowing me to search for the /definition/ of an Emacs package:
|
||
|
||
#+begin_src emacs-lisp
|
||
(defun ha-literate-definition (symb)
|
||
"Return list of `xref' objects of SYMB location in org files.
|
||
The location is based on a regular expression starting with
|
||
`(defxyz SYMB' where this can be `defun' or `defvar', etc."
|
||
(ha-literate--ripgrep-matches 'ha-literate--process-rg-line
|
||
(rx "("
|
||
(or "use-package"
|
||
(seq ; Match both defun and cl-defun:
|
||
(optional "cl-")
|
||
"def" (1+ (not space))))
|
||
(one-or-more space)
|
||
(literal symb)
|
||
word-boundary)))
|
||
#+end_src
|
||
|
||
The work of processing a match for the =ha-literate-definition= function. It calls =xref-make= to create an object for the Xref system. This takes two parameters, the text and the location. We create a location with =xref-make-file-location=.
|
||
|
||
#+begin_src emacs-lisp
|
||
(defun ha-literate--process-rg-line (rg-data-line)
|
||
"Return an `xref' structure based on the contents of RG-DATA-LINE.
|
||
The RG-DATA-LINE is a convert JSON data object from ripgrep.
|
||
The return data comes from `xref-make' and `xref-make-file-location'."
|
||
(when rg-data-line
|
||
(let-alist rg-data-line
|
||
(xref-make .data.lines.text
|
||
(xref-make-file-location .data.path.text
|
||
.data.line_number
|
||
(thread-last
|
||
(first .data.submatches)
|
||
(alist-get 'start)))))))
|
||
#+end_src
|
||
|
||
I really like the use of =let-alist= where the output from JSON can be parsed into a data structure that can then be accessible via /variables/, like =.data.path.text=.
|
||
|
||
We connect this function to the =xref-backend-definitions= list, so that it can be called when we type something like ~M-.~:
|
||
|
||
#+begin_src emacs-lisp
|
||
(cl-defmethod xref-backend-definitions ((_backend (eql org-babel)) symbol)
|
||
(ha-literate-definition symbol))
|
||
#+end_src
|
||
*** Apropos
|
||
The /apropos/ approach is anything, so the regular expression here is just the symbol, and we can re-use our processor:
|
||
|
||
#+begin_src emacs-lisp
|
||
(defun ha-literate-apropos (symb)
|
||
"Return an `xref' object for SYMB location in org files.
|
||
The location is based on a regular expression starting with
|
||
`(defxyz SYMB' where this can be `defun' or `defvar', etc."
|
||
(ha-literate--ripgrep-matches 'ha-literate--process-rg-line
|
||
(rx word-boundary
|
||
(literal symb)
|
||
word-boundary)))
|
||
#+end_src
|
||
|
||
And this to /hook it up/:
|
||
|
||
#+begin_src emacs-lisp
|
||
(cl-defmethod xref-backend-apropos ((_backend (eql org-babel)) symbol)
|
||
(ha-literate-apropos symbol))
|
||
#+end_src
|
||
*** References
|
||
While traditionally, =-apropos= can reference symbols in comments and documentation, searching for /references/ tend to be /calls/ and whatnot. What does that mean in the context of an org file? I’ve decided that references should only show symbols /within org blocks/.
|
||
|
||
How do we know we are /inside/ an org block?
|
||
|
||
I call =ripgrep= twice, once to get all the =begin_= and =end_src= lines and their line numbers.
|
||
The second =ripgrep= call gets the references.
|
||
|
||
#+begin_src emacs-lisp
|
||
(defun ha-literate-references (symb)
|
||
"Return list of `xref' objects for SYMB location in org files.
|
||
The location is limited only references in org blocks."
|
||
;; First, get and store the block line numbers:
|
||
(ha-literate--block-line-numbers)
|
||
;; Second, call `rg' again to get all matches of SYMB:
|
||
(ha-literate--ripgrep-matches 'ha-literate--process-rg-block
|
||
(rx word-boundary
|
||
(literal symb)
|
||
word-boundary)))
|
||
#+end_src
|
||
|
||
Notice for this function, we need a new processor that limits the results to only matches between the beginning and ending of a block, which I’ll describe later.
|
||
|
||
The =ha-literate--block-line-numbers= returns a hash where the keys are files, and the value is a series of begin/end line numbers. It calls =ripgrep=, but has a new processor.
|
||
|
||
#+begin_src emacs-lisp
|
||
(defun ha-literate--block-line-numbers ()
|
||
"Call `ripgrep' for org blocks and store results in a hash table.
|
||
See `ha-literate--process-src-refs'."
|
||
(clrhash ha-literate--process-src-refs)
|
||
(ha-literate--ripgrep-matches 'ha-literate--process-src-blocks
|
||
(rx line-start (zero-or-more blank)
|
||
"#+" (or "begin" "end") "_src")))
|
||
#+end_src
|
||
|
||
And the function to process the output simply attempts to connect the =begin_src= with the =end_src= lines. In true Emacs Lisp fashion (where we can’t easily, lexically nest functions), we use a global variable:
|
||
|
||
#+begin_src emacs-lisp
|
||
(defvar ha-literate--process-src-refs
|
||
(make-hash-table :test 'equal)
|
||
"Globabl variable storing results of processing
|
||
org-mode's block line numbers. The key in this table is a file
|
||
name, and the value is a list of line numbers marking #+begin_src
|
||
and #+end_src.")
|
||
|
||
(defvar ha-literate--process-begin-src nil
|
||
"Globabl variable storing the last entry of an
|
||
org-mode's `#+begin_src' line number.")
|
||
|
||
(defun ha-literate--process-src-blocks (rg-data-line)
|
||
"Return nil if RG-DATA-LINE contains a begin_src entry.
|
||
Otherwise return a list of previous begin_src, and the
|
||
current end_src line numbers."
|
||
(let-alist rg-data-line
|
||
(puthash .data.path.text ; filename is the key
|
||
(append
|
||
(gethash .data.path.text ha-literate--process-src-refs)
|
||
(list .data.line_number))
|
||
ha-literate--process-src-refs)))
|
||
#+end_src
|
||
|
||
With a collection of line numbers for all org-blocks in all org files in our project, we can process a particular match from =ripgrep= to see if the match is /within/ a block. Since the key is a file, and =.data.path.text= is the filename, that part is done, but we need a helper to walk down the list.
|
||
|
||
#+begin_src emacs-lisp
|
||
(defun ha-literate--process-rg-block (rg-data-line)
|
||
"Return an `xref' structure from the contents of RG-DATA-LINE.
|
||
Return nil if the match is _not_ with org source blocks.
|
||
Note that the line numbers of source blocks should be filled
|
||
in the hashmap, `ha-literate--process-src-refs'."
|
||
(let-alist rg-data-line
|
||
(let ((line-nums (thread-first .data.path.text
|
||
(gethash ha-literate--process-src-refs)
|
||
;; Turn list into series of tuples
|
||
(seq-partition 2))))
|
||
(when (ha-literate--process-in-block .data.line_number line-nums)
|
||
(ha-literate--process-rg-line rg-data-line)))))
|
||
|
||
(defun ha-literate--process-in-block (line-number line-numbers)
|
||
"Return non-nil if LINE-NUMBER is inclusive in LINE-NUMBERS.
|
||
The LINE-NUMBERS is a list of two element lists where the first
|
||
element is the starting line number of a block, and the second
|
||
is the ending line number."
|
||
(when line-numbers
|
||
(let ((block-lines (car line-numbers)))
|
||
(if (and (> line-number (car block-lines))
|
||
(< line-number (cadr block-lines)))
|
||
(car block-lines)
|
||
(ha-literate--process-in-block line-number (cdr line-numbers))))))
|
||
#+end_src
|
||
|
||
The helper function, =ha-literate--process-in-block= is a /recursive/ function that takes each tuple and sees if =line-number= is between them. If it isn’t between any tuple, and the list is empty, then we return =nil= to filter that out later.
|
||
|
||
Let’s connect the plumbing:
|
||
|
||
#+begin_src emacs-lisp
|
||
(cl-defmethod xref-backend-references ((_backend (eql org-babel)) symbol)
|
||
(ha-literate-references symbol))
|
||
#+end_src
|
||
|
||
Whew! This is pretty cool to jump out my literate code base as if it were actual =.el= files.
|
||
*** Identifier Completion Table
|
||
Need the completion table before we can find the references. It actually doesn’t even need to return anything purposeful:
|
||
|
||
#+begin_src emacs-lisp
|
||
(defun ha-literate-completion-table ())
|
||
#+end_src
|
||
|
||
Now we /hook this up/ to the rest of the system:
|
||
|
||
#+begin_src emacs-lisp
|
||
(cl-defmethod xref-backend-identifier-completion-table ((_backend (eql org-babel)))
|
||
(ha-literate-completion-table))
|
||
#+end_src
|
||
*** Activation of my Literate Searching
|
||
To finish the connections, we need to create a /hook/ that I only allow to turn on with org files:
|
||
|
||
#+begin_src emacs-lisp
|
||
(defun ha-literate-xref-activate ()
|
||
"Function to activate org-based literate backend.
|
||
Add this function to `xref-backend-functions' hook. "
|
||
(when (eq major-mode 'org-mode)
|
||
'org-babel))
|
||
|
||
(add-hook 'xref-backend-functions #'ha-literate-xref-activate)
|
||
#+end_src
|
||
|
||
At this point, we can jump to functions and variables that I define in my org file, or even references to standard symbols like =xref-make= or =xref-backend-functions=.
|
||
|
||
This is seriously cool to be able to jump around my literate code as if it were =.el= files. I may want to think about expanding the definitions to figure out the language of the destination.
|
||
** Searching by Header
|
||
:PROPERTIES:
|
||
:ID: de536693-f0b0-48d0-9b13-c29d7a8caa62
|
||
:END:
|
||
As large literate programming projects grow, I refine, re-organize and refactor content. I don’t always remember where I put particular code. For instance, in my Emacs configuration, did I configure /eww/, in [[file:ha-config.org][my default config]] file, or did I move it somewhere? Originally, after loading the file, I could issue a call to [[file:ha-general.org::*Consult][consult-imenu]] to get to the right location, but that assumes I have the correct file loaded.
|
||
|
||
The following section shows some code to use the fuzzy matching features of [[file:ha-config.org::*Orderless][Orderless]], to choose a headline in any of my Org files in a project, and then load that file and jump to that headline. The interface is =ha-hamacs-edit-file-heading=, and the supporting functions begin with =ha-hamacs-edit-=:
|
||
|
||
#+begin_src emacs-lisp
|
||
(defun ha-hamacs-edit-file-heading (&optional project-root)
|
||
"Edit a file based on a particular heading.
|
||
After presenting list of headings from all Org files,
|
||
it loads the file, and jumps to the line number where
|
||
the heading is located."
|
||
(interactive)
|
||
(let* ((default-directory (or project-root (project-root (project-current))))
|
||
(file-headings (ha-hamacs-edit--file-heading-list))
|
||
(file-choice (completing-read "Edit Heading: " file-headings))
|
||
(file-tuple (alist-get file-choice file-headings
|
||
nil nil 'string-equal)))
|
||
(find-file (first file-tuple))
|
||
(goto-line (second file-tuple))))
|
||
#+end_src
|
||
|
||
This function collects all possible headers by issuing a call to =ripgrep=, which returns something like:
|
||
|
||
#+begin_example
|
||
ha-applications.org:29:* Git and Magit
|
||
ha-applications.org:85:** Git Gutter
|
||
ha-applications.org:110:** Git Delta
|
||
ha-applications.org:136:** Git with Difftastic
|
||
...
|
||
"ha-applications.org:385:* Web Browsing
|
||
ha-applications.org:386:** EWW
|
||
...
|
||
#+end_example
|
||
|
||
We then filter out non-useful headers (with =ha-hamcs-edit—filter-heading=), and convert the headlines with =ha-hamcs-edit—process-entry= to be more presentable:
|
||
|
||
#+begin_src emacs-lisp
|
||
(defun ha-hamacs-edit--file-heading-list ()
|
||
"Return list of lists of headlines and file locations.
|
||
This is found by calling `ripgrep' in the `default-directory'.
|
||
Using the output from the shell command, `ha-hamacs-edit-ripgrep-headers',
|
||
it parses and returns something like:
|
||
|
||
'((\"Applications∷ Git and Magit\" \"ha-applications.org\" 29)
|
||
(\"Applications∷ Git and Magit ﹥ Git Gutter\" \"ha-applications.org\" 85)
|
||
(\"Applications∷ Git and Magit ﹥ Git Delta\" \"ha-applications.org\" 110)
|
||
(\"Applications∷ Git and Magit ﹥ Time Machine\" \"ha-applications.org\" 265)
|
||
...)"
|
||
(thread-last ha-hamacs-edit-ripgrep-headers
|
||
(shell-command-to-list)
|
||
;; Let's remove non-helpful, duplicate headings,
|
||
;; like Introduction:
|
||
(seq-remove 'ha-hamacs-edit--filter-heading)
|
||
;; Convert the results into both a displayable
|
||
;; string as well as the file and line structure:
|
||
(seq-map 'ha-hamacs-edit--process-entry)))
|
||
#+end_src
|
||
|
||
As the above function’s documentation string claims, I create a list that contains the data structure necessary for =completing-read= as well as the information I need to load/jump to a position in the file. This is a three-element list of the /headline/, /filename/ and /line number/ for each entry:
|
||
|
||
#+begin_src emacs-lisp :tangle no
|
||
'(("Applications∷ Git and Magit" "ha-applications.org" 29)
|
||
("Applications∷ Git and Magit ﹥ Git Gutter" "ha-applications.org" 85)
|
||
("Applications∷ Git and Magit ﹥ Git Delta" "ha-applications.org" 110)
|
||
("Applications∷ Git and Magit ﹥ Time Machine" "ha-applications.org" 265)
|
||
("Applications∷ Git and Magit ﹥ Gist" "ha-applications.org" 272)
|
||
("Applications∷ Git and Magit ﹥ Forge" "ha-applications.org" 296)
|
||
("Applications∷ Git and Magit ﹥ Pushing is Bad" "ha-applications.org" 334)
|
||
("Applications∷ Git and Magit ﹥ Github Search?" "ha-applications.org" 347)
|
||
("Applications∷ ediff" "ha-applications.org" 360)
|
||
("Applications∷ Web Browsing" "ha-applications.org" 385)
|
||
("Applications∷ Web Browsing ﹥ EWW" "ha-applications.org" 386)
|
||
;; ...
|
||
)
|
||
#+end_src
|
||
|
||
We’ll use this shell command to call =ripgrep= to search my collection of org files:
|
||
|
||
#+begin_src emacs-lisp
|
||
(defvar ha-hamacs-edit-ripgrep-headers
|
||
(concat "rg"
|
||
" --no-heading"
|
||
" --line-number"
|
||
;; " --max-depth 1"
|
||
" -e '^\\*+ '"
|
||
" *.org")
|
||
"A ripgrep shell call to search my headers.")
|
||
#+end_src
|
||
|
||
Not every header should be a destination, as many of my org files have duplicate headlines, like *Introduction* and *Technical Artifacts*, so I can create a regular expression to remove or flush entries:
|
||
|
||
#+begin_src emacs-lisp
|
||
(defvar ha-hamacs-edit-flush-headers
|
||
(rx "*" (one-or-more space)
|
||
(or "Introduction"
|
||
"Install"
|
||
"Overview"
|
||
"Summary"
|
||
"Technical Artifacts"))
|
||
"Regular expression matching headers to purge.")
|
||
#+end_src
|
||
|
||
Note: This variable should be set in the =.dir-locals.el= for a particular project, as in:
|
||
|
||
#+begin_src emacs-lisp :tangle no
|
||
((org-mode . ((ha-hamacs-edit-flush-headers .
|
||
"\\*[[:space:]]+\\(?:Background\\|Summary\\)"))))
|
||
#+end_src
|
||
|
||
And this next function is callable by the filter function, it uses the regular expression and returns true (well, non-nil) if the line entry given, =rg-input=, should be removed:
|
||
|
||
#+begin_src emacs-lisp
|
||
(defun ha-hamacs-edit--filter-heading (rg-input)
|
||
"Return non-nil if we should remove RG-INPUT.
|
||
These are headings with typical, non-unique entries,
|
||
like Introduction and Summary."
|
||
(string-match ha-hamacs-edit-flush-headers rg-input))
|
||
#+end_src
|
||
|
||
The =seq-map= needs to take each line from the =ripgrep= call and convert it to a list that I can use for the =completing-read= prompt. I love the combination of =seq-let= and =s-match= from Magnar’s [[https://github.com/magnars/s.el][String library]]. The built-in function, =string-match= returns the index in the string where the match occurs, and this is useful for positioning a prompt, in this case, I want the /contents/ of the matches, and =s-match= returns each /grouping/.
|
||
|
||
#+begin_src emacs-lisp
|
||
(defun ha-hamacs-edit--process-entry (rg-input)
|
||
"Return list of heading, file and line number.
|
||
Parses the line entry, RG-INPUT, from a call to `rg',
|
||
using the regular expression, `ha-hamacs-edit-rx-ripgrep'.
|
||
Returns something like:
|
||
|
||
(\"Some Heading\" \"some-file.org\" 42)"
|
||
(seq-let (_ file line level head)
|
||
(s-match ha-hamacs-edit-rx-ripgrep rg-input)
|
||
(list (ha-hamacs-edit--new-heading file head (length level))
|
||
file
|
||
(string-to-number line))))
|
||
#+end_src
|
||
|
||
Before we dive into the implementation of this function, let’s write a test to validate (and explain) what we expect to return:
|
||
|
||
#+begin_src emacs-lisp :tangle no
|
||
(ert-deftest ha-hamacs-edit--process-entry-test ()
|
||
(setq ha-hamacs-edit-prev-head-list '())
|
||
(should (equal
|
||
(ha-hamacs-edit--process-entry
|
||
"ha-somefile.org:42:* A Nice Headline :ignored:")
|
||
'("Somefile∷ A Nice Headline " "ha-somefile.org" 42)))
|
||
|
||
;; For second-level headlines, we need to keep track of its parent,
|
||
;; and for this, we use a global variable, which we can set for the
|
||
;; purposes of this test:
|
||
(setq ha-hamacs-edit-prev-head-list '("Parent"))
|
||
(should (equal
|
||
(ha-hamacs-edit--process-entry
|
||
"ha-somefile.org:73:** Another Headline")
|
||
'("Somefile∷ Parent﹥ Another Headline"
|
||
"ha-somefile.org" 73)))
|
||
|
||
(setq ha-hamacs-edit-prev-head-list '("Parent" "Subparent"))
|
||
(should (equal
|
||
(ha-hamacs-edit--process-entry
|
||
"ha-somefile.org:73:*** Deep Heading")
|
||
'("Somefile∷ Parent﹥ Subparent﹥ Deep Heading"
|
||
"ha-somefile.org" 73)))
|
||
|
||
(setq ha-hamacs-edit-prev-head-list '("Parent" "Subparent"
|
||
"Subby" "Deepsubby"))
|
||
(should (equal
|
||
(ha-hamacs-edit--process-entry
|
||
"ha-somefile.org:73:***** Deepest Heading")
|
||
'("Somefile∷ ... Deepest Heading"
|
||
"ha-somefile.org" 73))))
|
||
#+end_src
|
||
|
||
We next need a regular expression to pass to =s-match= to parse the output:
|
||
#+begin_src emacs-lisp
|
||
(defvar ha-hamacs-edit-rx-ripgrep
|
||
(rx (group (one-or-more (not ":"))) ":" ; filename
|
||
(group (one-or-more digit)) ":" ; line number
|
||
(group (one-or-more "*")) ; header asterisks
|
||
(one-or-more space)
|
||
(group (one-or-more (not ":")))) ; headline without tags
|
||
"Regular expression of ripgrep default output with groups.")
|
||
#+end_src
|
||
|
||
The =—new-heading= function will /prepend/ the name of the file and its parent headlines (if any) to the headline to be more useful in both understanding the relative context of the headline, as well as better to search using fuzzy matching.
|
||
|
||
This /context/ is especially important as =completing-read= will place the most recent choices at the top.
|
||
|
||
I found the use of =setf= to be quite helpful in manipulating the list of parents. Remember a =list= in a Lisp, is a /linked list/, and we can easily replace one or more parts, by pointing to an new list. This is my first iteration of this function, and I might come back and simplify it.
|
||
|
||
Essentially, if we get to a top-level headline, we set the list of parents to a list containing that new headline. If we get a second-level headine, =B=, and our parent list is =A=, we create a list =’(A B)= by setting the =cdr= of =’(A)= to the list =’(B)=. The advantage of this approach is that if the parent list is =’(A C D)=, the =setf= works the same, and the dangled /sublist/, =’(C D)= gets garbage collected.
|
||
|
||
#+begin_src emacs-lisp
|
||
(defun ha-hamacs-edit--new-heading (file head level)
|
||
"Return readable entry from FILE and org headline, HEAD.
|
||
The HEAD headline is, when LEVEL is greater than 1,
|
||
to include parent headlines. This is done by storing
|
||
the list of parents in `ha-hamacs-edit-prev-head-list'."
|
||
;; Reset the parent list to include the new HEAD:
|
||
(cond
|
||
((= level 1)
|
||
(setq ha-hamacs-edit-prev-head-list (list head)))
|
||
((= level 2)
|
||
(setf (cdr ha-hamacs-edit-prev-head-list) (list head)))
|
||
((= level 3)
|
||
(setf (cddr ha-hamacs-edit-prev-head-list) (list head)))
|
||
((= level 4)
|
||
(setf (cdddr ha-hamacs-edit-prev-head-list) (list head)))
|
||
((= level 5)
|
||
(setf (cddddr ha-hamacs-edit-prev-head-list) (list head))))
|
||
;; Let's never go any deeper than this...
|
||
|
||
(format "%s∷ %s"
|
||
(ha-hamacs-edit--file-title file)
|
||
(s-join "﹥ " ha-hamacs-edit-prev-head-list)))
|
||
#+end_src
|
||
|
||
The following test should pass some mustard and explain how this function works:
|
||
#+begin_src emacs-lisp :tangle no
|
||
(ert-deftest ha-hamacs-edit--new-heading-test ()
|
||
(should (equal
|
||
(ha-hamacs-edit--new-heading "ha-foobar.org" "Apples" 1)
|
||
"Foobar∷ Apples"))
|
||
(setq ha-hamacs-edit-prev-head-list '("Apples"))
|
||
(should (equal
|
||
(ha-hamacs-edit--new-heading "ha-foobar.org" "Oranges" 2)
|
||
"Foobar∷ Apples﹥ Oranges"))
|
||
(setq ha-hamacs-edit-prev-head-list '("Apples" "Oranges"))
|
||
(should (equal
|
||
(ha-hamacs-edit--new-heading "ha-foobar.org" "Bananas" 3)
|
||
"Foobar∷ Apples﹥ Oranges﹥ Bananas"))
|
||
(setq ha-hamacs-edit-prev-head-list '("Apples" "Oranges" "Bananas"))
|
||
(should (equal
|
||
(ha-hamacs-edit--new-heading "ha-foobar.org" "Cantaloupe" 4)
|
||
"Foobar∷ Apples﹥ Oranges﹥ Bananas﹥ Cantaloupe")))
|
||
#+end_src
|
||
|
||
I store the current list of parents, in the following (/gasp/) /global variable/:
|
||
|
||
#+begin_src emacs-lisp
|
||
(defvar ha-hamacs-edit-prev-head-list '("" "")
|
||
"The current parents of headlines as a list.")
|
||
#+end_src
|
||
|
||
I would like to make the /filename/ more readable, I use the =s-match= again, to get the groups of a regular expression, remove all the dashes, and use =s-titleize= to capitalize each word:
|
||
|
||
#+begin_src emacs-lisp
|
||
(defun ha-hamacs-edit--file-title (file)
|
||
"Return a more readable string from FILE."
|
||
(s-with file
|
||
(s-match ha-hamacs-edit-file-to-title)
|
||
(second)
|
||
(s-replace "-" " ")
|
||
(s-titleize)))
|
||
|
||
(defvar ha-hamacs-edit-file-to-title
|
||
(rx (optional (or "README-" "ha-"))
|
||
(group (one-or-more any)) ".org")
|
||
"Regular expression for extracting the interesting part of a
|
||
file to use as a title.")
|
||
#+end_src
|
||
|
||
So the following tests should pass:
|
||
|
||
#+begin_src emacs-lisp :tangle no
|
||
(ert-deftest ha-hamacs-edit-file-title-test ()
|
||
(should (equal (ha-hamacs-edit-file-title "ha-apples.org") "Apples"))
|
||
(should (equal (ha-hamacs-edit-file-title "apples.org") "Apples"))
|
||
(should (equal (ha-hamacs-edit-file-title "README-apples.org") "Apples"))
|
||
(should (equal (ha-hamacs-edit-file-title "README.org") "Readme")))
|
||
#+end_src
|
||
|
||
* Keybindings
|
||
:PROPERTIES:
|
||
:ID: 2412ef3b-b5d0-43a3-bd01-764fd92b0c3c
|
||
:END:
|
||
With a lovely collection of functions, we need to have a way to easily call them. I’ve been using the =pretty-hydra= feature of [[https://github.com/jerrypnz/major-mode-hydra.el][major-mode-hydra]]:
|
||
|
||
#+begin_src emacs-lisp
|
||
(defvar org-babel--title (concat (all-the-icons-faicon "pencil-square-o")
|
||
" Literate Programming Support"))
|
||
(pretty-hydra-define org-babel
|
||
(:title org-babel--title :color blue)
|
||
("Code Blocks"
|
||
(("g" avy-jump-org-block "Goto ")
|
||
("j" org-next-block "Previous" :color pink)
|
||
("k" org-previous-block "Next" :color pink))
|
||
"Evaluate"
|
||
(("o" avy-org-babel-execute-src-block "Block ")
|
||
("h" org-babel-execute-subtree "Section")
|
||
("b" org-babel-execute-buffer "Buffer"))
|
||
"Tangle"
|
||
(("t" org-babel-tangle "to Default")
|
||
("f" org-babel-tangle-file "choose File")
|
||
("T" org-babel-detangle "from File"))
|
||
"Misc"
|
||
(("e" avy-org-babel-edit-src-block "Edit Block "))))
|
||
#+end_src
|
||
|
||
And tie this hydra into the existing leader system:
|
||
#+begin_src emacs-lisp
|
||
(ha-leader :keymaps 'org-mode-map "o s" '("babel" . org-babel/body))
|
||
#+end_src
|
||
* Technical Artifacts :noexport:
|
||
Let's =provide= a name so we can =require= this file:
|
||
|
||
#+begin_src emacs-lisp :exports none
|
||
(provide 'ha-org-literate)
|
||
;;; ha-org-literate.el ends here
|
||
#+end_src
|
||
|
||
#+DESCRIPTION: literate programming in Emacs Org Files.
|
||
|
||
#+PROPERTY: header-args:sh :tangle no
|
||
#+PROPERTY: header-args:emacs-lisp :tangle yes
|
||
#+PROPERTY: header-args :results none :eval no-export :comments no mkdirp yes
|
||
|
||
#+OPTIONS: num:nil toc:t todo:nil tasks:nil tags:nil date:nil
|
||
#+OPTIONS: skip:nil author:nil email:nil creator:nil timestamp:nil
|
||
#+INFOJS_OPT: view:nil toc:nil ltoc:t mouse:underline buttons:0 path:http://orgmode.org/org-info.js
|
||
|
||
# Local Variables:
|
||
# jinx-local-words: "parseable"
|
||
# End:
|