diff --git a/README.org b/README.org index 1edef0c..0aee6d7 100644 --- a/README.org +++ b/README.org @@ -22,6 +22,7 @@ This creates [[file:~/.emacs.d/init.el][~/.emacs.d/init.el]] that starts the pro - [[file:ha-org-journaling.org][org-journaling]] :: for writing journal entries and tasks. - [[file:ha-org-publishing.org][org-publishing]] :: code for publishing my website, [[http://howardism.org][www.howardism.org]]. - [[file:ha-org-sprint.org][org-sprint]] :: functions for working with the my Org-focused sprint file. + - [[file:ha-data.org][data]] :: functions for dealing with a buffer-full of data. - [[file:ha-eshell.org][eshell]] :: customization and enhancement to the Emacs shell. - [[file:ha-remoting.org][remoting]] :: my interface to systems using SSH and Vterm. - [[file:ha-email.org][email]] :: reading email using =notmuch= in a *Hey* fashion. diff --git a/bootstrap.org b/bootstrap.org index f277572..2001e5d 100644 --- a/bootstrap.org +++ b/bootstrap.org @@ -155,7 +155,37 @@ The [[https://github.com/magnars/s.el][s.el]] project is a simpler string manipu Manipulate file paths with the [[https://github.com/rejeep/f.el][f.el]] project: #+begin_src emacs-lisp -(use-package f) + (use-package f) +#+end_src + +The [[help:shell-command][shell-command]] function is useful, but having it split the output into a list is a helpful abstraction: +#+begin_src emacs-lisp + (defun shell-command-to-list (command) + "Return list of lines from running COMMAND in shell." + (thread-last command + shell-command-to-string + s-lines + (-map 's-trim) + (-remove 's-blank-str?))) +#+end_src + +And let’s see the results: +#+begin_src emacs-lisp :tangle no + (ert-deftest shell-command-to-list-test () + (should (equal '("hello world") + (shell-command-to-list "echo hello world"))) + + ;; We don't need blank lines: + (should (equal '("hello world" "goodbye for now") + (shell-command-to-list "echo '\n\nhello world\n\ngoodbye for now\n\n'")) + + ;; No output? Return null: + (should (null (shell-command-to-list "echo"))) + + ;; No line should contain carriage returns: + (should (null (seq-filter + (lambda (line) (s-contains? "\n" line)) + (shell-command-to-list "ls"))))) #+end_src ** My Code Location Much of my more complicated code comes from my website essays and other projects. The destination shows up here: @@ -198,6 +228,7 @@ The following loads the rest of my org-mode literate files. I add new filesas t "ha-org-clipboard.org" "ha-capturing-notes.org" "ha-agendas.org" + "ha-data.org" "ha-passwords.org" "ha-eshell.org" "ha-remoting.org" diff --git a/ha-config.org b/ha-config.org index fe0d2e5..72a5e18 100644 --- a/ha-config.org +++ b/ha-config.org @@ -90,16 +90,6 @@ And some Mac-specific settings: (add-to-list 'default-frame-alist '(ns-appearance . dark))) #+end_src * Support Packages -** Piper -Rewriting my shell scripts in Emacs Lisp uses my [[https://gitlab.com/howardabrams/emacs-piper][emacs-piper project]], and this code spills into my configuration code, so let's load it now: -#+begin_src emacs-lisp - (use-package piper - :straight (:host gitlab :repo "howardabrams/emacs-piper") - :commands piper shell-command-to-list ; I use this function often - :bind (:map evil-normal-state-map - ("C-M-|" . piper) - ("C-|" . piper-user-interface))) -#+end_src ** Yet Another Snippet System (YASnippets) Using [[https://github.com/joaotavora/yasnippet][yasnippet]] to convert templates into text: diff --git a/ha-data.org b/ha-data.org new file mode 100644 index 0000000..189ab1f --- /dev/null +++ b/ha-data.org @@ -0,0 +1,239 @@ +#+TITLE: Editing Data Files +#+AUTHOR: Howard X. Abrams +#+DATE: 2022-10-14 +#+FILETAGS: :emacs: + +A literate programming file for configuring Emacs to edit files of data. + +#+begin_src emacs-lisp :exports none + ;;; ha-data --- edit data files. -*- lexical-binding: t; -*- + ;; + ;; © 2022 Howard X. Abrams + ;; Licensed under a Creative Commons Attribution 4.0 International License. + ;; See http://creativecommons.org/licenses/by/4.0/ + ;; + ;; Author: Howard X. Abrams + ;; Maintainer: Howard X. Abrams + ;; Created: October 14, 2022 + ;; + ;; While obvious, GNU Emacs does not include this file or project. + ;; + ;; *NB:* Do not edit this file. Instead, edit the original literate file at: + ;; /Users/howard.abrams/other/hamacs/ha-data.org + ;; And tangle the file to recreate this one. + ;; + ;;; Code: + #+end_src +* Introduction +Once upon a time, I [[https://www.youtube.com/watch?v=HKJMDJ4i-XI][gave a talk]] to EmacsConf 2019, about [[http://howardism.org/Technical/Emacs/piper-presentation-transcript.html][an interesting idea]] I called [[https://gitlab.com/howardabrams/emacs-piper][emacs-piper]]. I still like the idea of sometimes editing an Emacs buffer on the entire contents, as if it were a data file. This file contains what I feel are the best functions for that… oh, and a leader to call it (instead of the original Hydra). + +#+begin_src emacs-lisp + (ha-leader + "d" '(:ignore t :which-key "data") + "d |" '("pipe to shell" . replace-buffer-with-shell-command) + "d y" '("copy to clipboard" . ha-yank-buffer-contents)) +#+end_src + +* Line-Oriented Functions +These functions focus on the data in the buffer as a series of lines: +#+begin_src emacs-lisp + (ha-leader + "d l" '(:ignore t :which-key "on lines") + "d l f" '("flush lines" . flush-lines) + "d l k" '("keep lines" . keep-lines) + "d l s" '("sort lines" . ha-sort-lines) + "d l u" '("unique lines" . delete-duplicate-lines) + "d l b" '("flush blanks" . flush-blank-lines)) +#+end_src + +One issue I have is [[help:keep-lines][keep-lines]] operate on the lines /starting with the point/, not on the entire buffer. Let’s fix that: +#+begin_src emacs-lisp + (defun call-function-at-buffer-beginning (orig-fun &rest args) + "Call ORIG-FUN after moving point to beginning of buffer. + Point restored after completion. Good for advice." + (save-excursion + (goto-char (point-min)) + (apply orig-fun args))) + + (advice-add 'keep-lines :around #'call-function-at-buffer-beginning) + (advice-add 'flush-lines :around #'call-function-at-buffer-beginning) +#+end_src + +The [[help:sort-lines][sort-lines]] is useful, but insists on an /active/ region. Let’s made a data-focused version: +#+begin_src emacs-lisp + (defun ha-sort-lines (prefix) + "Sort the lines in a buffer or region (if active). + If PREFIX given, sort in reverse order." + (interactive "P") + (save-excursion + (if (region-active-p) + (sort-lines prefix (region-beginning) (region-end)) + (sort-lines prefix (point-min) (point-max))))) +#+end_src + +Getting rid of blank lines seems somewhat useful: +#+begin_src emacs-lisp + (defun flush-blank-lines () + "Delete all empty lines in the buffer. See `flush-lines'." + (save-excursion + (goto-char (point-min)) + (flush-lines (rx line-start (zero-or-more space) line-end)))) +#+end_src +* Table-Oriented Functions +These functions focus on the data in the buffer as a table consisting of columns and rows of some sort. +#+begin_src emacs-lisp + (ha-leader + "d t" '(:ignore t :which-key "on tables") + "d t k" '("keep columns" . keep-columns) + "d t f" '("flush columns" . flush-columns)) +#+end_src + +Each of the /table/ functions require a /table separator/ (for instance the =|= character) and often the columns to operate on. +The =keep-columns= removes all text, except for /indexed/ text between the separator: +#+begin_src emacs-lisp + (defun keep-columns (separator columns) + "Keep tabular text columns, deleting the rest in buffer or region. + Defined columns as text between SEPARATOR, not numerical + position. Note that text _before_ the separator is column 0. + + For instance, given the following table: + + apple : avocado : apricot + banana : blueberry : bramble + cantaloupe : cherry : courgette : cucumber + data : durian + + Calling this function with a `:' character, as columns: `0, 2' + results in the buffer text: + + apple : apricot + banana : bramble + cantaloupe : courgette + data : durian" + (interactive "sSeparator: \nsColumns to Keep: ") + (operate-columns separator columns t)) +#+end_src + +The =flush-columns= is similar, except that is deletes the given columns. +#+begin_src emacs-lisp + (defun flush-columns (separator columns) + "Delete tabular text columns in buffer or region. + Defined columns as text between SEPARATOR, not numerical + position. Note that text _before_ the separator is column 0. + + For instance, given the following table: + + apple : avocado : apricot + banana : blueberry : bramble + cantaloupe : cherry : courgette : cucumber + data : durian + + Calling this function with a `:' character, as columns: `1' + (remember the colums are 0-indexed), + results in the buffer text: + + apple : avocado : apricot + banana : blueberry : bramble + cantaloupe : cherry : courgette : cucumber + data : durian + + apple : apricot + banana : bramble + cantaloupe : courgette + data : durian" + (interactive "sSeparator: \nsColumns to Delete: ") + (operate-columns separator columns nil)) +#+end_src + +Both functions are similar, and their behavior comes from =operate-columns=, which walks through the buffer, line-by-line: +#+begin_src emacs-lisp + (defun operate-columns (separator columns-str keep?) + "Call `operate-columns-on-line' for each line in buffer. + First, convert string COLUMNS-STR to a list of number, then + search for SEPARATOR." + (let ((columns (numbers-to-number-list columns-str))) + (save-excursion + (when (region-active-p) + (narrow-to-region (region-beginning) (region-end))) + (goto-char (point-min)) + (while (re-search-forward (rx (literal separator)) nil t) + (operate-columns-on-line separator columns t) + (next-line))))) +#+end_src + +For each line, the =operate-columns= calls this function: +#+begin_src emacs-lisp + (defun operate-columns-on-line (separator columns keep?) + "Replace current line after keeping or deleting COLUMNS. + Keep the COLUMNS if KEEP? is non-nil, delete otherwise. + Defined columns as the text between SEPARATOR." + (cl-labels ((keep-oper (idx it) (if keep? + (when (member idx columns) it) + (unless (member idx columns) it)))) + (let* ((start (line-beginning-position)) + (end (line-end-position)) + (line (buffer-substring start end)) + (parts (thread-last (split-string line separator) + (--map-indexed (keep-oper it-index it)) + (-remove 'null))) + (nline (string-join parts separator))) + (delete-region start end) + (insert nline)))) +#+end_src + +I like the idea of the shell command, =cut=, where you can have an arbitrary character as a separator, and then either delete or keep the data between them, as columns. But I need a function that can convert a string of “columns”, for instance ="1, 4-7 9"= to an list of numbers, like ='(1 4 5 6 7 9)=: +#+begin_src emacs-lisp + (defun numbers-to-number-list (input) + "Convert the string, INPUT, to a list of numbers. + For instance: `1, 4-7 9' returns `(1 4 5 6 7 9)'" + (let* ((separator (rx (* space) (or "," space) (* space))) + (dashed (rx (* space) "-" (* space))) + (ranged (rx (group (+ digit)) (regexp dashed) (group (+ digit)))) + (str-list (split-string input separator t))) + (--reduce-from (append acc (if (string-match ranged it) + (number-sequence + (string-to-number (match-string 1 it)) + (string-to-number (match-string 2 it))) + (list (string-to-number it)))) + () str-list))) +#+end_src +Does this work? +#+begin_src emacs-lisp :tangle no + (ert-deftest numbers-to-number-list-test () + (should (equal (numbers-to-number-list "2") '(2))) + (should (equal (numbers-to-number-list "1, 2 3") '(1 2 3))) + (should (equal (numbers-to-number-list "1, 4-7 9") '(1 4 5 6 7 9)))) +#+end_src +* Buffer-Oriented Functions +If there is no specific function, but you can think of a shell command that will work, then +#+begin_src emacs-lisp +(defun replace-buffer-with-shell-command (command) + "Replaces the contents of the buffer, or the contents of the +selected region, with the output from running an external +executable, COMMAND. + +This is a wrapper around `shell-command-on-region'." + (interactive "sCommand: ") + (save-excursion + (save-restriction + (when (region-active-p) + (narrow-to-region (region-beginning) (region-end))) + (shell-command-on-region (point-min) (point-max) command nil t)))) +#+end_src + +* Technical Artifacts :noexport: +Let's =provide= a name so we can =require= this file: +#+begin_src emacs-lisp :exports none + (provide 'ha-data) + ;;; ha-data.el ends here + #+end_src + +#+DESCRIPTION: configuring Emacs to edit files of data. + +#+PROPERTY: header-args:sh :tangle no +#+PROPERTY: header-args:emacs-lisp :tangle yes +#+PROPERTY: header-args :results none :eval no-export :comments no mkdirp yes + +#+OPTIONS: num:nil toc:nil todo:nil tasks:nil tags:nil date:nil +#+OPTIONS: skip:nil author:nil email:nil creator:nil timestamp:nil +#+INFOJS_OPT: view:nil toc:nil ltoc:t mouse:underline buttons:0 path:http://orgmode.org/org-info.js diff --git a/ha-org-clipboard.org b/ha-org-clipboard.org index 084fa54..4070f9a 100644 --- a/ha-org-clipboard.org +++ b/ha-org-clipboard.org @@ -160,7 +160,7 @@ Let's define the clipboard for a Mac. The challenge here is that we need to bina (mapconcat #'byte-to-string byte-seq "") 'utf-8)))) #+end_src -And define the same interface for Linux. Keep in mind, we need the exit code from calling a process, so I am going to define/use a helper function (which I should move into the piper project). +And define the same interface for Linux. Keep in mind, we need the exit code from calling a process, so I am going to define/use a helper function. #+begin_src emacs-lisp (defun ha-get-linux-clipboard () "Return the clipbaard for a Unix-based system. See `ha-get-clipboard'." diff --git a/ha-org.org b/ha-org.org index e03693a..3e35c8c 100644 --- a/ha-org.org +++ b/ha-org.org @@ -281,7 +281,7 @@ This function calls the above-mentioned operating-system-specific functions, but (ha-search-notes--macos phrase path) (ha-search-notes--linux phrase path)))) (->> command - (shell-command-to-list) ; Function from piper! + (shell-command-to-list) (--remove (s-matches? "~$" it)) (--remove (s-matches? "#" it)) (--map (format "'%s'" it))