hamacs/ha-email.org

Configuring Emacs for Email with Notmuch

A literate configuration file for email using Notmuch.

Introduction

To use this system, begin with SPC a m (after a SPC a n to asychronously download new mail … which probably should be running regularly).

When the Notmuch interface up, hit J to jump to one of the Search boxes (described below). Typically, this is i for the Imbox, check out the focused message from people I care). Hit q to return.

Next type s to view and organize mail I've never seen before. We need to keep things focused, so regularly making auto filtering rules is important. Move the point to the message and hit one of the following to automatically move the sender to a pre-defined box:

I

screened stuff that important enough to go to my Imbox

S

spam … so much goes here

P

receipts go to this Paper Trail, which takes a tag as the name of the store

f

mailing lists and other things that might be nice to read go to The Feed

Installation and Basic Configuration

To begin, we need the code. On Ubuntu, this is:

sudo apt install -y notmuch

And on MacOS, we use brew:

brew install notmuch

Next, we need some basic configuration settings and some global keybindings:

  (use-package notmuch
    :init
    (setq mail-user-agent 'notmuch-user-agent

          notmuch-mail-dir (format "%s/%s" (getenv "HOME") ".mail")
          notmuch-hooks-dir (expand-file-name ".notmuch/hooks" notmuch-mail-dir)

          notmuch-show-logo nil
          notmuch-message-deleted-tags '("+deleted" "-inbox" "-unread")
          notmuch-archive-tags '("-inbox" "-unread" "+archived")
          notmuch-show-mark-read-tags '("-inbox" "-unread" "+archived")
          notmuch-search-oldest-first nil
          notmuch-show-indent-content nil)

    :bind (:map notmuch-hello-mode-map
                ("U" . notmuch-retrieve-messages)   ; Defined later
                ("C" . notmuch-mua-new-mail))
          (:map notmuch-tree-mode-map
                ("C" . notmuch-mua-new-mail))
    :config (ha-leader   ; Should I put these under an "m" heading?
              "a n" '("new mail" . notmuch-retrieve-messages)
              "a m" '("read mail" . notmuch)
              "a c" '("compose mail" . notmuch-mua-new-mail))
            <<hey-show-keybindings>>
            <<hey-search-keybindings>>)

Also, let's do some basic configuration of Emacs' mail system:

  (setq mm-text-html-renderer 'shr
        mail-specify-envelope-from t
        message-kill-buffer-on-exit t
        message-send-mail-function 'message-send-mail-with-sendmail
        message-sendmail-envelope-from 'header)

Configuration

Do I want to sign messages by default? Nope.

(add-hook 'message-setup-hook 'mml-secure-sign-pgpmime)

Addresses

I need to incorporate an address book again, but in the meantime, searching through a history of past email works well enough.

(setq notmuch-address-selection-function
      (lambda (prompt collection initial-input)
        (completing-read prompt
                         (cons initial-input collection)
                         nil
                         t
                         nil
                         'notmuch-address-history)))

Sending Messages

Do I need to set up MSMTP? No, as Notmuch will do that work. To do this, type c and select an option (including r to reply).

Retrieving Messages

When we start notmuch, we need to retrieve the email and then process it. Most of this is actually contained in the Notmuch configuration.

(defun notmuch-retrieve-messages ()
  "Retrieve and process my mail messages."
  (interactive)
  (async-shell-command "notmuch new"))

iSync Configuration

Using isync (or is it mbsync) for mail retrieval. Currently, I have a couple of Google Mail accounts that I want connected.

The file generally can have a Pass entry for the encrypted passcode, but in order to demonstrate how to connect to multiple accounts, I'm using a GPG daemon:

# Note: We now tangle this file from ~/other/hamacs/ha-email.org
Create Both
SyncState *
MaxMessages 300
Sync All # New ReNew Flags

#  PERSONAL ACCOUNT
IMAPAccount personal
Host imap.gmail.com
User howard@howardabrams.com
# I took out the --no-tty as we need to get the password to decrypt the password file.
# Note that the password is actually the token for the account.
PassCmd "gpg --quiet --for-your-eyes-only --no-tty --decrypt ~/.mailpass-personal.gpg"
SSLType IMAPS
AuthMechs LOGIN

IMAPStore personal-remote
Account personal

MaildirStore personal-local
Path ~/.mail/personal/
Inbox ~/.mail/personal/INBOX
Flatten .

Channel personal-inbox
Master :personal-remote:
Slave :personal-local:
Patterns * !"[Gmail]/Drafts" !"[Gmail]/Spam"
Expunge Both
# Patterns "inbox"
# ExpireUnread no

Channel personal-sent
Master :personal-remote:"[Gmail]/Sent Mail"
Slave :personal-local:sent
ExpireUnread yes

Channel personal-trash
Master :personal-remote:"[Gmail]/Trash"
Slave :personal-local:trash
ExpireUnread yes

#  GMAIL ACCOUNT
IMAPAccount gmail
Host imap.gmail.com
User howard.abrams@gmail.com
PassCmd "gpg -q --for-your-eyes-only --pinentry-mode loopback -d ~/.mailpass-google.gpg"
SSLType IMAPS
AuthMechs LOGIN

IMAPStore gmail-remote
Account gmail

MaildirStore gmail-local
Path ~/.mail/gmail/
Inbox ~/.mail/gmail/INBOX
Flatten .

Channel gmail-inbox
Master :gmail-remote:
Slave :gmail-local:
Patterns * !"[Gmail]/Drafts" !"[Gmail]/Spam"
Expunge Both
# Patterns "inbox"

Channel gmail-sent
Master :gmail-remote:"[Gmail]/Sent Mail"
Slave :gmail-local:sent
ExpireUnread yes

Channel gmail-trash
Master :gmail-remote:"[Gmail]/Trash"
Slave :gmail-local:trash
ExpireUnread yes

Notmuch Configuration

Notmuch requires a few configuration files.

notmuch-config

The general settings file that goes into ~/.notmuch-config:

# .notmuch-config - Configuration file for the notmuch mail system
# Note: We now tangle this file from ~/other/hamacs/ha-email.org
#
# For more information about notmuch, see https://notmuchmail.org

The commentary for each of the subsections came from their man page.

Database configuration

The only value supported here is 'path' which should be the top-level directory where your mail currently exists and to where mail will be delivered in the future. Files should be individual email messages. Notmuch will store its database within a sub-directory of the path configured here named ".notmuch".

[database]
path=.mail

User configuration

Here is where you can let notmuch know how you would like to be addressed. Valid settings are

name

Your full name.

primary_email

Your primary email address.

other_email

A list (separated by ;) of other email addresses at which you receive email.

Notmuch will use the various email addresses configured here when formatting replies. It will avoid including your own addresses in the recipient list of replies, and will set the From address based on the address to which the original email was addressed.

[user]
name=Howard Abrams
primary_email=howard.abrams@gmail.com
other_email=howard@howardabrams.com;howard@fuzzytoast.com;

Configuration for "notmuch new"

The following options are supported here:

tags

A list (separated by ;) of the tags that will be added to all messages incorporated by "notmuch new".

ignore

A list (separated by ;) of file and directory names that will not be searched for messages by "notmuch new".

NOTE: Every file/directory that goes by one of those names will be ignored, independent of its depth/location in the mail store.

[new]
tags=unread;inbox;
ignore=

Search configuration

The following option is supported here:

exclude_tags

A ;-separated list of tags that will be excluded from search results by default. Using an excluded tag in a query will override that exclusion.

[search]
exclude_tags=deleted;spam;

Maildir compatibility configuration

The following option is supported here:

synchronize_flags

Valid values are true and false. If true, then the following maildir flags (in message filenames) will be synchronized with the corresponding notmuch tags:

Flag	Tag
D	draft
F	flagged
P	passed
R	replied
S	unread (added when 'S' flag is not present)

The notmuch new command will notice flag changes in filenames and update tags, while the notmuch tag and notmuch restore commands will notice tag changes and update flags in filenames.

[maildir]
synchronize_flags=true

That should complete the Notmuch configuration.

pre-new

Then we need a shell script called when beginning a retrieval, pre-new that simply calls mbsync to download all the messages:

# More info about hooks: https://notmuchmail.org/manpages/notmuch-hooks-5/
# Note: We now tangle this file from ~/other/hamacs/ha-email.org

echo "Starting not-much 'pre-new' script"

mbsync -a

echo "Completing not-much 'pre-new' script"

post-new

And a post-new hook based on a filtering scheme that mimics the Hey.com workflow taken from this gist (note we have more to say on that later on) to filter and tag all messages after they have arrived:

# Based On: https://gist.githubusercontent.com/frozencemetery/5042526/raw/57195ba748e336de80c27519fe66e428e5003ab8/post-new
# Note: We now tangle this file from ~/other/hamacs/ha-email.org
#
# Install this by moving this file to <maildir>/.notmuch/hooks/post-new
# NOTE: you need to define your maildir in the vardiable nm_maildir (just a few lines below in this script)
# Also create empty files for:
# 1. thefeed.db (things you want to read every once in a while)
# 2. spam.db (things you never want to see)
# 3. screened.db (your inbox)
# 4. ledger.db (papertrail)
# in the hooks folder.
# More info about hooks: https://notmuchmail.org/manpages/notmuch-hooks-5/

# Note:
#    Old emails:  notmuch search --output summary NOT date:30d.. and tag:unread
#    Ignore old emails: notmuch tag -unread --output summary NOT date:30d.. and tag:unread

echo "Starting not-much 'post-new' script"
export nm_maildir="$HOME/.mail"
export start="-1"

echo Working from $nm_maildir

function timer_start {
    echo -n "    starting $1"
    export start=$(date +"%s")
}

function timer_end {
    end=$(date +"%s")
    delta=$(($end-$start))
    mins=$(($delta / 60))
    secs=$(($delta - ($mins*60)))
    echo " -- $1 completed: ${mins} minutes, ${secs} seconds"
    export start="-1" # sanity requires this or similar
}

timer_start "ledger"
while IFS= read -r line; do
    nm_tag=$(echo "$line" | cut -d' ' -f1 -)
    nm_entry=$(echo "$line" | cut -d' ' -f2 -)
    if [ -n "$nm_entry" ]
    then
        notmuch tag +archived +ledger/"$nm_tag" -inbox -- tag:inbox and tag:unread and from:"$nm_entry"
    fi
    echo -n "Handling entry: $nm_tag, $nm_entry"
done < $nm_maildir/.notmuch/hooks/ledger.db
timer_end "ledger"

timer_start "unsubscribable_spam"
for entry in $(cat $nm_maildir/.notmuch/hooks/spam.db)
do
    if [ -n "$entry" ]
    then
        notmuch tag +spam +deleted +archived -inbox -unread -- tag:inbox and tag:unread and from:"$entry"
    fi
done
timer_end "unsubscribable_spam"

timer_start "thefeed"
for entry in $(cat $nm_maildir/.notmuch/hooks/thefeed.db)
do
    if [ -n "$entry" ]
    then
        notmuch tag +thefeed +archived -inbox -- tag:inbox and tag:unread and from:"$entry"
    fi
done
timer_end "thefeed"

timer_start "Screened"

notmuch tag +screened 'subject:/\[Web\]/'
for entry in $(cat $nm_maildir/.notmuch/hooks/screened.db)
do
    if [ -n "$entry" ]
    then
        notmuch tag +screened -- from:"$entry" # tag:unread and tag:inbox and
    fi
done
timer_end "Screened"

# Projects...

timer_start "Old-Projects"
notmuch tag +old-project 'subject:/.*howardabrams\/node-mocks-http/'
notmuch tag +old-project 'subject:/.*Pigmice2733/'
timer_end "Old-Projects"

notmuch tag +screened 'subject:[Web]'

echo "Completing not-much 'post-new' script"

Hey

I originally took the following configuration from Vedang Manerikar's video, along with the code. The ideas brought out were to mimic the hey.com email workflow, and while not bad, I thought that maybe I could improve upon it slowly over time.

To allow me to keep Vedang's and my code side-by-side in the same Emacs variable state, I have renamed the prefix to hey-, however, if you are looking to steal my code, you may want to revisit the source.

Default Searches

A list of pre-defined searches act like "Folder buttons" at the top to quickly see files that match those buckets:

(setq notmuch-saved-searches '((:name "Imbox"
                                      :query "tag:inbox AND tag:screened AND tag:unread"
                                      :key "i"
                                      :search-type 'tree)
                               (:name "Previously Seen"
                                      :query "tag:screened AND NOT tag:unread"
                                      :key "I")
                               (:name "Unscreened"
                                      :query "tag:inbox AND tag:unread AND NOT tag:screened AND NOT date:..14d AND NOT tag:thefeed AND NOT tag:/ledger/ AND NOT tag:old-project"
                                      :key "s")
                               (:name "New Feed"
                                      :query "tag:thefeed AND tag:unread"
                                      :key "f"
                                      :search-type 'tree)
                               (:name "Old Feed"
                                      :query "tag:thefeed"
                                      :key "f"
                                      :search-type 'tree)
                               (:name "New Receipts"
                                      :query "tag:/ledger/ AND tag:unread"
                                      :key "p")
                               (:name "Papertrail"
                                      :query "tag:/ledger/"
                                      :key "P")

                               ;; (push '(:name "Projects"
                               ;;               :query "tag:project AND NOT tag:unread"
                               ;;               :key "x")
                               ;;       notmuch-saved-searches)
                               (:name "Old Projects"
                                      :query "tag:old-project AND NOT tag:unread"
                                      :key "X")))

Helper Functions

With good bucket definitions, we should be able to scan the mail quickly and deal with the entire lot of them:

(defun hey-notmuch-archive-all ()
  "Archive all the emails in the current view."
  (interactive)
  (notmuch-search-archive-thread nil (point-min) (point-max)))

(defun hey-notmuch-delete-all ()
  "Archive all the emails in the current view.
Mark them for deletion by cron job."
  (interactive)
  (notmuch-search-tag-all '("+deleted"))
  (hey-notmuch-archive-all))

(defun hey-notmuch-search-delete-and-archive-thread ()
  "Archive the currently selected thread. Add the deleted tag as well."
  (interactive)
  (notmuch-search-add-tag '("+deleted"))
  (notmuch-search-archive-thread))

(defun hey-notmuch-tag-and-archive (tag-changes &optional beg end)
  "Prompt the user for TAG-CHANGES.
Apply the TAG-CHANGES to region and also archive all the emails.
When called directly, BEG and END provide the region."
  (interactive (notmuch-search-interactive-tag-changes))
  (notmuch-search-tag tag-changes beg end)
  (notmuch-search-archive-thread nil beg end))

A key point in organizing emails with the Hey model, is looking at the "from" address:

(defun hey-notmuch-search-find-from ()
  "A helper function to find the email address for the given email."
  (let ((notmuch-addr-sexp (first
                            (notmuch-call-notmuch-sexp "address"
                                                       "--format=sexp"
                                                       "--format-version=1"
                                                       "--output=sender"
                                                       (notmuch-search-find-thread-id)))))
    (plist-get notmuch-addr-sexp :address)))

And we can create a filter, search and tagging based on this "from" function:

(defun hey-notmuch-filter-by-from ()
  "Filter the current search view to show all emails sent from the sender of the current thread."
  (interactive)
  (notmuch-search-filter (concat "from:" (hey-notmuch-search-find-from))))

(defun hey-notmuch-search-by-from (&optional no-display)
  "Show all emails sent from the sender of the current thread.
NO-DISPLAY is sent forward to `notmuch-search'."
  (interactive)
  (notmuch-search (concat "from:" (hey-notmuch-search-find-from))
                  notmuch-search-oldest-first nil nil no-display))

(defun hey-notmuch-tag-by-from (tag-changes &optional beg end refresh)
  "Apply TAG-CHANGES to all emails from the sender of the current thread.
BEG and END provide the region, but are ignored. They are defined
since `notmuch-search-interactive-tag-changes' returns them. If
REFRESH is true, refresh the buffer from which we started the
search."
  (interactive (notmuch-search-interactive-tag-changes))
  (let ((this-buf (current-buffer)))
    (hey-notmuch-search-by-from t)
    ;; This is a dirty hack since I can't find a way to run a
    ;; temporary hook on `notmuch-search' completion. So instead of
    ;; waiting on the search to complete in the background and then
    ;; making tag-changes on it, I will just sleep for a short amount
    ;; of time. This is generally good enough and works, but is not
    ;; guaranteed to work every time. I'm fine with this.
    (sleep-for 0.5)
    (notmuch-search-tag-all tag-changes)
    (when refresh
      (set-buffer this-buf)
      (notmuch-refresh-this-buffer))))

Moving Mail to Buckets

We based the Hey buckets on notmuch databases, we combine the hey-notmuch-add-addr-to-db with the hey-notmuch-tag-by-from functions to move messages.

(defun hey-notmuch-add-addr-to-db (nmaddr nmdbfile)
  "Add the email address NMADDR to the db-file NMDBFILE."
  (append-to-file (format "%s\n" nmaddr) nil nmdbfile))

(defun hey-notmuch-move-sender-to-thefeed ()
  "For the email at point, move the sender of that email to the feed.
This means:
1. All new email should go to the feed and skip the inbox altogether.
2. All existing email should be updated with the tag =thefeed=.
3. All existing email should be removed from the inbox."
  (interactive)
  (hey-notmuch-add-addr-to-db (hey-notmuch-search-find-from)
                              (format "%s/thefeed.db" notmuch-hooks-dir))
  (hey-notmuch-tag-by-from '("+thefeed" "+archived" "-inbox")))

(defun hey-notmuch-move-sender-to-papertrail (tag-name)
  "For the email at point, move the sender of that email to the papertrail.
This means:
1. All new email should go to the papertrail and skip the inbox altogether.
2. All existing email should be updated with the tag =ledger/TAG-NAME=.
3. All existing email should be removed from the inbox."
  (interactive "sTag Name: ")
  (hey-notmuch-add-addr-to-db (format "%s %s"
                                      tag-name
                                      (hey-notmuch-search-find-from))
                              (format "%s/ledger.db" notmuch-hooks-dir))
  (let ((tag-string (format "+ledger/%s" tag-name)))
    (hey-notmuch-tag-by-from (list tag-string "+archived" "-inbox" "-unread"))))

(defun hey-notmuch-move-sender-to-screened ()
  "For the email at point, move the sender of that email to Screened Emails.
This means:
1. All new email should be tagged =screened= and show up in the inbox.
2. All existing email should be updated to add the tag =screened=."
  (interactive)
  (hey-notmuch-add-addr-to-db (hey-notmuch-search-find-from)
                                 (format "%s/screened.db" notmuch-hooks-dir))
  (hey-notmuch-tag-by-from '("+screened")))

(defun hey-notmuch-move-sender-to-spam ()
  "For the email at point, move the sender of that email to spam.
This means:
1. All new email should go to =spam= and skip the inbox altogether.
2. All existing email should be updated with the tag =spam=.
3. All existing email should be removed from the inbox."
  (interactive)
  (hey-notmuch-add-addr-to-db (hey-notmuch-search-find-from)
                                 (format "%s/spam.db" notmuch-hooks-dir))
  (hey-notmuch-tag-by-from '("+spam" "+deleted" "+archived" "-inbox" "-unread" "-screened")))

(defun hey-notmuch-reply-later ()
  "Capture this email for replying later."
  (interactive)
  ;; You need `org-capture' to be set up for this to work. Add this
  ;; code somewhere in your init file after `org-cature' is loaded:

  ;; (push '("r" "Respond to email"
  ;;         entry (file org-default-notes-file)
  ;;         "* TODO Respond to %:from on %:subject  :email: \nSCHEDULED: %t\n%U\n%a\n"
  ;;         :clock-in t
  ;;         :clock-resume t
  ;;         :immediate-finish t)
  ;;       org-capture-templates)

  (org-capture nil "r")

  ;; The rest of this function is just a nice message in the modeline.
  (let* ((email-subject (format "%s..."
                                (substring (notmuch-show-get-subject) 0 15)))
         (email-from (format "%s..."
                             (substring (notmuch-show-get-from) 0 15)))
         (email-string (format "%s (From: %s)" email-subject email-from)))
    (message "Noted! Reply Later: %s" email-string)))

Bucket Keybindings

A series of keybindings to quickly send messages to one of the pre-defined buckets. The notmuch-show-mode is the … uhm

  (define-key notmuch-show-mode-map (kbd "C") 'hey-notmuch-reply-later)

The bindings in notmuch-search-mode are available when looking at a list of messages:

  (define-key notmuch-search-mode-map (kbd "r") 'notmuch-search-reply-to-thread)
  (define-key notmuch-search-mode-map (kbd "R") 'notmuch-search-reply-to-thread-sender)
  (define-key notmuch-search-mode-map (kbd "/") 'notmuch-search-filter)
  (define-key notmuch-search-mode-map (kbd "A") 'hey-notmuch-archive-all)
  (define-key notmuch-search-mode-map (kbd "D") 'hey-notmuch-delete-all)
  (define-key notmuch-search-mode-map (kbd "L") 'hey-notmuch-filter-by-from)
  (define-key notmuch-search-mode-map (kbd ";") 'hey-notmuch-search-by-from)
  (define-key notmuch-search-mode-map (kbd "d") 'hey-notmuch-search-delete-and-archive-thread)

  (define-key notmuch-search-mode-map (kbd "S") 'hey-notmuch-move-sender-to-spam)
  (define-key notmuch-search-mode-map (kbd "I") 'hey-notmuch-move-sender-to-screened)
  (define-key notmuch-search-mode-map (kbd "P") 'hey-notmuch-move-sender-to-papertrail)
  (define-key notmuch-search-mode-map (kbd "f") 'hey-notmuch-move-sender-to-thefeed)
  (define-key notmuch-search-mode-map (kbd "C") 'hey-notmuch-reply-later)

Org Integration

The gods ordained that Mail and Org should dance together, so step one is composing mail with org:

  (use-package org-mime
    :config
    (general-evil-define-key 'normal notmuch-message-mode-map
      :prefix "SPC m"
      "s" '("send" . notmuch-mua-send-and-exit)
      "m" '("mime it" . org-mime-htmlize)))

A new option is to use org-msg, so let's try it:

  (use-package org-msg
    :init
    (setq org-msg-options "html-postamble:nil H:5 num:nil ^:{} toc:nil author:nil email:nil \\n:t"
          org-msg-startup "hidestars indent inlineimages"
          org-msg-greeting-fmt "\nHi%s,\n\n"
          org-msg-recipient-names '(("howard.abrams@gmail.com" . "Howard Abrams"))
          org-msg-greeting-name-limit 3
          org-msg-default-alternatives '((new		. (text html))
                                         (reply-to-html	. (text html))
                                         (reply-to-text	. (text)))
          org-msg-convert-citation t
          org-msg-signature "

   Regards,

   ,#+begin_signature
   --
   ,*Howard*
   /One Emacs to rule them all/
   ,#+end_signature"))

The idea of linking org documents to email could be nice, however, the ol-notmuch package in the org-contrib package needs a maintainer.

  (use-package ol-notmuch
    :after org
    :straight (:type built-in)
    :config (add-to-list 'org-modules 'ol-notmuch))

To use, read a message and save a link to it with SPC o l. Next, in an org document, create a link with SPC m l. Now, you can return to the message from that document with SPC m o. Regardless, I may need to store a local copy when I upgrade Org.

Display Configuration

Using the Doom Modeline to add notifications:

  (setq doom-modeline-mu4e t)