hamacs/ha-remoting.org

#+title:  Remote Access to Systems
#+author: Howard X. Abrams
#+date:   2020-09-25
#+tags: emacs ssh shell

A literate configuration for accessing remote systems.

#+begin_src emacs-lisp :exports none
  ;;; ha-remoting --- Accessing remote systems. -*- lexical-binding: t; -*-
  ;;
  ;; © 2020-2023 Howard X. Abrams
  ;;   Licensed under a Creative Commons Attribution 4.0 International License.
  ;;   See http://creativecommons.org/licenses/by/4.0/
  ;;
  ;; Author: Howard X. Abrams <http://gitlab.com/howardabrams>
  ;; Maintainer: Howard X. Abrams
  ;; Created: September 25, 2020
  ;;
  ;; This file is not part of GNU Emacs.
  ;;
  ;; *NB:* Do not edit this file. Instead, edit the original literate file at:
  ;;            ~/other/hamacs/ha-remoting.org
  ;;       And tangle the file to recreate this one.
  ;;
  ;;; Code:
#+end_src
* Remote Editing with Tramp
[[https://www.emacswiki.org/emacs/TrampMode][Tramp]] allows almost all Emacs features to execute on a remote system.
#+begin_src emacs-lisp
  (use-package tramp
    :straight (:type built-in)

    :config
    ;; Use remote PATH on tramp (handy for eshell).
    (add-to-list 'tramp-remote-path 'tramp-own-remote-path)

    ;; Make sure version control system doesn't slow tramp:
    (setq vc-ignore-dir-regexp
          (format "%s\\|%s" vc-ignore-dir-regexp tramp-file-name-regexp)))
#+end_src

Will Schenk has [[https://willschenk.com/articles/2020/tramp_tricks/][a simple extension]] to allow editing of files /inside/ a Docker container:
#+begin_src emacs-lisp
  (use-package tramp
    :straight (:type built-in)
    :config
    (push '("docker" . ((tramp-login-program "docker")
                        (tramp-login-args (("exec" "-it") ("%h") ("/bin/sh")))
                        (tramp-remote-shell "/bin/sh")
                        (tramp-remote-shell-args ("-i") ("-c"))))
          tramp-methods)

    (defadvice tramp-completion-handle-file-name-all-completions
        (around dotemacs-completion-docker activate)
      "(tramp-completion-handle-file-name-all-completions \"\" \"/docker:\" returns
      a list of active Docker container names, followed by colons."
      (if (equal (ad-get-arg 1) "/docker:")
          (let* ((command "docker ps --format '{{.Names}}:'")
                 (dockernames-raw (shell-command-to-string command))
                 (dockernames (split-string dockernames-raw "\n")))
            (setq ad-return-value dockernames))
        ad-do-it)))
#+end_src
Keep in mind you need to /name/ your Docker session, with the =—name= option. I actually do more docker work on remote systems (as Docker seems to make my fans levitate my laptop over the desk). Granted, the =URL= is a bit lengthy, for instance:
#+begin_example
/ssh:kolla-compute1.cedev13.d501.eng.pdx.wd|sudo:kolla-compute1.cedev13.d501.eng.pdx.wd|docker:kolla_toolbox:/
#+end_example
Which means, I need to put it as a link in an org file.

*Note:* That we need to have Tramp SSH option comes from my personal [[file:~/.ssh/config][.ssh/config]] file instead of its internal cache:
 #+begin_src emacs-lisp
  (use-package tramp-sh
    :after tramp
    :straight (:type built-in)
    :custom (tramp-use-ssh-controlmaster-options nil))
#+end_src
* Remote Terminals
Sure =iTerm= is nice for connecting and running commands on remote systems, however, it lacks a command line option that allows you to select and manipulate the displayed text without a mouse. This is where Emacs can shine.

*Feature One:*

When calling the =ha-ssh= function, it opens a =vterm= window which, unlike other terminal emulators in Emacs, merges both Emacs and Terminal behaviors. Essentially, it just works. It =vterm= isn't installed, it falls back to =term=.

Preload a list of favorite/special hostnames with multiple calls to:

#+begin_src emacs-lisp :tangle no
(ha-ssh-add-favorite-host "Devbox 42" "10.0.1.42")
#+end_src

Then calling =ha-ssh= function, a list of hostnames is available to quickly jump on a system (with the possibility of fuzzy matching if you have Helm or Ivy installed).

This also has the ability to call OpenStack to gather the hostnames of dynamic systems (what I call "an Overcloud"), which is appended to the list of favorite hostnames. The call to OpenStack only needs to be called once, since the hosts are then cached, see =ha-ssh-overcloud-query-for-hosts=.

*Feature Two:*

Use the /favorite host/ list to quickly edit a file on a remote system using Tramp, by calling either =ha-ssh-find-file= and =ha-ssh-find-root=.

*Feature Three:*

Working with remote shell connections programmatically, for instance:

#+begin_src emacs-lisp :tangle no
(let ((win-name "some-host"))
  (ha-ssh "some-host.in.some.place" win-name)
  (ha-ssh-send "source ~/.bash_profile" win-name)
  (ha-ssh-send "clear" win-name))
;; ...
(ha-ssh-exit win-name)
#+end_src

Actually the =win-name= in this case is optional, as it will use a good default.


** VTerm
I'm not giving up on Eshell, but I am playing around with [[https://github.com/akermu/emacs-libvterm][vterm]], and it is pretty good, but I use it primarily as a more reliable approach for remote terminal sessions.

VTerm has an issue (at least for me) with ~M-Backspace~ not deleting the previous word, and yeah, I want to make sure that both keystrokes do the same thing.

#+begin_src emacs-lisp
  (use-package vterm
    :config
    (dolist (k '("<C-backspace>" "<M-backspace>"))
      (define-key vterm-mode-map (kbd k)
                  (lambda () (interactive) (vterm-send-key (kbd "C-w")))))

    ;; Enter copy mode? Go to Evil's normal state to move around:
    (advice-add 'vterm-copy-mode :after 'evil-normal-state)

    :hook
    (vterm-mode . (lambda ()
                    (setq-local evil-insert-state-cursor 'box)
                    (setq-local show-paren-mode nil)
                    (setf truncate-lines nil
                          vterm-use-vterm-prompt-detection-method nil
                          term-prompt-regexp "^.* $ ")
                    (flycheck-mode -1)
                    (yas-minor-mode -1)
                    (evil-insert-state))))
#+end_src

The advantage of running terminals in Emacs is the ability to copy text without a mouse. For that, hit ~C-c C-t~ to enter a special copy-mode. If I go into this mode, I might as well also go into normal mode to move the cursor. To exit the copy-mode (and copy the selected text to the clipboard), hit ~Return~.

** Variables
Let's begin by defining some variables used for communication between the functions.

#+begin_src emacs-lisp
  (defvar ha-latest-ssh-window-name nil
    "The window-name of the latest ssh session. Most commands default to the last session.")

  (defvar ha-ssh-host-history '() "List of hostnames we've previously connected.")

  (defvar ha-ssh-favorite-hostnames '()
    "A list of tuples (associate list) containing a hostname and its IP address.
  See =ha-ssh-add-favorite-host= for easily adding to this list.")
#+end_src

Also, let's make it easy for me to change my default shell:
#+begin_src emacs-lisp
  (defvar ha-ssh-shell (shell-command-to-string "type -p fish")
    "The executable to the shell I want to use locally.")
#+end_src

** Interactive Interface to Remote Systems

The function, =ha-ssh= pops up a list of /favorite hosts/ and then uses the =vterm= functions to automatically SSH into the chosen host:

#+begin_src emacs-lisp
(defun ha-ssh (hostname &optional window-name)
  "Start a SSH session to a given HOSTNAME (with an optionally specified WINDOW-NAME).
If called interactively, it presents the user with a list
returned by =ha-ssh-choose-host=."
  (interactive (list (ha-ssh-choose-host)))
  (unless window-name
    (setq window-name (format "ssh: %s" hostname)))
  (setq ha-latest-ssh-window-name (format "*%s*" window-name))

  ;; I really like this =vterm= interface, so if I've got it loaded, let's use it:
  (if (not (fboundp 'vterm))
      ;; Should we assume the =ssh= we want is on the PATH that started Emacs?
      (make-term window-name "ssh" nil hostname)
    (vterm ha-latest-ssh-window-name)
    (vterm-send-string (format "ssh %s" hostname))
    (vterm-send-return))

  (pop-to-buffer ha-latest-ssh-window-name))
#+end_src

Of course, we need a function that =interactive= can call to get that list, and my thought is to call =helm= if it is available, otherwise, assume that ido/ivy will take over the =completing-read= function:

#+begin_src emacs-lisp
(defun ha-ssh-choose-host ()
  "Prompts the user for a host, and if it is in the cache, return
its IP address, otherwise, return the input given.
This is used in calls to =interactive= to select a host."
  (let ((hostname
         ;; We call Helm directly if installed, only so that we can get better
         ;; labels in the window, otherwise, the =completing-read= call would be fine.
         (if (fboundp 'helm-comp-read)
             (helm-comp-read "Hostname: " ha-ssh-favorite-hostnames
                             :name "Hosts"
                             :fuzzy t :history ha-ssh-host-history)
           (completing-read "Hostname: " ha-ssh-favorite-hostnames nil 'confirm nil 'ha-ssh-host-history))))
    (alist-get hostname ha-ssh-favorite-hostnames hostname nil 'equal)))
#+end_src

Simply calling =vterm= fails to load my full environment, so this allows me to start the terminal in a particular directory (defaulting to the root of the current project):

#+begin_src emacs-lisp
  (defun ha-shell (&optional directory name)
    "Creates and tidies up a =vterm= terminal shell in side window."
    (interactive (list (read-directory-name "Starting Directory: " (projectile-project-root))))
    (let* ((win-name (or name (ha-shell--name-from-dir directory)))
           (buf-name (format "*%s*" win-name))
           (default-directory (or directory default-directory)))
      (setq ha-latest-ssh-window-name buf-name)
      (if (not (fboundp 'vterm))
          (make-term win-name ha-ssh-shell)
        (vterm buf-name))))
#+end_src

Before we leave this section, I realize that I would like a way to /add/ to my list of hosts:
#+begin_src emacs-lisp
(defun ha-ssh-add-favorite-host (hostname ip-address)
  "Add a favorite host to your list for easy pickin's."
  (interactive "sHostname: \nsIP Address: ")
  (add-to-list 'ha-ssh-favorite-hostnames (cons hostname ip-address)))
#+end_src

Let's have a quick way to bugger out of the terminal:
#+begin_src emacs-lisp
(defun ha-ssh-exit (&optional window-name)
  "End the SSH session specified by WINDOW-NAME (or if not, the latest session)."
  (interactive)
  (unless (string-match-p "v?term" (buffer-name))
    (unless window-name
      (setq window-name ha-latest-ssh-window-name))
    (pop-to-buffer window-name))

  (ignore-errors
    (term-send-eof))
  (kill-buffer window-name)
  (delete-window))
#+end_src
** Programmatic Interface
Now that Emacs can /host/ a Terminal shell, I would like to /programmatically/ send commands to the running terminal, e.g. =(ha-shell-send "ls *.py")=

Since every project perspective may have a shell terminal, let’s see if I can figure which shell buffer to send—based on the =current-directory=.
#+begin_src emacs-lisp
  (defun ha-shell-send (command &optional directory)
    "Send COMMAND to existing shell terminal based on DIRECTORY.
  If the shell doesn't already exist, start on up by calling
  the `ha-shell' function.

  The real work for this is done by `ha-ssh-send'.

  If DIRECTORY is nil, use the project root from projectile."
    (let ((buf (ha-shell--buf-from-dir directory)))
      (unless buf
        (setq buf (ha-shell directory)))
      (ha-ssh-send command buf)))

  (defun ha-shell--buf-from-dir (directory)
    "Return Terminal buffer associated with DIRECTORY.
  Or nil if no buffer has been found."
    (let* ((win-name (ha-shell--name-from-dir directory))
           (win-rx   (rx "*" (literal win-name) "*"))
           (bufs     (seq-filter (lambda (b) (when (string-match win-rx (buffer-name b)) b))
                                 (buffer-list))))
       (first bufs)))

  (defun ha-shell--name-from-dir (&optional directory)
    "Return an appropriate title for a terminal based on DIRECTORY.
  If DIRECTORY is nil, use the `projectile-project-name'."
    (unless directory
      (setq directory (projectile-project-name)))
    (let ((name
           ;; Most of the time I just want the base project name, but in
           ;; my "work" directory, the projects are too similar, and I
           ;; need two levels of directories to distinguish them as a
           ;; project.
           (if (s-contains? "/work/" directory)
               (thread-last directory
                            (s-split "/")
                            (-remove 's-blank-str?)
                            (-take-last 2)
                            (s-join "/"))
             (file-name-base (directory-file-name directory)))))
      (format "Terminal: %s" name)))
#+end_src

Perhaps a Unit test is in order:
#+begin_src emacs-lisp :tangle no
  (ert-deftest ha--terminal-name-from-dir-test ()
    (should
     (string= (ha-shell--name-from-dir "~/other/hamacs/") "Terminal: hamacs"))
    (should
     (string= (ha-shell--name-from-dir "~/work/foo/bar") "Terminal: foo/bar"))
    (should
     (string= (ha-shell--name-from-dir) "Terminal: hamacs")))
#+end_src

The previous functions (as well as my own end of sprint demonstrations) often need to issue some commands to a running terminal session, which is a simple wrapper around a /send text/ and /send return/ sequence:
#+begin_src emacs-lisp
  (defun ha-ssh-send (phrase &optional window-name)
    "Send command PHRASE to the currently running SSH instance.
  If you want to refer to another session, specify the correct WINDOW-NAME.
  This is really useful for scripts and demonstrations."
    (unless window-name
      (setq window-name ha-latest-ssh-window-name))
    (save-window-excursion
      (pop-to-buffer window-name)

      (if (fboundp 'vterm)
          (progn
            (vterm-send-string phrase)
            (vterm-send-return))
        (progn
          (term-send-raw-string phrase)
          (term-send-input)))))
#+end_src

As you may know, I’m big into /literate devops/ where I put my shell commands in org files. However, I also work as part of a team that for some reason, doesn’t accept Emacs as their One True Editor. At least, I am able to talk them into describing commands in Markdown files, e.g. =README.md=. Instead of /copying-pasting/ into the shell, could I /send/ the /current command/ to that shell?
#+begin_src emacs-lisp
  (defun ha-ssh-send-line (prefix)
    "Copy the contents of the current line in the current buffer,
  and call =ha-ssh-send= with it. After sending the contents, it
  returns to the current line."
    (interactive "P")
    ;; The function =save-excursion= doesn't seem to work...
    (let ((buf (current-buffer)))
      (dolist (line (ha-ssh--line-or-block prefix))
        ;; (sit-for 0.25)
        (ha-ssh-send line))
      (pop-to-buffer buf)))
#+end_src

What does /current command/ mean? The current line? A good fall back. Selected region? Sure, if active, but that seems like more work. In a Markdown file, I can gather the entire source code block, just like in an Org file.
So the following function may be a bit complicated in determining what is this /current code/:
#+begin_src emacs-lisp
  (defun ha-ssh--line-or-block (num-lines)
    "Return a list of the NUM-LINES from current buffer.
  If NUM-LINES is nil, then follow these rules:
  If the region is active, return the lines from that.
  If in an org-mode block, return that block.
  If in a Markdown file, return the triple-back-tick code,
  or the indented code, or the inline code between single ticks.
  Otherwise, just return the current line."
    (ha-ssh--line-cleanup
     (cond
      ((and num-lines (numberp num-lines))
       (buffer-substring-no-properties
        (line-beginning-position) (line-end-position num-lines)))

      ;; Region active?
      ((region-active-p)
       (buffer-substring-no-properties
        (region-beginning) (region-end)))

      ;; In org? Use the block
      ((and (eq major-mode 'org-mode) (org-in-src-block-p))
       (org-element-property :value (org-element-at-point)))

      ;; In Markdown block?
      ((and (eq major-mode 'markdown-mode) (markdown-code-block-at-point-p))
       (buffer-substring-no-properties
        (car (markdown-code-block-at-point-p))
        (cadr (markdown-code-block-at-point-p))))

      ;; In Markdown code that is just on part of the line?
      ((and (eq major-mode 'markdown-mode) (markdown-inline-code-at-point-p))
       (buffer-substring-no-properties
        (car (markdown-inline-code-at-point-p))
        (cadr (markdown-inline-code-at-point-p))))

      (t      ; Otherwise, just grab the current line:
       (buffer-substring-no-properties
        (line-beginning-position) (line-end-position))))))
#+end_src

In Markdown (and org), I might have initial spaces that should be removed (but not all initial spaces):
#+begin_src emacs-lisp
  (defun ha-ssh--line-cleanup (str)
    "Return STR as a list of strings."
    (let* ((lst-contents (thread-last str
                                      (s-split "\n")
                                      (-remove 's-blank-str-p)))
           (first-line   (car lst-contents))
           (trim-amount  (when (string-match (rx bol (group (* space))) first-line)
                           (length (match-string 1 first-line)))))
      (mapcar (lambda (line) (substring line trim-amount)) lst-contents)))

  (ert-deftest ha-ssh--line-cleanup-test ()
    (should (equal (ha-ssh--line-cleanup "bob") '("bob")))
    (should (equal (ha-ssh--line-cleanup "  bob") '("bob")))
    (should (equal (ha-ssh--line-cleanup "bob\nfoo") '("bob" "foo")))
    (should (equal (ha-ssh--line-cleanup "  bob\n  foo") '("bob" "foo")))
    (should (equal (ha-ssh--line-cleanup "  bob\n    foo") '("bob" "  foo"))))
#+end_src

** Editing Remote Files
TRAMP, when it works, is amazing that we can give it a reference to a remote directory, and have =find-file= magically autocomplete.

#+begin_src emacs-lisp
(defun ha-ssh-find-file (hostname)
  "Constructs a ssh-based, tramp-focus, file reference, and then calls =find-file=."
  (interactive (list (ha-ssh-choose-host)))
  (let ((tramp-ssh-ref (format "/ssh:%s:" hostname))
        (other-window (when (equal current-prefix-arg '(4)) t)))
    (ha-ssh--find-file tramp-ssh-ref other-window)))

(defun ha-ssh--find-file (tramp-ssh-ref &optional other-window)
  "Calls =find-file= after internally completing a file reference based on TRAMP-SSH-REF."
  (let ((tramp-file (read-file-name "Find file: " tramp-ssh-ref)))
    (if other-window
        (find-file-other-window tramp-file)
      (find-file tramp-file))))
#+end_src

We can even edit it as root:

#+begin_src emacs-lisp
(defun ha-ssh-find-root (hostname)
  "Constructs a ssh-based, tramp-focus, file reference, and then calls =find-file=."
  (interactive (list (ha-ssh-choose-host)))
  (let ((tramp-ssh-ref (format "/ssh:%s|sudo:%s:" hostname hostname))
        (other-window (when (equal current-prefix-arg '(4)) t)))
    (ha-ssh--find-file tramp-ssh-ref other-window)))
#+end_src

** OpenStack Interface

Instead of making sure I have a list of remote systems already in the favorite hosts cache, I can pre-populate it with a call to OpenStack (my current VM system I'm using). These calls to the =openstack= CLI assume that the environment is already filled with the credentials. Hey, it is my local laptop ...

We'll give =openstack= CLI a =--format json= option to make it easier for parsing:

#+begin_src emacs-lisp
(use-package json)
#+end_src

Need a variable to hold all our interesting hosts. Notice I use the word /overcloud/, but this is a name I've used for years to refer to /my virtual machines/ that I can get a listing of, and not get other VMs that I don't own.

#+begin_src emacs-lisp
(defvar ha-ssh-overcloud-cache-data nil
  "A vector of associated lists containing the servers in an Overcloud.")
#+end_src

If our cache data is empty, we could automatically retrieve this information, but only on the first time we attempt to connect. To do this, we'll =advice= the =ha-ssh-choose-host= function defined earlier:

#+begin_src emacs-lisp
  (defun ha-ssh-overcloud-query-for-hosts ()
    "If the overcloud cache hasn't be populated, ask the user if we want to run the command."
    (when (not ha-ssh-overcloud-cache-data)
      (when (y-or-n-p "Cache of Overcloud hosts aren't populated. Retrieve hosts?")
        (call-interactively 'ha-ssh-overcloud-cache-populate))))

  (advice-add 'ha-ssh-choose-host :before 'ha-ssh-overcloud-query-for-hosts)
#+end_src

We'll do the work of getting the /server list/ with this function:

#+begin_src emacs-lisp
  (defun ha-ssh-overcloud-cache-populate (cluster)
    "Given an `os-cloud' entry, stores all available hostnames.
  Calls `ha-ssh-add-favorite-host' for each host found."
    (interactive (list (completing-read "Cluster: " '(devprod1 devprod501 devprod502))))
    (message "Calling the `openstack' command...this will take a while. Grab a coffee, eh?")
    (let* ((command (format "openstack --os-cloud %s server list --no-name-lookup --insecure -f json" cluster))
           (json-data (thread-last command
                                   (shell-command-to-string)
                                   (json-read-from-string))))
      (dolist (entry (seq--into-list json-data))
        (ha-ssh-add-favorite-host (alist-get 'Name entry)
                                  (thread-last entry
                                               (alist-get 'Networks)
                                               (alist-get 'cedev13)
                                               (seq-first))))
      (message "Call to `openstack' complete. Found %d hosts." (length json-data))))
#+end_src

In case I change my virtual machines, I can repopulate that cache:

#+begin_src emacs-lisp
(defun ha-ssh-overcloud-cache-repopulate ()
  "Repopulate the cache based on redeployment of my overcloud."
  (interactive)
  (setq ha-ssh-overcloud-cache-data nil)
  (call-interactively 'ha-ssh-overcloud-cache-populate))
#+end_src

The primary interface:

#+begin_src emacs-lisp
(defun ha-ssh-overcloud (hostname)
  "Log into an overcloud host given by HOSTNAME. Works better if
you have previously run =ssh-copy-id= on the host. Remember, to
make it behave like a real terminal (instead of a window in
Emacs), hit =C-c C-k=."
  (interactive (list (ha-ssh-choose-host)))
  (when (not (string-match-p "\." hostname))
    (setq hostname (format "%s.%s" hostname (getenv "OS_PROJECT_NAME"))))

  (let ((window-label (or (-some->> ha-ssh-favorite-hostnames
                                    (rassoc hostname)
                                    car)
                          hostname)))
    (ha-ssh hostname window-label)
    (sit-for 1)
    (ha-ssh-send "sudo -i")
    (ha-ssh-send (format "export PS1='\\[\\e[34m\\]%s\\[\e[m\\] \\[\\e[33m\\]\\$\\[\\e[m\\] '"
                         window-label))
    (ha-ssh-send "clear")))
#+end_src
* Keybindings
This file, so far, as been good-enough for a Vanilla Emacs installation, but to hook into Doom's leader for some sequence binding, this code isn't:

#+begin_src emacs-lisp
  (ha-leader
    "a s"  '(:ignore t :which-key "ssh")
    "a s o" '("overcloud"     . ha-ssh-overcloud)
    "a s l" '("local shell"   . ha-shell)
    "a s s" '("remote shell"  . ha-ssh)
    "a s p" '("project shell" . (lambda () (interactive) (ha-shell (projectile-project-root))))
    "a s q" '("quit shell"    . ha-ssh-exit)
    "a s f" '("find-file"     . ha-ssh-find-file)
    "a s r" '("find-root"     . ha-ssh-find-root)
    "a s b" '("send line"     . ha-ssh-send-line)

    "p t"   '("project vterm" . (lambda () (interactive) (ha-shell (projectile-project-root)))))
#+end_src
* Technical Artifacts                                :noexport:
Provide a name so we can =require= the file:
#+begin_src emacs-lisp :exports none
  (provide 'ha-remoting)
  ;;; ha-remoting.el ends here
#+end_src

Before you can build this on a new system, make sure that you put the cursor over any of these properties, and hit: ~C-c C-c~

#+DESCRIPTION: A literate configuration for accessing remote systems.

#+PROPERTY:    header-args:sh :tangle no
#+PROPERTY:    header-args:emacs-lisp :tangle yes
#+PROPERTY:    header-args    :results none :eval no-export :comments no mkdirp yes

#+OPTIONS:     num:nil toc:nil todo:nil tasks:nil tags:nil date:nil
#+OPTIONS:     skip:nil author:nil email:nil creator:nil timestamp:nil
#+INFOJS_OPT:  view:nil toc:nil ltoc:t mouse:underline buttons:0 path:http://orgmode.org/org-info.js