#+TITLE:  Remote Access to Systems
#+AUTHOR: Howard X. Abrams
#+EMAIL:  howard.abrams@gmail.com
#+DATE:   2020-09-25
#+FILETAGS: :emacs:

A literate configuration for accessing remote systems.

#+BEGIN_SRC emacs-lisp :exports none
;;; ha-remoting.el --- A literate configuration for accessing remote systems. -*- lexical-binding: t; -*-
;;
;; Copyright (C) 2020 Howard X. Abrams
;;
;; Author: Howard X. Abrams <http://gitlab.com/howardabrams>
;; Maintainer: Howard X. Abrams <howard.abrams@gmail.com>
;; Created: September 25, 2020
;;
;; This file is not part of GNU Emacs.
;;
;; *NB:* Do not edit this file. Instead, edit the original literate file at:
;;            ~/other/hamacs/ha-remoting.org
;;       And tangle the file to recreate this one.
;;
;;; Code:
#+END_SRC
* Introduction
Sure =iTerm= is nice for connecting and running commands on remote systems, however, it lacks a command line option that allows you to select and manipulate the displayed text without a mouse. This is where Emacs can shine.

*Feature One:*

When calling the =ha-ssh= function, it opens a =vterm= window which, unlike other terminal emulators in Emacs, merges both Emacs and Terminal behaviors. Essentially, it just works. It =vterm= isn't installed, it falls back to =term=.

Preload a list of favorite/special hostnames with multiple calls to:

#+BEGIN_SRC emacs-lisp :tangle no
(ha-ssh-add-favorite-host "Devbox 42" "10.0.1.42")
#+END_SRC

Then calling =ha-ssh= function, a list of hostnames is available to quickly jump on a system (with the possibility of fuzzy matching if you have Helm or Ivy installed).

This also has the ability to call OpenStack to gather the hostnames of dynamic systems (what I call "an Overcloud"), which is appended to the list of favorite hostnames. The call to OpenStack only needs to be called once, since the hosts are then cached, see =ha-ssh-overcloud-query-for-hosts=.

*Feature Two:*

Use the /favorite host/ list to quickly edit a file on a remote system using Tramp, by calling either =ha-ssh-find-file= and =ha-ssh-find-root=.

*Feature Three:*

Working with remote shell connections programmatically, for instance:

#+BEGIN_SRC emacs-lisp :tangle no
(let ((win-name "some-host"))
  (ha-ssh "some-host.in.some.place" win-name)
  (ha-ssh-send "source ~/.bash_profile" win-name)
  (ha-ssh-send "clear" win-name))
;; ...
(ha-ssh-exit win-name)
#+END_SRC

Actually the =win-name= in this case is optional, as it will use a good default.

* Variables

Let's begin by defining some variables used for communication between the functions.

#+BEGIN_SRC emacs-lisp
(defvar ha-latest-ssh-window-name nil
  "The window-name of the latest ssh session. Most commands default to the last session.")

(defvar ha-ssh-host-history '() "List of hostnames we've previously connected.")

(defvar ha-ssh-favorite-hostnames '()
  "A list of tuples (associate list) containing a hostname and its IP address.
See =ha-ssh-add-favorite-host= for easily adding to this list.")
#+END_SRC

Also, let's make it easy for me to change my default shell:
#+BEGIN_SRC emacs-lisp
(defvar ha-ssh-shell (shell-command-to-string "type -p fish")
  "The executable to the shell I want to use locally.")
#+END_SRC

* Interactive Interface to Remote Systems

The function, =ha-ssh= pops up a list of /favorite hosts/ and then uses the =vterm= functions to automatically SSH into the chosen host:

#+BEGIN_SRC emacs-lisp
(defun ha-ssh (hostname &optional window-name)
  "Start a SSH session to a given HOSTNAME (with an optionally specified WINDOW-NAME).
If called interactively, it presents the user with a list
returned by =ha-ssh-choose-host=."
  (interactive (list (ha-ssh-choose-host)))
  (unless window-name
    (setq window-name (format "ssh: %s" hostname)))
  (setq ha-latest-ssh-window-name (format "*%s*" window-name))

  ;; I really like this =vterm= interface, so if I've got it loaded, let's use it:
  (if (not (fboundp 'vterm))
      ;; Should we assume the =ssh= we want is on the PATH that started Emacs?
      (make-term window-name "ssh" nil hostname)
    (vterm ha-latest-ssh-window-name)
    (vterm-send-string (format "ssh %s" hostname))
    (vterm-send-return))

  (pop-to-buffer ha-latest-ssh-window-name))
#+END_SRC

Of course, we need a function that =interactive= can call to get that list, and my thought is to call =helm= if it is available, otherwise, assume that ido/ivy will take over the =completing-read= function:

#+BEGIN_SRC emacs-lisp
(defun ha-ssh-choose-host ()
  "Prompts the user for a host, and if it is in the cache, return
its IP address, otherwise, return the input given.
This is used in calls to =interactive= to select a host."
  (let ((hostname
         ;; We call Helm directly if installed, only so that we can get better
         ;; labels in the window, otherwise, the =completing-read= call would be fine.
         (if (fboundp 'helm-comp-read)
             (helm-comp-read "Hostname: " ha-ssh-favorite-hostnames
                             :name "Hosts"
                             :fuzzy t :history ha-ssh-host-history)
           (completing-read "Hostname: " ha-ssh-favorite-hostnames nil 'confirm nil 'ha-ssh-host-history))))
    (alist-get hostname ha-ssh-favorite-hostnames hostname nil 'equal)))
#+END_SRC

Simply calling =vterm= fails to load my full environment, so this allows me to start the terminal in a particular directory (defaulting to the root of the current project):

#+BEGIN_SRC emacs-lisp
(defun ha-shell (&optional directory)
  "Creates and tidies up a =vterm= terminal shell in side window."
  (interactive (list (counsel-read-directory-name "Starting Directory:" (projectile-project-root))))
  (let* ((win-name "Terminal")
         (buf-name (format "*%s*" win-name))
         (default-directory (or directory default-directory)))
    (setq ha-latest-ssh-window-name buf-name)
    (if (not (fboundp 'vterm))
        (make-term win-name ha-ssh-shell)
      (vterm buf-name)
      ;; (ha-ssh-send "source ~/.bash_profile" buf-name)
      ;; (ha-ssh-send "clear" buf-name)
      )))
#+END_SRC

Before we leave this section, I realize that I would like a way to /add/ to my list of hosts:

#+BEGIN_SRC emacs-lisp
(defun ha-ssh-add-favorite-host (hostname ip-address)
  "Add a favorite host to your list for easy pickin's."
  (interactive "sHostname: \nsIP Address: ")
  (add-to-list 'ha-ssh-favorite-hostnames (cons hostname ip-address)))
#+END_SRC

* Programmatic Interface

The previous functions (as well as my own end of sprint demonstrations) often need to issue some commands to a running terminal session, which is a simple wrapper around a /send text/ and /send return/ sequence:

#+BEGIN_SRC emacs-lisp
(defun ha-ssh-send (phrase &optional window-name)
  "Send command PHRASE to the currently running SSH instance.
If you want to refer to another session, specify the correct WINDOW-NAME.
This is really useful for scripts and demonstrations."
  (unless window-name
    (setq window-name ha-latest-ssh-window-name))

  (pop-to-buffer window-name)

  (if (fboundp 'vterm)
      (progn
        (vterm-send-string phrase)
        (vterm-send-return))
    (progn
      (term-send-raw-string phrase)
      (term-send-input))))
#+END_SRC

On the rare occasion that I write a shell script, or at least, need to execute some one-line shell commands from some document, I have a function that combines a /read line from buffer/ and then send it to the currently running terminal:

#+BEGIN_SRC emacs-lisp
(defun ha-ssh-send-line ()
  "Copy the contents of the current line in the current buffer,
and call =ha-ssh-send= with it. After sending the contents, it
returns to the current line."
  (interactive)
  ;; The function =save-excursion= doesn't seem to work...
  (let* ((buf (current-buffer))
         (cmd-line (buffer-substring-no-properties
                    (line-beginning-position) (line-end-position)))
         (trim-cmd (s-trim cmd-line)))
    (ha-ssh-send trim-cmd)
    ;; (sit-for 0.25)
    (pop-to-buffer buf)))
#+END_SRC

Let's have a quick way to bugger out of the terminal:

#+BEGIN_SRC emacs-lisp
(defun ha-ssh-exit (&optional window-name)
  "End the SSH session specified by WINDOW-NAME (or if not, the latest session)."
  (interactive)
  (unless (string-match-p "v?term" (buffer-name))
    (unless window-name
      (setq window-name ha-latest-ssh-window-name))
    (pop-to-buffer window-name))

  (ignore-errors
    (term-send-eof))
  (kill-buffer window-name)
  (delete-window))
#+END_SRC

* Editing Remote Files

TRAMP, when it works, is amazing that we can give it a reference to a remote directory, and have =find-file= magically autocomplete.

#+BEGIN_SRC emacs-lisp
(defun ha-ssh-find-file (hostname)
  "Constructs a ssh-based, tramp-focus, file reference, and then calls =find-file=."
  (interactive (list (ha-ssh-choose-host)))
  (let ((tramp-ssh-ref (format "/ssh:%s:" hostname))
        (other-window (when (equal current-prefix-arg '(4)) t)))
    (ha-ssh--find-file tramp-ssh-ref other-window)))

(defun ha-ssh--find-file (tramp-ssh-ref &optional other-window)
  "Calls =find-file= after internally completing a file reference based on TRAMP-SSH-REF."
  (let ((tramp-file (read-file-name "Find file: " tramp-ssh-ref)))
    (if other-window
        (find-file-other-window tramp-file)
      (find-file tramp-file))))
#+END_SRC

We can even edit it as root:

#+BEGIN_SRC emacs-lisp
(defun ha-ssh-find-root (hostname)
  "Constructs a ssh-based, tramp-focus, file reference, and then calls =find-file=."
  (interactive (list (ha-ssh-choose-host)))
  (let ((tramp-ssh-ref (format "/ssh:%s|sudo:%s:" hostname hostname))
        (other-window (when (equal current-prefix-arg '(4)) t)))
    (ha-ssh--find-file tramp-ssh-ref other-window)))
#+END_SRC

* OpenStack Interface

Instead of making sure I have a list of remote systems already in the favorite hosts cache, I can pre-populate it with a call to OpenStack (my current VM system I'm using). These calls to the =openstack= CLI assume that the environment is already filled with the credentials. Hey, it is my local laptop ...

We'll give =openstack= CLI a =--format json= option to make it easier for parsing:

#+BEGIN_SRC emacs-lisp
(use-package json)
#+END_SRC

Need a variable to hold all our interesting hosts. Notice I use the word /overcloud/, but this is a name I've used for years to refer to /my virtual machines/ that I can get a listing of, and not get other VMs that I don't own.

#+BEGIN_SRC emacs-lisp
(defvar ha-ssh-overcloud-cache-data nil
  "A vector of associated lists containing the servers in an Overcloud.")
#+END_SRC

If our cache data is empty, we could automatically retrieve this information, but only on the first time we attempt to connect. To do this, we'll =advice= the =ha-ssh-choose-host= function defined earlier:

#+BEGIN_SRC emacs-lisp
  (defun ha-ssh-overcloud-query-for-hosts ()
    "If the overcloud cache hasn't be populated, ask the user if we want to run the command."
    (when (not ha-ssh-overcloud-cache-data)
      (when (y-or-n-p "Cache of Overcloud hosts aren't populated. Retrieve hosts?")
        (call-interactively 'ha-ssh-overcloud-cache-populate))))

  (advice-add 'ha-ssh-choose-host :before 'ha-ssh-overcloud-query-for-hosts)
#+END_SRC

We'll do the work of getting the /server list/ with this function:

#+BEGIN_SRC emacs-lisp
  (defun ha-ssh-overcloud-cache-populate (cluster)
    (interactive (list (completing-read "Cluster: " '(devprod1 devprod501 devprod502))))
    (message "Calling the =openstack= command...this will take a while. Grab a coffee, eh?")
    (let* ((command (format "openstack --os-cloud %s server list --no-name-lookup --insecure -f json" cluster))
           (json-data (shell-command-to-string command)))
      (message "Call to openstack returned: %s" json-data)
      (setq ha-ssh-overcloud-cache-data (json-read-from-string json-data))
      (let* ((hostnames (->> ha-ssh-overcloud-cache-data
                             (--map (alist-get 'Name it))))
             (addresses (->> ha-ssh-overcloud-cache-data
                             (--map (alist-get 'Networks it))
                             (--map (replace-regexp-in-string ".*?=" "" it)))))
        (setq ha-ssh-favorite-hostnames (append ha-ssh-favorite-hostnames
                                                (-zip-pair hostnames addresses)))))
    (message "Call to =openstack= complete. Found %d hosts." (length ha-ssh-overcloud-cache-data)))
#+END_SRC

In case I change my virtual machines, I can repopulate that cache:

#+BEGIN_SRC emacs-lisp
(defun ha-ssh-overcloud-cache-repopulate ()
  "Repopulate the cache based on redeployment of my overcloud."
  (interactive)
  (setq ha-ssh-overcloud-cache-data nil)
  (call-interactively 'ha-ssh-overcloud-cache-populate))
#+END_SRC

The primary interface:

#+BEGIN_SRC emacs-lisp
(defun ha-ssh-overcloud (hostname)
  "Log into an overcloud host given by HOSTNAME. Works better if
you have previously run =ssh-copy-id= on the host. Remember, to
make it behave like a real terminal (instead of a window in
Emacs), hit =C-c C-k=."
  (interactive (list (ha-ssh-choose-host)))
  (when (not (string-match-p "\." hostname))
    (setq hostname (format "%s.%s" hostname (getenv "OS_PROJECT_NAME"))))

  (let ((window-label (or (-some->> ha-ssh-favorite-hostnames
                                    (rassoc hostname)
                                    car)
                          hostname)))
    (ha-ssh hostname window-label)
    (sit-for 1)
    (ha-ssh-send "sudo -i")
    (ha-ssh-send (format "export PS1='\\[\\e[34m\\]%s\\[\e[m\\] \\[\\e[33m\\]\\$\\[\\e[m\\] '"
                         window-label))
    (ha-ssh-send "clear")))
#+END_SRC
* Keybindings
This file, so far, as been good-enough for a Vanilla Emacs installation, but to hook into Doom's leader for some sequence binding, this code isn't:

#+BEGIN_SRC emacs-lisp
(ha-leader
   "a s"  '(:ignore t :which-key "ssh")
   "a s v" '("vterm"         . vterm)
   "a s o" '("overcloud"     . ha-ssh-overcloud)
   "a s l" '("local shell"   . ha-shell)
   "a s s" '("remote shell"  . ha-ssh)
   "a s q" '("quit shell"    . ha-ssh-exit)
   "a s f" '("find-file"     . ha-ssh-find-file)
   "a s r" '("find-root"     . ha-ssh-find-root))
#+END_SRC
* Technical Artifacts                                :noexport:

Provide a name so we can =require= the file:

#+BEGIN_SRC emacs-lisp :exports none
(provide 'ha-remoting)
;;; ha-remoting.el ends here
#+END_SRC

Before you can build this on a new system, make sure that you put the cursor over any of these properties, and hit: ~C-c C-c~

#+DESCRIPTION: A literate configuration for accessing remote systems.

#+PROPERTY:    header-args:sh :tangle no
#+PROPERTY:    header-args:emacs-lisp :tangle yes
#+PROPERTY:    header-args    :results none :eval no-export :comments no mkdirp yes

#+OPTIONS:     num:nil toc:nil todo:nil tasks:nil tags:nil date:nil
#+OPTIONS:     skip:nil author:nil email:nil creator:nil timestamp:nil
#+INFOJS_OPT:  view:nil toc:nil ltoc:t mouse:underline buttons:0 path:http://orgmode.org/org-info.js