Configuring Python in Emacs

Introduction
LSP Integration of Python
Project Configuration

A literate programming file for configuring Python.

Introduction

The critical part of Python integration with Emacs is running LSP in Python using direnv. And the question to ask is if the Python we run it in Docker or in a virtual environment.

While Emacs supplies a Python editing environment, we’ll still use use-package to grab the latest:

(use-package python
  :after flycheck
  :mode ("[./]flake8\\'" . conf-mode)
  :mode ("/Pipfile\\'" . conf-mode)
  :init
  (setq python-indent-guess-indent-offset-verbose nil
        flycheck-flake8-maximum-line-length 120)
  :config
  (when (and (executable-find "python3")
             (string= python-shell-interpreter "python"))
      (setq python-shell-interpreter "python3"))

  (flycheck-add-next-checker 'python-pylint 'python-pycompile 'append))

Note: Install the following checks:

pip install flake8 pylint pyright mypy pycompile

Or better yet, add those to the requirements-dev.txt file.

Virtual Environment

For a local virtual machine, put the following in your .envrc file:

layout_python3

That is pretty slick and simple.

The old way, that we still use if you need a particular version of Python, is to install pyenv globally:

pip install pyenv

And have this in your .envrc file:

use python 3.7.1

Also, you need the following in your ~/.config/direnv/direnvrc file (which I have):

use_python() {
  local python_root=$(pyenv root)/versions/$1
  load_prefix "$python_root"
  if [[ -x "$python_root/bin/python" ]]; then
    layout python "$python_root/bin/python"
  else
    echo "Error: $python_root/bin/python can't be executed."
    exit
  fi
}

Editing Python Code

Let’s integrate this Python support for evil-text-object project:

(when (fboundp 'evil-define-text-object)
  (use-package evil-text-object-python
    :hook (python-mode . evil-text-object-python-add-bindings)))

This allows me to delete a Python “block” using dal.

Docker Environment

Docker really allows you to isolate your project's environment. The downside is that you are using Docker and probably a bloated container. On my work laptop, a Mac, this creates a behemoth virtual machine that immediately spins the fans like a wind tunnel.

But, but… think of the dependencies!

Enough of the rant (I go back and forth), after getting Docker installed and running (ooo Podman … shiny), and you've created a Dockerfile for your project, let's install container-env.

Your project's .envrc file would contain something like:

CONTAINER_NAME=my-docker-container
CONTAINER_WRAPPERS=(python3 pip3 yamllint)
CONTAINER_EXTRA_ARGS="--env SOME_ENV_VAR=${SOME_ENV_VAR}"

container_layout

Unit Tests

(use-package python-pytest
  :after python
  :commands python-pytest-dispatch
  :init
  (ha-local-leader :keymaps 'python-mode-map
    "t" '(:ignore t :which-key "tests")
    "t a" '("all" . python-pytest)
    "t f" '("file dwim" . python-pytest-file-dwim)
    "t F" '("file" . python-pytest-file)
    "t t" '("function-dwim" . python-pytest-function-dwim)
    "t T" '("function" . python-pytest-function)
    "t r" '("repeat" . python-pytest-repeat)
    "t p" '("dispatch" . python-pytest-dispatch)))

Python Dependencies

Each Python project's requirements-dev.txt file would reference the python-lsp-server (not the unmaintained project, python-language-server):

python-lsp-server[all]

Note: This does mean, you would have a tox.ini with this line:

[tox]
minversion = 1.6
skipsdist = True
envlist = linters
ignore_basepython_conflict = True

[testenv]
basepython = python3
install_command = pip install {opts} {packages}
deps = -r{toxinidir}/test-requirements.txt
commands = stestr run {posargs}
           stestr slowest
# ...

Pyright

I’m using the Microsoft-supported pyright package instead. Adding this to my requirements.txt files:

pyright

The pyright package works with LSP.

(use-package lsp-pyright
    :hook (python-mode . (lambda () (require 'lsp-pyright)))
    :init (when (executable-find "python3")
              (setq lsp-pyright-python-executable-cmd "python3")))

LSP Integration of Python

Now that the LSP Integration is complete, we can stitch the two projects together, by calling lsp. I oscillate between automatically turning on LSP mode with every Python file, but I sometimes run into issues when starting, so I turn it on with , w s.

(use-package lsp-mode
  ;; :hook ((python-mode . lsp)))
  :config
  (ha-local-leader :keymaps 'lsp-mode-map
    "0" '("treemacs" . lsp-treemacs-symbols)

    "/" '("complete" . completion-at-point)
    "k" '("check code" . python-check)
    "]" '("shift left" . python-indent-shift-left)
    "[" '("shift right" . python-indent-shift-right)

    ;; actions
    "a" '(:ignore t :which-key "code actions")
    "aa" '("code actions" . lsp-execute-code-action)
    "ah" '("highlight symbol" . lsp-document-highlight)
    "al" '("lens" . lsp-avy-lens)

    ;; formatting
    "=" '(:ignore t :which-key "formatting")
    "==" '("format buffer" . lsp-format-buffer)
    "=r" '("format region" . lsp-format-region)

    "e" '(:ignore t :which-key "eval")
    "e P" '("run python" . run-python)
    "e e" '("send statement" . python-shell-send-statement)
    "e b" '("send buffer" . python-shell-send-buffer)
    "e f" '("send defun" . python-shell-send-defun)
    "e F" '("send file" . python-shell-send-file)
    "e r" '("send region" . python-shell-send-region)
    "e ;" '("expression" . python-shell-send-string)
    "e p" '("switch-to-shell" . python-shell-switch-to-shell)

    ;; folders
    "F" '(:ignore t :which-key "folders")
    "Fa" '("add folder" . lsp-workspace-folders-add)
    "Fb" '("un-blacklist folder" . lsp-workspace-blacklist-remove)
    "Fr" '("remove folder" . lsp-workspace-folders-remove)

    ;; goto
    "g" '(:ignore t :which-key "goto")
    "ga" '("find symbol in workspace" . xref-find-apropos)
    "gd" '("find declarations" . lsp-find-declaration)
    "ge" '("show errors" . lsp-treemacs-errors-list)
    "gg" '("find definitions" . lsp-find-definition)
    "gh" '("call hierarchy" . lsp-treemacs-call-hierarchy)
    "gi" '("find implementations" . lsp-find-implementation)
    "gm" '("imenu" . lsp-ui-imenu)
    "gr" '("find references" . lsp-find-references)
    "gt" '("find type definition" . lsp-find-type-definition)

    ;; peeks
    "G" '(:ignore t :which-key "peek")
    "Gg" '("peek definitions" . lsp-ui-peek-find-definitions)
    "Gi" '("peek implementations" . lsp-ui-peek-find-implementation)
    "Gr" '("peek references" . lsp-ui-peek-find-references)
    "Gs" '("peek workspace symbol" . lsp-ui-peek-find-workspace-symbol)

    ;; help
    "h" '(:ignore t :which-key "help")
    "he" '("eldoc" . python-eldoc-at-point)
    "hg" '("glance symbol" . lsp-ui-doc-glance)
    "hh" '("describe symbol at point" . lsp-describe-thing-at-point)
    "gH" '("describe python symbol" . python-describe-at-point)
    "hs" '("signature help" . lsp-signature-activate)

    "i" 'imenu

    ;; refactoring
    "r" '(:ignore t :which-key "refactor")
    "ro" '("organize imports" . lsp-organize-imports)
    "rr" '("rename" . lsp-rename)

    ;; toggles
    "t" '(:ignore t :which-key "toggle")
    "tD" '("toggle modeline diagnostics" . lsp-modeline-diagnostics-mode)
    "tL" '("toggle log io" . lsp-toggle-trace-io)
    "tS" '("toggle sideline" . lsp-ui-sideline-mode)
    "tT" '("toggle treemacs integration" . lsp-treemacs-sync-mode)
    "ta" '("toggle modeline code actions" . lsp-modeline-code-actions-mode)
    "tb" '("toggle breadcrumb" . lsp-headerline-breadcrumb-mode)
    "td" '("toggle documentation popup" . lsp-ui-doc-mode)
    "tf" '("toggle on type formatting" . lsp-toggle-on-type-formatting)
    "th" '("toggle highlighting" . lsp-toggle-symbol-highlight)
    "tl" '("toggle lenses" . lsp-lens-mode)
    "ts" '("toggle signature" . lsp-toggle-signature-auto-activate)

    ;; workspaces
    "w" '(:ignore t :which-key "workspaces")
    "wD" '("disconnect" . lsp-disconnect)
    "wd" '("describe session" . lsp-describe-session)
    "wq" '("shutdown server" . lsp-workspace-shutdown)
    "wr" '("restart server" . lsp-workspace-restart)
    "ws" '("start server" . lsp)))

Project Configuration

I work with a lot of projects with my team where I need to configure the project such that LSP and my Emacs setup works. Let's suppose I could point a function at a project directory, and have it set it up:

(defun ha-python-configure-project (proj-directory)
  "Configure PROJ-DIRECTORY for LSP and Python."
  (interactive "DPython Project: ")

  (let ((default-directory proj-directory))
    (unless (f-exists? ".envrc")
      (message "Configuring direnv")
      (with-temp-file ".envrc"
        ;; (insert "use_python 3.7.4\n")
        (insert "layout_python3\n"))
      (direnv-allow))

    (unless (f-exists? ".pip.conf")
      (message "Configuring pip")
      (with-temp-file ".pip.conf"
        (insert "[global]\n")
        (insert "index-url = https://pypi.python.org/simple\n"))
      (shell-command "pipconf --local")
      (shell-command "pip install --upgrade pip"))

    (message "Configuring pip for LSP")
    (with-temp-file "requirements-dev.txt"
      (insert "python-lsp-server[all]\n")

      ;; Let's install these extra packages individually ...
      (insert "pyls-flake8\n")
      ;; (insert "pylsp-mypy")
      ;; (insert "pyls-isort")
      ;; (insert "python-lsp-black")
      ;; (insert "pyls-memestra")
      (insert "pylsp-rope\n"))
    (shell-command "pip install -r requirements-dev.txt")))