Configuring Python in Emacs

Introduction
LSP Integration of Python
Project Configuration

A literate programming file for configuring Python.

Introduction

The critical part of Python integration with Emacs is running LSP in Python using . And the question to ask is if the Python we run it in Docker or in a virtual environment.

While Emacs supplies a Python editing environment, we’ll still use use-package to grab the latest:

(use-package python
  :after flycheck
  :mode ("[./]flake8\\'" . conf-mode)
  :mode ("/Pipfile\\'" . conf-mode)
  :init
  (setq python-indent-guess-indent-offset-verbose nil
        flycheck-flake8-maximum-line-length 120)
  :config
  (when (and (executable-find "python3")
             (string= python-shell-interpreter "python"))
      (setq python-shell-interpreter "python3"))

  (flycheck-add-next-checker 'python-pylint 'python-pycompile 'append))

Note: Install the following checks:

pip install flake8 pylint pyright mypy pycompile

Or better yet, add those to the requirements-dev.txt file.

Virtual Environment

For a local virtual machine, put the following in your .envrc file:

layout_python3

That is pretty slick and simple.

The old way, that we still use if you need a particular version of Python, is to install pyenv globally:

pip install pyenv

And have this in your .envrc file:

use python 3.7.1

Also, you need the following in your ~/.config/direnv/direnvrc file (which I have):

use_python() {
  local python_root=$(pyenv root)/versions/$1
  load_prefix "$python_root"
  if [[ -x "$python_root/bin/python" ]]; then
    layout python "$python_root/bin/python"
  else
    echo "Error: $python_root/bin/python can't be executed."
    exit
  fi
}

Editing Python Code

Let’s integrate this Python support for evil-text-object project:

(when (fboundp 'evil-define-text-object)
  (use-package evil-text-object-python
    :hook (python-mode . evil-text-object-python-add-bindings)))

This allows me to delete a Python “block” using dal.

Docker Environment

Docker really allows you to isolate your project’s environment. The downside is that you are using Docker and probably a bloated container. On my work laptop, a Mac, this creates a behemoth virtual machine that immediately spins the fans like a wind tunnel.

But, but… think of the dependencies!

Enough of the rant (I go back and forth), after getting Docker installed and running (ooo Podman … shiny), and you’ve created a Dockerfile for your project, let’s install container-env.

Your project’s .envrc file would contain something like:

CONTAINER_NAME=my-docker-container
CONTAINER_WRAPPERS=(python3 pip3 yamllint)
CONTAINER_EXTRA_ARGS="--env SOME_ENV_VAR=${SOME_ENV_VAR}"

container_layout

Unit Tests

(use-package python-pytest
  :after python
  :commands python-pytest-dispatch
  :init
  (ha-local-leader :keymaps 'python-mode-map
    "t" '(:ignore t :which-key "tests")
    "t a" '("all" . python-pytest)
    "t f" '("file dwim" . python-pytest-file-dwim)
    "t F" '("file" . python-pytest-file)
    "t t" '("function-dwim" . python-pytest-function-dwim)
    "t T" '("function" . python-pytest-function)
    "t r" '("repeat" . python-pytest-repeat)
    "t p" '("dispatch" . python-pytest-dispatch)))

Python Dependencies

Each Python project’s requirements-dev.txt file would reference the python-lsp-server (not the unmaintained project, python-language-server):

python-lsp-server[all]

Note: This does mean, you would have a tox.ini with this line:

[tox]
minversion = 1.6
skipsdist = True
envlist = linters
ignore_basepython_conflict = True

[testenv]
basepython = python3
install_command = pip install {opts} {packages}
deps = -r{toxinidir}/test-requirements.txt
commands = stestr run {posargs}
           stestr slowest
# ...

Pyright

I’m using the Microsoft-supported pyright package instead. Adding this to my requirements.txt files:

pyright

The pyright package works with LSP.

(use-package lsp-pyright
    :hook (python-mode . (lambda () (require 'lsp-pyright)))
    :init (when (executable-find "python3")
              (setq lsp-pyright-python-executable-cmd "python3")))

LSP Integration of Python

Now that the LSP Integration is complete, we can stitch the two projects together, by calling lsp. I oscillate between automatically turning on LSP mode with every Python file, but I sometimes run into issues when starting, so I turn it on with , w s.

(use-package lsp-mode
  ;; :hook ((python-mode . lsp)))
  :config
  (ha-local-leader :keymaps 'lsp-mode-map
    "0" '("treemacs" . lsp-treemacs-symbols)

    "/" '("complete" . completion-at-point)
    "k" '("check code" . python-check)
    "]" '("shift left" . python-indent-shift-left)
    "[" '("shift right" . python-indent-shift-right)

    ;; actions
    "a" '(:ignore t :which-key "code actions")
    "aa" '("code actions" . lsp-execute-code-action)
    "ah" '("highlight symbol" . lsp-document-highlight)
    "al" '("lens" . lsp-avy-lens)

    ;; formatting
    "=" '(:ignore t :which-key "formatting")
    "==" '("format buffer" . lsp-format-buffer)
    "=r" '("format region" . lsp-format-region)

    "e" '(:ignore t :which-key "eval")
    "e P" '("run python" . run-python)
    "e e" '("send statement" . python-shell-send-statement)
    "e b" '("send buffer" . python-shell-send-buffer)
    "e f" '("send defun" . python-shell-send-defun)
    "e F" '("send file" . python-shell-send-file)
    "e r" '("send region" . python-shell-send-region)
    "e ;" '("expression" . python-shell-send-string)
    "e p" '("switch-to-shell" . python-shell-switch-to-shell)

    ;; folders
    "F" '(:ignore t :which-key "folders")
    "Fa" '("add folder" . lsp-workspace-folders-add)
    "Fb" '("un-blacklist folder" . lsp-workspace-blacklist-remove)
    "Fr" '("remove folder" . lsp-workspace-folders-remove)

    ;; goto
    "g" '(:ignore t :which-key "goto")
    "ga" '("find symbol in workspace" . xref-find-apropos)
    "gd" '("find declarations" . lsp-find-declaration)
    "ge" '("show errors" . lsp-treemacs-errors-list)
    "gg" '("find definitions" . lsp-find-definition)
    "gh" '("call hierarchy" . lsp-treemacs-call-hierarchy)
    "gi" '("find implementations" . lsp-find-implementation)
    "gm" '("imenu" . lsp-ui-imenu)
    "gr" '("find references" . lsp-find-references)
    "gt" '("find type definition" . lsp-find-type-definition)

    ;; peeks
    "G" '(:ignore t :which-key "peek")
    "Gg" '("peek definitions" . lsp-ui-peek-find-definitions)
    "Gi" '("peek implementations" . lsp-ui-peek-find-implementation)
    "Gr" '("peek references" . lsp-ui-peek-find-references)
    "Gs" '("peek workspace symbol" . lsp-ui-peek-find-workspace-symbol)

    ;; help
    "h" '(:ignore t :which-key "help")
    "he" '("eldoc" . python-eldoc-at-point)
    "hg" '("glance symbol" . lsp-ui-doc-glance)
    "hh" '("describe symbol at point" . lsp-describe-thing-at-point)
    "gH" '("describe python symbol" . python-describe-at-point)
    "hs" '("signature help" . lsp-signature-activate)

    "i" 'imenu

    ;; refactoring
    "r" '(:ignore t :which-key "refactor")
    "ro" '("organize imports" . lsp-organize-imports)
    "rr" '("rename" . lsp-rename)

    ;; toggles
    "t" '(:ignore t :which-key "toggle")
    "tD" '("toggle modeline diagnostics" . lsp-modeline-diagnostics-mode)
    "tL" '("toggle log io" . lsp-toggle-trace-io)
    "tS" '("toggle sideline" . lsp-ui-sideline-mode)
    "tT" '("toggle treemacs integration" . lsp-treemacs-sync-mode)
    "ta" '("toggle modeline code actions" . lsp-modeline-code-actions-mode)
    "tb" '("toggle breadcrumb" . lsp-headerline-breadcrumb-mode)
    "td" '("toggle documentation popup" . lsp-ui-doc-mode)
    "tf" '("toggle on type formatting" . lsp-toggle-on-type-formatting)
    "th" '("toggle highlighting" . lsp-toggle-symbol-highlight)
    "tl" '("toggle lenses" . lsp-lens-mode)
    "ts" '("toggle signature" . lsp-toggle-signature-auto-activate)

    ;; workspaces
    "w" '(:ignore t :which-key "workspaces")
    "wD" '("disconnect" . lsp-disconnect)
    "wd" '("describe session" . lsp-describe-session)
    "wq" '("shutdown server" . lsp-workspace-shutdown)
    "wr" '("restart server" . lsp-workspace-restart)
    "ws" '("start server" . lsp)))

Project Configuration

I work with a lot of projects with my team where I need to configure the project such that LSP and my Emacs setup works. Let’s suppose I could point a function at a project directory, and have it set it up:

(defun ha-python-configure-project (proj-directory)
  "Configure PROJ-DIRECTORY for LSP and Python."
  (interactive "DPython Project: ")

  (let ((default-directory proj-directory))
    (unless (f-exists? ".envrc")
      (message "Configuring direnv")
      (with-temp-file ".envrc"
        ;; (insert "use_python 3.7.4\n")
        (insert "layout_python3\n"))
      (direnv-allow))

    (unless (f-exists? ".pip.conf")
      (message "Configuring pip")
      (with-temp-file ".pip.conf"
        (insert "[global]\n")
        (insert "index-url = https://pypi.python.org/simple\n"))
      (shell-command "pipconf --local")
      (shell-command "pip install --upgrade pip"))

    (message "Configuring pip for LSP")
    (with-temp-file "requirements-dev.txt"
      (insert "python-lsp-server[all]\n")

      ;; Let's install these extra packages individually ...
      (insert "pyls-flake8\n")
      ;; (insert "pylsp-mypy")
      ;; (insert "pyls-isort")
      ;; (insert "python-lsp-black")
      ;; (insert "pyls-memestra")
      (insert "pylsp-rope\n"))
    (shell-command "pip install -r requirements-dev.txt")))