Initial attempt at a background for the book

This is just a brain dump and the Background and Advantages section
need to be proof-read.

We should also expand on the general outline and table of contents, as
the headers here are just a start.

.gitignore

@@ -49,3 +49,5 @@ flycheck_*.el
 /network-security.data
 
 
+/lp-in-org.info
+/lp-in-org.texi

README.org

@@ -1,3 +1,27 @@
-# lp-book
+#+title:  Literate Programming in Org
+#+date:   2024-08-03 August
+#+tags:   book literate-programming lp emacs org org-mode
 
-A short book on literate programming in Emacs using Org Mode
+# (progn (find-file (org-texinfo-export-to-info)) (Info-mode) (Info-top-node))
+
+This is a book on /Literate Programming in Emacs/ using Org Mode.
+
+The goal is to write the book as an org document, and then /export/ it as:
+  - Info, readable in Emacs
+  - PDF and Epub, readable on devices
+  - HTML to be viewable as a web page in EWW
+  - Org, downloadable and useful when this project is cloned
+
+A question on /interactive-ness/. Since the goal is to teach LP through Emacs (and not just a general guide on LP), we want the text viewable in Emacs, so that code can be easily copy/pasted into their own notes, as well as allowing the user to enter ~C-x C-e~ on any s-expression.
+
+To render and read [[file:lp-in-org.org][the book]] in Info, execute the following:
+
+#+begin_src emacs-lisp
+  (progn
+    (find-file "lp-in-org.org")
+    (find-file (org-texinfo-export-to-info ))
+    (Info-mode)
+    (Info-top-node))
+#+end_src
+
+#+RESULTS:

lp-in-org.org

@@ -0,0 +1,147 @@
+#+TITLE:  Literate Programming in Org
+#+AUTHOR: Howard Abrams
+#+EMAIL:  howard.abrams@gmail.com
+#+DATE:   2016 Oct 06
+#+DESCRIPTION: This file is used as the basis for the Info documentation
+#+OPTIONS: ':t toc:t author:t email:t
+#+LANGUAGE: en
+#+MACRO: version 1.0
+#+MACRO: updated last updated 1 August  2024
+
+#+TEXINFO_FILENAME: lp-in-org.info
+#+TEXINFO_HEADER: @syncodeindex pg cp
+#+TEXINFO_HEADER: @syncodeindex vr cp
+
+# NOTES: To create the info, call M-x org-texinfo-export-to-info
+#
+# Next, load the file, currently called lp-in-org.info
+#
+# Turn on Info-mode (notice the capital) or run:
+#    (add-to-list 'auto-mode-alist '("\\.info\\'" . Info-mode))
+#
+# Hit C-j to run ‘Info-forward-node’ to start browsing the results.
+#
+# Or simply execute the following s-expression with `C-x C-e':
+#
+# (progn (find-file (org-texinfo-export-to-info)) (Info-mode) (Info-top-node))
+
+This is a book on /Literate Programming in Emacs/ using Org Mode.
+
+Use literate programming as a /style/ to aid in discovery, exploration and clarity of code.
+
+#+begin_quote
+Let us change our traditional attitude to the construction of programs: Instead of imagining that our main task is to instruct a computer what to do, let us concentrate rather on explaining to human beings what we want a computer to do.
+
+    —Donald Knuth in "Literate Programming", The Computer Journal 27 (1984), p. 97. (Reprinted in /Literate Programming/, 1992, p. 99.)
+#+end_quote
+* Introduction
+In a computer program (no matter what the computer language), we write code, as a first class citizen, without ornamentation, but we /comment/ the code with some sort of marker, e.g. a symbol to signify the start and end, like =/* and =*/=, or a single symbol, like =#= or =//= to highlight the rest of the line.
+
+Literate programming is a style of coding where we change the paradigm as what would normally be the comments is the focus, and the code is ornamented. When Donald Knuth originally proposed the idea in 1984, text editing was still in an infant stage, and writing LP was klunky. However, with modern editors, like Emacs (can I really claim, with a straight face, that Emacs is modern), literate programming in org files can be smooth.
+
+We assume the reader of this book to be fairly proficient with [[info:Emacs][Emacs]] keybindings, and at least, a passing familiarity with [[info:Org][editing Org Mode files]], but we don’t assume, you’ve grokked the [[info:org#Working with Source Code][literate programming features]] of Org.
+
+As you probably know, Org is large, and the features for writing, evaluating and connecting blocks of source code in a document are extensive, and documenting them all is a daunting task. This book attempts to both guide and inspire a programmer to enjoy coding in a /iterate way/.
+** Background
+
+Literate Programming was first invented by Donald Knuth in 1980’s in an attempt to emphasize communication to other members on your team. He [[http://www.brainyquote.com/quotes/authors/d/donald_knuth.html#0RwBBIoWjqiKPb2Y.99][once wrote]]:
+
+#+begin_quote
+I believe that the time is ripe for significantly better documentation of programs, and that we can best achieve this by considering programs to be works of literature. Hence, my title: "Literate Programming."
+#+end_quote
+
+He had been playing around with the idea that a "program" shouldn't be a bunch of computer instructions, but more like /literature/. He called his approach, [[http://en.wikipedia.org/wiki/Literate_programming][literate programming]]:
+
+#+BEGIN_QUOTE
+The practitioner of literate programming can be regarded as an essayist, whose main concern is with exposition and excellence of style. Such an author, with thesaurus in hand, chooses the names of variables carefully and explains what each variable means. He or she strives for a program that is comprehensible because its concepts have been introduced in an order that is best for human understanding, using a mixture of formal and informal methods that reinforce each other.
+#+END_QUOTE
+
+
+Wanting programs to be written for human understanding, with the order based on logic of the problem, and not constrained to deficiencies in the programming language, we create a /literate programming document/ that generates a document for people *and* the source code files.
+
+The idea is to invert /code/ peppered with /comments/ to /prose/ interjected with /code/.
+A pre-processing program would then write the code blocks out into a source code file (called /tangling/) and create a published document of both the prose and the code formatted for reading (called /weaving/).
+
+What happened to his concept and why don’t we program this way?
+
+After introducing the concept in a white paper, he expanded the idea by publishing an example of how the source code would be written in [[http://en.wikipedia.org/wiki/Jon_Bentley][Jon Bentley]]’s “Programming Pearls” column [Communications of the ACM 29, 5 (May 1986), 364-3691].
+[[http://en.wikipedia.org/wiki/Douglas_McIlroy][Doug McIlroy]] added a /rebuttal/ where he boiled Knuth’s example into a single (now famous) shell command:
+
+#+begin_src bash
+  tr -cs A-Za-z '\n' |
+  tr A-Z a-z |
+  sort |
+  uniq -c |
+  sort -rn |
+  sed ${1}q
+#+end_src
+
+McIlroy invented the shell pipe as well as many of those command line tools. He’s quoted as saying:
+
+#+begin_quote
+A wise engineering solution would produce—or better, exploit—reusable parts.
+#+end_quote
+
+His example proved his point.
+
+While the resulting source code /tangled/ from a literate programming document, may look the same as a source file coded directly, this idea did not significantly change our industry
+However, given a complex problem without the necessary components, perhaps composing your initial solution in a literate program is helpful?
+
+Perhaps this process was a bit too much writing for most engineers, who view code comments as unnecessary, oversized baggage that needs to be maintained.
+Java's [[http://www.oracle.com/technetwork/java/javase/documentation/index-jsp-135444.html][Javadoc]], [[http://www.stack.nl/~dimitri/doxygen/][Doxygen]], [[http://jashkenas.github.com/docco/][Docco]] and other similar projects that can extract an API from the comments of the source code could be viewed as a /step/ toward literate programming. [[https://wiki.haskell.org/Literate_programming][Haskell]] has a partial implementation built into the compiler so that it doesn't require a special comment syntax or an external macro system.
+
+What most of these systems lack is that the code, not the logic, drives the presentation order. For instance, many languages require imports, variable definitions and functions to be declared before they are used. Knuth's original "WEB" program allowed a code block to refer (include) another code block in no particular order... you could describe your code in any order that made the most sense.
+
+Knuth's original /literate programming/ approach was text with minimal editor support, as he only wrote the WEB program to create (/weave)/ the documentation and write (/tangle)/ the source code.
+
+From my perspective, literate programming can only be useful with help from an editor, for instance the [[http://ipython.org/notebook.html][iPython's notebook]] (now expanded as the [[https://jupyter.org/][Jupyter Project]]) is quite popular. However, unlike iPython's storage of the files in JSON format, I think a literate file should be readable text, as [[http://transcriptvids.com/v/oJTwQvgfgMM.html][Carsten Dominik]], the creator of Org, wrote:
+
+#+begin_quote
+“In the third millennium, does it still make sense to work with text files? Text files are the only truly portable format for files. The data will never get lost.”
+#+end_quote
+
+An Org file, with its readable syntax, and amazing support from Emacs, gives a programmer a good environment to discover, explore and clarify complex code.
+
+*Further Reading:*
+
+  - [[http://orgmode.org/worg/org-contrib/babel/how-to-use-Org-Babel-for-R.html][Introduction to org-mode's Babel Project]] for teaching Emacs to do Literate Programming
+  - [[http://orgmode.org/worg/org-contrib/babel/intro.html][Reference material for the Babel Project]]
+  - [[http://www.leancrew.com/all-this/2011/12/more-shell-less-egg/][More Shell, Less Egg]] is a good historical essay on this subject
+  - [[http://www.hazyblue.me/2014/02/where-have-all-the-literate-programs-gone/][Where have all the Literate Programmers gone?]]
+** Advantages of LP
+
+Some of the advantages of literate programming for your source code include:
+
+   - Clarification of your thoughts of complicated situations
+   - Better documentation for your source code
+   - Great for team communication for issues and problems
+   - Inter-language facility for using the /best tool for the job/ (for instance, querying a database and then manipulating it with a general purpose language)
+
+The advantages of literate programming in Org is the advantage of Org itself.
+
+  - Text formatting, like emphasized text and lists
+  - Org's /organizational/ features, like embedded heading sections marking subtrees
+  - Tasks management, like Agendas, embedded /with your code/
+  - Note-oriented REPL for investigating new libraries and APIs
+
+I made this last point as part of my essays on [[https://howardism.org/Technical/Emacs/literate-devops.html][Literate Devops]] ideas. Briefly, REPLs can be a wonderful approach to discovering features of libraries and modules, as one types expressions, and sees the results. You can view a shell running in a terminal as a REPL. The problem is a REPL is /transient/. The output can be lost in the Terminal’s scroll buffer.
+
+With LP in Org, you can still type an expression, evaluate it, but the /print/ is embedded back into your file buffer. As an added bonus, the results can be named, and used as input variables in other block of code (and these code blocks can be written in a different computer language).
+
+However, if you are reading this book, you probably see the advantages, and now want to learn how to master this tool yourself.
+* Getting Started
+
+At this point, we should assume basic familiarity with Org … just not working with Code.
+
+Since Emacs Lisp is /built-in/, this *Getting Started* guide will use that language. In subsequent chapters, we will assume different languages.
+** Create a File
+Create an org file. Set some variables.
+
+Create a code block.
+* Working with Python
+* Calling out to the Shell
+Can we do both Bash, Fish and Powershell?
+* Creating Illustrations
+** Graphviz
+** PlantUML
+** Pikchr