How is Lisp's read-eval-print loop different than Python's?

In support of Stallman's position, Python does not do the same thing as typical Lisp systems in the following areas:

• The read function in Lisp reads an S-expression, which represents an arbitrary data structure that can either be treated as data, or evaluated as code. The closest thing in Python reads a single string, which you would have to parse yourself if you want it to mean anything.

• The eval function in Lisp can execute any Lisp code. The eval function in Python evaluates only expressions, and needs the exec statement to run statements. But both these work with Python source code represented as text, and you have to jump through a bunch of hoops to "eval" a Python AST.

• The print function in Lisp writes out an S-expression in exactly the same form that read accepts. print in Python prints out something defined by the data you're trying to print, which is certainly not always reversible.

Stallman's statement is a bit disingenuous, because clearly Python does have functions named exactly eval and print, but they do something different (and inferior) to what he expects.

In my opinion, Python does have some aspects similar to Lisp, and I can understand why people might have recommended that Stallman look into Python. However, as Paul Graham argues in What Made Lisp Different, any programming language that includes all the capabilities of Lisp, must also be Lisp.

Stallman's point is that not implementing an explicit "reader" makes Python's REPL appear crippled compared to Lisps because it removes a crucial step from the REPL process. Reader is the component that transforms a textual input stream into the memory — think of something like an XML parser built into the language and used for both source code and for data. This is useful not only for writing macros (which would in theory be possible in Python with the ast module), but also for debugging and introspection.

Say you're interested in how the incf special form is implemented. You can test it like this:

[4]> (macroexpand '(incf a))(SETQ A (+ A 1)) ;

But incf can do much more than incrementing symbol values. What exactly does it do when asked to increment a hash table entry? Let's see:

[2]> (macroexpand '(incf (gethash htable key)))(LET* ((#:G3069 HTABLE) (#:G3070 KEY) (#:G3071 (+ (GETHASH #:G3069 #:G3070) 1))) (SYSTEM::PUTHASH #:G3069 #:G3070 #:G3071)) ;

Here we learn that incf calls a system-specific puthash function, which is an implementation detail of this Common Lisp system. Note how the "printer" is making use of features known to the "reader", such as introducing anonymous symbols with the #: syntax, and referring to the same symbols within the scope of the expanded expression. Emulating this kind of inspection in Python would be much more verbose and less accessible.

In addition to the obvious uses at the REPL, experienced Lispers use print and read in the code as a simple and readily available serialization tool, comparable to XML or json. While Python has the str function, equivalent to Lisp's print, it lacks the equivalent of read, the closest equivalent being eval. eval of course conflates two different concepts, parsing and evaluation, which leads to problems like this and solutions like this and is a recurring topic on Python forums. This would not be an issue in Lisp precisely because the reader and the evaluator are cleanly separated.

Finally, advanced features of the reader facility enable the programmer to extend the language in ways that even macros could not otherwise provide. A perfect example of such making hard things possible is the infix package by Mark Kantrowitz, implementing a full-featured infix syntax as a reader macro.

In a Lisp-based system one typically develops the program while it is running from the REPL (read eval print loop). So it integrates a bunch of tools: completion, editor, command-line-interpreter, debugger, ... The default is to have that. Type an expression with an error - you are in another REPL level with some debugging commands enabled. You actually have to do something to get rid of this behavior.

You can have two different meanings of the REPL concept:

• the Read Eval Print Loop like in Lisp (or a few other similar languages). It reads programs and data, it evaluates and prints the result data. Python does not work this way. Lisp's REPL allows you to work directly in a meta-programming way, writing code which generates (code), check the expansions, transform actual code, etc.. Lisp has read/eval/print as the top loop. Python has something like readstring/evaluate/printstring as the top-loop.

• the Command Line Interface. An interactive shell. See for example for IPython. Compare that to Common Lisp's SLIME.

The default shell of Python in default mode is not really that powerful for interactive use:

Python 2.7.2 (default, Jun 20 2012, 16:23:33) [GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwinType "help", "copyright", "credits" or "license" for more information.>>> a+2Traceback (most recent call last):File "<stdin>", line 1, in <module>NameError: name 'a' is not defined>>> 

You get an error message and that's it.

Compare that to the CLISP REPL:

rjmba:~ joswig$ clisp  i i i i i i i       ooooo    o        ooooooo   ooooo   ooooo  I I I I I I I      8     8   8           8     8     o  8    8  I  \ `+' /  I      8         8           8     8        8    8   \  `-+-'  /       8         8           8      ooooo   8oooo    `-__|__-'        8         8           8           8  8        |            8     o   8           8     o     8  8  ------+------       ooooo    8oooooo  ooo8ooo   ooooo   8Welcome to GNU CLISP 2.49 (2010-07-07) <http://clisp.cons.org/>Copyright (c) Bruno Haible, Michael Stoll 1992, 1993Copyright (c) Bruno Haible, Marcus Daniels 1994-1997Copyright (c) Bruno Haible, Pierpaolo Bernardi, Sam Steingold 1998Copyright (c) Bruno Haible, Sam Steingold 1999-2000Copyright (c) Sam Steingold, Bruno Haible 2001-2010Type :h and hit Enter for context help.[1]> (+ a 2)*** - SYSTEM::READ-EVAL-PRINT: variable A has no valueThe following restarts are available:USE-VALUE      :R1      Input a value to be used instead of A.STORE-VALUE    :R2      Input a new value for A.ABORT          :R3      Abort main loopBreak 1 [2]> 

CLISP uses Lisp's condition system to break into a debugger REPL. It presents some restarts. Within the error context, the new REPL provides extended commands.

Let's use the :R1 restart:

Break 1 [2]> :r1Use instead of A> 24[3]> 

Thus you get interactive repair of programs and execution runs...