This document is a deeper dive on what’s going on when you use Extempore; if you’re just after the quick version then the quickstart guide might be what you’re after.
The best way to think about programming and evaluating code in Extempore is to
think of it as a compiler-as-a-service (CaaS). The compiler (provided by the
extempore executable) runs in a terminal, and you connect to it via a TCP
socket connection. When the compiler receives any code over this connection it
compiles and executes it. The general term for this is ‘evaluating’ the code.
There are some nuances to this process, but in general the programmer interacts
with a running Extempore process over a TCP connection.
The ‘chunks’ of code (serialised as strings) which are sent to the xtlang
compiler may be anything from small one-liners to whole source files (via the
load function). The functions and data (often referred to collectively as the
environment) persist for as long as the
extempore process is running. As
discussed in the philosophy page, the Extempore compiler actually compiles
two languages: Scheme and xtlang (Extempore’s own programming language). The
process of evaluating them is the same—just sending strings to the compiler
over the TCP port.
So, to do anything in Extempore you need a text editor which can
- open up a TCP connection to the compiler
- create a string which represents a valid chunk of Scheme or xtlang code
- send that string over the TCP connection
There are already Extempore modes/plugins for VSCode, Emacs, Atom, vim and Sublime Text (see editor support). If you already have a favourite text editor, then you’ll probably want to use that one. If you don’t, then VSCode is a good choice. In the end it doesn’t matter too much which editor you use, so pick the one that makes you happiest.
The instructions for starting an Extempore “session” are listed on the quickstart page, so we won’t duplicate them here.
One thing about the Extempore startup process which is relevant here is the way
the compiler prints some information about starting up some processes, namely a
primary process on port
7099 and a
utility process on port
are the TCP ports we’ll send our code to. A running Extempore binary can provide
multiple Extempore processes (kindof like threads) as well as connecting to
multiple other Extempore processes, potentially running on remote hosts. This
forms the basis for Extempore’s powerful distributed processing capability. For
the moment, though, you don’t have to worry about multiple processes, just
connect and interact with the
Connecting to the Extempore compiler
So far, all the stuff we’ve done has been in a terminal. The
process, which provides the Extempore compiler, is just sitting there idle,
waiting to be given some code to evaluate. That’s where the text editor part of
the equation comes in.
When you open up a file ending in
.xtm (Extempore’s default file extension),
should detect that you’re editing Extempore source code, and load the
appropriate Extempore plugin. Here’s a (short) example file containing some
The content of the file is at the top, and I’ve also included a representation of the “echo area” at the bottom. This is a part of your editor which displays information about the results of different editor commands, and may also be where the feedback from the Extempore compiler is “echoed” (printed out). It’s blank at the moment.
Now that we have
- an editor open with some Extempore code
- an Extempore (editor) plugin loaded
extemporeprocess still running
we can open up the TCP connection. In Emacs, this is done with M-x
extempore-connect. In VSCode, it’s ctrl+return. In Atom,
with alt+O. In ST2, use the menu item
Tools > Extempore >
Connect.... The default host and port arguments will be
respectively. If the connection is made successfully, then Extempore will echo
back the string “Welcome to extempore!”.
Once everything’s hooked up, then the compiler is just waiting there for you to
give it some code to evaluate. So, from a ‘blank slate’
.xtm file, let’s start
with some basic Scheme arithmetic. If you’re playing along, you can write
2) into your file somewhere.
This is where the ‘Compiler as a Service’ (CaaS) thing starts to get real.
Currently, the code
(+ 1 2) is just text sitting in your editor. It won’t get
compiled until you send it for evaluation. The easiest way to do this is to move
your cursor somewhere inside the code
(+ 1 2) and hit
ctrl+return (in VSCode) or
C-M-x (in Emacs). In ST2, you have to highlight
the code you want to evaluate and hit Ctrl+E. This takes
the whole expression
(+ 1 2) and sends it (as a string) to the running
The orange ‘box’ in the diagram indicates code that has been sent for
evaluation. See how the code string (in grey) is sent over the connection, and
the result is sent back (also as a string) and displayed in the echo area.
Nothing is printed in the terminal where
extempore is running.
Congratulations—you’ve just evaluated your first Extempore code!
We can write some more code to
bind-val a global variable
myPI, which is an
xtlang global variable of type
double. If you evaluate this what happens is
One difference from the previous (Scheme) example is that the
compiler now prints a message to the terminal:
Bound myPI >>> double
Evaluating xtlang code will always print a message to the log about the name
and type of the variables. Also, notice how the string that is echoed back is
“#t”, which is the Scheme/xtlang literal for boolean
true. This is what the
compiler returns if the value is ‘
bind-val‘ed successfully. It’s worth
observing that what the
extempore compiler prints to the log isn’t the same as
the result it echoes back to the editor over the TCP connection.
How about compiling an xtlang closure?
circle_area is an xtlang closure which takes a (
representing the radius of a circle and returns the area of that circle (another
double). It also uses the global variable
myPI which we evaluated earlier.
The closure compiled successfully, and the compiler prints
Compiled circle_area >>> [double,double]*
to the log. If there was a problem with the compilation, then the compiler would have printed a (hopefully helpful) compile error to the log instead.
Let’s find out the area of a circle of radius
5.0 units. We need to call
circle_area with the argument
When we evaluate the
(circle_area 5.0) expression, a couple of things happen.
The code is sent to the compiler, which returns the value
78.539816 to the
editor. In addition, a message about creating a new memory
printed to the log. That’s because this is the first time we’ve called some
xtlang code, and so a memory zone needs to be set up to provide any
memory. This zone allocation won’t happen if we evaluate the same code again,
because the default zone already exists. The compiler in this ‘created default
zone’ message is just telling us helpful things about the state of our Extempore
As another example of the difference between the return value of an xtlang expression and any side effects it may introduce, have a think about how you would get the circle’s area printed to the log view, rather than returned and shown in the echo area.
The answer: we can wrap the call to the
circle_area closure in a call to
println is a built-in function which prints (to the log) a string
representation of whatever arguments it is passed.
This time, the result (
78.539816) is printed to the log. And the result
returned to the editor is different, too—it’s now
#t. That’s because the
println function returns a value, indicating whether it was successful in
printing its arguments to the log or not. The actual printing is a ‘side
effect’ of the
println function—behaviour that happens during the course of
the function’s execution.
As a final basic example, we can send code to the compiler more than ‘one closure at a time’. Let’s write another closure, this time for figuring out the area of a ‘doughnut’:
Because we already have a closure (
circle_area) for figuring out the area of a
circle, it makes sense to use that closure in our
doughnut_area closure. The
area of the doughnut is the area of the outer circle (radius
r1) minus the
area of the inner circle (radius
See how this time both the definition of the
doughnut_area closure and the
(doughnut_area 5.0 2.5) are sent to the compiler in the same ‘chunk’,
meaning that they were both highlighted in the editor before giving the
evaluation command. The results of this evaluation indicate that the two parts
of the code were both evaluated successfully: the
compiled successfully, and the result
58.904862 was returned to the editor.
The power (and danger) of CaaS
Thus we’ve only evaluated code in the order it appears in the file. Closures
which use other closures or globals have all worked fine. But when we kill the
extempore process (i.e. with
SIGINT), the Extempore environment we’ve ‘built
up’ isn’t saved—it’s destroyed.
After restarting the
extempore process above, and reconnecting the editor to
it, let’s try compiling the
doughnut_area closure first:
circle_area closure isn’t there anymore, and so the compiler throws an
error (and no value is returned to the editor). Because the compiler is a
‘service’, it’ll just evaluate the code and build up the environment in whatever
order you throw code at it. The source code isn’t necessarily a linear
representation of the evolution of the environment—it all depends on the
‘evaluation trajectory’ that you take through the code.
So, if we go back and evaluate all the necessary code, everything works properly
One other thing you can do is redefine the behaviour of existing functions and
variables. For example, say we wanted to change our
circle_area function to
use an ancient Egyptian approximation for the area of a circle described on the
Rhind papyrus (c. 1800BC).
In the editor, change the code for the
circle_area closure and re-evaluate:
The result is (slightly) different, but not too far off—not bad for a 4000
year old formula. But the main thing is that the code to call
didn’t change—only the definition did. The new closure definition has to have
the same signature as the old one, so that any code which calls the existing
closure will still work ok (type-signature wise). This re-configurability in the
behaviour of the code lies at the heart of live coding, a
practice which has informed much of the design of Extempore.
This should be a serious challenge to any notion you may have had about the source code being the canonical definition of how an Extempore ‘program’ behaves. In live programming, the programmer is constantly both building new code and data structures, and also redefining and re-evaluating old bits of code to fit better with the current execution and environmental context. There are lots of deep implications of this way of thinking about programming, and I won’t go into them here, but hopefully this has been helpful for thinking about what programming in Extempore looks like.
Now, if you want to code everything up in source files which are evaluated
linearly from start to finish (e.g. with a call to
load) then you can still do
that, too. All of the Extempore libraries (including those for DSP and graphics)
work that way, and Extempore still works great in that paradigm. But you have
the ability to dive in and change things if you need to, and that opens up some
This is really just the tip of the compiler-as-a-service (CaaS) iceberg. Extempore’s CaaS will also let you do things like query for all bound symbols, print all closures of a particular signature type, return the abstract syntax tree of a particular closure, etc… In fact the Extempore compiler itself is fully runtime modifiable!