Building Tools for Sharing Physics

On building your own telescope before looking up at the stars.

It’s time for an update! I seem to have forgotten that we’re doing this together, and that this means I need to resist the urge to endlessly polish before I share.

I sketched out a rough path in my previous letter. I decided to start with a beautiful book called “Structure and Interpretation of Classical Mechanics”, or SICM. This is a graduate-level textbook on “classical mechanics”, which is an unfortunate name for, really, all of physics where quantum effects don’t come into play. (This still doesn’t tell you what it is. I’ll get to that soon.)

Structure and Interpretation of Classical Mechanics

SICM is not a normal textbook. The authors have implemented all of the physics in the book in the MIT Scheme programming language (a variant of Lisp), using a software library called “scmutils” that they built over the course of decades. Imagine, MIT’s AI Lab in the 80s, the throbbing heart of the movement to build artificial minds by encoding everything humans know in computer-interpretable symbolic expressions. Many of the leading lights of computing put a huge amount of effort into building systems that could do calculus, “symbolic computation”, and simulations of physical systems on a computer.

With much of School-physics, it’s standard to study systems and models that are “solvable”, or simple enough that you can use pen and paper to figure out how they evolve through time. We study pendulums swinging, or two planets rotating around each other, or balls rolling down ramps. But most systems aren’t simple. The equations for seemingly-straightforward systems quickly become far too difficult to manage. Here’s a screenshot of the “equations of motion” for a bead moving around on an ellipse, generated by the scmutils library:

The right side’s missing because I couldn’t zoom out far enough. The physical system is nice and simple to imagine, but the math is a brutal slog.

SICM’s software lets you dodge that and bulldoze past the usual boundary into the study of systems that are very difficult to analyze on paper, and still easy on the computer.

This book hooked me for a few other reasons:

• I’m obsessed with Lisp. Clojure, a modern version of Lisp, gets my brain singing like nothing else, and launched my career in software.

• My guess, not quite correct, was that it would be much easier to share work with you all that I’d built up in code. If something here excites you (you!), you should be able to run the code I used to generate the math or trigger a simulation. If all I give you is equations, you might as well read a textbook by someone better than me at teaching.

• The book has a sequel called “Functional Differential Geometry” (PDF here) that uses the same software to take us through the math behind general relativity. Talk about high leverage.

I’m 100 pages into SICM. Why haven’t I shared anything? I’m caught in a fit of obsession. Let me tell you about it.

Shareable Physics

I started the SICM journey by working my way through its first chapter two times. I took extensive notes in a Roam Research database that I’ve kept for months. These notes are quite valuable as summaries of what you should expect inside the book, but they’re at a level of detail that’s only interesting if you want to work through the book yourself. Still, feel free to dig in if you’re curious, and please send me a note if you do.

I decided to start working through the exercises in code, figuring that a few of these would be gems that I could share. This is where the trouble started.

My goal was to be able to publish posts for you all that:

• Had great-looking inline math that I could mix freely with images and words. This is all storytelling, after all.

• If I generated any math, results or animations with code, both the code and the results needed to live in the document and look good.

• The goal is to inspire you to play with this stuff and learn more, and share it again. So if you see something in one of these posts that you like, you need to be able to “effortlessly” pick up where I left off, and start tweaking and modifying and playing with the actual code. You should certainly be able to recreate anything I did in the post.

If you do get hooked on anything I’m exploring, I need you to be able to use the exact same tools as I did with minimal fuss to tell stories, or do your own research. It’s not useful if I make tools that transform the way I work and leave you with nothing.

I was surprised to find that no one has built anything like this for SICM. The SICM book itself was generated this way! But the source code for the book isn’t available, so we can’t know how Sussman and Wisdom actually worked to make it. Dozens of grad students take MIT’s 6.946 course every year. How are they sharing their work?

Building Tools

I embarked on an odyssey of tool-building. I knew the experience I wanted to generate, and it wasn’t possible. And I happen to have the skills to make it work. Do we need any further reason to make things? It’s very easy to talk yourself out of making something interesting by needing to know why. But any reason you generate is a just-so story, a plausible justification you can file to let yourself just make the damned thing. The why is, because you felt the urge to make a thing. That urge is delicate and precious, I think.

I built a system that let me generate posts like this:

Optics and the Principle of Least Time

I have close to 30 of the exercises in SICM completed and hosted in Github. I’m writing each of the exercises in a large “Org Mode” file in Emacs, and then publishing them out to Github-compatible markdown using a custom export engine.

If you visit that link you’ll notice that no math renders. Github can’t do math, so I had to write a Chrome plugin that will render it for you. Install the Chrome plugin and reload that page and you’ll see the whole thing come alive with glorious equations.

Some of the most exciting exercises generate animations like this:

To record those, I had to find GiFox, a paid-only tool that can generate GIFs off of screen recordings.

Every one of those exercises, in addition to generating nice Markdown documents, exports its code out into Scheme source code files, with all of the text and math rendered as comments inline around the code. Here’s the source for the exercise that generates the animation above, and here's the folder with all the rest.

Can you run this code yourself? Yes, you can! That wasn’t easy either. It’s not easy at all to get this software running on a computer. I wrote up the many-stepped process, but it was too depressing to imagine no one else executing these exercises. So I used a piece of software called “Docker” to wrap up the MIT Scheme language and the scmutils library uses in their own custom Docker containers.

You can’t get away with knowing nothing about computers, but if you follow these instructions, you can quickly get to a place where you can run a command like:

bin/mechanics

get a “REPL” (read-eval-print loop, a classic Lisp idea), type

(load "ch1/ex1-5.scm")

and see the animation from above play on your computer.

I wrote up everything I had to build at this page. It was so, so much. It was exhausting. But I’m closer now to something that’s tolerable for writing and teaching about the material in the textbook. And, for a later post, I understand now what I need to build to make this work accessible to probably 100x the audience it has now.

To be clear: this system is not yet nice to use for anyone but me. But I’m going to get it there, and share it, and anyone who chooses to spend their time exploring classical mechanics and differential geometry will find it a little easier to work with the material and share their excitement with others.

A note to address the obvious question. There are, of course, existing systems that attempt to make publishing interactive documents possible:

• Jupyter notebooks that look like this

• Mathematica , the closed source giant in the room

• org-babel, the system I built on for this work

But none of them support MIT Scheme out of the box, which means that they can’t expose the treasure trove of incredible results in SICM.

Concluding, for now

I started this newsletter because I’m interested and intrigued by why it’s so difficult to learn high-level math and physics, and why so many of the folks who do end up mastering these fields seem to have such a difficult time sharing what they’ve learned. So much knowledge is trapped in PDFs. This is the technology we used to share results in the 1800s. Could it be possible that it’s still the best way?

I really don’t think so.

I’m going to start sharing the work that I’ve generated already with this system, of course. But it’s still not good enough. A wall of math and code is not the way that we’re going to absorb this stuff together. I think I know a better way.

I need to talk to you about Mindstorms, and Seymour Papert, and the Children’s Machine. We need to talk about mechanics, yes, but we also need to talk about microworlds, and about getting to know classical mechanics, not like memorizing a series of equations, but more like you’d get to know a friend.

I would love comments and feedback. I know I’ve mostly hinted at what’s coming; but if this urge to create better ways of communicating and teaching resonates with you, please write me and tell me why. I suspect that many folks interested in this newsletter are coming back to this material for a second time. I know we can do better.

Until then!