System Stack

Home About Fiction Links RSS

Org mode, but more

This started out as a Mastodon thread in which I got to thinking about Org mode and Emacs, and how complicated all of it is, about the same time that I was reading so much about Plan 9 that I finally decided to install 9front on my old Thinkpad X200.

I need to start this out with an important caveat: the way I use Org is not the way most people use it. I originally came to Emacs because I was writing a ton of Markdown and starting to use Linux as much as I was my Mac (a PowerBook G4 at that time) and Windows (on a Thinkpad T22). markdown-mode was and is very nice, but I’ve always been a bit of a nerd for outliners, so I gave Org a try. That was probably 2007, or maybe even earlier. I’ve been using it ever since.

Org does a lot of stuff.

My ~/.emacs.d/elpa/org-20210301 directory contains 128,898 lines of code in the .el files in the repo… this is obviously a very heavyweight program running on top of what, in Emacs, is already a very heavyweight and specific environment. It’s a great deal of complexity. You already know this; if you’re reading this at all I assume you’re familiar with how big the manual is.

This, in and of itself, is not such a bad thing. Emacs is a very versatile tool, that can do almost literally anything. Org allows a lot of different ways of working, and it can do, and it lets people basically run their lives without ever leaving their text editor of choice.

That’s all fine, it’s just a lot more complexity than I need for how I use the tool, and carting around a massive Emacs config, while kind of cool, is also a symptom of what I’m getting at here.

What does Plan 9 have to do with it?

I like to think of Plan 9 as “Unix, but more.” What “more” actually means there is less: fewer system calls, a radically simplified way of using and interacting with the machine and the network (but which enables more powerful uses of those machines and networks), everything boiled down to the most simplified version of itself that can still be powerful. Namespaces, everything-as-a-file, bind and factotum and rio and Acme.

Plan 9 is an iteration on the concept of Unix, not just a new or stripped-down version of it. That it is actually stripped-down is more of a byproduct of the argument for simplicity being made; the things it can do and the experience of actually using it for a while shows the capabilities there.

And, of course, it’s not that these capabilities are free, but they are legible. I’m a terrible C programmer, but on Plan 9 I can read a man page, and if I don’t find the answer to my question, I can just go look at the source code, and I can generally understand what it’s doing. Even the way it’s written takes nothing for granted, and tries to do the simplest thing that will work.

So, naturally, when I got to thinking about Org in this way, and Emacs under it, and all the work that goes into getting Emacs set up on Windows (At work, I actually had to get Emacs 27.1 for Windows approved for use by my IT department, which was no small task), Linux, and macOS, the GNU-isms that come along with that, and generally the expansiveness of GNU and GCC and Linux and whatever else is at the bottom of that truly massive, incomprehensible stack of computer code, I thought about Plan 9, and how much cleaner and nicer everything is there. That is what I want my tools to be like. I can use the big hairy ugly ones for “real” work (like VS Code, which I use to work on this here Jekyll site). But for writing, and for thinking, I want something more like 9. I want it to be “Org, but more,” except I want to boil Org down to its purest, most potent essence, and then evolve it. Which begs the question: what is that essence?

What I do with Org

I really use Org for two things:

  1. Writing long documents with some sort of structure.
  2. Making lists and project plans.

I’ve always been a big fan of outliners; I came of age as a computer user on the early 2000’s Mac, the late blossom of MacOS 9 and the early origins of Mac OS X. There were all kinds of interesting outliners for these platforms, made by little one- or several-person software companies trying to find little interesting niches for tools. Things like Frontier, Palimpsest, OmniOutliner.

These things, to me, trace a wiggly line all the way back to things like the Memex and Project Xanadu, or to the Mother of all Demos (funded though it was by DARPA and all sorts of dark forces of US elite imperial power), through the Homebrew Computer Club. Tools like those outliners, and really like Org, are the closest we’ve been able to get to this idea of the computer as an external brain, a tool for processing thought and empowering people to make more than they’d have been able to do without the tools.

Outliners are a quentissential expression of this idea of the computer as a tool. Outlines, whether digital or paper, have always been a native language of thought for me, and as such, I’ve had at least one (and usually many) go-to outliner on every iteration of my computer environment.

I’ve spent some time thinking about what an “Outliner” really is and what functionality has to go in to one for me to want to use it seriously.

The Minimum Viable Outliner

These are things that an Org replacement would have to do for it to actually be a replacement, at a minimum.

  • Creating and deleting headings
  • Promoting and demoting those headings to different levels of the document structure
  • Moving those headings and their associated subtrees around, in what Org calls a “refile” but I just call “using an outliner”
  • Having arbitrary sections of formatted (Org or Markdown) text associated with those headings, so that a heading can collapse or expand and wherever the heading goes, the text goes with it (for instance, moving chapters or sections around by moving its header around.)
  • The ability to attach a state to a heading, like TODO or DONE. This allows the outline to also serve as a task list, or a project plan, or even for a document to carry within it information about the completion state of the document (Org lets you define your own statuses on a per-file basis, so things like DRAFT and REVISED can also be states that you can toggle through).

There are probably more things that should go on this list, but I don’t know what they are yet. I suspect that as I get further into this exploration, a true set of requirements will start to coalesce.

Task management or document tool?

The correct answer here is “both,” but it’s worth noting that some people use Org in a totally different way: they strictly use it as a task-management tool. Org can track tags and priorities for tasks (TODO items), logging and archiving of closed items (which I do use in my “Open Loops” and “Weekly Outcomes” documents, more about which later). It allows dates and times to be associated with tasks, and has an agenda which is core to how a lot of people use it.

I’ve never really understood what the agenda is or how to use it; it’s just now how my brain works with outlines. They can blur the lines between “document” and “collection of tasks,” but I’ve never been comfortable putting everything into it, maybe because I don’t really like the mobile clients that are available. Emacs, of course, is not available in the App Store.

File storage

It’s important to me that textfiles are the medium of exchange here. For something strictly to be used as a “to do” list or a task management system, that doesn’t make as much sense, but since I’m using Org to write this, and also using it to track all of the things I want to accomplish this week, one emphasis can’t take precendence over the other.

The other miraculous thing about Org, that hasn’t existed on any previous iteration of this system for me, is ox-pandoc, an incredible mode that pipes your Org files into Pandoc so that your outline, which then expands to become your finished draft, can seamlessly be turned into Markdown, Word, LaTeX, a PDF, or any number of other file formats. The ability to natively dump into Pandoc what was previously a textfile with a bunch of bulleted points in it is transformational.

It doesn’t follow, though, that any future more minimal version of this tool wouldn’t have the same power. It’d just have to follow some of the conventions of the Org file format (using the number of asterisks to signify the depth of a heading down the hierarchy, for example).

Some possible paths forward

There are several potential futures here. The first is that I just keep using Org in Emacs and deal with the overhead of installing Emacs and maintaining a configuration and waiting on it to load a huge Org file (which takes forever on Linux and takes even longer on Windows). That’s not fun, though. What other options are there?

A new Emacs mode, a “lite Org”

This solves one issue, that Org has a whole bunch of stuff I don’t use and loads slowly. But it’s a lot of work to end up at the same point.

A new text-based outline editor around these features

This feels like a lose-lose; we’d have to implement a new text editor when Emacs is already done, and we’d have to reimplement Org features when Org is already done. And the end result is still bound to Emacs, which is the slow piece of the puzzle here and the thing I’m trying not to have to rely on.

A bunch of rc scripts in Acme

Part of this whole Plan 9 experiment lately has been a real exposure to Acme in its native environment. Acme is great. It’s brilliant, and it makes the mouse make sense in a way that makes C-x 3 and C-c $ and all these insane Emacs keyboard shortcuts immediately feel archaic. The Plan 9 environment as a whole is really compelling to me.

Acme is radically different from any other editor I’ve ever used, in a way that is very intuitive and I think great, but it’s not so great for authoring long prose documents. (For one, I can’t figure out how to do word-wrapping; it seems I’m supposed to use newlines for that.) There are limitations in Acme that feel like they’re there because its author(s) wanted to prove a point, not because they were obviously the best choice. I wonder what they used to write all those Plan 9 man pages? Surely they missed word wrap?

The minimalism of Plan 9 is inspiring. I wonder what the smallest possible tool I could write would be that would carry forward these features and serve as a basis for building more, out in the open?

A text-based outlining UI designed for e-Ink

What if there’s a whole new category of device that needs to exist to support this kind of tool or this kind of environment? Can it be built out of today’s cheap-but-powerful single board computers? Are the raw materials for this thing already sitting around?

This is maybe the most exciting but least-developed possibility here for me, maybe something that’s years away. This is certainly not where I want to try to start.

Next actions

I’m a project manager, at the end of the day. I conclude things and then I figure out what the next steps are. Here’s what I’d like to do and what I’ll do about it next.

I’d like to build something like this for myself, either in scripts for use with Acme or in some other technology that will work on Plan 9. I don’t want to port Emacs to Plan 9, and I have a suspicion nobody else in the Plan 9 world wants Emacs ported to Plan 9 either. But the fact remains: I think there should be a radically simpler tool to do this kind of text-based outlining and planning than Org, even if Org is the gold standard. I think we can do better, and leave Org to the people who really need its complexity.

Part of this is my interest in things like Cyberdecks and Raspberry Pi hardware and old Palm Pilots and whatever else (my M5Paper development kit still isn’t here). I want to find the ideal tool for thinking and writing, and I’m pretty convinced it’s not a full-fledged personal computer with a big high-color screen. Emacs and Org are heavy pieces of equipment, not meant for these sorts of environments, really. There’s a lot of baggage there, bringing a Lisp machine with an editor on top of it and a massive Lisp package in Org on top of that. All of these devices and paradigms require something easier. Even just Plan 9 on a Core 2 Duo Thinkpad forces that issue.

The fact of the matter is that right now I’m not a good enough programmer to make this thing a reality. This is going to involve a lot of trial and error on my part to even develop the competence needed to build such a tool.

I’ll keep researching and mining this particular vein of thought—a lot of smarter people than me have done a lot of thinking and writing about outliners and about text editors and computing simplicity. Over time, I hope I can set out some basic requirements and start learning how to build a system that meets them, even if I’m the only person who ever uses it.

I’m convinced there’s something to this idea. There’s a way to do this that results in a clear vision for a new iteration of my favorite kind of computer tool.