9 months of being paid to make things by strangers on the internet!
Here is what I did in the last quarter:
- In imp:
- Added a live repl (demo video here)
- Redesigned the syntax to be much more concise (before and after)
- Added backwards links from each compiler stage to the previous stage (used for eg figuring out error locations in the repl)
- Added the ability to set memory limits and to interrupt the interpreter while it's running (important for embedding in other tools).
- Drastically changed the internal representations. This was setup work for decorrelation of higher-order functions and iteration, but as a side effect it also made closures much smaller and fixed some nasty edge cases in type inference and in the interpreter, bringing the runtime for integration tests from ~30s down to ~0.1s.
- In dida:
- The core library now actually manages memory instead of never freeing.
- Many more tests and corresponding bug fixes.
- A compile-time switch to support dumping all internal events and state changes.
- The beginnings of the interactive debugger - currently it can run an example in wasm and dump the event/state log to a div.
- In focus:
- Added syntax highlighting (see this thread on color models and final results here).
- Added error squigglies (used for the imp live repl).
- Added a simple make mode that will run a bash command (autocompleting from .bash_history) and restart it on every save.
- Many minor bug fixes.
- Started publishing notes on the design:
- Mutant, a library which parses zig code and inserts random bugs. I was thinking to use it for some structured debugging practice.
- New writing:
- Reflections on a decade of coding. Very tentative advice on progressing past the novice stage.
- Implicit ordering in relational languages. Seems like a silly problem at first, but exposes a deep weakness in SQL and a challenge for incremental maintenance.
- Better operator precedence. A slightly nicer way to handle precedence in Pratt parsers. Also allows having a non-total order on binding power. Derived from imp, where I mostly want to avoid having precedence rules at all but they are crucial for a few very common combinations of operators.
- Thoughts on benchmarking streaming systems. I considered doing some benchmarking but concluded that it was far too large and amorphous a problem to be worthwhile.
- Notes on Ultralearning. I'm still clinging to the idea that I'll maintain this note-publishing habit despite being two years behind my current reading.
- Three more editions of my sponsors newsletter.
- Updated some older posts:
- I got so many responses to Against SQL that I added an FAQ to the post instead of replying individually.
- In How Materialize and other databases optimize SQL subqueries I didn't actually test oracle (because it looked arduous to install) and just liked to their docs. After some conversation with an engineer at oracle:
- I've updated the link to point to the official docs rather than an older mirror.
- The official docs are apparently also not totally accurate, but I've not been able to get an exact description of what the current limitations are.
- I found out that the versions of oracle which I would be able to install locally are all old anyway - the newer versions are cloud-only.
- But I didn't manage to test the free tier of the oracle cloud because it didn't like my prepaid credit card.
- In Looking for debugger I tested all the linux gui debuggers I could find. The author of gede reached out and I've now added some EDITs to the gede section. Some bugs have been fixed, some I can no longer reproduce, and the author confirmed that many of the limitations are intrinsic to the awful gdb api.
- I keep seeing How safe is zig? used in silly flamewars. I've now added much more context to the beginning, so you now have to at least read below the fold to get to the flamewar fuel.
Here is what my income (in USD / month) looks like over time:
And the distribution of sponsorship amounts:
$1 x 3
$2 x 7
$4 x 1
$5 x 17
$10 x 11
$15 x 2
$20 x 46
$50 x 2
$100 x 1
$128 x 1
$200 x 3
I think it's interesting that even with pay-what-you-want the majority of new signups are for $20. So the default does seem to make a difference.
I'm now making the equivalent of ~31.5k CAD per year. That's above minimum wage, just slightly below the mean phd income at UBC and about 70% (pre-tax) of my 2020 budget.
The list of things I did this quarter bears no resemblance to the list of things I planned to do, so why even have plans?
But here are some things I might do next quarter:
- Imp:
- I really want to get decorrelation working with higher-order functions and fix/reduce. It's been aiming to do it for a long time and it would open up a lot more interesting downstream projects, like efficient translation to sql.
- I want to add a lot more examples in order to figure out what works and what doesn't in the language design. A major prerequisite to that is deciding how to import data - should I try to do eg automatic imports of sqlite databases?
- Dida:
- I have several known bugs that need to be fixed, one of which might require some architectural changes.
- I need to figure out better workflows for debugging. The symptoms for most bugs occur far away from their symptoms (in both time and code). Building a simple vizualizer/debugger would help a lot with following the unfolding of complex bugs, but I can probably also introduce heavy debug-mode validation of invariants like "no aliased rows".
- I still want to add animated vizualizations to the explanation of how dida works, but I'm really dreading having to glue stuff together through the wasm abi and package it into some javascript project.
- Focus:
- I want to extend the make mode to allow jumping to error locations. This is the last major feature that I miss from emacs.
- There are tons of minor quality-of-life changes I could make eg make ripgrep mode async so it doesn't chug in large directories, add syntax highlighters for more languages, add clipboard history, indent wrapped lines instead of starting from the edge of the screen etc.
- I'd like to continue writing up notes on the design. I have a vague notion that this could eventually be a book, along the same lines as crafting interpreters or chidb.
- I have a lot more outlined for the reflections series. I still feel somewhat dubious about the value of posthoc generalized advice, but people keep encouraging me to finish it.
- I have some vague plans to do some structured debugging practice using mutant. Maybe I should record the sessions? Or maybe just write a condensed summary of anything improvements I figure out.
I have a ton of other ideas but I feel the need to pop at least one of the above projects off my mental stack before starting anything new.