0015: imp internals, reflections, precedence, make mode, mutant, q3, error recovery, tonsky ui, subtext 10, factfulness, benchmarking advice, dependency hubs, independent research, zig wayland, retool, observable dependencies, ugly buildings, without scihub, wasm virtual memory, huawei breakdown, infrastructure langauges, stencil vectors, chiX

Published 2021-10-12

New stuff:

The author of rust-analyzer made a video about error recovery in their recursive descent parser.

The bulk of the strategy seems to be whenever you would return an error, just record it instead and continue as if you'd seen the expected token.

If that was all they did then errors in half-written code could the parse of finished code later in the file.

struct fo| // cursor here

fn foo() usize {
    // this is a valid function, want to parse this correctly instead of using the curly braces to finish the struct definition
    return 42;

That's handled by recovery sets (discussed here). I don't really follow the implementation yet though. I think it just continues to record errors without ever eating any tokens from the recovery set.

Two ideas for experiments:

I also really like the offhand reference to recursive descent as 'immediate-mode parsing'.

Tonsky is writing a UI library. I'm glad more and more people are taking seriously the idea that the web is not the final end state of UI programming.

Jonathan Edwards published the design document for subtext 10, his latest experiment in end-user programming.

It's a good sign for a design doc when there are so many interesting ideas that I have the urge to implement it just so that I don't have to wait to play with it.

I read Factfulness. I don't have much to add to the abstract except that it's a good unicorn chaser for, uh, the internet.

I get amazing spam:

...uses the latest technology to make learning engaging and accessible. It's an extremely sophisticated edtech platform that uses the power of AI, facial recognition, smart chat, and blockchain technology to fix the issues of today's Education & Training Industries.

The power of buzzword, buzzword, buzzword and an irreversible way to send us money...

A phd student emailed me to ask for advice on benchmarking streaming systems. My advice was to pick a different subject (I don't think that went down very well).

I think the scope of the problem is probably on par with benchmarking databases, except that the hardware costs are higher and the workloads much less well understood. The latter is especially problematic if you don't have access to some large tech company's internal workloads.

Very few of the academic work I've seen on the subject was worth doing at all.



...you can perhaps maintain your own fork of Perl, but you cannot fork its hub position in the network, nor can you reasonably maintain forks of its 7964 dependants. If the Perl maintainers introduce a breaking change, those 7964 dependents will either adapt or disappear. Hypothetically, a large number of them could together envisage maintaining their own fork. But there are no good coordination mechanisms among developers of unrelated Open Source projects, and therefore this doesn't happen in practice.


From the author of Working in Public:

These days, if you say you work in research, most people assume you work in academia. But it's sort of odd that we assume you need someone's permission to do research.

Imagine studying something that nobody else is studying, for reasons you can't really articulate, without knowing what the outcome of your work will be. For the truly obsessed person, the need for validation isn't about ego; it's about sanity. You want to know there's some meaning behind the dizzying mental labyrinth that you simultaneously can't escape and also never want to leave.


On writing a wayland client in zig. The main interest is that the zig api manages to be significantly harder to misuse without being harder to use.

Retool looks like a really nice database-to-UI glue. From one of the authors of subform and sketch systems.

Observable has a gorgeous UI for tracking dependencies between nodes.


How did this happen? Even rich people often live in terribly ugly homes. With all the advantages of technology, we rarely build beautiful structures. All the progress in mechanization and materials seems to be little match for the colossal bad taste that permeates modern environments. It is far easier to move the earth than it was 200 years ago, yet the end result gives a feeling that nobody is really trying their best.

This extends beyond structures. The machine age has let us do so much, but it has also degraded work and craft into a means, taking something away from us in the process. What it has done to work it has done to art and leisure as well. If we may have another renaissance, it will be a recapturing of beauty.


In his book The Systems Bible, Gall defined the operational fallacy as a situation where 'the system itself does not do what it says it is doing.' In other words, what a system calls itself is not always a reliable indicator of its true function. In this case, the name of the 'academic publishing industry' implies that it is supposed to be involved in the dissemination of scholarship. But the effective function of the academic publishing industry as it actually exists is to prevent the dissemination of scholarly work.

Few people today are priced out of streaming films or buying MP3s, but access to scientific information has become a true luxury item.


Interesting discussion of how wasm interacts with virtual memory. Many of the underlying capabilities of the OS and hardware are not exposed.


Obstensibly a breakdown of the components of a single phone, but also contains a lot of an in-depth analysis of the shape of the mobile hardware industry as a whole and the long-term strategies of various manufacturers.

Examination of the insides of Huawei’s phones shows that Huawei is relentlessly focused on controlling its own destiny and building up its own ecosystem. Samsung and Apple are famous for their control of their own ecosystems, but teardowns of their flagship phones show that they actually use a lower proportion of their own chips than Huawei. 50% (20 out 40) of the ICs in the Mate 20 X (5G) are designed by HiSilicon, whereas 40% (12 out of 30) of the ICs in the Galaxy S10+ are designed and fabbed by Samsung and 14% (5 out of 36) of the ICs in the iPhone 11 Pro Max are designed by Apple. Huawei is at a distinct disadvantage compared to Samsung, since it doesn’t make its own RAM, Flash memory and image sensors like the South Korean company, but in half a decade Huawei learned to make many of the components in its phones, whereas Samsung has been working on its components for close to 3 decades.

An interesting comment:


Infrastructure languages have a set of requirements that slightly overlap both of these. They need to be safe because they're aimed at network-facing workloads that are going to be exposed to the public internet. A single security bug in the kinds of things that they target can cost millions of dollars. At the same time, they have strict resource (latency, memory overhead, downtime) constraints that also cost a lot of money if you violate. Programmers need to be able to reason about memory consumption and tail latency (which you can't do easily in a world with global GC) because they have SLAs that cost money if they are not met.


Racket added a new type to their runtime to improve the performance of Hash Array Mapped Tries. It seems like the main benefit is better layout and special-cased interaction with the garbage collector, so it would only be useful for managed languages.

I mentioned chidb in the last update, but I just realized that it's part of a whole suite of didactic projects.