There is something cyberpunk about learning about operating systems.
Before I started, I assumed the materials would be very systematic and well laid out: start with how chips are designed, and then on to assembly language, and then on to compilers, and then finally how the operating system lives on top of that.
But it is not like this at all. An implicit knowledge of chips, assembly language, compilers etc. are assumed. The reader is just thrown to some C code, which implements various fundamental features an operating systems must have.
Many of these features (so basic that people never notice) are about limiting the damage done by bad code. Access to “bare metal” is jealously protected and regulated: you want different processes to be able to talk to one another, but you don’t want them to be able to (for example) modify and break how the operating system boots up.
OS-level security is in a sense why we allow other software to operate on the model of mild chaos (hence the cyberpunk analogy). So long as they do not shut the whole system down, user applications are (by and large) left to do what they like.
All of this makes (vague) sense on the conceptual level. But the implementation details add a lot of complexity.
For example, as explained above, the operating system is an abstraction from assembly language, which is just one level above chips. Does it mean I need a physical chip to learn about it?
The answer is “no”. A solution that doesn’t require additional hardware is to use a simulator called QEMU, which emulates a computer’s processor for you. One processor that is popular among textbooks is RISC-V, which is designed to be beginner friendly. See OS in 1,000 lines and this Dan Luu recommendation. Setting up RISC-V on MacOS is not straightforward: the MIT repository suggest building from source, but this blogpost details a simpler way forward.
Running these systems gives one a visceral feel of how much work is required to get even the most basic functions to work. At the end of OS in 1,000 lines, all that can be achieved is a shell that can read and write to a file on disk (the file is fixed; there is no facility to create new files yet).
It boggles the mind that behind every “smart” device, there is something like this at work. The complexity also adds flesh to the claim that, somewhere in the code, there are probably vulnerabilities waiting to be discovered and exploited.