UEFI is an interface implemented by firmware (literally, Unified Extensible Firmware Interface), it's not the firmware itself. Saying "it starts the machine" is a bit of a nomenclature faux pas. The firmware starts the machine, you talk to the firmware via UEFI.
This post skips all the interesting things in the modern firmware dance. Not the least of which is when you call ExitBootServices() you're already in long mode. There's no need for the journey through real and protected.
firmwares are the real IoT nobody talks about that that actually happened. infinite little computers and code churning away from view. and not being updated.
next 'snowden leaks' will have that word on every page.
If you're curious go walk through the bootloaders of an embedded platform, they serve roughly the same role as firmware that implements UEFI on traditional x86 systems.
Example, start with bl1 (first stage boot loader) for a Rasberry Pi here:
I'm currently trying to learn more about how the boot process works, specifically with UEFI. Yesterday I watched a video on how to program a UEFI bootloader so I was happy when I saw this article this morning.
The article jumps over so many steps and leaves out so many mechanics that I had to stop reading, because it just opened more and more questions without closing any. For example, right at the beginning:
> Right after reset the CPU jumps to a special address called the reset vector at 0xFFFFFFF0. Think of it as a permanent bookmark that says “start here.” There is room for almost nothing at that address, so manufacturers put a short jump there that passes control to the firmware on your motherboard.
What does that even mean? It jumps there means that it executed whatever is at that address, but it failed to explain how the instruction at that location is even put there in the first place.
Reading a little more I truly am confused for who this was even written. It seems you need a lot of prior knowledge to understand what the author talks about and then it also makes me wonder what additional information this article conveys if you already can fill in the gaps.
Really nice post, some months ago I also wrote about linux boot, but a bit more focused on the IO side (what's on disk, how is it loaded), here: https://blog.davidv.dev/posts/booting-x86-64/
Oh wow! Combined with the explaination of iso files which can be a single file which can have things like fat and bzimage and rootfs etc. with the bootloader and combined with software like dd / gnome-disks and your article and the HN thread article, this finally makes me understand how Linux starts and how we can install things from https / internet and use software to fundamentally change the operating system
Its magic.
I was recently trying to build my own linux operating system and I felt so frustrated as to how to build an iso image from bzImage / linux kernel + initramfs but I was able to withstand its pain somehow and it took me a week to actually learn a lot more about the fundamentals of linux which are so fascinating.
Now, I was trying to do some mix and matching (imagine tinycorelinux initramfs and buildroot bzImage and merging them) and doing some other shenanigans.
Sometimes my computer kernel didn't work or the userspace didn't work in the vm and I guess now WHEN I AM WRITING THIS POST, I realized that it was because of the difference in 32 bit and 64 bit, I think I had 64 bit kernel and 32 bit userspace which had some issues
There were some different things as well, which had caused some issues which made me want to scratch my remaining hair but it was so worth it (maybe i used some llm assistance in the end as well), its all just makes sense now, I always wanted to learn how computers work from scratch, now I feel like I have a very basic idea on how things work and they are complex indeed :sob:
I was looking for a post like yours yesterday and so I am still glad I found it when the topic is still fresh.
I faced so many issues trying to build my own iso but I don't know if it was my skill issue which streched it to a week. But the whole process of building your own iso seems less daunting to me now knowing what it really is
I may have skill issues indeed but I fixed those, I feel like that there is a lot to learn from those errors that I missed but maybe that's also because they got too overwhelming. But I now know a path which works and some explainations of why they work. I feel more confident in my ability to make my own custom distro even, but my appreciation for linux grows so much.
Its just such a massive rabbit hole and most of linux is literally just "it just works", linux is amazing.
It's a weird article for me. On one side it is an interesting topic. On the other hand why are we explaining what a hex number is? Who is interested in this level of detail but doesn't know hex? Maybe I'm overanalyzing.
At the same time this doesn't address my biggest open question on the topic - how do we get from the physical push to the reset vector? Somehow that magic works in HW, physics and electronics - how?
HN is not just read by IT professionals. And they might be a bit curious about Linux startup without necessarily recalling what they know or never knew about hexadecimals.
It's fine to explain things to people that are not into that topic. But I also wonder what the target audience is. On one hand, he explains what a hexadecimal ist, but then jumps over essential information needed to understand the boot process. It's like: "Learning to swim: This is water, it is wet, if you jump in you sink and in water you cannot breath. Then you start moving your body parts and then you swim."
Does anyone have a similar article with more detail? I don't quite want to read the datasheet of your favorite microprocessor, but I would like a decent amount more detail than what's provided. Especially before UEFI/BIOS.
At a very high level, the processor initialization procedure is similar to most microcontrollers:
- processor is powered up with RESET pin asserted, making sure that all circuits are in a known-safe state.
- when RESET is deasserted, all autonomous circuits (e.g. core control units, cache controllers, ALUs) go through a hardware setup phase.
- the processor accesses the boot firmware according to the platform design. There's many options here, but the three most common are 1) the boot ROM is permanently mapped to a dedicated hardware address, and the processor uses its memory controller to access that memory; 2) the boot ROM is directly connected to the processor via dedicated I/O pins; 3) the initial boot code is located on the processor itself (factory-programmed EEPROM).
- if the boot ROM is not mapped to a memory address, the data is loaded into CPU cache (external RAM needs to be initialized first, and that task is usually performed by the firmware).
- the primary cpu core starts executing boot code (UEFI/BIOS/etc) from said predefined hardware address. Other cores (and processors) are usually left uninitialized throughout the startup process, and it is left to the operating system to bring up the remaining parts of the system.
> When power stabilizes, the CPU resets itself to a tiny, old‑fashioned mode called real mode. Real mode dates back to the original 8086 chip. The rules are simple on purpose. Memory addresses are built from two values the CPU keeps in special fast storage called registers. You combine a segment and an offset like this:
physical_address = (segment << 4) + offset
Your grandmother sounds unusually proficient with this sort of thing.
I dont know, i just don't like the tone. This is a complex subject where the target audience should probably already know what is an hexadecimal number or an interrupt and the explanation of a cpu register ought to be better than:
"A register is a tiny slot inside the CPU. It holds a number the CPU is using right now."
If the subject interest you, you deserve better.
There's nothing wrong with it but most people striving to do this would make each bit clickable to detail or something so that you can read at your level. I imagine this was a constraint of the site framework or the author's writing style.
But hey what the heck, it's fine. An LLM can rewrite it to whatever level of knowledge you like so the deepest level is optimal.
One of the things we were taught in uni was audience analysis. I think about it a lot. What's expected to already be known? What acronyms or phrases need defining? Etc. This is an art I'm far from perfect at and it seems a lot of tech writers are too
I wonder whether the power button really directly starts the CPU. I wouldn't be surprised if the Intel Management Engine (or AMD equivalent), a CPU that's running all the time, is what gets the power button signal and starts the CPU.
I think that is the job of the embedded controller, which is before the bootloader bringing up CPU. On Chromebooks, that part is open source along with the coreboot bootloader.
As should be clear from the reset vector, the 80286 and its successors actually boot in unreal mode. On the 80386, the base address of the code segment is 0xffff0000, which cannot be obtained by shifting the 16-bit CS register by 4. The descriptor cache simply gets loaded with the correct value at reset. Writing to CS in real mode overwrites the cached value with CS * 16.
This isn't quite right either. That's not what unreal mode is. Unreal mode has 32-bit data segments limit, not code segment limits.
In this case the code segment still has a 16 bit limit, it's just that its base address in the descriptor cache is outside the bounds typically associated with real mode.
Related to this topic, what is the best way to replace the code involved in the entire boot process? This is useful when sanitizing a system received from a provider that may not be trustworthy, as malware could be hidden at low levels.
The disk could be wiped from the BIOS. One could also run “fwupdmgr update” from a live USB to update the motherboard firmware and then reinstall the operating system. However, I’m not sure if this would completely clear the system.
Kinda missing the point of UEFI if you don't mention that it skips the first 3 sections and allows your kernel to boot directly from 64-bit mode ("long mode") with paging, a framebuffer, and a function to read files from your boot drive, already set up for you. Removes the need for GRUB entirely.
ARM and lots of non-x86 architectures often use a series of bootloaders to kick up ram, wake up parts of the hardware, blah blah, and read devicetree blobs to know what the hardware looks like
Based on all the negative comments, and being a non-expert, I had an LLM generate an article for that same title. I suspect it was more approachable and more technically accurate than this. In fact I asked it to compare its article with this one and the negative comments.
The author of any technical blog post should be using an LLM to vet things before posting to find gaps or inconsistencies.
This post made me remember the technical interview by phone I had with Facebook circa 2010 for a Production Engineer role (but maybe it wasn't called yet that way back then) where they asked me exactly this. Well, they actually asked "explain me the boot process of a Linux server", with no more hints and "go into the details you feel are important" as the only help.
Anyway I should have moved to Dublin and something something surveillance capitalism so the grape wasn't ripe enough anyway.
fascinating how it's all over the place wrt level of detail. and absolutely unreadable. luckily the layout is simple and reader mode works.
> Hex is base 16
i would argue that someone that understand bases (in the first place), understands what the << operator does (context where base 16 is explained), but doesn't understand what base 16 is, doesn't exist. this is the kind of haphazard approach of this article i'm talking about. even the author's name, 0xkato, is an example of this.
as to the content, i wish it had touched on TPM, PCRs, UEFI secure boot, and ME pre-boot.
i'm forgiving all the actual errors since it is a pretty broad overview.
i'm guessing first-year uni student.
rather amazed a post like this can make it to the #1 spot.
Video device initialization is intimately intertwined and a dependency for all this early boot stuff. I was hoping to learn more but it's not even mentioned. Still, neat.
It's because on a brief period of old PC hardware (1994-1998), when Linux was starting off, high-resolution video initialization (SVGA+) was easier to do in 16-bit mode, before the bootloader switched the CPU over to 32-bit mode. Because GRUB handled this transition, GRUB became responsible for video initialization. After 1998 GPU manufacturers added 32-bit support for initialization, but the long tail of users with hardware with that limitation really entrenched this responsibility in GRUB.
It's not a dependency for Linux boot at all. You can do well with serial port alone, as anyone who brought up eg. an ARM SoC in Linux will attest to.
Also it's not very interesting either. At simplest, Linux just needs to take a pointer to a beginning of a framebuffer and some metadata, and will write to the framebuffer whenever there's something to update.
Soekris (rip) had an x86 network device. Four 10/100s and the disk was a CF. Could only serial console that thing - or SSH once it's running. Best router I ever had.
Also, in 2000 when Windows crashed you could get a serial debugger. Wonder if they still do that?
Maybe not linux specifically, but POST requires a video device software (BIOS Option ROM or UEFI GOP Drivers) of some sort does it not? That's been my experience with all PCs for 30 years. But maybe there are cases where it doesn't?
edit: Apparently it's a desktop motherboard firmware thing. Ubiquitous but not technically a requirement for POSTing a computer.
It was sorta required by IBM PC compatibles, but is not required by UEFI which supports serial console (or other interfaces).
IBM PC assumed existence of graphic output and sometimes Option ROMs did really too crazy things with it, I still have shivers when I hear "intel raid card" because of that one with possibly Win3.x in ROM...
Nice to see the good old hacker energy & independent blogs explaining things showing up on top of hacker news. Welcome change from insufferable agent this and vibe that
Very much overdue, i would say! Lately HN was turning into just adware for AI news.
Does anybody have pointers to similar low-level details specific to iOS/Android systems? Given that most of us by now have a few of these lying around the house gathering dust, they can make a nice DUT to hack/study/learn a few technical things. I would be quite interested in knowing more about how the RF subsystem works and maybe repurposed as an SDR for example.
This post skips all the interesting things in the modern firmware dance. Not the least of which is when you call ExitBootServices() you're already in long mode. There's no need for the journey through real and protected.
next 'snowden leaks' will have that word on every page.
Its certainly not something regularly taught in higher education.
Example, start with bl1 (first stage boot loader) for a Rasberry Pi here:
https://github.com/raspberrypi/arm-trusted-firmware
The Pi's hardware (register map, etc) is well documented so it's a good way to learn this stuff.
The article jumps over so many steps and leaves out so many mechanics that I had to stop reading, because it just opened more and more questions without closing any. For example, right at the beginning:
> Right after reset the CPU jumps to a special address called the reset vector at 0xFFFFFFF0. Think of it as a permanent bookmark that says “start here.” There is room for almost nothing at that address, so manufacturers put a short jump there that passes control to the firmware on your motherboard.
What does that even mean? It jumps there means that it executed whatever is at that address, but it failed to explain how the instruction at that location is even put there in the first place.
Reading a little more I truly am confused for who this was even written. It seems you need a lot of prior knowledge to understand what the author talks about and then it also makes me wonder what additional information this article conveys if you already can fill in the gaps.
Its magic.
I was recently trying to build my own linux operating system and I felt so frustrated as to how to build an iso image from bzImage / linux kernel + initramfs but I was able to withstand its pain somehow and it took me a week to actually learn a lot more about the fundamentals of linux which are so fascinating.
Now, I was trying to do some mix and matching (imagine tinycorelinux initramfs and buildroot bzImage and merging them) and doing some other shenanigans.
Sometimes my computer kernel didn't work or the userspace didn't work in the vm and I guess now WHEN I AM WRITING THIS POST, I realized that it was because of the difference in 32 bit and 64 bit, I think I had 64 bit kernel and 32 bit userspace which had some issues
There were some different things as well, which had caused some issues which made me want to scratch my remaining hair but it was so worth it (maybe i used some llm assistance in the end as well), its all just makes sense now, I always wanted to learn how computers work from scratch, now I feel like I have a very basic idea on how things work and they are complex indeed :sob:
I was looking for a post like yours yesterday and so I am still glad I found it when the topic is still fresh.
I faced so many issues trying to build my own iso but I don't know if it was my skill issue which streched it to a week. But the whole process of building your own iso seems less daunting to me now knowing what it really is
I may have skill issues indeed but I fixed those, I feel like that there is a lot to learn from those errors that I missed but maybe that's also because they got too overwhelming. But I now know a path which works and some explainations of why they work. I feel more confident in my ability to make my own custom distro even, but my appreciation for linux grows so much.
Its just such a massive rabbit hole and most of linux is literally just "it just works", linux is amazing.
At the same time this doesn't address my biggest open question on the topic - how do we get from the physical push to the reset vector? Somehow that magic works in HW, physics and electronics - how?
- processor is powered up with RESET pin asserted, making sure that all circuits are in a known-safe state.
- when RESET is deasserted, all autonomous circuits (e.g. core control units, cache controllers, ALUs) go through a hardware setup phase.
- the processor accesses the boot firmware according to the platform design. There's many options here, but the three most common are 1) the boot ROM is permanently mapped to a dedicated hardware address, and the processor uses its memory controller to access that memory; 2) the boot ROM is directly connected to the processor via dedicated I/O pins; 3) the initial boot code is located on the processor itself (factory-programmed EEPROM).
- if the boot ROM is not mapped to a memory address, the data is loaded into CPU cache (external RAM needs to be initialized first, and that task is usually performed by the firmware).
- the primary cpu core starts executing boot code (UEFI/BIOS/etc) from said predefined hardware address. Other cores (and processors) are usually left uninitialized throughout the startup process, and it is left to the operating system to bring up the remaining parts of the system.
Specifically for Intel, see https://binarydebt.wordpress.com/2018/10/06/how-does-an-x86-... . Or see https://electronics.stackexchange.com/a/726918 and its comments to get an idea of the differences between platforms.
What's your issue with this? Would you prefer it mentioned the x86 register doesn't always correspond to the same place in the register file?
But hey what the heck, it's fine. An LLM can rewrite it to whatever level of knowledge you like so the deepest level is optimal.
These blog posts really annoy me because I feel like with 20% more effort you could have something worth reading.
If you want a more thorough explanation, go read a book. Many are available for free on sites such as archive.org and programming-motherfucker.com
Here are some details: https://www.pixelbeat.org/docs/disk/
https://chromium.googlesource.com/chromiumos/platform/ec/+/H...
In this case the code segment still has a 16 bit limit, it's just that its base address in the descriptor cache is outside the bounds typically associated with real mode.
There is a well praised post on HN: https://www.nan.fyi/database, built with the framework: https://github.com/nandanmen/NotANumber
The disk could be wiped from the BIOS. One could also run “fwupdmgr update” from a live USB to update the motherboard firmware and then reinstall the operating system. However, I’m not sure if this would completely clear the system.
The author of any technical blog post should be using an LLM to vet things before posting to find gaps or inconsistencies.
Anyway I should have moved to Dublin and something something surveillance capitalism so the grape wasn't ripe enough anyway.
> Hex is base 16
i would argue that someone that understand bases (in the first place), understands what the << operator does (context where base 16 is explained), but doesn't understand what base 16 is, doesn't exist. this is the kind of haphazard approach of this article i'm talking about. even the author's name, 0xkato, is an example of this.
as to the content, i wish it had touched on TPM, PCRs, UEFI secure boot, and ME pre-boot.
i'm forgiving all the actual errors since it is a pretty broad overview.
i'm guessing first-year uni student.
rather amazed a post like this can make it to the #1 spot.
https://webaim.org/resources/contrastchecker/
(this is the site: https://webaim.org/resources/contrastchecker/?fcolor=D0D0D0&...)
UEFI does not require at all and supports and UI framework to enable work in both graphical and textual (with so-called VT-UTF8) mode.
https://www.amazon.com/-/he/Developing-32-Bit-Operating-Syst...
Also it's not very interesting either. At simplest, Linux just needs to take a pointer to a beginning of a framebuffer and some metadata, and will write to the framebuffer whenever there's something to update.
It is a unique monstrosity that boots from the video / GPU core instead of one of the ARM cores. It has an arcane undocumented architecture.
Thankfully, pretty much everyone else just uses U-Boot.
Also, in 2000 when Windows crashed you could get a serial debugger. Wonder if they still do that?
edit: Apparently it's a desktop motherboard firmware thing. Ubiquitous but not technically a requirement for POSTing a computer.
IBM PC assumed existence of graphic output and sometimes Option ROMs did really too crazy things with it, I still have shivers when I hear "intel raid card" because of that one with possibly Win3.x in ROM...
Does anybody have pointers to similar low-level details specific to iOS/Android systems? Given that most of us by now have a few of these lying around the house gathering dust, they can make a nice DUT to hack/study/learn a few technical things. I would be quite interested in knowing more about how the RF subsystem works and maybe repurposed as an SDR for example.