as many of you should know, I am currently developing an operating system. Currently I am working through paging and to get better panics, I decided to support DWARF debug info in the panic function. At the beginning, it seemed relatively easy: I just added the DWARF sections to the linker script, got some slices from the addresses my symbols (dwarf_info_start, dwarf_info_end and so on) contained and initialized the DWARF debug info. But when running everything, std.dwarf.openDwarfDebugInfo() threw an EndOfBuffer error.
I looked at the ELF file using bloaty and it says:
So apparently, there is some debug info. But the section types donāt really look like it is the debug info I am searching for.
My code is at https://codeberg.org/loup-os/kernel (branch paging). If you want to run the kernel, I recommend you to clone https://codeberg.org/loup-os/dev-env, then run setup.sh, cd into sources/kernel, git checkout paging, cd back into dev-env and then run zig build run (dependencies: qemu). If you scroll through the entire log, youāll find the error somewhere.
cd sources/
cd kernel/
git checkout paging ā¦branch āpagingā set up to track āorigin/pagingā.
Switched to a new branch 'paging
zig run build ā¦ Error message
Error log below as well
[error] (debug): Error occurred during opening debug info: EndOfBuffer
I think that the git change maybe the problem.
Grin
If you need me to try anything else please advise.
// ----------------------------------------
// Senior Moo says: "I like the errors being in red"
// ----------------------------------------
// \
// \ ^__^
// \ (oo)\_______
// (__)\ )\/\
// ||----w |
// || ||
All the best
Working log
WARNING: Image format was not specified for 'json:{"fat-type": 0, "dir": "/media/sgeorge/usb_hd_ext4/00_bits/Computers/Zig/Play/20240802_kernel_ziggit/dev-env/zig-out/systemroot", "driver": "vvfat", "floppy": false, "rw": true, "write-target": {"driver": "qcow", "file": {"driver": "file", "filename": "/var/tmp/vl.LUZMR2"}}}' and probing guessed raw.
Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
Specify the 'raw' format explicitly to remove the restrictions.
BdsDxe: loading Boot0001 "UEFI QEMU HARDDISK QM00004 " from PciRoot(0x0)/Pci(0x1,0x1)/Ata(Secondary,Slave,0x0)
BdsDxe: starting Boot0001 "UEFI QEMU HARDDISK QM00004 " from PciRoot(0x0)/Pci(0x1,0x1)/Ata(Secondary,Slave,0x0)
[debug] (main): Locating graphics output protocol
[debug] (main): Querying graphics mode info
[info] (main): Current graphics mode = 0
[info] (main): Resolution and pixel format: 1280x800 BlueGreenRedReserved8BitPerColor
[debug] (main): Locating simple file system protocol
[debug] (main): Opening root volume
[debug] (main): Getting memory map to find free addresses
[debug] (main): Finding free kernel base address
[debug] (main): mem_count is 103
[debug] (main): mem_index is 0
[debug] (main): mem_index is 1
[debug] (main): mem_index is 2
[debug] (main): Found 1792 free pages at 0x100000
[info] (main): Loading kernel image
[debug] (loader): Opening kernel image
[debug] (loader): Checking ELF identity
[debug] (loader): ELF identity is good; continuing loading
[debug] (loader): Loading ELF header
[debug] (loader): Loading ELF header succeeded; entry point is 0x100010
[debug] (loader): Loading program headers
[debug] (loader): Loading 5 segments
[debug] (loader): Loading program segment 0
[debug] (loader): Set kernel start address to 0x100000 and base address difference to 0x0
[debug] (loader): Allocating 70 pages at address '0x100000'
[debug] (loader): Reading segment data with file size '0x4521a'
[debug] (loader): Loading program segment 1
[debug] (loader): Allocating 7 pages at address '0x146000'
[debug] (loader): Reading segment data with file size '0x6a98'
[debug] (loader): Loading program segment 2
[debug] (loader): Allocating 9 pages at address '0x14d000'
[debug] (loader): Reading segment data with file size '0x1088'
[debug] (loader): Zero-filling 28664 bytes at address '0x14e088'
[debug] (main): Kernel Entry Point is: '0x0000000000100010'
[debug] (main): Kernel Start Address is: '0x0000000000100000'
[debug] (main): Disabling watchdog timer
[info] (main): Getting memory map and trying to exit boot services
Not working log
WARNING: Image format was not specified for 'json:{"fat-type": 0, "dir": "/media/sgeorge/usb_hd_ext4/00_bits/Computers/Zig/Play/20240802_kernel_ziggit/dev-env/zig-out/systemroot", "driver": "vvfat", "floppy": false, "rw": true, "write-target": {"driver": "qcow", "file": {"driver": "file", "filename": "/var/tmp/vl.XUILR2"}}}' and probing guessed raw.
Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
Specify the 'raw' format explicitly to remove the restrictions.
BdsDxe: loading Boot0001 "UEFI QEMU HARDDISK QM00004 " from PciRoot(0x0)/Pci(0x1,0x1)/Ata(Secondary,Slave,0x0)
BdsDxe: starting Boot0001 "UEFI QEMU HARDDISK QM00004 " from PciRoot(0x0)/Pci(0x1,0x1)/Ata(Secondary,Slave,0x0)
[debug] (main): Locating graphics output protocol
[debug] (main): Querying graphics mode info
[info] (main): Current graphics mode = 0
[info] (main): Resolution and pixel format: 1280x800 BlueGreenRedReserved8BitPerColor
[debug] (main): Locating simple file system protocol
[debug] (main): Opening root volume
[debug] (main): Getting memory map to find free addresses
[debug] (main): Finding free kernel base address
[debug] (main): mem_count is 107
[debug] (main): mem_index is 0
[debug] (main): mem_index is 1
[debug] (main): mem_index is 2
[debug] (main): Found 1792 free pages at 0x100000
[info] (main): Loading kernel image
[debug] (loader): Opening kernel image
[debug] (loader): Checking ELF identity
[debug] (loader): ELF identity is good; continuing loading
[debug] (loader): Loading ELF header
[debug] (loader): Loading ELF header succeeded; entry point is 0x1001d0
[debug] (loader): Loading program headers
[debug] (loader): Loading 5 segments
[debug] (loader): Loading program segment 0
[debug] (loader): Set kernel start address to 0x100000 and base address difference to 0x0
[debug] (loader): Allocating 132 pages at address '0x100000'
[debug] (loader): Reading segment data with file size '0x83a9a'
[debug] (loader): Loading program segment 1
[debug] (loader): Allocating 199 pages at address '0x184000'
[debug] (loader): Reading segment data with file size '0xc6b69'
[debug] (loader): Loading program segment 2
[debug] (loader): Allocating 10 pages at address '0x24b000'
[debug] (loader): Reading segment data with file size '0x12a8'
[debug] (loader): Zero-filling 32152 bytes at address '0x24c2a8'
[debug] (main): Kernel Entry Point is: '0x00000000001001d0'
[debug] (main): Kernel Start Address is: '0x0000000000100000'
[debug] (main): Disabling watchdog timer
[info] (main): Getting memory map and trying to exit boot services
[info] (kmain): Zig OS Kernel
[info] (kmain): =============
[info] (kp_alloc): Kernel Page Allocator initialization...
[debug] (kp_alloc): Bootstrapping the allocator...
[debug] (kp_alloc): 1451 free pages found at 0x255000
[debug] (kp_alloc): 3 free pages found at 0x808000
[debug] (kp_alloc): 4 free pages found at 0x80c000
[debug] (kp_alloc): 9205 free pages found at 0x1780000
[debug] (kp_alloc): 9615 free pages found at 0x3b95000
[debug] (kp_alloc): 29 free pages found at 0x6143000
[debug] (kp_alloc): 1 free pages found at 0x6251000
[debug] (kp_alloc): 1 free pages found at 0x63df000
[debug] (kp_alloc): 1 free pages found at 0x63e8000
[debug] (kp_alloc): 1 free pages found at 0x6969000
[debug] (kp_alloc): 219 free pages found at 0x7e00000
[debug] (kp_alloc): 81 pages needed to store map
[debug] (kp_alloc): Reslicing the map to 0x7af000
[debug] (kp_alloc): Migrating map contents from bootstrap to new slice
[debug] (kp_alloc): Allocating 1 pages at 0x00000000007ae000
[debug] (kp_alloc): Test allocation has 1
[info] (kp_alloc): Kernel Page Allocator initialization successful!
[debug] (debug): Panic and debug info initialization...
[debug] (debug): Debug INFO from 0x000000000018eab0 to 0x00000000001e1133 (len = 337539)
[debug] (debug): Debug ABBREV from 0x00000000001e1133 to 0x00000000001e18ef (len = 1980)
[debug] (debug): Debug STR from 0x00000000001e18ef to 0x0000000000201fed (len = 132862)
[debug] (debug): Debug LINE from 0x0000000000201fed to 0x0000000000239149 (len = 225628)
[debug] (debug): Debug RANGES from 0x0000000000239149 to 0x000000000024ab69 (len = 72224)
[error] (debug): Error occurred during opening debug info: EndOfBuffer
[info] (acpi): ACPI initialization...
[debug] (acpi): RSDP 1.0 Address: 0x777e000
[debug] (acpi): RSDP 2.0 Address: 0x777e014
[debug] (acpi): RSDP Signature is valid!
[debug] (acpi): RSDP Checksum is valid!
[debug] (acpi): XSDT Address is 0x777d0e8
[debug] (acpi): XSDT Signature is valid!
[debug] (acpi): XSDT Checksum is valid!
[debug] (acpi): XSDT has 5 entries
[debug] (acpi_xsdt): Entry is FACP
[debug] (acpi_xsdt): Entry is APIC
[debug] (acpi): Found MADT at 0x7779000
[debug] (acpi): MADT Checksum is valid!
[debug] (acpi_madt): MADT Entry tag is lapic
[debug] (acpi_madt): MADT Entry tag is io_apic
[debug] (acpi): I/O APIC Entry is acpi.madt.IOAPIC{ .apic_id = 0, .res1 = 0, .ioapic_addr = 4273995776, .glob_sys_int_base = 0 }
[info] (acpi): ACPI initialization successful!
[info] (arch_platform): Platform-specific initialization...
[info] (arch_gdt): GDT initialization...
[info] (arch_gdt): GDT initialization successful!
[info] (arch_idt): IDT initialization...
[info] (arch_idt): IDT initialization successful!
[info] (arch_platform): PIC disabling...
[info] (arch_platform): PIC disabling successful!
[info] (arch_paging): Paging initialization...
[debug] (arch_paging): Allocating a Page Map Level 4 Table
[debug] (kp_alloc): Allocating 1 pages at 0x00000000007ad000
[debug] (arch_paging): UEFI Memory Map Contents:
[debug] (arch_paging): V: 0x0-0xfff PG: 1 T: boot_services
[debug] (arch_paging): V: 0x1000-0x9ff61 PG: 159 T: conventional
[debug] (arch_paging): V: 0x100000-0x254eab PG: 341 T: loader
[debug] (arch_paging): V: 0x255000-0x7ffa55 PG: 1451 T: conventional
[debug] (arch_paging): V: 0x800000-0x807ff8 PG: 8 T: acpi
[debug] (arch_paging): V: 0x808000-0x80affd PG: 3 T: conventional
[debug] (arch_paging): V: 0x80b000-0x80bfff PG: 1 T: acpi
[debug] (arch_paging): V: 0x80c000-0x80fffc PG: 4 T: conventional
[debug] (arch_paging): V: 0x810000-0x8fff10 PG: 240 T: acpi
[debug] (arch_paging): V: 0x900000-0x177f180 PG: 3712 T: boot_services
[debug] (arch_paging): V: 0x1780000-0x3b72c0b PG: 9205 T: conventional
[debug] (arch_paging): V: 0x3b75000-0x3b94fe0 PG: 32 T: boot_services
[debug] (arch_paging): V: 0x3b95000-0x6121a71 PG: 9615 T: conventional
[debug] (arch_paging): V: 0x6124000-0x6142fe1 PG: 31 T: loader
[debug] (arch_paging): V: 0x6143000-0x615ffe3 PG: 29 T: conventional
[debug] (arch_paging): V: 0x6160000-0x6250f0f PG: 241 T: boot_services
[debug] (arch_paging): V: 0x6251000-0x6251fff PG: 1 T: conventional
[debug] (arch_paging): V: 0x6252000-0x63dee73 PG: 397 T: boot_services
[debug] (arch_paging): V: 0x63df000-0x63dffff PG: 1 T: conventional
[debug] (arch_paging): V: 0x63e0000-0x63e7ff8 PG: 8 T: boot_services
[debug] (arch_paging): V: 0x63e8000-0x63e8fff PG: 1 T: conventional
[debug] (arch_paging): V: 0x63e9000-0x6968fea PG: 1408 T: boot_services
[debug] (arch_paging): V: 0x6969000-0x6969fff PG: 1 T: conventional
[debug] (arch_paging): V: 0x696a000-0x74ecd71 PG: 2947 T: boot_services
[debug] (arch_paging): V: 0x74ed000-0x76ecf00 PG: 512 T: runtime_services
[debug] (arch_paging): V: 0x76ed000-0x776cf80 PG: 128 T: reserved
[debug] (arch_paging): V: 0x776d000-0x77fef80 PG: 146 T: acpi
[debug] (arch_paging): V: 0x77ff000-0x7dff9ff PG: 1537 T: boot_services
[debug] (arch_paging): V: 0x7e00000-0x7edaf25 PG: 219 T: conventional
[debug] (arch_paging): V: 0x7edb000-0x7f57fe7 PG: 125 T: boot_services
[debug] (arch_paging): V: 0x7f58000-0x7f77fe0 PG: 32 T: runtime_services
[debug] (arch_paging): V: 0x7f78000-0x7ffff78 PG: 136 T: acpi
[debug] (arch_paging): Kernel Physical Start: 0x0000000000100000
[debug] (arch_paging): Kernel page size: 341 pages
[info] (arch_paging): Paging initialization successful!
[info] (arch_lapic): APIC initialization...
[debug] (arch_lapic): CPU has builtin APIC
[debug] (arch_lapic): APIC Base is 0xfee00000
[debug] (arch_lapic): Physical from virtual (hex): null
[debug] (arch_lapic): Register local_apic_version has value 0x50014
[debug] (arch_lapic): Local APIC Version: arch.x86-64.lapic.LapicVersion{ .version = 20, .res1 = 0, .max_lvt = 5, .eoi_broadcast_suppression = false, .res2 = 0 }
[info] (arch_lapic): APIC initialization successful!
[info] (arch_ioapic): I/O APIC Initialization...
[debug] (arch_ioapic): Disabling IRQ 0 => VEC 48
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Enabling IRQ 1 => VEC 49
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Disabling IRQ 2 => VEC 50
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Disabling IRQ 3 => VEC 51
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Disabling IRQ 4 => VEC 52
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Disabling IRQ 5 => VEC 53
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Disabling IRQ 6 => VEC 54
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Disabling IRQ 7 => VEC 55
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Disabling IRQ 8 => VEC 56
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Disabling IRQ 9 => VEC 57
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Disabling IRQ 10 => VEC 58
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Disabling IRQ 11 => VEC 59
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Disabling IRQ 12 => VEC 60
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Disabling IRQ 13 => VEC 61
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Disabling IRQ 14 => VEC 62
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Disabling IRQ 15 => VEC 63
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Disabling IRQ 16 => VEC 64
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Disabling IRQ 17 => VEC 65
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Disabling IRQ 18 => VEC 66
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Disabling IRQ 19 => VEC 67
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Disabling IRQ 20 => VEC 68
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Disabling IRQ 21 => VEC 69
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Disabling IRQ 22 => VEC 70
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): Disabling IRQ 23 => VEC 71
[debug] (arch_lapic): Register local_apic_id has value 0x0
[debug] (arch_ioapic): I/O APIC Version: 0x170020
[info] (arch_ioapic): I/O APIC Initialization successful!
[debug] (arch_platform): CR0 contents: arch.x86-64.registers.CR0{ .pe = true, .mp = true, .em = false, .ts = false, .et = true, .ne = true, .res1 = 0, .wp = true, .res2 = 0, .am = false, .res3 = 0, .nw = false, .cd = false, .pg = true, .res4 = 0 }
[info] (arch_platform): Platform-specific initialization successful!
[debug] (acpi_madt): MADT Entry tag is lapic
[debug] (acpi_madt): MADT Entry tag is io_apic
[debug] (acpi_madt): MADT Entry tag is io_apic_int_src_override
[debug] (arch_ioapic): I/O APIC Source Override: 0x0000000000020000
[debug] (acpi_madt): MADT Entry tag is io_apic_int_src_override
[debug] (arch_ioapic): I/O APIC Source Override: 0x000d000000050500
[debug] (acpi_madt): MADT Entry tag is io_apic_int_src_override
[debug] (arch_ioapic): I/O APIC Source Override: 0x000d000000090900
[debug] (acpi_madt): MADT Entry tag is io_apic_int_src_override
[debug] (arch_ioapic): I/O APIC Source Override: 0x000d0000000a0a00
[debug] (acpi_madt): MADT Entry tag is io_apic_int_src_override
[debug] (arch_ioapic): I/O APIC Source Override: 0x000d0000000b0b00
[debug] (acpi_madt): MADT Entry tag is lapic_int_no_mask
[debug] (kmain): kernel_start = 0x100000, kernel_end = 0x254040
[debug] (term_fbcon): Found control sequence!
[debug] (term_fbcon): Arguments:
[debug] (term_fbcon): Number: 38
[debug] (term_fbcon): Number: 5
[debug] (term_fbcon): Number: 100
[debug] (term_fbcon): Command: m
[debug] (term_fbcon): Trying to handle it now
[warning] (term_fbcon_sgr): Unknown control sequence argument, skipping
[debug] (term_fbcon): Found control sequence!
[debug] (term_fbcon): Arguments:
[debug] (term_fbcon): Number: 2
[debug] (term_fbcon): Command: J
[debug] (term_fbcon): Trying to handle it now
[debug] (term_fbcon): Found control sequence!
[debug] (term_fbcon): Arguments:
[debug] (term_fbcon): Number: 1
[debug] (term_fbcon): Number: 4
[debug] (term_fbcon): Number: 7
[debug] (term_fbcon): Number: 35
[debug] (term_fbcon): Command: m
[debug] (term_fbcon): Trying to handle it now
[debug] (term_fbcon): Found control sequence!
[debug] (term_fbcon): Arguments:
[debug] (term_fbcon): Number: 0
[debug] (term_fbcon): Number: 35
[debug] (term_fbcon): Command: m
[debug] (term_fbcon): Trying to handle it now
[debug] (term_fbcon): Found control sequence!
[debug] (term_fbcon): Arguments:
[debug] (term_fbcon): Number: 0
[debug] (term_fbcon): Command: m
[debug] (term_fbcon): Trying to handle it now
On the Zig master branch, there have been some changes in DWARF debug info (namespace and so on). Iāll use that to re-think another time about my problemā¦
On another note. Why are you building a kernel in zig? Is it for fun? Do you expect to test out an idea? Do you want to make it commercial? Is it a learning exercise? Just wondered what you hope to achieve with the competition being well advanced in the case of the Linux kernel.
I suppose what I am trying to get at is this. Is this going to be an āalso ranā? And if not what makes it special? Whatās itās niche?
At the moment, itās just āyet another kernelā. But somehow I feel like I want to start from scratch with another operating system written entirely in Zig. And itās a great learning experience for me.
And I have some interesting things:
A kernel shell (a shell built in the kernel) which will accept human-readable commands for low-level things. I plan to even include this shell in kernel panics, so that people with a keyboard and some intelligence could even ārecoverā the kernel from a panic.
There are many hobby OS developers and even many kernels written in Zig. However, during my beginnings with a kernel I found (I think) no OS that was up-to-date with Zig and has an own UEFI bootloader. My OS features both of them.
(2) is already in there in my opinion, (1) is planned.
But it can be an āalso ranā too, because I plan POSIX system calls and ELF executables (so most linux programs could run on it).
I just inspected it and observed some weird behaviour of QEMU:
When I run zig build install and then objdump --all-headers, everything is fine. But after running the QEMU command (using zig build run or manually), objdump complains. Does anyone know why?
Anyways, I inspected the (working) binary using Cutter (software for reverse engineering) and went through the error return trace.
It seems like there is something with debug_abbrev (I got the error in some function related to that). The size of debug_abbrev seems to be around 1.8KB, while the other debug infos are all bigger than 70KB (look at this post). What am I doing wrong? Or is that normal?
It seems like the small size of debug_abbrev is normal. However, I dumped the first 16 bytes of it, compared it a bit with some examples from the OSDev Wiki and found that it seems to be valid data. That makes the problem even more complicated.
Now it works. My solution was to stop getting the debug info in the kernel and instead just do everything in the bootloader.
I just put all the debug info in separate sections and went over those sections in the bootloader to get slices of section contents. Then, I opened the debug info inside the bootloader and passed it to the kernel using my KernelBootInfo.
When you say keyboard do you mean UART / serial connection? Many electronic boards will have a Rx and TX copper contact on their board (test pads). This allows a cheap serial connection. Ie you can now ādumb terminalā into the board ie vt100 etc. Is this what you envisioned? If so pretty cool.
As of now, all my logging works through UART, so itās already used. But in special cases (like a kernel panic) it might be useful to do that command line via UART.
With keyboard, I meant standard PS/2, but that might change in the future.
I didnāt have your idea, but it looks cool. When my OS is in a good state, probably Iāll ask people like you some questions what you wish was there.