So it all depends on buffer size:
It’s a bit strange.
I have enough space in any case, why can I cast to HdrElf * only for some buf sizes?
It looks like the buffer size must be multiple of @sizeof(ElfHdr), which is in turn may not be “true” size (8 instead of 7).
it’s kinda disappointing, isn’t it?
So does this really mean you cannot have packed structs that are odd size? That would be really strange indeed.
To add to your experiment, also a strange thing is that the original Elf64ExecutionHeader packed struct is 64 bytes in size (so even number). And it still fails if you want to print out certain fields with alignment issue, even though const header: *Elf64ExecutionHeader = @ptrCast(@alignCast(&buffer)); produced no errors. Crash only happens when I try to print it out (then I get alignment issue).
std.debug.print("Object file type: {}\n", .{header.type});
std.debug.print("Architecture: {}\n", .{header.machine});
std.debug.print("Object file version: 0x{X:0>2}\n", .{header.version});
std.debug.print("Entry point virtual address: 0x{X:0>2}\n", .{header.entry});
std.debug.print("Program header table file offset: {}\n", .{header.phoff});
std.debug.print("Section header table file offset: {}\n", .{header.shoff});
std.debug.print("Processor-specific flags: 0x{X:0>2}\n", .{header.flags});
std.debug.print("ELF header size in bytes: {}\n", .{header.ehsize});
std.debug.print("Program header table entry size: {}\n", .{header.phentsize});
// std.debug.print("Program header table entry count: {}\n", .{header.phnum});
// std.debug.print("Section header table entry size: {}\n", .{header.shentsize});
// std.debug.print("Section header table entry count: {}\n", .{header.shnum});
// std.debug.print("Section header string table index: {}\n", .{header.shstrndx});
I intentionally don’t print out the bottom 4 fields because the program crashes.
zig run main.zig
thread 2199375 panic: incorrect alignment
/home/m/Vault/projects/probe/zig-elf/main.zig:36:43: 0x103920a in main (main)
const header: *Elf64ExecutionHeader = @ptrCast(@alignCast(&buffer));
^
/home/m/.local/bin/zig-linux-x86_64-0.14.0-dev.1587+feaee2ba1/lib/std/start.zig:619:37: 0x1038eb2 in posixCallMainAndExit (main)
const result = root.main() catch |err| {
^
/home/m/.local/bin/zig-linux-x86_64-0.14.0-dev.1587+feaee2ba1/lib/std/start.zig:250:5: 0x1038a8f in _start (main)
asm volatile (switch (native_arch) {
^
???:?:?: 0x0 in ??? (???)
This crash happens when I try to print std.debug.print("Program header table entry count: {}\n", .{header.phnum}); that is commented out from my example above.
Additionally, I added @sizeOf and @bitSizeOf to check the struct size just to be sure, and they are the same.
sizeOf(Elf64ExecutionHeader) = 64
bitSizeOf(Elf64ExecutionHeader) = 512 (512/8=64)
I am baffled by this. There must be an explanation for this.
Just a guess - maybe those problematic fields appeared to be (naturally) unaligned?..
But Intel CPUs AFAIK allows f32, u16, u32 to be unaligned by the cost of access time.
(
I remember I run into similar problems on some ARM,
de-referencing unaligned pointer to f32 gives weird value if this float,
and program does not get SIGBUS, the pointer is just gets aligned under the hood,
thus giving wrong value.
)
(With respect the alignment issues.) Note in this error (attached below), Zig is trying to tell you that the @alingCast asserts that the alignment matches. But since the u8-array has an alignment requirement of 1, that’s probably not always true (its probably always 8-byte aligned, but only sometimes 16-byte aligned, depending on what else is on the stack?).
I think you want to add align(@alignOf(Elf64ExecutionHeader)) to the array declaration:
var buffer: [64]u8 align(@alignOf(Elf64ExecutionHeader)) = undefined;
A continuation. Change buffer size in program 1 to 9:
const std = @import("std");
const ElfHdr = packed struct {
sign: u32, // 7F 45 4C 46, '[DEL]ELF'
clas: u8, // 2 for 64 bit
data: u8, // endianess, 1 for LE
vers: u8, // == 1
};
pub fn main() !void {
std.debug.print("sizeof(ElfHdr) = {}\n", .{@sizeOf(ElfHdr)});
var file = try std.fs.cwd().openFile("./elf", .{});
defer file.close();
var buffer: [9]u8 = undefined;
_ = try file.read(buffer[0..]);
const h: *ElfHdr = @ptrCast(@alignCast(&buffer));
std.debug.print("hdr = {any}\n", .{h});
}
Crashed.
Now, compile with zig build-exe elf.zig -O ReleaseSmall.
Everything is ok:
$ ./elf
sizeof(ElfHdr) = 8
hdr = elf.ElfHdr{ .sign = 1179403647, .clas = 2, .data = 1, .vers = 1 }
That panic happens only when compiling in default (Debug) mode,
It does not happen when compiling in Fast, Safe and Small mode
(and the program produces correct results).
Misaligned memory access is undefined behavior in Zig. @alignCast checks for that in debug builds. In release, the check doesn’t happen. If you’re using x86, most times things will still work, since x86 is very relaxed with alignment. Sometimes things can break though.
Yeah, I see, but I do not quite understand, why (in debug) casting address of 9 bytes array to ElfHdr* is incorrect alignment, and casting address of 8,16… bytes array is not.
It just can happen to be correct accidentially depending on how stuff is placed in memory, because you don’t declare an alignment for the array.
You need to do this:
The @alignCast isn’t a directive to tell the compiler “please align this”, it only tells the compiler to check (in debug/safe) for a difference in alignment, the above code actually declares the correct alignment for the array.
If you don’t declare the correct alignment you still can get it by chance, but it is unreliable.
Yeees, thanx to @rootbeer and to @Sze !
Now the prog works fine even with 7 byte buffer and with Debug build:
const std = @import("std");
const ElfHdr = packed struct {
sign: u32, // 7F 45 4C 46, '[DEL]ELF'
clas: u8, // 2 for 64 bit
data: u8, // endianness, 1 for LE
vers: u8, // == 1
};
pub fn main() !void {
var file = try std.fs.cwd().openFile("./elf", .{});
defer file.close();
var buffer: [7]u8 align(@alignOf(ElfHdr)) = undefined;
_ = try file.read(buffer[0..@sizeOf(ElfHdr) - 1]);
const hdr: *ElfHdr = @ptrCast(&buffer); // no @alignCast() this time
std.debug.print("hdr = {any}\n", .{hdr});
}
Looks exactly like this.
Size and alignment of packed struct {a: u8, b: u8} is 2,
whereas size and alignment of packed struct {a: u8, b: u8, c: u8} is 4.
I guess it is somehow related with “pointers alignment”.
2-byte aligned pointer always has zero least significant bit.
4-byte aligned pointer always has 2 zero least significant bits and so on.
Since you can’t have 1.5 LSB to be zero, you can’t have 3-byte aligned pointer.
And hence you can’t have a pointee to have alignment equal to 3.
Something like that 
@dee0xeed this actually makes sense now.
For anybody interested in the solution, I am posting one below.
Thank you to everybody for helping clarify this for me.
const std = @import("std");
const Elf64ExecutionHeader = packed struct {
ident: u128,
type: u16,
machine: u16,
version: u32,
entry: u64,
phoff: u64,
shoff: u64,
flags: u32,
ehsize: u16,
phentsize: u16,
phnum: u16,
shentsize: u16,
shnum: u16,
shstrndx: u16,
};
pub fn main() !void {
var file = try std.fs.cwd().openFile("./elf", .{});
defer file.close();
var buffer: [64]u8 align(@alignOf(Elf64ExecutionHeader)) = undefined;
_ = try file.read(buffer[0..@sizeOf(Elf64ExecutionHeader)]);
const header: *Elf64ExecutionHeader = @ptrCast(&buffer);
std.debug.print("sizeOf(Elf64ExecutionHeader) = {}\n", .{@sizeOf(Elf64ExecutionHeader)});
std.debug.print("bitSizeOf(Elf64ExecutionHeader) = {} ({}/8={d})\n", .{
@bitSizeOf(Elf64ExecutionHeader),
@bitSizeOf(Elf64ExecutionHeader),
@bitSizeOf(Elf64ExecutionHeader) / 8,
});
std.debug.print("Object file type: {}\n", .{header.type});
std.debug.print("Architecture: {}\n", .{header.machine});
std.debug.print("Object file version: 0x{X:0>2}\n", .{header.version});
std.debug.print("Entry point virtual address: 0x{X:0>2}\n", .{header.entry});
std.debug.print("Program header table file offset: {}\n", .{header.phoff});
std.debug.print("Section header table file offset: {}\n", .{header.shoff});
std.debug.print("Processor-specific flags: 0x{X:0>2}\n", .{header.flags});
std.debug.print("ELF header size in bytes: {}\n", .{header.ehsize});
std.debug.print("Program header table entry size: {}\n", .{header.phentsize});
std.debug.print("Program header table entry count: {}\n", .{header.phnum});
std.debug.print("Section header table entry size: {}\n", .{header.shentsize});
std.debug.print("Section header table entry count: {}\n", .{header.shnum});
std.debug.print("Section header string table index: {}\n", .{header.shstrndx});
}
Kinda yes, but I’m still confused a bit by that padding.
In C I can easly do things like
#include <stdio.h>
#include <string.h>
struct s {
char x;
char y;
char z;
} __attribute__((packed));
char *buf = "123456789";
int main(void) {
printf("sizeof(struct s) = %lu\n", sizeof(struct s));
struct s *p = (struct s*)buf;
int k = 0;
for (; k < strlen(buf) / sizeof(*p); k++, p++)
printf("s[%d] @ %p = {'%c','%c','%c'}\n", k, p, p->x, p->y, p->z);
}
and it works as intended.
__attribute__((packed)) and Zig’s packed struct are different things, it’s likely that the latter will have a name change at some point.
A packed struct is an integer with field access to bit widths within it. Zig ‘emulates’ (not a great word) integers which aren’t natural widths by effectively rounding up. It would be very inefficient to bit-pack an array of u21, so those have a stride of 4 and an illegal range of values dictated by the width. For one u21 this is dictated by the hardware, there’s no other possibility, but arrays could be exactly 21 bits per unit, at a great cost to performance.
So packed struct(u21) is a u21, including the part where it’s emulated with a u32. It will also have a stride/@sizeOf/alignment of 4. Within the struct there is no padding.
__attribute__((packed)) is different: it guarantees no padding between fields, and that arrays will be laid out by the minimum stride, and the user accepts whatever inefficiency this creates (in exchange for the compactness of data). Since it’s C, the fields are all ≥ sizeof(char), which is 1. How many bits is that? Implementation defined, of course!
Byte-aligned access is not so expensive on x86 in particular, ARMs don’t like it much though. So this has its place.
Zig doesn’t have an equivalent of __attribute__((packed)), although there are proposals to add it.
But apparently it is possible with so called extern structs, thanks to @IntegratedQuantum.
For example
const S = extern struct {
a: u8 align(1),
b: u8,
c: u8,
};
has size 3 and alignment 1.
That’s very cool actually, I didn’t know that setting the first field like that would work, this prints 9 and I would have guessed 12:
const Extern = extern struct {
a: u8 align(1),
b: u8,
c: u8,
};
test "stride of extern" {
std.debug.print("three externs is {d}\n", .{@sizeOf([3]Extern)});
}
So that’s closer to an __attribute__((packed)) than I would have thought. Might even cover all the bases, let’s check:
const Extern2 = extern struct {
a: u8 align(1),
b: u32 align(1),
};
test "stride of extern2" {
std.debug.print("three extern2s is {d}\n", .{@sizeOf([3]Extern2)});
}
Is 15, so, yep. That’s that. Restricted to extern compatible types, but you can’t have everything, or at least not all at once…
There is something similar in D language:
align (1) struct EpollEvent {
align(1):
uint event_mask;
EventSource es;
/* just do not want to use that union, epoll_data_t */
}
static assert(EpollEvent.sizeof == 12);
There are bit fields.
I rarely (if at all) use them, because AFAIK bit order is the same as machine byte order
(I may be wrong here) and I do not want to care about it. Bit shifts and bit-wise ops do their job.
Well, it is names packed and extern that confuse every C programmer, I guess.
extern (which by itself is a veeeery weird modifier for a type) combined with per field aligns is C’s packed (or very close to), and packed is a… a combination of C’s packed and C’s bit fields?..
Btw, /opt/zig-0.14/lib/std$ grep -rI "extern struct" | grep -i ELF shows that everything in there is extern struct, not packed struct.
So chances are that if you are going to read/write some binary files with already designed format or to use existing binary network protocols, you should, most likely, use extern struct. And packed struct is exclusively for internal (aha! that’s where extern came from!) use within a program.
Bit fields do exist, but careful with them, the layout is implementation defined. So there’s no mechanism like a Zig packed struct, which has a layout guarantee: every bit in a packed struct is in machine-specific least-significant-bit order, and there is never any padding between fields (but there is padding when needed at the logical end of the struct).
The part of me which likes to be able to do things just because they’re possible is interested in a “packed array” for Zig, where a packed struct(u17) in a four-unit array has a sizeof 9. Realistically, the implementation complexity and performance impact is probably not worthwhile, and I come up blanks trying to think of a genuine use case for it, but that could be failure of imagination on my part.
Zig even has a special pointer type which allows for taking the address of a bit field in a packed struct. It’s incompatible with ordinary pointers for what I hope are obvious reasons, but the raw material for packed arrays is there. So if anyone comes up with a truly compelling use for it…
I’ve already written above - I personally avoid this muggy bit-level-endiannes stuff.
This whole topic illuminated a ton of things I was not considering, so I consider this thread to be a success.