Raw socket with the new std.Io.net interfaces

Help
1

I am trying to implement an API that uses raw sockets to send data. I want to be able to specify all of the content in a packet, from the ethernet headers to the packet payload.

This document outlines the binary protocol that I am trying to implement: docs/saprus_proto_design.md · main · C2 Games / Red Team / Saprus · GitLab

This function is a basic example of how they create a raw socket and send data in the existing implementation of the protocol: utils/relay_message.go · dev · C2 Games / Red Team / Saprus · GitLab

Here is what I have tried doing using std.Io.net:

const ip: std.Io.net.IpAddress = .{ .ip4 = .unspecified(0) };
const socket = try ip.bind(init.io, .{ .mode = .raw, .protocol = .ethernet }); // I have also tried .protocol = .raw and gotten the same behavior.
defer socket.close(init.io);

try socket.send(init.io, &.{ .ip4 = try .parse("255.255.255.255", 8888) }, "foo");

Where I have foo I want to write data starting with the ethernet headers, then the IP headers, then the UDP headers, then my payload data. If I run my program as a non-root user, the program fails at the try ip.bind(...), which is what I would expect and what I want. If I run as root, I fail with the following stack trace:

unexpected errno: 13
/home/robby/downloads/zig-x86_64-linux-0.16.0-dev.2223+4f16e80ce/lib/std/posix.zig:2216:40: 0x10645e0 in unexpectedErrno (std.zig)
        std.debug.dumpCurrentStackTrace(.{});
                                       ^
/home/robby/downloads/zig-x86_64-linux-0.16.0-dev.2223+4f16e80ce/lib/std/Io/Threaded.zig:11305:63: 0x11bfb46 in netSendMany (std.zig)
                    else => |err| return posix.unexpectedErrno(err),
                                                              ^
/home/robby/downloads/zig-x86_64-linux-0.16.0-dev.2223+4f16e80ce/lib/std/Io/Threaded.zig:11099:29: 0x11b3c7f in netSendPosix (std.zig)
            i += netSendMany(handle, messages[i..], posix_flags) catch |err| return .{ err, i };
                            ^
/home/robby/downloads/zig-x86_64-linux-0.16.0-dev.2223+4f16e80ce/lib/std/Io/net.zig:1101:47: 0x11a7e72 in send (std.zig)
        const err, const n = io.vtable.netSend(io.userdata, s.handle, (&message)[0..1], .{});
                                              ^
/home/robby/src/zaprus/src/main.zig:100:20: 0x11a35ce in main (main.zig)
    try socket.send(init.io, &.{ .ip4 = try .parse("255.255.255.255", 8888) }, "foo");
                   ^
/home/robby/downloads/zig-x86_64-linux-0.16.0-dev.2223+4f16e80ce/lib/std/start.zig:718:30: 0x11a43eb in callMain (std.zig)
    return wrapMain(root.main(.{
                             ^
/home/robby/downloads/zig-x86_64-linux-0.16.0-dev.2223+4f16e80ce/lib/std/start.zig:190:5: 0x11a2a81 in _start (std.zig)
    asm volatile (switch (native_arch) {
    ^
error: Unexpected
/home/robby/downloads/zig-x86_64-linux-0.16.0-dev.2223+4f16e80ce/lib/std/Io/net.zig:1102:21: 0x11a7f0e in send (std.zig)
        if (n != 1) return err.?;
                    ^
/home/robby/src/zaprus/src/main.zig:100:5: 0x11a365f in main (main.zig)
    try socket.send(init.io, &.{ .ip4 = try .parse("255.255.255.255", 8888) }, "foo");
    ^

(errno 13 is std.os.linux.E.ACCES (Permission denied), which should probably be an expected error)

Am I trying to do something that is simply unsupported by the current Io interface? Or am I misunderstanding how to do what I am trying to do? It seems like, instead of specifying an IP address and port that is not actually used, I should be able to get a raw socket by specifying the std.Io.net.Interface, but there does not seem to be any way to do that right now.

2

Have you look via strace which syscall fails?

3

Man page says:

       EACCES (For UNIX domain sockets, which are identified by pathname) Write permission is denied on the des‐
              tination  socket  file, or search permission is denied for one of the directories the path prefix.
              (See path_resolution(7).)

              (For UDP sockets) An attempt was made to send to a network/broadcast address as though  it  was  a
              unicast address.

Thanks for reporting the unexpected errno, I put up #30874 - std.Io.Threaded: sendmmsg can return EACCES - ziglang/zig - Codeberg.org for the missing error code.

4

IPPROTO_RAW, which I think is what your code is currently using, is different from an AF_PACKET socket.

IPPROTO_RAW:

AF_PACKET:

IPPROTO_RAW is for implementing protocols on top of IPv4. You will not be able to control the MAC address etc at this layer.

AF_PACKET is for implementing protocols on top of ethernet (layer 2). At this layer you can control the MAC header etc.

I haven’t looks too closely yet into how to implement AF_PACKET (“raw sockets”) on the io interface yet. It may not be possible, considering that it is impossible to accomplish on windows without a kernel level driver like npcap. (Windows doesn’t think I should have control over my own machine).

The IO interface would need to do the equivalent of the following:

create a socket of type AF_PACKET and SOCK_RAW, with protocol of your choice. The choice of protocol is a filter applied to your packet socket for ingested packets. Only packets with the selected protocol (a 2 byte value in the ethernet header) will pass into your socket. Man page for packet says:

By default, all packets of the specified protocol type are passed
       to a packet socket.

In this example we filter for EtherCAT packets, but you can put anything you want there.

const socket: std.posix.socket_t = try std.posix.socket(
            std.posix.AF.PACKET,
            std.posix.SOCK.RAW,
            std.mem.nativeToBig(u32, ETH_P_ETHERCAT),
        );

You will then need to “bind to an interface”. This means bind to an ethernet port like eth0, or eno1 etc. It depends on your distribution on how they name the interfaces. You can list the available ethernet interfaces on your system with ip addr command.

var ifr: std.posix.ifreq = std.mem.zeroInit(std.posix.ifreq, .{});
        @memcpy(ifr.ifrn.name[0..ifname.len], ifname);
        ifr.ifrn.name[ifname.len] = 0;
        try std.posix.ioctl_SIOCGIFINDEX(socket, &ifr);
        const ifindex: i32 = ifr.ifru.ivalue;

        var rval = std.posix.errno(std.os.linux.ioctl(socket, std.os.linux.SIOCGIFFLAGS, @intFromPtr(&ifr)));
        switch (rval) {
            .SUCCESS => {},
            else => {
                return error.NicError;
            },
        }
        ifr.ifru.flags.BROADCAST = true;
        ifr.ifru.flags.PROMISC = true;
        rval = std.posix.errno(std.os.linux.ioctl(socket, std.os.linux.SIOCSIFFLAGS, @intFromPtr(&ifr)));
        switch (rval) {
            .SUCCESS => {},
            else => {
                return error.NicError;
            },
        }
        const sockaddr_ll = std.posix.sockaddr.ll{
            .family = std.posix.AF.PACKET,
            .ifindex = ifindex,
            .protocol = std.mem.nativeToBig(u16, @as(u16, ETH_P_ETHERCAT)),
            .halen = 0, //not used
            .addr = .{ 0, 0, 0, 0, 0, 0, 0, 0 }, //not used
            .pkttype = 0, //not used
            .hatype = 0, //not used
        };
        try std.posix.bind(socket, @ptrCast(&sockaddr_ll), @sizeOf(@TypeOf(sockaddr_ll)));

These code examples came from my library gatorcat which uses raw sockets to send EtherCAT frames on an ethernet interface: Could at least make sure that all gods with.

I wrote this code a long long time ago. I should clean it up and not use std.posix anymore, instead using the linux syscalls directly. But the concepts should be the same.

2 Likes
5 
bind(3, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("0.0.0.0")}, 16) = 0
getsockname(3, {sa_family=AF_INET, sin_port=htons(255), sin_addr=inet_addr("0.0.0.0")}, [16]) = 0
sendmmsg(3, [{msg_hdr={msg_name={sa_family=AF_INET, sin_port=htons(8888), sin_addr=inet_addr("255.255.255.255")}, msg_namelen=16, msg_iov=[{iov_base="foo", iov_len=3}], msg_iovlen=1, msg_controllen=0, msg_flags=0}}], 1, MSG_NOSIGNAL) = -1 EACCES (Permission denied)

With the man page snippet that @andrewrk added, it makes sense that I’m getting this error for using a broadcast address with a socket that is set up for unicast traffic. I’m a little confused as to how one would use normal UDP broadcast traffic, since out of curiosity I tried various combinations for .mode and .protocol that I would have expected to work (mainly .dgram and .udp respectively), but I always got the same permission denied error. This isn’t my end goal, but I wanted to see if I could get that to work, and I couldn’t.

6

Thanks, I’ve got another one for you :slight_smile:

socket(AF_INET, SOCK_RDM|SOCK_CLOEXEC, IPPROTO_RAW) = -1 ESOCKTNOSUPPORT (Socket type not supported)

I got this by doing:

const ip: std.Io.net.IpAddress = .{ .ip4 = .unspecified(0) };
const socket = try ip.bind(init.io, .{ .mode = .rdm, .protocol = .raw });
defer socket.close(init.io);

try socket.send(init.io, &.{ .ip4 = .{ .bytes = @splat(255), .port = 8888 } }, "foo");

Basically I’ve just been trying different BindOptions to see if there is anything that does what I want, but it doesn’t seem like there is. Can you confirm if one should be able to make an AF_PACKET socket using the std.Io interface, or should I go direct to the OS API like @kj4tmp demonstrated? (Isn’t that slated for removal? Would it be better to just call out to libc?)

FWIW, I was actually depending on gatorcat for doing this in my application before, since the alternative was basically doing the exact same thing myself. If there is no way to use std.Io to do this, I think going back to using gatorcat would be the best for my needs.

7

You should definitely do it yourself and not depend on me. One reason is I think you were unintentionally putting EtherCAT in you socket creation by using my library. Also I only made that vtable thing so unknown people could implement Ethernet interface on something like embedded.

If I have some more time this weekend I’ll try to deep dive on your code and the io interface to provide more strictly helpful and applicable examples.

1 Like
8

I appreciate that! (and if you don’t get time, I really appreciate the help you’ve given me already!)

My full repo can be found here: main.zig « src - zaprus - do THINGS with NETWORKS in ZIG

It’s in a bit of a messy state since I’m in the middle of moving a bunch of things around, but I linked directly to where I’m trying to do the raw socket connection.

Edit: switched link to point to the commit so I can delete the branch

9

To answer this question, std.Io is intended to cover 100% of the networking APIs. It’s not clear to me what you’re missing right now. What does calling out to libc accomplish? I mean specifically, what does it do, so that we can add that missing API to std.Io. Would be helpful to see an example of working strace bits to understand your goal.

10

Here is a functioning example that lets me write data how I want. This program must either be run as root, or using the CAP_NET_RAW capability (man)

const std = @import("std");
const Io = std.Io;

const zig_raw_socket = @import("zig_raw_socket");

pub fn main() !void {
    const socket: i32 = @intCast(std.os.linux.socket(
        std.posix.AF.PACKET,
        std.posix.SOCK.RAW,
        0,
    ));

    // hardcoded interface name for test
    const ifname = "enp7s0";

    var ifr: std.posix.ifreq = std.mem.zeroInit(std.posix.ifreq, .{});
    @memcpy(ifr.ifrn.name[0..ifname.len], ifname);
    ifr.ifrn.name[ifname.len] = 0;
    try std.posix.ioctl_SIOCGIFINDEX(socket, &ifr);
    const ifindex: i32 = ifr.ifru.ivalue;

    var rval = std.posix.errno(std.os.linux.ioctl(socket, std.os.linux.SIOCGIFFLAGS, @intFromPtr(&ifr)));
    switch (rval) {
        .SUCCESS => {},
        else => {
            return error.NicError;
        },
    }
    ifr.ifru.flags.BROADCAST = true;
    ifr.ifru.flags.PROMISC = true;
    rval = std.posix.errno(std.os.linux.ioctl(socket, std.os.linux.SIOCSIFFLAGS, @intFromPtr(&ifr)));
    switch (rval) {
        .SUCCESS => {},
        else => {
            return error.NicError;
        },
    }
    const sockaddr_ll = std.posix.sockaddr.ll{
        .family = std.posix.AF.PACKET,
        .ifindex = ifindex,
        .protocol = std.mem.nativeToBig(u16, @as(u16, std.os.linux.ETH.P.IP)),
        .halen = 0, //not used
        .addr = .{ 0, 0, 0, 0, 0, 0, 0, 0 }, //not used
        .pkttype = 0, //not used
        .hatype = 0, //not used
    };
    _ = std.os.linux.bind(socket, @ptrCast(&sockaddr_ll), @sizeOf(@TypeOf(sockaddr_ll)));

    const payload: []const u8 = "raw bytes sent over the wire";
    _ = std.os.linux.sendto(
        socket,
        payload.ptr,
        payload.len,
        0,
        @ptrCast(&sockaddr_ll),
        @sizeOf(@TypeOf(sockaddr_ll)),
    );
}

Here is the full strace output:

execve("./zig-out/bin/zig_raw_socket", ["./zig-out/bin/zig_raw_socket"], 0x7fff87e84380 /* 24 vars */) = 0
mmap(NULL, 262223, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7efee2aa2000
arch_prctl(ARCH_SET_FS, 0x7efee2ae2028) = 0
prlimit64(0, RLIMIT_STACK, NULL, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
prlimit64(0, RLIMIT_STACK, {rlim_cur=16384*1024, rlim_max=RLIM64_INFINITY}, NULL) = 0
sigaltstack({ss_sp=0x7efee2aa200a, ss_flags=0, ss_size=262144}, NULL) = 0
rt_sigaction(SIGSEGV, {sa_handler=0x11a5210, sa_mask=[], sa_flags=SA_RESTORER|SA_ONSTACK|SA_RESTART|SA_RESETHAND|SA_SIGINFO, sa_restorer=0x10d8780}, NULL, 8) = 0
rt_sigaction(SIGILL, {sa_handler=0x11a5210, sa_mask=[], sa_flags=SA_RESTORER|SA_ONSTACK|SA_RESTART|SA_RESETHAND|SA_SIGINFO, sa_restorer=0x10d8780}, NULL, 8) = 0
rt_sigaction(SIGBUS, {sa_handler=0x11a5210, sa_mask=[], sa_flags=SA_RESTORER|SA_ONSTACK|SA_RESTART|SA_RESETHAND|SA_SIGINFO, sa_restorer=0x10d8780}, NULL, 8) = 0
rt_sigaction(SIGFPE, {sa_handler=0x11a5210, sa_mask=[], sa_flags=SA_RESTORER|SA_ONSTACK|SA_RESTART|SA_RESETHAND|SA_SIGINFO, sa_restorer=0x10d8780}, NULL, 8) = 0
socket(AF_PACKET, SOCK_RAW, htons(0 /* ETH_P_??? */)) = 3
ioctl(3, SIOCGIFINDEX, {ifr_name="enp7s0", ifr_ifindex=2}) = 0
ioctl(3, SIOCGIFFLAGS, {ifr_name="enp7s0", ifr_flags=IFF_UP|IFF_BROADCAST|IFF_RUNNING|IFF_PROMISC|IFF_MULTICAST}) = 0
ioctl(3, SIOCSIFFLAGS, {ifr_name="enp7s0", ifr_flags=IFF_UP|IFF_BROADCAST|IFF_RUNNING|IFF_PROMISC|IFF_MULTICAST}) = 0
bind(3, {sa_family=AF_PACKET, sll_protocol=htons(ETH_P_IP), sll_ifindex=if_nametoindex("enp7s0"), sll_hatype=ARPHRD_NETROM, sll_pkttype=PACKET_HOST, sll_halen=0}, 20) = 0
sendto(3, "raw bytes sent over the wire", 28, 0, {sa_family=AF_PACKET, sll_protocol=htons(ETH_P_IP), sll_ifindex=if_nametoindex("enp7s0"), sll_hatype=ARPHRD_NETROM, sll_pkttype=PACKET_HOST, sll_halen=0}, 20) = 28
exit_group(0)                           = ?
+++ exited with 0 +++

I also captured a more complete strace from running this Go program, which shows both writing to and reading from raw sockets how I would like to be able to.

conn-message-trace.txt (42.5 KB)

Here is a screenshot of Wireshark showing the full packet that is sent:

image
image1920×1080 304 KB

As you can see, the bytes starting from raw bytes are interpreted as the ethernet headers.

Edit: and to be clear, I mentioned calling out to libc not because it cannot be done with the standard library right now (as shown), but because I wasn’t sure if the APIs required to do it are expected to be removed, since the POSIX APIs that @kj4tmp used have been removed on master.

11

Thanks this is perfect. I can take a detailed look at this later, but at first glance, it seems like you only need ioctl? That’s one of the bullet points of #30150 - Migrate all applicable I/O APIs to `std.Io` - ziglang/zig - Codeberg.org, the main issue I’ve been prioritizing before wrapping up the 0.16.x release cycle.

1 Like
12

ioctl actually shouldn’t be a requirement for what I’m trying to do. The strace dump of the more complete example in Go does not include any ioctl calls.

I think the most important syscalls are:

socket(AF_PACKET, SOCK_RAW|SOCK_CLOEXEC|SOCK_NONBLOCK, htons(0 /* ETH_P_??? */))

This isn’t demonstrated in my example, but it would be important to be able to set the BPF filter like:

getsockopt(3, SOL_SOCKET, SO_TYPE, [3], [4])
setsockopt(3, SOL_SOCKET, SO_ATTACH_FILTER, {len=4, filter=0xc000196000}, 16)

bind(3, {sa_family=AF_PACKET, sll_protocol=htons(ETH_P_IP), sll_ifindex=if_nametoindex("eth0"), sll_hatype=ARPHRD_NETROM, sll_pkttype=PACKET_HOST, sll_halen=0}, 20)

sendto(3, "\377\377\377\377\377\377\274$\21\311\261\347\10\0E\0\0>\0\0\0\0@\21z\260\0\0\0\0\377\377"..., 76, 0, {sa_family=AF_PACKET, sll_protocol=htons(ETH_P_IP), sll_ifindex=if_nametoindex("eth0"), sll_hatype=ARPHRD_NETROM, sll_pkttype=PACKET_HOST, sll_halen=6, sll_addr=[0xff, 0xff, 0xff, 0xff, 0xff, 0xff]}, 20)

and

recvfrom(3, "\377\377\377\377\377\377\274$\21\324e>\10\0E\0\0002\0\0\0\0@\21z\274\0\0\0\0\377\377"..., 2048, 0, {sa_family=AF_PACKET, sll_protocol=htons(ETH_P_IP), sll_ifindex=if_nametoindex("eth0"), sll_hatype=ARPHRD_ETHER, sll_pkttype=PACKET_BROADCAST, sll_halen=6, sll_addr=[0xbc, 0x24, 0x11, 0xd4, 0x65, 0x3e]}, [112 => 20])

Importantly, all of these use sa_family=AF_PACKET and SOCK_RAW as applicable.

1 Like
13

I see - the main problem is OutgoingMessage hard-codes AF_INET/AF_INET6.

14

Tracking issue:

7 Likes