Real Time Industrial Ethernet Protocol (EtherCAT) Library Design

Help
1

I’m attempting to write a zig library / application for a binary protocol (EtherCAT) that is transmitted on standard ethernet hardware.

The protocol makes heavy use of bit fields and is real-time critical (sometimes sending frames at 20 kHz), which makes me think zig may be the most enjoyable language for this project.

Frames are sent into a ring of connected subdevices. Subdevices contain ASIC chips that modify the frames in flight, writing and reading data on the frame and recalculating the checksum on the fly, and then the frame is returned to the main device (running the application) where the results are read.

I’m struggling with designing my API. Because this is a real-time application, heap allocations should generally only occur at startup, and if possible, the majority of the memory should be stack-allocated (to avoid page faults). Most applications of the library will fit within a typical stack (<1 MB) and may not require any heap allocations at all.

The primary challenge I am facing is serialization and de-serialization of the binary protocol. The protocol makes heavy use of bit fields to optimize utilization of the network bandwidth. For example:

/// Datagram Header
///
/// Ref: IEC 61158-4-12:2019 5.4.1.2
pub const DatagramHeader = packed struct {
    /// service command, APRD etc.
    command: Command,
    /// used my maindevice to identify duplicate or lost datagrams
    idx: u8,
    /// auto-increment, configured station, or logical address
    /// when position addressing
    address: u32,
    /// length of following data, in bytes
    length: u11,
    /// reserved, 0
    reserved: u3 = 0,
    /// true when frame has circulated at least once, else false
    circulating: bool,
    /// multiple datagrams, true when more datagrams follow, else false
    next: bool,
    /// EtherCAT event request register of all subdevices combined with
    /// a logical OR. Two byte bitmask (IEC 61131-3 WORD)
    irq: u16,
};

I guess my main questions are:

  1. How to handle endianness of my hosts? I first reached for packed structs to best represent the binary protocol, but I don’t think they actually provide much use because I will need to do endianness handling on serialization and de-serialization anyway, so they might as well be regular structs.
  2. Are serialization and de-serialization methods generally struct methods?
  3. For serialization, I assume I should take a target byte slice as a parameter to serialize into and make heavy use of the bit writer or other writers from the standard library.
  4. For de-serialization, I assume I will need to take an allocator as a parameter, but the frame structure is rather nested, with slices to structs containing slices to other structs, so it may be difficult for callers to free the returned data. Should I just put a comment that callers should pass an arena allocator that they should free themselves?
  5. I would like to support multiple frames in flight, which means I will need to send frames and have a place pre-allocated to receive the frame back into. I guess the send frame API will need to accept both the frame struct to send as well as a pointer to where the frame should be deserialized into upon the frame returning, some microseconds later? Ideally multiple threads should be able to send and receive frames, and the access to the shared resource that is the network interface should have mutexes etc inside it to handle this.
3 Likes
2

For your layout/serialization concerns, I think your best friend here is the language reference. You’ll find information about the layout of structs, endian-ness of packed structs, etc: Documentation - The Zig Programming Language

For endian-ness of packed structs, it’s always lowest-to-highest:

Fields remain in the order declared, least to most significant.

Normal structs do not have defined layouts. You may have to use extern here to get what you want: Documentation - The Zig Programming Language

If well-defined in-memory layout is not required, struct is a better choice because it places fewer restrictions on the compiler.

3 Likes
3

Hi, very cool endeavour!
Here’re my takes on these points:

  • (1.) Packed structs should be good here since you can use @byteSwap/std.mem.byteSwapAllFields to change endianness. However, you have to be extra mindful with these: it’s better to specify the backing integer type explicitly, e.g. packed struct (u32), carefully order the fields and assert intended alignment.
  • (2. and 3.) I’d imagine serialization/deserialization should really come down to just using std.mem.writePackedInt/std.mem.readPackedInt functions and their specialized variants. In general, there’re all kinds of handy byte-casting in std.mem.
  • (4. and 5.) Yeah, you should try to devise upper bounds for everything and design around pre-allocating all memory once, at the very start, removing dynamic allocation all together.
7 Likes
4

The thing about std.mem.byteSwapAllFields which is used by writeStructEndian is that it cant byteswap a u11. (non-byte divisible fields don’t really make sense when it comes to endianness / byte swapping).

/home/jeff/zig/zig-linux-x86_64-0.14.0-dev.66+1fdf13a14/lib/std/mem.zig:2023:57: error: @byteSwap requires the number of bits to be evenly divisible by 8, but u11 has 11 bits

@AndrewCodeDev are you suggesting that I cast the packed struct to the underlying backing integer and then byteswap that?

5

That’s my first instinct, yes. I can’t speak entirely to your use case, but that strikes me as a thing to try. The other thing is to define types based on endian-ness but that can become quite verbose. I’m interested to hear if you run into any snags with the casting approach.

6

I’m trying

const datagram_header_as_int: u80 = @bitCast(datagram.header);
try writer.writeInt(
              u80,
              datagram_header_as_int,
              little,
          );

we shall see if I have any footguns!

And yes, the protocol embeds little endian data into ethernet frames, isnt that fun :slight_smile: