Hi folks,
I am Java dev who depends on Java collection heavily. I ported Eclipse Collections to Zig, it is about 500 files with most features from EC. It passes tests, performance is ok. I would like to talk with someone, who has Java,Scala or Kotlin exposure and is experienced with Zig, before I publicly release this.
Thanks
1 Like
Sounds like a mountain of work… I have a hard time imagining how Java collections would work on Zig. Feels a bit antithetical (if this is a word). Would be really curious to see that though.
I did a lot of work in Java, in recent times more Kotlin. But when working with Zig my mindset completely changes from abstraction seeker to bit pusher. I enjoy both but mixing them feels like quite a feat of mental acrobatics.
5 Likes
Converting the code was easy part. I already had harness for Golang port, and it worked on Zig quite well (good type system). Hard part is to make it usable on Zig, Now it uses unmanaged arrays and error handling is bad.
Is there a way to contact you? Drop me DM at twitter handle, or email jan at kotek net
How well do you know Zig? How would you describe your experience level with it?
From the sounds of it, you just used AI to generate a ~500 file project with a minimal understanding of the language, and are now asking for others to review it in order to make it usable? My sincere apologies if that is inaccurate, but that is what I am gathering so far, hence I am a bit taken back by audacity of the request.
7 Likes
Not much, just learning. My background is in JVM and data structures.
Yes, it is machine translation. But it took some time.
I am not asking to review 500 files of code. I can do that myself. I need to understand what is actually needed. So I am asking for someone to discus this stuff with.
In Java we just mostly crash program if memory allocation fails, Zig handles this very differently. But there are many options, like overflowing to swap file.
1 Like
zig lets you choose! you can crash the program if you want, in some domains thats the better option or even needed.
instead of translating over java collections, implement stuff as you need them.
If you’re wanting to make a library for others, ask them directly. That should be a new topic and in the Brainstorming category.
1 Like
I believe you would have a better reception if you ‘soft launched’ your collections and solicited public feedback, rather than trying to lure specific people off-site for a private discussion.
That’s what we do here, look at code and talk about it. Perhaps a few representative examples? 500 types is a lot.
Welcome to the forum!
3 Likes
It’s great that you wanted to attempt this. One thing you can do is look closely at interfaces to collection type things in the zig standard library. And then think about what your data structures would be like if they were more conformant to the design patterns present there which demonstrate zig style and idioms.
What was your translation process and what kind of inputs do you steer it with?
1 Like
Welcome to ziggit!
I feel it’s worth complementing these comments with this: zig is great at things that other languages aren’t, and of course vice-versa is true. I’ve loved blindly relying on garbage-collection in Python for years, but I would seriously pause if somebody asked me to implement some Python collections in zig, because (besides having to implement GC) I’d feel that I’d lose a lot of the advantages of zig, just making a zig collection API look as close-as-possible to Python. I want zig because I want the things Python can’t do wrt/ memory management and interface ergonomics. I’d fail in other ways, too, because pythonic things (e.g., some operators) are just not available at the language level in zig, and some patterns are not a great fit (usually with some good reason). A lot of basic collections exist in zig’s std, and probably do all the “usual” stuff you need, anyway, so…. But I’m sure it was hard work, and perhaps the remaining work is exciting for you - great. Keep an open mind, though, to the limitations you’ll encounter.
5 Likes
i can’t speak to Java collections, but the conventional way in zig would be to take an allocator as a parameter to the functions that need it. Then your collection doesn’t need to worry about things like swap files or handling OOM, the caller can decide the best way for their use case.
2 Likes
The Java collections API is designed around interfaces such as Map, that are implemented by types such as HashMap. This allows swapping out different types that implement Map with miminal changes, i.e., a type of polymorphism.
“Porting” this design to Zig may be impractical. In Zig you would use comptime interfaces, meaning duck typing, for collections since you wouldn’t want the overhead of a vtable indirection. Zig doesn’t have comptime interfaces per se that are part of the language, so these would be largish duck-typed interfaces that would be described in documentation only, not using a language declaration.
In Zig, as far as I’ve seen, it is not a goal to abstract collection interfaces such that they can be swappable/polymorphic. So this may be considered contrary to usual Zig patterns. One reason for this bias is probably that interfaces are not built into the language. You could scrap the interfaces, but then I question the value of porting the design.
2 Likes
As an ex-kotlin dev, moving over to zig was one of the best thing’s I’ve done for my dx - build, test, startup, memory, and response times have dropped through the floor from using spring/gradle to zig and httpz.
LLM assistance has been instrumental in my transition for helping me mentally move over from GC and abstraction obliviousness into knowing the true cost of everything I do - and making the tradeoffs obvious.
The mental shift from inheritence to vtables/fatpointer switches and the awe inspiring power of comptime has been eye opening about the costs that I used to pay without thinking. Another part of letting the jvm go is understanding that common crutches, like checked exceptions with payloads and stacktraces don’t exist, errors, yah, but if you care hard about stacktraces and exception payloads, you can get close w/ diagnostic side-channels on a threadlocal or task struct.
However, I would take a moment before blindly converting the collections libs, and think about what you actually want to do, why it might not fit in zig’s philosophy or methodology, and maybe, just maybe, getting to know zig enough to make it even more powerful or simple.
Next time you want to reach for the utility of the java collections, ask yourself why, and ask a LLM what the zig of achieving your goal would look like (and why). You’ll find massive dx/runtime speedups and a more aware perspective.
4 Likes
Here is Eclipse Collections port to Zig:
1 Like
// Code generated by mapdb-codegen. DO NOT EDIT.
Any chance you plan to also open source mapdb-codegen? I’d like to study it.
Forgot to remove the note.
Not a chance to release generator, it is a total mess. Git repo with 5GB history, deb packages,build fragments, personal notes… Most of it was generated on termux tablet while I was on holiday…
Original EC uses code generator for Java (have a look at that). I made changes to it, to support multiple languages. However it does not really make a sense with LLM agents. Code generator just adds another layer of complexity. Perhaps there are same minor savings in specialized/boxed collections.
Plan is to sunset the code generator, and apply changes (and PRs) to source code directly. I will extract stuff out of codegen into separate projects, that will be opensource.
If you ever decide to try to use code generator this way, the most important part was to use code formatter (linter) on generated files. It greatly simplified code generator and validation.
Hi, I wrote my last lines of Java more than a decade ago… and while I do not consider myself an experienced zig programmer, I think my comments might still help you getting into zig.
The whole thing must have been quite some work, even if generated! (I would be interested in that generator, too 
)
I only had a look at some of your ArrayList variants.
So my comments only refer to those:
- Some functions do not make sense and are obviously only there, because the generator does not care, e.g.
BoolArraList.detectorBoolArraList.maxand there are other issues here and there. The library would definitely need quite some additional work (but I suggest to first invest the time learning zig and the standard library). - I do not like function aliases. They are confusing.
- A lot of functions are just one line wrappers arround
std.ArrayList - Most of the remaining functions could be written by using one or two functions from
std.ArrayListand/orstd.mem, which provides a lot of the implemented functionality (e.g. count, eql, min, max, reverse, sort…). Actually, you used std.mem.sort, but none of the the other functions. Why?
- I think the few remaining functions would be more useful, if you implemented them similar to the functions in
std.mem, i.e. using standard slices and generic types.
I think for the most part the provided functionality is already in the std library, maybe just not as a method of ArrayList so you may not have seen it yet. Have a closer look at std.mem!
2 Likes
I see, was just curious how you made it. Could you clarify though - the file comment and the first part of your reply indicates you wrote a generator, while the last part indicate it’s a large amount of llm output. Just nice to know what codegen means before spending time on it.
1 Like
It is both.
Porting EC to other languages meant modifying EC code generator. This step was not necessary, just grabbing generated java sources, and porting those would have been easier.
It is also large amount of LLM output. I used git to track specs and work progress, while agent worked semi autonomously. I found it much easier then agent harness.
Agent worked alone for some time, and committed every step into git. If it failed, I just rolled back to older commit, told it what was wrong in the older failed branch, and let it figure it out. Agent had access to all previous failed attempts.
There is value in code generator, and it will take some time to dig out (for example Float.NaN handling).. But nothing really specific to zig lang.
Thanks, appreciate the clarification