Fair enough, I’ll leave my comment as is, otherwise the ensuing discussion makes no sense.
1 Like
In particular situation, it’s impossible to make even semi-reliable predictions like that. So all you can do is make the choices you think are right, and keep an eye on what’s happening.
Uh oh, It isn’t?! I guess I’ll be leaving now.
1 Like
Async/Await for projects. AI is coloring our lives.
What are the rules for the llm tag? Should a project like this be tagged?
Based on the previous discussion, I guess it means use the tag if you used LLM anywhere in the development of the project.
1 Like
I was asking myself the same question when I initially did the post. I felt like it was misleading as I handwrite 99% of my code and only use LSP autocompletion. But I don’t mind adding the tag, its no problem
2 Likes
I fully agree with you, but in the past, I’ve heard the opinion that any LLM use should be marked, as people don’t want to deal with the boundary, so I’m not sure.
1 Like
+1 to the policies. I use LLMs in some projects (including at work) but not everywhere. Being open about is good, and feels like the Zig and Ziggit spirit. I like it.
The only thing I’d flag is the boolean perspective of LLM on/off. There are multiple tiers of “using LLM/AI” and the differences can be big.
My admittedly simplified scale is:
- Human-written / zero to zero-ish AI: AI may help with research, questions, understanding code, or bouncing ideas (just like a forum could), but it does not write the code. Only a human does.
- AI-assisted: AI writes small, specific pieces: “add a function with this signature that does X.” You review, adjust, and own the result. This is like auto-complete in an IDE. (Some IDEs like IntelliJ actually suggest auto-completing the entire function if they detect it’s a very standard function, and that isn’t using any AI/LLMs.)
- AI-written, human-led: Getting closer to vibe coding. You are leading the project, but you really aren’t reviewing every line of code. You give very strong guidance, you’re at the helm, but you’re not doing most of the coding, if any.
- Vibe coding and AI slop: You understand nothing about the code. You provide vague, general guidance. You give the design, tone, maybe directionality, but the code is as transparent to you as a black door painted a darker black, while wearing a sleeping mask and sunglasses. This could be brilliant work or a pile of garbage - you couldn’t discern one from the other.
Or using contracting work as explanation:
- You’re a contractor who builds everything on the house yourself. Good or bad, it’s fully you. But you’re allowed to get advice from others.
- You’re the contractor but hire people to help build. You tell them exactly what to do. Some of them may do things better than you, but your hand is in everything.
- You’re a contractor living in a different continent who explains on the phone in approximate terms what should be done.
- You are a pianist in Switzerland explaining to your Brazilian buddy using awkward metaphors and hand gestures how to perform surgery. The line cuts off. The phone hasn’t rung since. So… probably fine? shrug
I wouldn’t easily trust #4 if ever - I think this is the one people want to avoid the most. I prefer not to use #3 (but it might not be a disaster). I don’t mind #2 (and I don’t think people should be shamed for it). And one could argue #1 could benefit from AI in testing, reviewing, etc. (curl, for example, uses AI for review. It is assisted by AI, very openly. It finds bugs, suggests fixes, etc. But nothing is done on the code without a human doing it.)
The line between #1 and #2 is getting blurry anyway. Is autocomplete AI? Maybe. What about Grammarly? What about Grammarly after it adopted AI? How much LLM involvement makes something “AI-assisted”?
In short, it’s messy. I’d say we should be clear about whether AI was used, and more importantly, how it was used. That lets a reader/user decide what the level of risk is, and also doesn’t attempt to shame any use of anything that has LLM in it (except pure slop).
11 Likes
I would just add to this, or create a preceding entry in this scale: AI for review. In and of itself, this would seem like the least controversial use of AI that materially benefits end-users, particularly when talking about security-sensitive applications.
3 Likes
Thanks for emphasising that no LLM use gives users more freedom. I also read your blog post regarding Contributor Poker. However, there is still uncertainty in my mind regarding what constitutes LLM use - would you able to provide a definition for this? If the answer is something along the lines of no code is generated as tokens from an LLM, that would be fine with me, although I suspect that such code has already made its way into the codebase. I just want Zig to be the best it can be for users and in my opinion more clarity on this would only be of benefit.
1 Like
In this context it’s up to the admins of Ziggit to come up with their criteria for the llm and llm-deps badges.
For the Zig compiler project the rejection is complete (no LLM involvement at all). For the forum maybe (maybe) it might make sense to use a more relaxed set of criteria more close to probability of something being slop. But I genuinely don’t know.
Very possible, if the PR was normal looking and the change self-contained and non-controversial, it’s possible that it just got merged without ceremony. But even if this was the case, it is still correct to not relax the wording of the policy, there is no point in debating in PRs about whether a given threshold was surpassed or not in the creation of the PR.
5 Likes
If you used AI to write any code (I don’t consider zls autocomplete as AI) then you should use the label. That is the easy to define rule, and so why I think it works. For questions about using AI for ideation, review, or research, for now that doesn’t need the label, unless you feel that the use was so much that it would be dishonest to not disclose it.
This is a discussion we had as mods. We went with the binary label to start as we think it might take time to coalesce what are good ‘sub’ groupings. Users are able to add their own tags as well (tags don’t need to be created by moderators), so f you want to add those sub labels you are welcome and invited to do so, As long as the two ‘official’ ones are used as well. In the future we might come up with an official hierarchy, though i doubt we will remove the llm and llm-deps tag requirement.
9 Likes
Thank you for the clarification, it’s much appreciated. Personally, the phrase ‘no LLM involvement’ is much clearer to me than ‘no LLMs’, but maybe that’s just me.
See below several quotes from latest llm showcases
It’s hard to beleive that after such discloses we will see discussions like:
- how to use cheep subscription for my hobby zig project
- how i can cross-check code using different llm
- how to organize development of large zig project using llm
- …
Development with llm assistance is fight
I have been coding since 1977, so I will win
But I have no clue how to explain/help juniors learn programming in AI era
Saying “don’t use AI” is not enough
May be we need ‘additional tweaks’ ?
This resonates with me. I started in 87 but still I win. It’s harder to use ai (to produce quality) but alot faster.
Maybe make a whole llm/ai category where showcases can be posted and questions can be discussed, while keep the possibility to simply ignore the llm stuff category for those who don’t want to read about it. Since, if I remember it correctly, this possibility of filtering llm stuff out of someone’s view, was the original reason for introducing these tags.