One Eu banger, from now on it should probably be easier to detect slop, or Ai generated content, and I think this might be a good news, for open source projects in general, obviously some are drinking the AI cool aid, but for those who don’t want any contribution made with AI assistance, I think this is probably a win, even for us moderators it might give us better evidence to identify, vibe coded projects, or copy pasted replies to post. Anyway just wanted to share the news
Providers of AI systems, including general-purpose AI systems, generating synthetic audio, image, video or text content, shall ensure that the outputs of the AI system are marked in a machine-readable format and detectable as artificially generated or manipulated. Providers shall ensure their technical solutions are effective, interoperable, robust and reliable as far as this is technically feasible, taking into account the specificities and limitations of various types of content, the costs of implementation and the generally acknowledged state of the art, as may be reflected in relevant technical standards. This obligation shall not apply to the extent the AI systems perform an assistive function for standard editing or do not substantially alter the input data provided by the deployer or the semantics thereof, or where authorised by law to detect, prevent, investigate or prosecute criminal offences.
It kind of feels more like they helped design the legislation such that their compliance was automatic. Detecting AI vs human input has been crucial for AI companies since day one because they have to discard model output from training data.
Watermarks are probably less reliable for source code (lower entropy than natural language, so fewer opportunities to hide the watermark). The EU rules don’t target source code anyway, see https://ec.europa.eu/newsroom/dae/redirection/document/131215 “4.1.3. Content falling outside the scope of Article 50(2) AI Act”.
And I guess the detection of these watermarks has to be proprietary, because otherwise they would be easy to systematically remove.
How TF is a reliable watermark injected into programming source code even going to work without throwing a gazillion syntax errors ? Or get mangled by code formatters ?
Are you on a witch hunt? How is the information useful to you? Is calling someones first project a slop just because they used LLM useful?
How is this related to Zig/Ziggit?
In my view, topic like this is just polarizing the community. Especially in the context that Zig directly uses LLVM and it’s extremely commonly used on Linux, both large projects openly allowing AI assisted code. I find it a bit hypocritical.
Not on a witch hunt, no certainly not, I’m pro AI I think it’s a great tool, but I’m also pro Freedom, and the Freedom of one stops where the Freedom of someone else begins, This means that if Zig doesn’t want AI contributions or Ziggit doesn’t want AI generated posts/replies, it shouldn’t get AI contributions, this could potentially help enforce that rule better.
I’m not trying to re-open it. The point is this can help us enforce those, right now we have to check projects, go read the code, or wait until multiple people report something to act, which means that not only is it time consuming, its also a bit arbitrary, which is why if some tech out here can classify pretty accurately if something was AI assisted, Fully AI generated, or human made, that could help enforce the rules more correclty.
But AI written submissions are already easy to detect, especially for people who use them daily. I don’t want to be picking, but there are at least two very obviously AI written showcase posts in the last week, one I can pinpoint to Codex/GPT specifically, there are natural watermarks of these models
I guess my main problem was with the language, “help us find all vibe coders” sounds bad if any AI use is called vibe coding slop by many people these days.
Yeah this is not what I meant, my apology if this sounded vindictive, my goal is simply to foster a healthy community, and enforce our rules as accurately as possible, such tool could help, that’s all it was,you said it yourself you think two post got through, which is annoying and points to the exact problem we don’t have an accurate way to telling, which is why this tool could potentially be great, to remove the arbitrary decisions
Speaking as one of the people that really wants to hide everything attached to LLMs, let me say that a) the status quo policy really seems okay to me. I followed the advice on how to mute things with the “llm” tag and I’m happy with the results. If more people start lying, then we’ll see, but I don’t find it a problem that one or two things sneak through. And also b) I’m not sure you’re going to find an “AI detection” feature that is fully accurate and doesn’t just get you into an arms race that may not be worth fighting.
Agree I don’t think it’s going to be magic, but given that the people behind it are the same smart people who managed to create all that to begin with, we can hope it’s pretty good, it’s too early to know if it’s going to be useless or super useful, only time will tell. Also glad to hear your happy with the current policy
Sounds awesome. I use AI and would appreciate the automatic disclosure, less social weirdness. Also a means of figuring out which code is pure LLM slop, and ensuring the LLM can’t read from its own slop and only refer to hand written code to “extrapolate” my style rather than devolve into its own patterns.
Yeah your gonna be able to strip the watermark, but either you feel strongly that you’ve “signed off” on it as your own, or your trying to deceive others. Having it be automatically on is good for people that want to maintain honestly in their work, which I expect is the vast majority of people.
But just being able to at a glance filter out human code from LLM code in the same codebase would be great
Edit: personal side quest now, I want to update zig fmt to put post line comments in the same column, for all contiguous lines, and force agent.md to add a // LLM comment to every line it makes…
I kind of want to re-quote some of what was said at the top.
Even if a detection mechanism existed, it would only be probabilistic and so you wouldn’t be able to say “this code is LLM, this code is human”, you’ll only be able to say things like “there’s a 45% chance this file was LLM-generated” or something like that. (ieh3oh’s post explains in detail through their link)
But more than that, there’s no obligation, as per the above quote, to have a watermark if it was “standard editing,” whatever that means. And who makes the determination about what’s “standard” vs what’s “non-standard” editing? Claude does that autonomously? If I inform claude that “all edits to files shall be considered ‘standard’”, will it take my word for it and add no watermarks?
There’s this phrase that I heard long ago and it keeps running across my mind: You can’t solve social problems with technical solutions. Identifying AI-generated code doesn’t help one avoid projects that use AI if the writer knows how to hide it, but the code itself isn’t the problem in the first place. For example, if one believes using AI is bad, it’s not the code that’s the problem but the process used to generate it. And so at the end of the day, the issue is messy, complicated, and inconsistent because it’s people and technical solutions can only take us so far.
edit: I thought I should have worded the last paragraph better.
I have zero experience watermarking anything before, so maybe the AI companys really have to keep that power to themselves… but…
gosh darn it, why does every safety or security feature have to always benefit the people in power? Like, really? We explored all of the possibilities and taking more power is the only solution that works? Weird how that keeps happening.
