Internals Category?

[Sze]

The great answers in topic linked below made me think maybe we should have an internals category, that is specifically for topics exploring more of the zig internals and tricks and tips about diving deeper into the implementation of zig itself?:

Emit "monomorphized" zig code

Hello, the title probably doesn’t make much sense as I don’t think zig calls it monomorphization. Lately I’ve been playing around with comptime (really enjoying this take on metaprogramming). I don’t understand the inner workings of the compiler yet, but I’m guessing there is a step for interpreting the compile-time parts of a Zig program and emitting the “monomorphized” version of it? If so is it possible to look at the code during this compilation step, ie. a compile option for zig build-obj? …

Part of me is interested in getting more involved with zig, but because of very limited time I always end up avoiding it, I think exploring certain things about the zig code base through discussions and sharing, could be a way to get people to dip their toe into zig language development, without diving right in and also absorb some more knowledge of internals over time.

That category could also help in building a bit more of a map towards different resources and places that already exist somewhere that explain parts of the internals.

Maybe we could even start creating a sort of guide towards what are the easier vs more difficult to understand parts, which parts are likely to stay relatively unchanged vs are highly experimental.

[AndrewCodeDev]

If we go forward with this idea, I could take some time to go through existing threads and mark them with the internals tag.

An issue that I’ve seen here is that the help category tends to be a catch-all. Typically when people ask about builds/linking (which is our most requested help topics), they go to the help category because they tend to have a question. Same goes with internals - people usually reach for more information there when they want an answer about something.

So if we make another category tag, we’d need to know what is the structure of that category - does it work like help? In other words, does each thread have a solution? If we’re just talking about making searching easier, adding an internals tag makes that easier because you can filter by tags.

If we can talk about it here, maybe we’ll find something that makes sense beyond a tag or another reason to use the help category.

[AndrewCodeDev]

As an addendum - let me give you an example to clarify.

Say for example I’m trying to figure something out about the compiler and I want to know “how does x get processed into y by the compiler” (or something similar).

I wouldn’t want to post that as brainstorming because I’m looking for a specific answer. It’s certainly not a showcase… so where does that leave me?

I ran some rudimentary statistics over our forum topics quite some time ago using some SQL queries and given what I’ve seen here, I think they still hold up.

The number of posts about the compiler and its internals that are not question-related are small enough to be an error margin. See my concern?

I think a better roadmap to the internals is a great idea (and the sooner that can get started, the better). I’d like to make sure that it’s not abandoned by default. Something like this sounds like a good github project but if we can find a way to also support that with the forum then I’m all for it.

[Sze]

@AndrewCodeDev

does it work like help? In other words, does each thread have a solution? If we’re just talking about making searching easier, adding an internals tag makes that easier because you can filter by tags.

@AndrewCodeDev

Say for example I’m trying to figure something out about the compiler and I want to know “how does x get processed into y by the compiler” (or something similar).

I wouldn’t want to post that as brainstorming because I’m looking for a specific answer.

I think you have found an interesting point here I would say what I had in mind was something more like brainstorming (free flowing discussion, that is a bit more open ended) except that it is explicitly not about how things could be, but about how things currently are and possibly why they are that way, maybe linking to issues where certain insights were gained resulting in a particular direction, but also about insights somebody has gained about zig internals.

One thing I also have noticed is that we often have discussions about things, that when you search for a bit you can find a bunch of issues that already had a very similar discussion, so partly I also thought that mapping this out a bit would give us more context, so that not everyone individually starts asking themselves why was this done this way.

Basically I want to understand how and why things are the way they are, because only then I have the ability to think about my own ideas and whether they fit with that together.

Regarding whether it needs to be a category or just a tag, I am not sure, I thought category would give it more of its own vibe like we are collaborating on trying to explain things to each other and we aren’t necessarily asking for help in the sense of I have a specific question, maybe instead I just started to look into how the frontend works and want to describe it a bit and have others add their knowledge.

Creating a github project instead might make sense too.
Apparently discourse also has wiki posts that can be edited by everyone and a category could be configured to automatically use wiki posts, however it seems concurrent modification is buggy.

So maybe creating a github project and using issues for discussion would be better?
Or we could discuss here with tag internalsmap or something like that and collect the results of the discussion in a repo.

[AndrewCodeDev]

Interesting… so tell me what you think about this…

This week, I’ll spend some time going through our existing threads and I’ll come up with a solution for how we can aggregate the information we already have. If there’s some kind of common “identifier”, I can write some SQL queries and maybe @dude_the_builder can run them.

At the very least, I can start tagging threads with internals that have something to do with this project so we can make searching them easier. If you see any that need tagging, just send me a direct message and I’ll add tags there.

If you go to this page, you’ll see existing tags used for topics: Ziggit

We can probably find several tags that are similar to what we’re looking for here… compiler seems promising. Anyhow, that should help the data-mining effort and we can hopefully collect information more easily. That way, if a github project is in the cards, we can use the information we excavate here.

[dude_the_builder]

The way I see it, a category allows for the free-flowing discussion in a central place about a specific area of knowledge like @Sze had in mind. Tags tend to be more of a cross-cutting concerns nature, where you can have posts in very different categories all sharing the same tag. The Help category in a programming language forum is essential and will gather up most of the posts, but should be mainly for actually helping someone that’s currently stuck because of something they don’t know; more of an “I’m stuck because I don’t know this” versus “I would like to know that”. In reality, the lines are not precisely defined, every time you’re dealing with taxonomies and information architecture, you’re bound to find gray areas with overlapping boundaries.

So we could have a Zig Internals category and a zig-internals or compiler-internals tag at the same time. With the category you get a one-click listing of specific discussion in that area, and with the tag you get filtering in search across all categories. Could be the best of both worlds.

As in many areas of technology, the weakest link is the user; if they don’t select the appropriate category or assign the right tag, the whole scheme comes crumbling down. But our community has done a pretty good job up til now and the moderators and I try to re-categorize and tag whenever we can.

[AndrewCodeDev]

I see your point - it sounds like a “looking for more information” kind of category. If I’m following correctly, it’s not quite like help (where there is one predominate solution). It bridges on general discussion but with an overarching subject.

From what @Sze was saying here:

@Sze

Apparently discourse also has wiki posts that can be edited by everyone and a category could be configured to automatically use wiki posts, however it seems concurrent modification is buggy.

…and by the nature of this request, it almost sounds more like a wiki-page. I think what people are looking for is better documentation? I’m just trying to puzzle out what we’re really looking for, ya know? That’s why I recommend a github page because it sounds like a project or a direct resource.

I guess I should ask the straight question - @Sze (and everyone who is interested in this subject) are you looking for a place to ask questions/share-info or a wiki-like reference to look up information? We could certainly do both, like @dude_the_builder is suggesting.

The difference imo is that a wiki-page (or an instructional of some sort) is orderly… a beginner could start at the top, read to the bottom, and expect to learn something through examples and explanation. A forum is more a place to talk about something in general, so maybe we need both?

[slonik-az]

@AndrewCodeDev

So if we make another category tag, we’d need to know what is the structure of that category - does it work like help? In other words, does each thread have a solution? If we’re just talking about making searching easier, adding an internals tag makes that easier because you can filter by tags.

I would say that Help category is more geared towards using Zig, its stdlib API and answer question “How to use X to solve Y problem”. Internals category is more about “How X works inside” and does not directly require solving any “Y problem”. Sometimes Help and Internals could overlap but they are conceptually distinct.

[mscott9437]

So from what I can tell there is one issue here which is having an additional category on the forum. And the second issue which is something that would need to be addressed by a github project. From my experience with this forum I can say that there are a lot of posts which are very advanced in nature, which tend to lead to a lot of discussion. At the same time the help category is also covering a lot of questions from people who are either just getting into Zig and need some basic information, or maybe even they are just getting into programming for the first time. So it seems reasonable that you could have sort of an “advanced topics” category, which may or may not require a direct solution. This would leave the Help category reserved more for questions which have obvious answers and direct solutions. Also this new category would cover everything in internals and whatever else you can think of really (and could possibly have some new tags to go with it). You can also anticipate that there will be more people over time needing help with basic things, and thus more resources will need to go towards that.

Having a wiki I think would be be getting more into the docs which ideally should be coordinated with the existing official Zig docs. So I think here we are getting more into the github project, which is obviously more complex than simply adding a category. Personally for me I think it would be good to sort out the easier questions from the harder ones. Since I tend to look for what I can provide an immediate response to, without doing too much research or reading a lot of code, so in this way it would make it easier to be helpful.

[Sze]

@AndrewCodeDev

I guess I should ask the straight question - @Sze (and everyone who is interested in this subject) are you looking for a place to ask questions/share-info or a wiki-like reference to look up information?

I think for wiki like things we can see whether it would have a place in the zig wiki and if it is a bit too offtopic for the official wiki, we can create either similar wikis or repos on demand just by what people think makes sense or what they want to create on their own (you can always aggregate/link things later).

So while I think the results collecting part is certainly valuable and makes sense, I think it is a separate issue, first we need to kick up some dust and actually gain some insights.

@mscott9437

So from what I can tell there is one issue here which is having an additional category on the forum. And the second issue which is something that would need to be addressed by a github project.

I think you are right that there are two separate things here, I think the category may be more general.

---

One of the things I would be interested in posting:
It is just that I wouldn’t want to post a “Hey lets explore this part of the zig code base” topic in any of the existing categories. If I wanted to improve some part of the code base than I could use the brainstorming category.

But my idea is more like lets be tourists that lost their tour guide and stumble together through an unfamiliar place, until some of us start to understand the language of the place and begin being able to help the group to navigate and find new places to explore.

I think the part that could be helpful about that particular idea is that we don’t know that much / too much, thus whatever we will end up explaining to another will also help others that don’t know that yet.

Maybe it is also a bit about finding the things we didn’t knew we knew, through interaction with others (additionally to things we don’t know at all).

If we instead would go the route of finding an appropriate tour guide first, the process would lead to different results. I guess I suggest more of a research expedition/hike, instead of a talk where somebody needs to spend more time upfront in preparing something.

I sometimes take photos, however after a while I got somewhat bored with it, recently I started going to “photo walks” (a bunch of people walking around taking photos of whatever they find interesting) and it immediately made it more interesting to me again, my idea is more about having something like that, without having a full in person meetup (those are great too). I guess another analogy would be, it is like a book club, but our books are zig code.

After all that I wonder, maybe the category should be simply named ‘Community’ and it is for social discussions and projects of different kinds, that the zig community comes up with?

[dude_the_builder]

@Sze

maybe the category should be simply named ‘Community’ and it is for social discussions and projects of different kinds, that the zig community comes up with?

I think this is another good idea, but one separate from the original Internals category idea. A Community category would have pretty much a wide open scope when it comes to possible topics, and the original idea of having a place to discuss Zig internals would be diluted in a sea of other more general topics.

[dude_the_builder]

Maybe to broaden the scope a bit but not too much, isntead of Internals or Zig Internals, we could name the category:

• How and Why
• How it Works
• Explaining Zig
• Please Explain

Please reply with more suggestions!

[z1fire]

Those category names sound more intuitive, coming from a newbie perspecitve

[AndrewCodeDev]

I think the best approach is to look at the topics we already have and build something that addresses the traffic we get. After doing this, I honestly think the best word is just Explain (or Explanations if we want to emphasize response plurality). Here’s a few examples from existing topics:

Explain: Is it Ok to use std.heap.page_allocator directly?

Explain: Understanding "sentinels" (slices, arrays, pointers)

Explain: What is the purpose of std.testing.refAllDecls()?

Explain: How to use toOwnedSlice() and alternatives

There are many help topics that don’t end up with a marked solution - if you look at the question, it doesn’t lend itself to a single answer.

The word Explain also includes design philosophies, best practices, language/library direction, etc.

Help → One solution, prefer code examples, focused on solving a definite problem.

Explain → No general solution, optional code examples, open-ended conversation.

This can be used for descriptive topics like “Explain: How to understand the Zig Intermediate Representations” or even prescriptive topics such as “Explain: When should we prefer x over y?”

[Sze]

Another popular phrase that is somewhat related is ‘Today I Learned’ often abbreviated TIL

[dude_the_builder]

@AndrewCodeDev

Explain: Understanding “sentinels” (slices, arrays, pointers)

This one brings to mind a category name of Understanding Zig.

[dude_the_builder]

Let’s vote for the new category name! This poll will remain open until Thursday January 4, 2024 23:45 PM.

• Explain
• Explanations
• How and Why
• How it Works
• Internals
• Today I Learned
• Understanding Zig

[dude_the_builder]

The people have spoken; Explain it is!