How to get the screen buffer from the Raylib image to be able to draw pixels and "push it" to the window?

Help
,
1

Hello everyone, I’m currently trying to port a library from my school to a new backend more versatile than X11, I’ve choosen to use Raylib, as it’s fairly simple and cross platform. So far so good. One of the restriction of that library is that the user should only be able to draw from scratch which means that you can only have a function to get a buffer of []u32 (for the colors) draw pixel by pixel, and then push it to the window. The current problem that I have is that I can’t find a way of doing that I’ve tried several function from raylib, but none got me the expected result. I know drawing this way is light years aways from being optimal and kind of negate the usefulness of using something like Raylib, but I’m trying to be faithful to the original API.

So TLDR:

1 - How to get the raw screen buffer of pixel ?
2 - How to push it to be displayed to the screen once it has been filled ?

2

Just restating what you already said, this kind of negates using raylib in the first place.

You might be able to use rlBlitFramebuffer raylib/src/rlgl.h at 46cb6af4375da00563c68c6bbe4c24dcdf885b60 · raysan5/raylib · GitHub
I think that should work, but you would have to research it a bit using raylib in such a way is quite unconventional, most of the time applications that just want to push out a framebuffer would use opengl directly, or some windowing library.

It also may make sense to use some other library, I think it is probably possible to do this with raylib it is just that it is rare for raylib to be used this way, so examples for other libraries are probably more readily available.

If you don’t care about outputting to the real framebuffer you could just setup a render target texture, fill the texture and then render it. But updating that render texture completely on every frame might be pretty slow, usually you only update what actually got changed, but detecting that from a pure pixel buffer interface is also not easily doable.

If the resolution of what you want to output is fairly small (with the render texture you can decide its resolution) than it might not be too slow for what you are trying to build. Here is a raylib example that uses a render texture

You also can create a Image and then create a Texture via ray.LoadTextureFromImage(image); and then use ray.UpdateTexture(texture, image.data); for future updates.
But for big textures that is really really slow.

1 Like
3

Yes I know this is very much not conventional as a matter of fact I only went with Raylib because it’s my favorite graphics library to use, and the one I know best, which is why I was a bit confused when I didn’t see anything to grab a buffer. Because there is a drawPixel that work but nothing that returns that pixelBuffer at least not to my knowledge. But the y of my problem is more what is out there in terms of library that can let me grap a buffer of pixels, inputs, is cross-platform and not a hassle to setup ? Because I’ve looked into a lot of different library and none of them feel as simple as Raylib. Granted I’m not a gamedev at all. But if you know anything I’d be great.

4

Ok I think I have found something that seems to work. I have switched to the great GLFW wrapper from the mach team, and it seems to have everything wanted so It’s probably going to take some use to but it should at least work within my expectation. Just last question do you know if it’s normal that the autocomplete for the lsp doesn’t work with it ? I had this issue previously when using modules, and I don’t know how to solve that, I can live without but the lsp is often very useful to explore the API.

5

I think one approach to find something for that is to search for examples of software rendering / cpu raytracing, because that is essentially what you are doing. And look what those do for the splatting the data into the framebuffer part.

Do you intend to read data out of the framebuffer too or do you only want to write to it?

Image in raylib is just cpu image data, you can upload that to a texture to put it on the gpu and then render that texture. You also can have RenderTexture those are used when you use the gpu to render to a texture and then use that texture further. You can also load an Image from a texture via LoadImageFromTexture. Generally you don’t really want to read data from the gpu unless you have a really good reason to do so.

Doing that isn’t difficult, the problem is that the naive way of doing that gives you probably not even 32 fps, if the resolution is high enough or the hardware is old enough. So it really is more about defining what you really need and figuring out whether you really need the most dumbed down version of pixel access or whether we can cut some corners.

I think it would be helpful if you could show what kind of API you need to provide to the user at the end, because that would give hints towards what is possible to avoid.

Having to resubmit the whole buffer to the gpu every time is only necessary when we don’t know what changed, that is why framebuffer as memory block is a bad interface for matching that to “modern” apis.

Having the user specify sub-rectangles to update would already allow for way better performance because that way the user could update small parts of the buffer that lives on the gpu without having to re-copy all the things that didn’t change.

Personally I find it a bit silly to use such an old way (especially if it is make believe and everything is running / emulated on modern computers).
May I ask what the point is? Is it just nostalgia for a simpler time, or are there more practical reasons like getting this to run on hardware devices that still operate that way?

(If it is just nostalgia, then I question this method of teaching, because I don’t find it too helpful to teach ancient ways of getting graphics on the screen)

6

I imagine that is probably a better fit for the level you intend to interact with the graphics backend.

I haven’t used mach exept run a few examples, so I am not quite sure whether that is some temporary issue, or whether there is some more fundamental issue that trips up the lsp. Generally lsp has worked pretty well for me lately, at least for raylib. If you have an example I could try to run it and see whether the lsp works for me.

Maybe you could take a look at GitHub - foxnne/pixi: Pixel art editor made with Zig.
and see whether the lsp works in that project for you, if it is at all comparable to what you do?

7

Ok so to be more precise, the previous graphics API that our school mandates us too used is based mainly on X11, but the API itself is really really bad, the documentation is poor and being written in C it’s often a nightmare to build, so because I’ve suffer so much with it I decided to try to port it to something more modern while keeping 100% within the actual pedagogical intent of this library.

It’s meant to be a very dumb down graphics API, for very lightweight graphics project, It has to be able to be used with C, it needs to be cross-platform to Linux/Windows/Macos and is expected to be used on somewhat recent hardware nothing exotic.

The previous API had those functions

void	*mlx_new_window(void *mlx_ptr, int size_x, int size_y, char *title);
int	mlx_clear_window(void *mlx_ptr, void *win_ptr);
int	mlx_pixel_put(void *mlx_ptr, void *win_ptr, int x, int y, int color);
void	*mlx_new_image(void *mlx_ptr,int width,int height);
char	*mlx_get_data_addr(void *img_ptr, int *bits_per_pixel,
			   int *size_line, int *endian);
int	mlx_put_image_to_window(void *mlx_ptr, void *win_ptr, void *img_ptr,
				int x, int y);
int	mlx_get_color_value(void *mlx_ptr, int color);
int	mlx_mouse_hook (void *win_ptr, int (*funct_ptr)(), void *param);
int	mlx_key_hook (void *win_ptr, int (*funct_ptr)(), void *param);
int	mlx_expose_hook (void *win_ptr, int (*funct_ptr)(), void *param);
int	mlx_loop_hook (void *mlx_ptr, int (*funct_ptr)(), void *param);
int	mlx_loop (void *mlx_ptr);
int mlx_loop_end (void *mlx_ptr);
int	mlx_string_put(void *mlx_ptr, void *win_ptr, int x, int y, int color,
		       char *string);
void	mlx_set_font(void *mlx_ptr, void *win_ptr, char *name);
void	*mlx_xpm_to_image(void *mlx_ptr, char **xpm_data, int *width, int *height);
void	*mlx_xpm_file_to_image(void *mlx_ptr, char *filename, int *width, int *height);
int	mlx_destroy_window(void *mlx_ptr, void *win_ptr);
int	mlx_destroy_image(void *mlx_ptr, void *img_ptr);
int	mlx_destroy_display(void *mlx_ptr);
int	mlx_hook(void *win_ptr, int x_event, int x_mask, int (*funct)(), void *param);
int	mlx_do_key_autorepeatoff(void *mlx_ptr);
int	mlx_do_key_autorepeaton(void *mlx_ptr);
int	mlx_do_sync(void *mlx_ptr);
int	mlx_mouse_get_pos(void *mlx_ptr, void *win_ptr, int *x, int *y);
int	mlx_mouse_move(void *mlx_ptr, void *win_ptr, int x, int y);
int	mlx_mouse_hide(void *mlx_ptr, void *win_ptr);
int	mlx_mouse_show(void *mlx_ptr, void *win_ptr);
int	mlx_get_screen_size(void *mlx_ptr, int *sizex, int *sizey);

As you can see it’s very basic, you can create windows, an image, get the buffer of the image, put the image to the window, get some hooks etc etc. While I fully agree with you that using the GPU would be much more efficient and pleasant I intend to make a library that follows those guidelines, of only rendering within the CPU, using an “image-based” / “pixel-buffer” approach. This is meant to be stupid simple just to teach everyone how to do basic computer graphics.

ps : i think I’ve missed a few prototypes but you get the idea

1 Like
8

mlx_new_window I think that function alone is a good reason to use something different from raylib, the vanilla raylib is a single window only (there is a wiki entry about changing the raylib source to create a multi window version but that would be annoying (using a custom fork)).

That is quite the puzzle matching this specific api to some other api, good luck with that.
Might make sense to setup something where you can switch between your implementation and the old one and compare the rendered/image/screenshot output of both.

1 Like
9

Just to give some context you usually create one window at the beginning and use it for the rest of your program, I’ve never once had multiple windows, but the GLFW wrapper that I found seems to have something a bit similar but I have to explore it more, thanks anyway for your time :slight_smile:

In any case if anyone else has some suggestions of how to build a cross platform API, to support my use case It would be of great help.

1 Like
10

I think using Raylib is fine if that’s what you are comfortable with. It will be inefficient for the reasons mentioned above, but most direct framebuffer software rendering will be.

I just played around and did this with Raylib in C:

#include <raylib.h>

#define WIDTH 480
#define HEIGHT 480

Color pixels[WIDTH * HEIGHT];

int main(void) {
    Image fb_image = {
        .data = pixels,
        .width = WIDTH,
        .height = HEIGHT,
        .mipmaps = 1,
        .format = PIXELFORMAT_UNCOMPRESSED_R8G8B8A8,
    };

    SetTargetFPS(60);
    InitWindow(WIDTH, HEIGHT, "framebuffer texture");

    Texture2D fb_texture = LoadTextureFromImage(fb_image);

    unsigned int frames = 0;
    while (!WindowShouldClose()) {
        unsigned char color_offset = (unsigned char)frames;
        frames += 1;

        for (int y = 0; y < HEIGHT; y += 1) {
            for (int x = 0; x < WIDTH; x += 1) {
                pixels[y * WIDTH + x] = (Color){
                    .r = color_offset + ((unsigned char)(x) & 0xff),
                    .g = color_offset + ((unsigned char)y & 0xff),
                    .b  = color_offset + ((unsigned char)(x + y) & 0xff),
                    .a = 0xff,
                };
            }
        }

        UpdateTexture(fb_texture, pixels);

        BeginDrawing();
        ClearBackground(RAYWHITE);
        DrawTexture(fb_texture, 0, 0, WHITE);
        EndDrawing();
    }
}
2 Likes
11

Yeah I’ve played a bit with that but couldn’t make it work, I wasn’t sure if there wasn’t another method available that I didn’t know I was trying to loadFromScreen() hopping it would give me some kind of drawable buffer, but I keep that In mind I want to take the opportunity to try other things too just to be sure, but would gladly resort to using Raylib ahah.