How do I post a file to a website?

My code is

const std = @import("std");

test {
    const allocator = std.testing.allocator;

    const file = @embedFile("test.zig");

    var client = std.http.Client{ .allocator = allocator };
    defer client.deinit();

    const uri = try std.Uri.parse("www.example.com");
    var buf: [1024 * 1024]u8 = undefined;
    var req = try client.open(.POST, uri, .{
        .server_header_buffer = &buf,
    });
    defer req.deinit();

    req.transfer_encoding = .{ .content_length = file.len };

    try req.send();
    try req.writeAll(file);
    try req.finish();
    try req.wait();

    try std.testing.expectEqual(.ok, req.response.status);

    const mssg = try req.reader().readAllAlloc(allocator, std.math.maxInt(usize));
    defer allocator.free(mssg);
    try std.io.getStdErr().writeAll(mssg);
}

And the server replies with a message saying that there was an error while uploading the file, which does make sense since I am never specifying what the name even is.

My question: How do I correctly upload a file using the stdlib using a post request.

Current problem: How do I post a file to a website? - #12 by markus

There are a lot ways to do this. The way to do it depends on the expectations of the server side.


The most common way is with a "multipart/form-data" POST response. This is the case where there is a form, with a control that select a file. The response is the values for all the fields in the form, including the contents of the file.
If the form contains only one control for file named "file" the response must be:

POST url HTTP/1.1
Host: hostname
Content-Length: 123
Content-Type: multipart/form-data; boundary=----Random

----Random
Content-Disposition: form-data; name="file"; filename="test.zig"
Content-Type: text/plain

Content of the file replaces this line<<<
----Random

The simplest way is to send a PUT or POST response to a url, with the contents of the file.

I am sorry for being so helpless, but all this http stuff is new to me. Could you provide a zig code example?

I personally find the zig http api to be very unintuitive

Your code is mostly correct. The server is probably unhappy due to the absence of a Content-Type header.

Try adding this line:

    req.headers.content_type = .{ .override = "text/plain" };

It still does not work, the server keeps returning this html:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>File Upload Service</title>
    <style>
        body {
            font-family: Arial, sans-serif;
            color: #333;
            max-width: 600px;
            margin: 0 auto;
            padding: 20px;
            background-color: #f9f9f9;
        }
        .container {
            background-color: #fff;
            padding: 20px;
            border: 1px solid #ddd;
            border-radius: 4px;
        }
        h1 {
            font-size: 24px;
            text-align: center;
        }
        form {
            margin-top: 20px;
            text-align: center;
        }
        input[type="file"] {
            margin: 10px 0;
        }
        input[type="submit"] {
            background-color: #3498db;
            color: white;
            padding: 10px 20px;
            border: none;
            border-radius: 4px;
            cursor: pointer;
            font-size: 16px;
        }
        input[type="submit"]:hover {
            background-color: #2980b9;
        }
        .message {
            margin-top: 20px;
            padding: 10px;
            border-radius: 4px;
            text-align: center;
        }
        .success {
            background-color: #d4edda;
            color: #155724;
            border: 1px solid #c3e6cb;
        }
        .error {
            background-color: #f8d7da;
            color: #721c24;
            border: 1px solid #f5c6cb;
        }
        .progress-bar {
            width: 100%;
            background-color: #f3f3f3;
            border: 1px solid #ddd;
            border-radius: 4px;
            overflow: hidden;
            margin-top: 20px;
        }
        .progress-bar-fill {
            height: 20px;
            background-color: #3498db;
            width: 0%;
            transition: width 0.25s;
        }
    </style>
</head>
<body>
    <div class="container">
        <h1>Upload</h1>
        <form id="uploadForm" action="/upload.php" method="post" enctype="multipart/form-data">
            <input type="file" name="fileToUpload" id="fileToUpload" required>
            <input type="submit" value="Upload File" name="submit">
        </form>

        <div class="progress-bar">
            <div class="progress-bar-fill"></div>
        </div>

                    <div class="message error">
                No file was uploaded.                            </div>
            </div>

    <script>
        const form = document.getElementById('uploadForm');
        const progressBarFill = document.querySelector('.progress-bar-fill');

        form.addEventListener('submit', function(e) {
            e.preventDefault();
            const formData = new FormData(form);
            const xhr = new XMLHttpRequest();

            xhr.upload.addEventListener('progress', function(e) {
                const percent = e.lengthComputable ? (e.loaded / e.total) * 100 : 0;
                progressBarFill.style.width = percent + '%';
            });

            xhr.open('POST', form.action, true);
            xhr.onload = function() {
                if (xhr.status === 200) {
                    document.body.innerHTML = xhr.responseText;
                } else {
                    alert('An error occurred while uploading the file.');
                }
            };
            xhr.send(formData);
        });
    </script>
</body>
</html>
response = requests.post(
    "www.example.com/upload.php",
    files={"fileToUpload": (file_name, file_data)},
)
response.raise_for_status()

Whatever these two lines of python do different, they work lol

A little bit of background: he made a python script with chatgpt that uploades files to his website which he then compiled so his friends can use it. In my naivete I said f this imma do this in a language thats supposed to actually be compiled and here I am fighting issues I never expected to encounter lol

Yet again though, Ive already learned a bunch of stuff

Ah, it’s expecting a multipart/form-data request. You’ll need to encode the file in manner that @dimdin described. Since the file content is comptime known, simple concatenation should do the job here:

    const header =
        \\------eb542ed298bc07fa2f58d09191f02dbbffbaa477
        \\Content-Disposition: form-data; name="fileToUpload"; filename="test.zig"
        \\Content-Type: text/plain
        \\
    ;
    const trailer = 
        \\------eb542ed298bc07fa2f58d09191f02dbbffbaa477--
    ;
    const content = header ++ file ++ trailer;

    req.headers.content_type = .{ .override = "multipart/form-data; boundary=----eb542ed298bc07fa2f58d09191f02dbbffbaa477" };
    req.transfer_encoding = .{ .content_length = content.len };

    try req.send();
    try req.writeAll(content);

EDIT: Added two extra dashes to multi-part boundaries

2 Likes

In which way is that different to a “regular one” and why cant that be done using the stdlib internal stuff?

How do most clients deal with this if they dont have that information yet? Do they just try so many post requests until it works?

Multipart form data is a bit of a relic from the time of HTTP 1.0. We generally use regular HTTP POST/PUT for file uploading these days. Works better for things like upload progress and streaming to disk.

Lol okay so I can conclude that my friends website uses outdated php?

It still does not work btw, I get the same error as before.

An uneducated guess: could that be because I am uploading just literally the file itself so that theres a boundary before before the actual ending?

Nop, still does not work.

My code now is

const std = @import("std");

test {
    const allocator = std.testing.allocator;

    const file = @embedFile("index.html");

    var client = std.http.Client{ .allocator = allocator };
    defer client.deinit();

    const uri = try std.Uri.parse("www.example.com/upload.php");
    var buf: [1024 * 1024]u8 = undefined;
    var req = try client.open(.POST, uri, .{ .server_header_buffer = &buf });
    defer req.deinit();

    const header =
        \\----eb542ed298bc07fa2f58d09191f02dbbffbaa477
        \\Content-Disposition: form-data; name="fileToUpload"; filename="test.zig"
        \\Content-Type: text/plain
        \\
    ;
    const trailer =
        \\----eb542ed298bc07fa2f58d09191f02dbbffbaa477--
    ;
    const content = header ++ file ++ trailer;

    req.headers.content_type = .{ .override = "multipart/form-data; boundary=----eb542ed298bc07fa2f58d09191f02dbbffbaa477" };
    req.transfer_encoding = .{ .content_length = content.len };

    try req.send();
    try req.writeAll(content);
    try req.finish();
    try req.wait();

    try std.testing.expectEqual(.ok, req.response.status);

    const mssg = try req.reader().readAllAlloc(allocator, std.math.maxInt(usize));
    defer allocator.free(mssg);
    try std.io.getStdErr().writeAll(mssg);
}

Oops. I forgot that there’re two extra dashes in the boundary:

const header =
        \\------eb542ed298bc07fa2f58d09191f02dbbffbaa477
        \\Content-Disposition: form-data; name="fileToUpload"; filename="test.zig"
        \\Content-Type: text/plain
        \\
    ;

Try uploading the file through the browser and then study the request in the development console.

1 Like

Alright I was out eating and took a break from this lmao but now I have this:

When I upload a file I first select it and then upload. More than one file is at least via gui not possible.

Working example performed by the browser

The request payload then looks like this:

-----------------------------273344254425326470862221127403
Content-Disposition: form-data; name="fileToUpload"; filename="stab.h"
Content-Type: text/x-chdr

#ifndef __GNU_STAB__

/* Indicate the GNU stab.h is in use.  */

#define __GNU_STAB__

#define __define_stab(NAME, CODE, STRING) NAME=CODE,

enum __stab_debug_code
{
#include <bits/stab.def>
LAST_UNUSED_STAB_CODE
};

#undef __define_stab

#endif /* __GNU_STAB_ */
-----------------------------273344254425326470862221127403
Content-Disposition: form-data; name="submit"

Upload File
-----------------------------273344254425326470862221127403--

The file here is just something I found laying around.

The headers are

POST /upload.php HTTP/2
Host: www.example.com
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:128.0) Gecko/20100101 Firefox/128.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/png,image/svg+xml,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br, zstd
Content-Type: multipart/form-data; boundary=---------------------------273344254425326470862221127403
Content-Length: 613
Origin: www.example.com
DNT: 1
Connection: keep-alive
Referer: https://example.com/upload.php
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: same-origin
Sec-Fetch-User: ?1
Priority: u=0, i

My code

const std = @import("std");

test {
    const allocator = std.testing.allocator;

    const filename = "file1.txt";
    const file =
        \\some 
        \\test
        \\file
        \\
    ;

    var client = std.http.Client{ .allocator = allocator };
    defer client.deinit();

    const uri = try std.Uri.parse("https://example.com/upload.php");
    var buf: [1024 * 1024]u8 = undefined;
    var req = try client.open(.POST, uri, .{ .server_header_buffer = &buf });
    defer req.deinit();

    const boundary = "-----------------------------1760513608948131895337362114";
    const content = std.fmt.comptimePrint(
        \\{[boundary]s}
        \\Content-Disposition: form-data; name="fileToUpload"; filename="{[filename]s}"
        \\Content-Type: text/x-chdr
        \\
        \\{[file]s}
        \\{[boundary]s}
        \\Content-Disposition: form-data; name="submit"
        \\
        \\Upload File
        \\{[boundary]s}--
    , .{ .boundary = boundary, .filename = filename, .file = file });

    std.debug.print("{s}\n", .{content});

    req.headers.content_type = .{ .override = "multipart/form-data; boundary=" ++ boundary };
    req.transfer_encoding = .{ .content_length = content.len };

    try req.send();
    try req.writeAll(content);
    try req.finish();
    try req.wait();

    try std.testing.expectEqual(.ok, req.response.status);

    const mssg = try req.reader().readAllAlloc(allocator, std.math.maxInt(usize));
    defer allocator.free(mssg);

    const out_file = try std.fs.cwd().createFile("index.html", .{});
    defer out_file.close();
    try out_file.writeAll(mssg);
}

My program receives the http code 200 (== .ok), however index.html is displaying the site with a notification saying that no files were uploaded.

Additional errors I have found: the line breaks need to all be \r\n, except those part of the data itself. My code now is

const std = @import("std");

const boundary = "-----------------------------13549909921538775893949405532";
const file = .{
    .name = "lastlog.h",
    .str =
    \\/* This header file is used in 4.3BSD to define `struct lastlog',
    \\   which we define in <bits/utmp.h>.  */
    \\
    \\#include <utmp.h>
    ,
};

const form = blk: {
    var buf: [4 * 1024 * 1024]u8 = undefined;
    const unix_form =
        \\{[boundary]s}
        \\Content-Disposition: form-data; name="fileToUpload"; filename="{[filename]s}"
        \\Content-Type: text/x-chdr
        \\
        \\{[file]s}
        \\{[boundary]s}--
        \\
    ;
    const lendiff = std.mem.replace(u8, unix_form, "\n", "\r\n", &buf);
    break :blk buf[0 .. unix_form.len + lendiff].*;
};

test {
    const allocator = std.testing.allocator;

    var client = std.http.Client{ .allocator = allocator };
    defer client.deinit();

    const uri = try std.Uri.parse("https://example.com/upload.php");
    var header_buf: [1024 * 1024]u8 = undefined;
    var req = try client.open(.POST, uri, .{ .server_header_buffer = &header_buf });
    defer req.deinit();

    const content = std.fmt.comptimePrint(&form, .{
        .boundary = boundary,
        .filename = file.name,
        .file = file.str,
    });

    std.debug.print("{s}\n", .{content});
    std.debug.print("length: {d}\n", .{content.len});

    req.headers.content_type = .{ .override = "multipart/form-data; boundary=" ++ boundary };
    req.transfer_encoding = .{ .content_length = content.len };

    try req.send();
    try req.writeAll(content);
    try req.finish();
    try req.wait();

    try std.testing.expectEqual(.ok, req.response.status);

    const mssg = try req.reader().readAllAlloc(allocator, std.math.maxInt(usize));
    defer allocator.free(mssg);

    const out_file = try std.fs.cwd().createFile("index.html", .{});
    defer out_file.close();
    try out_file.writeAll(mssg);
}

The body now is identical with a working example from my browser, down to the byte. The site still returns some html saying that no files was uploaded, which is confirmed when checking its upload folder.

Headers remain the same as in How do I post a file to a website? - #12 by markus, only difference obviously being the fact that the length obviously varies, but I have made sure that its correct.

Is there a chance that zigs lack of http1.1 is the issue here? Every working instance of uploads to the site used http2

Edit: Switching to http fixed it

Switching to http made everything work!