So I’ve been working on this more or less non-stop for the last couple of months. My goal for this ‘release’ was to have a reliable way to process different syntaxes, so that I could adjust the rules on the fly as I’m improving this, without causing any catastrophic breakage. I also wanted to be able to do this with a repl. from here I can easily add the ability load external scripts. I am releasing this with a lot of room for improvements. But I think my string parsing algorithm is solid. There’s a lot of things I could add at this point without too much effort (i.e. reading json configs to define the destination host). So for now I am just restricting the http request to the one hardcoded in the source…
So a little bit about what I’m trying to do. It’s basically to have a scripting language to accept variables and objects and pass them into different kindof of http requests. Sort of a way to plug into a remote API. Also being able to query and process JSON responses, which I got a pretty got start on, but here I am simply outputting a constant JSON response body.
For example, I would like to have a script like this
$obj = { hello: "okay" }
$query = 'hello.id[5]'
.post ($obj.JSON) { $query }
this would process a JavaScript style object stored in a variable, converting it to a JSON syntax and sending it as the request body over a post request, with the destination host defined in the config file. The incoming JSON response would be processed by the query variable, and output appropriately. For now it’s only supporting string variable declarations (which should validate with single or double quotes). These variables are automatically formatted into a url querystring and sent as a GET request to a minimal node server.
I was hoping to have a little big more, but unfortunately I hit a lot of dead-ends along the way, so I will be taking a break from it for a little while in order to gain more perspective. The upside is that I learned a lot about Zig, everything in std.mem, and the different kinds of lists and ways to iterate over them, when a for loop is more useful and when a while loop is useful. Also continue expressions in while loops… very useful. I ended up writing a simple custom “Forward” iterator to keep things simple. So i can just use like it.next() in the while loops without having to worry about tracking an index.
Anyway at this point I’m relieved just to be able to throw this out there. At least now I got past the hard parts (i think), so I should just be able to improve this bit by bit without too much effort.
To use the repl, you just put in string variables line by line like so…
$hello=‘okay’
$okay=“hello”
a double enter will trigger the script to process and send the http request based on the variables stored, converting them to url parameters in the process
you can also do multiline input by appending a \ at the end of each line. these lines will all be concatenated before processing.
definitely a lot of loose threads here, but it does work as is, with the provided javascript server.
source code for server:
const http = require('http');
const url = require('url')
const port = 3000;
const host = 'localhost';
const user = { hello: 'okay' };
http.createServer((req, res) => {
console.log(url.parse(req.url).query);
res.writeHead(200, { 'Content-Type': 'application/json' });
res.end(JSON.stringify(user));
}).listen(port, (err) => {
if (err) throw err;
console.log(`${host}:${port}`);
});
and the main src…
const std = @import("std");
const Value = union {
null: void,
bool: bool,
integer: i64,
float: f64,
string: []const u8,
array: std.ArrayList(Value),
object: std.StringArrayHashMap(Value),
};
const ForwardIterator = struct {
haystack: *[]u8,
index: usize = 0,
fn next(self: *ForwardIterator) ?*u8 {
if (self.index < self.haystack.*.len) {
self.index += 1;
return &self.haystack.*[self.index - 1];
}
return null;
}
};
const State = struct {
lines: []u8,
query: *std.ArrayList(u8),
value: *std.ArrayList(u8),
vars: *std.StringArrayHashMap(Value),
fn init(lines: []u8, query: *std.ArrayList(u8), value: *std.ArrayList(u8), vars: *std.StringArrayHashMap(Value)) State {
return State{
.lines = lines,
.query = query,
.value = value,
.vars = vars,
};
}
fn exec(self: *State) !void {
var it = ForwardIterator{ .haystack = &self.lines };
outer: while (it.next()) |a| {
switch (a.*) {
'$' => {
first: while (it.next()) |b| {
switch (b.*) {
'\n' => {
std.debug.print("INVALID::\\n\n", .{});
break :outer;
},
'=' => {
while (it.next()) |c| {
switch (c.*) {
'\n' => {
std.debug.print("INVALID::\\n\n", .{});
break :outer;
},
'\'' => {
while (it.next()) |d| {
switch (d.*) {
'\n' => {
std.debug.print("INVALID::\\n\n", .{});
break :outer;
},
'\\' => {
while (it.next()) |e| {
switch (e.*) {
'\\' => {
try self.value.append(e.*);
break;
},
'\'' => {
try self.value.append(e.*);
break;
},
else => {
std.debug.print("INVALID::{c}\n", .{e.*});
break :outer;
},
}
}
},
'\'' => {
while (it.next()) |e| {
switch (e.*) {
'\n' => {
try self.vars.put(try self.query.toOwnedSlice(), Value{ .string = try self.value.toOwnedSlice() });
break :first;
},
else => {
std.debug.print("INVALID::{c}\n", .{e.*});
break :outer;
},
}
}
},
else => {
try self.value.append(d.*);
},
}
}
},
'"' => {
while (it.next()) |d| {
switch (d.*) {
'\n' => {
std.debug.print("INVALID::\\n\n", .{});
break :outer;
},
'\\' => {
while (it.next()) |e| {
switch (e.*) {
'\\' => {
try self.value.append(e.*);
break;
},
'"' => {
try self.value.append(e.*);
break;
},
else => {
std.debug.print("INVALID::{c}\n", .{e.*});
break :outer;
},
}
}
},
'"' => {
while (it.next()) |e| {
switch (e.*) {
'\n' => {
try self.vars.put(try self.query.toOwnedSlice(), Value{ .string = try self.value.toOwnedSlice() });
break :first;
},
else => {
std.debug.print("INVALID::{c}\n", .{e.*});
break :outer;
},
}
}
},
else => {
try self.value.append(d.*);
},
}
}
},
else => {
std.debug.print("INVALID::{c}\n", .{c.*});
break :outer;
},
}
}
},
else => {
switch (b.*) {
'_' => {
try self.query.append(b.*);
},
else => {
if (std.ascii.isAlphanumeric(b.*)) {
try self.query.append(b.*);
} else {
std.debug.print("INVALID::{c}\n", .{b.*});
break :outer;
}
},
}
},
}
}
},
else => {
std.debug.print("INVALID::{c}\n", .{a.*});
break :outer;
},
}
}
}
};
test {
const allocator = std.testing.allocator;
std.debug.print("\n~\n> ", .{});
const stdin = std.io.getStdIn();
var reader = stdin.reader();
var input = std.ArrayList(u8).init(allocator);
var multi = std.ArrayList(u8).init(allocator);
var lines = std.ArrayList(u8).init(allocator);
var query = std.ArrayList(u8).init(allocator);
defer query.deinit();
var value = std.ArrayList(u8).init(allocator);
defer value.deinit();
var vars = std.StringArrayHashMap(Value).init(allocator);
defer vars.deinit();
while (reader.streamUntilDelimiter(input.writer(), '\n', 1000) != error.EndOfStream) {
if (input.items.len > 0 and input.items[input.items.len - 1] == '\r') {
input.shrinkRetainingCapacity(input.items.len - 1);
}
while (input.items.len > 0 and input.items[input.items.len - 1] == '\\' and reader.streamUntilDelimiter(multi.writer(), '\n', 1000) != error.EndOfStream) {
input.shrinkRetainingCapacity(input.items.len - 1);
if (multi.items.len > 0 and multi.items[multi.items.len - 1] == '\r') {
multi.shrinkRetainingCapacity(multi.items.len - 1);
}
try input.appendSlice(try multi.toOwnedSlice());
}
if (input.items.len == 0) {
var s: State = State.init(try lines.toOwnedSlice(), &query, &value, &vars);
try s.exec();
var it = s.vars.iterator();
var url: []const u8 = "http://127.0.0.1:3000/?";
while (it.next()) |kv| {
url = try std.fmt.allocPrint(allocator, "{s}{s}={s}&", .{ url, try std.Uri.escapeString(allocator, kv.key_ptr.*), try std.Uri.escapeString(allocator, kv.value_ptr.*.string) });
}
std.debug.print("{s}\n", .{url});
const uri = std.Uri.parse(url) catch unreachable;
var client: std.http.Client = .{ .allocator = allocator };
defer client.deinit();
var headers = std.http.Headers{ .allocator = allocator };
defer headers.deinit();
try headers.append("accept", "*/*");
var request = try client.request(.GET, uri, headers, .{});
defer request.deinit();
try request.start();
try request.wait();
try std.testing.expect(request.response.status == .ok);
const body = request.reader().readAllAlloc(allocator, 8192) catch unreachable;
defer allocator.free(body);
std.debug.print("{s}\n", .{body});
query.clearRetainingCapacity();
value.clearRetainingCapacity();
vars.clearRetainingCapacity();
} else {
try input.append('\n');
try lines.appendSlice(try input.toOwnedSlice());
}
std.debug.print("> ", .{});
}
std.debug.print("//EOF\n", .{});
}
not quite ready to release this on github, so i will just drop the license here as well…
ISC License
Copyright (c) 2023, Michael S. Scott (mscott9437@gmail.com)
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
A big thanks to all the Zig team, for building a language interesting enough to keep my attention. Definitely any feedback is appreciated, especially as to how I might optimize this, or if there is any easier way to do what i’m already doing.
Edit: just to clarify the official name of the project is ‘rr’ which stands for request/response, and I am planning to implement the processing of script files with the .rr extension