DFS: The first true 2-way sync-template dotfiles manager

This is a dotfiles manager that can template and pull changes from rendered files. My old approach was using symlinks and the Git repository, and I am very spoiled by upstream changes (because I don’t have to track them-git does that for me). But maintaining different branches for different systems and architectures is a drag. Plus it looks like Ansible on BSD is slowing down, and I am slowly losing my testing tools, one by one (stale molecule plugins, etc.). So I decided to switch from my Ansible dotfiles manager to something more robust, with templates and stuff, so I could merge my dotfiles branches into one. It turned out none of the currently available tools can do what I need. There are symlink-based ones and template solutions as well. But none can track changes in the rendered file.

So I wrote a new tool from the ground up—DFS! It has a template engine that can conditionally render strings, and it tracks changes in the rendered files, translating them back into the template like it was edited there.

The source config must be a Git repository that mirrors $HOME. You can set a custom repository destination as well as a custom deployment directory (defaults to $HOME).

It is pretty raw; I just finished rewriting the engine after polishing the foundation (logistics and CLI), so the library still needs refactoring. Only OS conditionals are implemented right now—architecture and cool userland stuff will follow shortly.

test applyTemplate {
    if (builtin.os.tag != .freebsd or builtin.cpu.arch != .x86_64) return error.SkipZigTest;
    var gpa = std.testing.allocator;

    const template =
        \\{> if SYSTEM.os == linux <}
        \\val="Foo"
        \\{> elif SYSTEM.os == freebsd <}
        \\val="Bar"
        \\{> else <}
        \\val="Else"
        \\{> end <}
        \\{> if SYSTEM.arch == x86_64 <}
        \\val="test0"
        \\{> else <}
        \\val="test1"
        \\{> end <}
        \\
        \\{> if SYSTEM.hostname == not_my_machine <}
        \\val="HOST2"
        \\{> else <}
        \\val="HOST1"
        \\{> end <}
        \\
    ;

    const rendered_expected =
        \\val="Bar"
        \\val="test0"
        \\
        \\val="HOST1"
        \\
    ;

    const rendered = try applyTemplate(gpa, template);
    defer gpa.free(rendered);
    try std.testing.expectEqualStrings(rendered_expected, rendered);
}

test "back-template (fbsd)" {
    if (builtin.os.tag != .freebsd) return error.SkipZigTest;
    var gpa = std.testing.allocator;

    const template =
        \\{> if SYSTEM.os == linux <}
        \\val="Foo"
        \\{> elif SYSTEM.os == freebsd <}
        \\val="Bar"
        \\{> else <}
        \\val="Else"
        \\{> end <}
        \\
    ;

    const rendered_user_edit =
        \\val="Zoot"
        \\
    ;

    const reversed = try reverseTemplate(gpa, rendered_user_edit, template);
    defer gpa.free(reversed);

    const expected_template =
        \\{> if SYSTEM.os == linux <}
        \\val="Foo"
        \\{> elif SYSTEM.os == freebsd <}
        \\val="Zoot"
        \\{> else <}
        \\val="Else"
        \\{> end <}
        \\
    ;
    try std.testing.expectEqualStrings(expected_template, reversed);
}

test "back-no_template (fbsd)" {
    if (builtin.os.tag != .freebsd) return error.SkipZigTest;
    var gpa = std.testing.allocator;

    const template =
        \\FOO
        \\{> if SYSTEM.os == linux <}
        \\val="Foo"
        \\{> elif SYSTEM.os == freebsd <}
        \\val="Bar"
        \\{> else <}
        \\val="Else"
        \\{> end <}
        \\
    ;

    const rendered_user_edit =
        \\BAR
        \\val="Bar"
        \\
    ;

    const reversed = try reverseTemplate(gpa, rendered_user_edit, template);
    defer gpa.free(reversed);

    const expected_template =
        \\BAR
        \\{> if SYSTEM.os == linux <}
        \\val="Foo"
        \\{> elif SYSTEM.os == freebsd <}
        \\val="Bar"
        \\{> else <}
        \\val="Else"
        \\{> end <}
        \\
    ;
    try std.testing.expectEqualStrings(expected_template, reversed);
}
3 Likes
┌───┐
│dfs│
└───┘
____________ _____
|  _  \  ___/  ___|
| | | | |_  \ `--.
| | | |  _|  `--. \
| |/ /| |   /\__/ /
|___/ \_|   \____/

    dfs
    A true 2-way sync dotfiles manager with reverse template translator

    EXAMPLE:
        dfs init -h

    SUBCOMMANDS:
        version: Show the 'dfs' version.
        init: Initialize the configuration. The deployed layout mirrors
    the source completely (except assets in the ignore list).
    So it is recommended to structure the source repository as $HOME.

    (requires git)
        sync: Run synchronization.
        usage: Show the 'dfs' usage display.
        help: Show the 'dfs' help display.

    OPTIONS:
        Source:
            -s, --source <path>
            Override the source directory.
        Destination:
            -d, --destination <path>
            Override the destination directory.
        Usage:
            -u, --usage <bool>
            Show the 'dfs' usage display.
        Help:
            -h, --help <bool>
            Show the 'dfs' help display.

via cova

1 Like

Moved the app data, refactored lib fns, and fixed inlined templates.

The dfs sync --dry output is rough (for now) but extensive.

no rest for the wicked!! figuring out the conflict resolver and going for the release Commits · charlesrocket/dfs · GitHub

3 Likes

Fixed the multiblock bug my subconscious tried to forget (anticipated way less LOC). Gonna get some rest and maybe implement a bootstrap command before releasing later this evening. Finally, I can slow down and pick up on what’s going on in jinja-reverse—there must be some good moves I could learn from (my test cases are not as wide as I want them to be, but it is enough for 0.1).

0.1.0

Bug Fixes

  • Handle missing meta
  • Indicate dry-run
  • Increase template file buffer
  • Correct input enum
  • Move sync files to datadir
  • Add separator
  • Clone submodules
  • Drop evalIfBlock()
  • Set stdout prints
  • Improve stdout prints
  • Ignore changelog
  • Set file size
  • Enforce file size type
  • Improve diff/raw format
  • Improve copy output
  • Handle rendered conflicts
  • Handle multiblocks

Documentation

  • Add readme
  • Add roadmap
  • Fix md
  • Update Conditionals
  • Update roadmap
  • Update example
  • Update init description
  • Roadmap backups
  • Update description
  • Add header
  • Close roadmap items
  • Move roadmap

Features

  • Add directory scanner
  • Template mechanics
  • Add recordLastSync()
  • Process sync records
  • Add lastMod()
  • Add reverseTemplate()
  • Expose template functions
  • Add cli
  • Add assets
  • Add config infrastructure
  • Add options
  • Add dry-run
  • Check binary data
  • Add init subcommand
  • Change template syntax
  • Rewrite engine
  • Handle inlined templates
  • Add file count
  • Add SYSTEM.hostname
  • Add SYSTEM.arch
  • Add config option
  • Add colors
  • Add verbose mode

Miscellaneous tasks

  • Add license
  • Add gitignore
  • Ignore docs
  • Move library
  • Add changelog

Operations

  • Add integration files
  • Enable test coverage
  • Bump actions/attest-build-provenance from 2 to 3
  • Add library label
  • Deploy docs
  • Fix ghp permissions
  • Bump actions/labeler from 5 to 6
  • Update labels
  • Add cli label

Refactor

  • Optimize template functions
  • Move Config
  • Move Dotfile
  • Move main functions
  • Move getUserInput()
  • Move dotfile functions

Styling

  • Fix formatting
  • Fix formatting
  • Fix missing newline
  • Fix print formatting

Testing

  • Add linux cases
  • Add mixed
  • Add inline cases
  • Add reverseTemplate
  • Add block cases

Build

  • Add dependencies
  • Add clean step
  • Fix man page section
  • Add docs step
  • Edit docs step
  • Update fingerprint

Fixed linux issues and expanded CLI a little

0.2.0

Bug Fixes

  • Handle relative paths
  • Fix linux paths
  • Change json options
  • Add newline to stdout

Features

  • Add json option
  • Add bootstrap command
  • Add platform-specific ignore list
  • [breaking] Add ignore list

Miscellaneous tasks

  • Ignore kcov

Operations

  • Fix release name
  • Update label list

Refactor

  • renderedrender

Testing

  • Add sync

Build

  • Fix coverage