DFS: The first true 2-way sync dotfiles manager

This is a dotfiles manager that can template and pull changes from rendered files. My old approach was using symlinks and the Git repository, and I am very spoiled by upstream changes (because I don’t have to track them-git does that for me). But maintaining different branches for different systems and architectures is a drag. Plus it looks like Ansible on BSD is slowing down, and I am slowly losing my testing tools, one by one (stale molecule plugins, etc.). So I decided to switch from my Ansible dotfiles manager to something more robust, with templates and stuff, so I could merge my dotfiles branches into one. It turned out none of the currently available tools can do what I need. There are symlink-based ones and template solutions as well. But none can track changes in the rendered file.

So I wrote a new tool from the ground up—DFS! It has a template engine that can conditionally render strings, and it tracks changes in the rendered files, translating them back into the template like it was edited there.

The source config must be a Git repository that mirrors $HOME. You can set a custom repository destination as well as a custom deployment directory (defaults to $HOME).

It is pretty raw; I just finished rewriting the engine after polishing the foundation (logistics and CLI), so the library still needs refactoring. Only OS conditionals are implemented right now—architecture and cool userland stuff will follow shortly.

test applyTemplate {
    if (builtin.os.tag != .freebsd or builtin.cpu.arch != .x86_64) return error.SkipZigTest;
    var gpa = std.testing.allocator;

    const template =
        \\{> if SYSTEM.os == linux <}
        \\val="Foo"
        \\{> elif SYSTEM.os == freebsd <}
        \\val="Bar"
        \\{> else <}
        \\val="Else"
        \\{> end <}
        \\{> if SYSTEM.arch == x86_64 <}
        \\val="test0"
        \\{> else <}
        \\val="test1"
        \\{> end <}
        \\
        \\{> if SYSTEM.hostname == not_my_machine <}
        \\val="HOST2"
        \\{> else <}
        \\val="HOST1"
        \\{> end <}
        \\
    ;

    const rendered_expected =
        \\val="Bar"
        \\val="test0"
        \\
        \\val="HOST1"
        \\
    ;

    const rendered = try applyTemplate(gpa, template);
    defer gpa.free(rendered);
    try std.testing.expectEqualStrings(rendered_expected, rendered);
}

test "back-template (fbsd)" {
    if (builtin.os.tag != .freebsd) return error.SkipZigTest;
    var gpa = std.testing.allocator;

    const template =
        \\{> if SYSTEM.os == linux <}
        \\val="Foo"
        \\{> elif SYSTEM.os == freebsd <}
        \\val="Bar"
        \\{> else <}
        \\val="Else"
        \\{> end <}
        \\
    ;

    const rendered_user_edit =
        \\val="Zoot"
        \\
    ;

    const reversed = try reverseTemplate(gpa, rendered_user_edit, template);
    defer gpa.free(reversed);

    const expected_template =
        \\{> if SYSTEM.os == linux <}
        \\val="Foo"
        \\{> elif SYSTEM.os == freebsd <}
        \\val="Zoot"
        \\{> else <}
        \\val="Else"
        \\{> end <}
        \\
    ;
    try std.testing.expectEqualStrings(expected_template, reversed);
}

test "back-no_template (fbsd)" {
    if (builtin.os.tag != .freebsd) return error.SkipZigTest;
    var gpa = std.testing.allocator;

    const template =
        \\FOO
        \\{> if SYSTEM.os == linux <}
        \\val="Foo"
        \\{> elif SYSTEM.os == freebsd <}
        \\val="Bar"
        \\{> else <}
        \\val="Else"
        \\{> end <}
        \\
    ;

    const rendered_user_edit =
        \\BAR
        \\val="Bar"
        \\
    ;

    const reversed = try reverseTemplate(gpa, rendered_user_edit, template);
    defer gpa.free(reversed);

    const expected_template =
        \\BAR
        \\{> if SYSTEM.os == linux <}
        \\val="Foo"
        \\{> elif SYSTEM.os == freebsd <}
        \\val="Bar"
        \\{> else <}
        \\val="Else"
        \\{> end <}
        \\
    ;
    try std.testing.expectEqualStrings(expected_template, reversed);
}
4 Likes
____________ _____
|  _  \  ___/  ___|
| | | | |_  \ `--.
| | | |  _|  `--. \
| |/ /| |   /\__/ /
|___/ \_|   \____/

    dfs
    A configuration (dotfiles) manager with a template engine and
    a true 2-way synchronization. The deployed layout recreates the
    source completely (except assets in the ignore list). So it is
    recommended to structure the source repository as a $HOME mirror.

    EXAMPLE:
        dfs init -h

    SUBCOMMANDS:
        version: Show the 'dfs' version.
        init: Initialize the configuration (requires git).
        sync: Run synchronization.
        validate: Run template validation.
        daemon: Start the daemon.
        purge: Delete application data (meta, backups, logs).
        usage: Show the 'dfs' usage display.
        help: Show the 'dfs' help display.

    OPTIONS:
        Config:
            -c, --config <path>
            Configuration path.
        Source:
            -s, --source <path>
            Override the source directory.
        Target:
            -t, --target, --destination <path>
            Override the target directory.
        Notifications:
            -n, --notifications <bool>
            Show desktop notifications.
        Json:
            --json <bool>
            Output JSON status string.
        Usage:
            -u, --usage <bool>
            Show the 'dfs' usage display.
        Help:
            -h, --help <bool>
            Show the 'dfs' help display.

via cova

1 Like

Moved the app data, refactored lib fns, and fixed inlined templates.

The dfs sync --dry output is rough (for now) but extensive.

no rest for the wicked!! figuring out the conflict resolver and going for the release Commits · charlesrocket/dfs · GitHub

3 Likes

Fixed the multiblock bug my subconscious tried to forget (anticipated way less LOC). Gonna get some rest and maybe implement a bootstrap command before releasing later this evening. Finally, I can slow down and pick up on what’s going on in jinja-reverse—there must be some good moves I could learn from (my test cases are not as wide as I want them to be, but it is enough for 0.1).

0.1.0

Bug Fixes

  • Handle missing meta
  • Indicate dry-run
  • Increase template file buffer
  • Correct input enum
  • Move sync files to datadir
  • Add separator
  • Clone submodules
  • Drop evalIfBlock()
  • Set stdout prints
  • Improve stdout prints
  • Ignore changelog
  • Set file size
  • Enforce file size type
  • Improve diff/raw format
  • Improve copy output
  • Handle rendered conflicts
  • Handle multiblocks

Documentation

  • Add readme
  • Add roadmap
  • Fix md
  • Update Conditionals
  • Update roadmap
  • Update example
  • Update init description
  • Roadmap backups
  • Update description
  • Add header
  • Close roadmap items
  • Move roadmap

Features

  • Add directory scanner
  • Template mechanics
  • Add recordLastSync()
  • Process sync records
  • Add lastMod()
  • Add reverseTemplate()
  • Expose template functions
  • Add cli
  • Add assets
  • Add config infrastructure
  • Add options
  • Add dry-run
  • Check binary data
  • Add init subcommand
  • Change template syntax
  • Rewrite engine
  • Handle inlined templates
  • Add file count
  • Add SYSTEM.hostname
  • Add SYSTEM.arch
  • Add config option
  • Add colors
  • Add verbose mode

Miscellaneous tasks

  • Add license
  • Add gitignore
  • Ignore docs
  • Move library
  • Add changelog

Operations

  • Add integration files
  • Enable test coverage
  • Bump actions/attest-build-provenance from 2 to 3
  • Add library label
  • Deploy docs
  • Fix ghp permissions
  • Bump actions/labeler from 5 to 6
  • Update labels
  • Add cli label

Refactor

  • Optimize template functions
  • Move Config
  • Move Dotfile
  • Move main functions
  • Move getUserInput()
  • Move dotfile functions

Styling

  • Fix formatting
  • Fix formatting
  • Fix missing newline
  • Fix print formatting

Testing

  • Add linux cases
  • Add mixed
  • Add inline cases
  • Add reverseTemplate
  • Add block cases

Build

  • Add dependencies
  • Add clean step
  • Fix man page section
  • Add docs step
  • Edit docs step
  • Update fingerprint

Fixed linux issues and expanded CLI a little

0.2.0

Bug Fixes

  • Handle relative paths
  • Fix linux paths
  • Change json options
  • Add newline to stdout

Features

  • Add json option
  • Add bootstrap command
  • Add platform-specific ignore list
  • [breaking] Add ignore list

Miscellaneous tasks

  • Ignore kcov

Operations

  • Fix release name
  • Update label list

Refactor

  • renderedrender

Testing

  • Add sync

Build

  • Fix coverage

tons of tests and improvements

0.3.0

Bug Fixes

  • [breaking] Rename errors
  • Switch to gpa
  • Catch template errors
  • Fix meta file path
  • Switch to std.fs.Dir.walk()
  • Fix dotfile.lastMod()
  • Fix file errors
  • Get stderr
  • Use absolute template path
  • Fix lastMod()
  • Keep relative source string
  • Improve write ops
  • Improve change detection
  • Drop fs.realpathAlloc()
  • Properly handle first sync
  • Catch empty templates

Features

  • Support env vars

Miscellaneous tasks

  • Ignore dest*

Refactor

  • Fix signatures
  • Define tag
  • Drop redundant len

Styling

  • Fix formatting

Testing

  • Add token tests
  • Ignore dest
  • Add processFile
  • Fix unit cases
  • Fix sync
  • Fix merge step
  • Add sync-dry
  • Add config cases
  • Fix processFile
  • Add sync-back
  • Add binary file
  • Fix sync-dry
  • Add blocks-mixed
  • Add parseTag
  • Add parseBody
  • Add trimming cases
  • Add indexOfTag
  • Add nextTag
  • Add splitWhitespace
  • Add extractChangeChunk
  • Add findAnchorLiteral
  • Fix sync cases
  • Add copyWithWhitespace
  • Add normalizeTrailing
  • Add evalIfGroup
  • Add evalCondition
  • Merge cases
1 Like

0.4.0

Bug Fixes

  • Update summary
  • Handle file permissions
  • Set config size
  • Update example_config
  • Set initial delay
  • Update description
  • Update I/O
  • Update opt_usage
  • Update for zig 0.15
  • Update arrays

Documentation

  • Add Usage
  • Add Configuration

Features

  • Add validator
  • Add backups
  • Add purge command
  • Add progress status

Operations

  • Bump zig to 0.15.1
  • Update coverage

Testing

  • Improve validate
  • Fix config
  • Update runner
  • Fix sync-back

Build

  • Update cova + ghext
  • Add integration_tests_mod
  • Fix coverage
1 Like

0.5.0

Bug Fixes

  • Add missing counters
  • Correct writing positions
  • Update config errors
  • Handle ignore_list allocations
  • Clone during bootstrap
  • Move stdout prints
  • Fix pathFormat() deallocations
  • Improve returns
  • Purge logs
  • Edit config error
  • Fix cloneRepo() path
  • Make cloneRepo() silent
  • [breaking] Move bootstrap command
  • Fix zon parser leak
  • Drop url option
  • Improve config handling

Documentation

  • Update config
  • .destination.target

Features

  • Add direction option
  • Add repository
  • Migrate old formats
  • Automatic config updates
  • Add logger
  • Use custom logger
  • [breaking] Add target

Miscellaneous tasks

  • Note deallocations for path functions

Operations

  • Bump zig to 0.15.2

Refactor

  • Split processFile()
  • Move bootstrap()
  • Add cloneRepo()
  • Restructure Dotfile/Config
  • Add core module
  • Improve direction assignment
  • Move IGNORE_LIST

Styling

  • Fix getUserInput() formatting
  • Fix description

Testing

  • Add sync-back-forced
  • Add sync-forward-forced
  • Update processFile
  • Update configs
  • Add migrateConfig
  • Add pathFormat
  • Reformat cases
  • Reformat processFile
  • Update config cases
  • Add log
  • Fix config bad

Build

  • Bump MSZV to 0.15.1
  • Update fingerprint
  • Bump cova to 08d92ce
2 Likes

0.6.0

Bug Fixes

  • Highlight summary
  • [breaking] Switch to immutable strings
  • [breaking] ValidationErrorTemplateError
  • [breaking] Update error set
  • Update validation messages
  • Improve conditionals
  • Fix position tracking
  • Remove splitWhitespace()
  • Optimize parseCondition()
  • Make sendNotification() silent
  • Gate kqueue functions
  • [breaking] Move config/app data
  • Update bootstrap example
  • Update daemon description
  • Check the rendered output
  • Properly handle literal changes
  • Reflect sync status
  • Adjust registration procedure
  • Fix xdg desktop string

Documentation

  • Comment main functions
  • Comment validate()
  • Update example config
  • Add tray setting

Features

  • Add desktop notifications
  • Prepare daemon
  • Add spawnTray()
  • Implement file watcher
  • Add Configuration item
  • Add tray
  • Add sync time tooltip
  • Set named icons
  • Add Tray.menu_icons
  • Add SYSTEM.desktop

Operations

  • Install dbus
  • Add daemon label
  • Update daemon label
  • Add asset label

Performance

  • Drop redundant check
  • Drop redundant trimTag()

Refactor

  • Utilize parseTag()
  • Improve conditional fns
  • Add SYSTEM
  • Rewrite evalIfGroup()
  • Update reverseIfGroup()
  • Remove countTrail()
  • Add LOG_SIZE_MAX
  • Add evalBranch()
  • Fix notifications check
  • Add scan()
  • Add sync()
  • Move daemon

Testing

  • Update layout
  • Reformat interpret
  • Fix parseTag
  • Update evalIfGroup
  • Update validate
  • Update log
  • Add defaultConfigPath

Build

  • Add libstray
  • Add dbus option
  • Add icons
  • Bump libstray to 9c4c412
  • Bump libstray to 0fba7fb