I started with Gentoo. If you know, you know: USE flags, emerge --sync, watching a browser compile for six hours, and the quiet satisfaction of a system shaped entirely to your will. It was maximum configuration at maximum cost - every package exactly how you wanted it, every kernel option hand-chosen, every dependency chain personally audited. It was also completely unreproducible. The system on disk was the result of hundreds of commands typed across months. If the disk died, so did the configuration. You could document it, but documentation and reality drift apart fast.

After a few years of that, I migrated my servers to Debian. Less customization, fewer surprises, faster iteration. Debian is rock solid. But the gap between “I’ve configured this” and “this is reproducible” never closed. The configuration lived partly in files, partly in database state, partly in my memory. Upgrading a server meant hoping nothing drift-collided with the new packages. I learned to fear dist-upgrades.

Multi-machine management made this worse. When I had three servers to keep consistent, I reached for Ansible. Later tried Salt. Both are genuinely useful tools - they brought the configuration into version control, made state more explicit, handled idempotent runs reasonably well. But they’re scaffolding over a fundamentally mutable OS. Ansible describes actions to take, not the state to achieve. The playbook says “ensure this package is installed” and “write this file,” but it says nothing about what happens to the thousand other files the system has accumulated over the years. You run the playbook, the system mostly matches what you expect, and you’re left with a lingering uncertainty: what else is in there?

Building a WireGuard mesh with Salt illustrates the problem. You write YAML templates, manage state files, think carefully about idempotency edge cases, and you still have hosts where salt-minion mysteriously fell out of sync six months ago and nobody noticed. The tool is fighting the problem. You’re layering abstraction on top of a mutable foundation and hoping the accretion doesn’t bite you.

NixOS is different in kind, not just in degree. The whole system is declared as a pure function over text files. Run the same config on two machines and you get the same system. Roll back to yesterday’s config and you get yesterday’s system. The “what else is in there” question dissolves because the system is a direct consequence of what’s in the repo.

That framing clicked for me when I started this config, now covering a dozen machines across servers, desktops, a gaming rig, and a VR setup. This isn’t a single desktop experiment. It’s a homelab, and the whole thing lives in one git repository.

What NixOS Actually Is

The central insight: your entire NixOS system is a pure function. Given the same inputs - configuration files, package hashes, module options - you get the same, bit-for-bit reproducible system. Not “mostly the same.” The same.

Three consequences fall out of this:

Reproducibility. If a coworker asks for the same toolchain you use, you share a Nix expression instead of a wiki page. It works the first time.

Rollback. Every nixos-rebuild switch creates a new system generation. If the new kernel panics at boot, you select the previous generation from the bootloader menu. The old generation is still there on disk, unchanged. You lose nothing.

Fearless experimentation. Want to try a different init setup, a new compositor, a service you’ve never configured before? Add it, switch, try it. If it’s wrong, roll back. The cost of experimentation drops to nearly zero.

It’s worth being clear about what NixOS is not. It’s not just a package manager (though it has one). It’s not Docker - there are no containers or images involved in basic NixOS usage, and your system packages live in the Nix store, not a layered filesystem. It’s not Ansible - there are no playbooks, no target hosts, no SSH connections during a normal rebuild. The system is built from a description; it doesn’t describe operations to run against a running system.

The honest note on the learning curve: Nix the language is genuinely strange. It’s lazy and functional, variables are declarations rather than assignments, and error messages can be opaque. The first two weeks feel like reading documentation written for people who already understand it. That phase ends, and what’s on the other side is worth it. But it is a real ramp.

The Flake: One Entry Point for Everything

A flake is Nix’s mechanism for pinned, reproducible builds. A flake.nix declares what your project depends on (inputs) and what it produces (outputs). The flake.lock file pins every input to an exact commit hash and content hash, giving you a complete bill of materials for your infrastructure.

The inputs section of this config shows the pinning strategy clearly:

inputs = {
  nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";
  home-manager = {
    url = "github:nix-community/home-manager";
    inputs.nixpkgs.follows = "nixpkgs";
  };
  sops = {
    url = "github:Mic92/sops-nix";
    inputs.nixpkgs.follows = "nixpkgs";
  };
  disko = {
    url = "github:nix-community/disko";
    inputs.nixpkgs.follows = "nixpkgs";
  };
  hyprland = {
    url = "github:hyprwm/Hyprland/v0.54.2";
    inputs.nixpkgs.follows = "nixpkgs";
  };
  hy3 = {
    url = "github:outfoxxed/hy3?ref=hl0.54.2";
    inputs.hyprland.follows = "hyprland";
  };
  lanzaboote = {
    url = "github:nix-community/lanzaboote";
    inputs.nixpkgs.follows = "nixpkgs";
  };
};

Notice two things: inputs.nixpkgs.follows = "nixpkgs" propagates throughout, ensuring all these tools share one nixpkgs version rather than each pulling their own. And Hyprland is pinned to v0.54.2 with hy3 at the matching hl0.54.2 ref - intentional, because the Hyprland plugin API changes between releases.

For that last point, pinning isn’t enough if you need the pinned version available system-wide. The overlay pattern solves it. An overlay in Nix is a function that takes the final and previous package sets and returns overrides. The hyprland overlay replaces the nixpkgs hyprland package with the pinned one:

hyprlandOverlay = final: prev:
  let
    hyprlandPkgs = inputs.hyprland.packages.${system};
    hy3Pkgs = inputs.hy3.packages.${system};
    hyprlandPkg = hyprlandPkgs.hyprland or hyprlandPkgs.default;
    hy3Pkg = hy3Pkgs.hy3 or hy3Pkgs.default;
  in
  {
    hyprland = hyprlandPkg;
    hyprland-unwrapped = hyprlandPkgs.hyprland-unwrapped or hyprlandPkg;
    hyprlandPackages = hyprlandPkgs;
    hyprlandPlugins = (prev.hyprlandPlugins or { }) // { hy3 = hy3Pkg; };
    hy3 = hy3Pkg;
  };

This overlay gets applied in nixpkgs.overlays, so every module in the entire config that references pkgs.hyprland gets the pinned version. No special imports, no passing versions around - the whole system agrees on one build.

The most satisfying part of the flake setup is how new hosts appear automatically. There’s no host registration, no list to update, no imports to add:

hostDirs = lib.attrNames (lib.filterAttrs (_: v: v == "directory") (builtins.readDir ./hosts));
mkHosts = lib.genAttrs hostDirs;

builtins.readDir ./hosts returns an attrset mapping names to types ("directory", "regular", etc.). Filter to directories, take the names, and generate a NixOS configuration for each one. Every directory inside hosts/ automatically becomes a machine. Add a folder, add a default.nix inside it, run nixos-rebuild or deploy - that’s it.

Roles: One Module System for 12 Different Machines

Twelve machines sound like twelve configs to maintain. They’re not. Machines are variations on a theme, and the theme is expressed as roles.

The role system starts with a typed option in modules/default.nix:

options.ox.roles = mkOption {
  default = [ "desktop" ];
  type = with lib.types; listOf (enum [ "desktop" "laptop" "server" "games" "vr" "nas" ]);
  description = ''
    The machine roles.
  '';
};

options.ox.role =
  lib.genAttrs [ "desktop" "laptop" "server" "games" "vr" "nas" ] (name:
    mkOption {
      type = lib.types.bool;
      readOnly = true;
      default = lib.elem name config.ox.roles;
      description = "Whether this machine has the ${name} role.";
    }
  ) // {
    enduser = mkOption {
      type = lib.types.bool;
      readOnly = true;
      default = config.ox.role.desktop || config.ox.role.laptop;
      description = "Whether this machine has a desktop or laptop role.";
    };
  };

ox.roles is a list you set per-host: ["server"], ["desktop" "games" "vr"], whatever the machine is. ox.role derives boolean attributes from that list automatically - role.server, role.desktop, and so on - plus the computed role.enduser which is true for any machine a human sits in front of.

Those booleans then drive defaults for every feature in the system:

ox = {
  auth.enable = lib.mkDefault true;
  boot.enable = lib.mkDefault true;
  samba.enable = lib.mkDefault role.enduser;
  bluetooth.enable = lib.mkDefault role.laptop;
  printing.enable = lib.mkDefault role.enduser;
  gui.enable = lib.mkDefault role.enduser;
  k8s.enable = lib.mkDefault (role.desktop || role.laptop || role.server);
  k8s.server.enable = lib.mkDefault role.server;
  ffmpeg.enable = lib.mkDefault role.enduser;
  hyprland.enable = lib.mkDefault role.enduser;
  packages.nixEnable = lib.mkDefault role.enduser;
  ssh.enable = lib.mkDefault true;
};

When a machine is a server, gui.enable, hyprland.enable, bluetooth.enable, printing.enable all default to false. When it’s a laptop, bluetooth turns on. When it’s an enduser machine, you get Hyprland, Samba mounts, printing, and the Nix development packages. All without writing those conditions in twelve separate host configs. mkDefault means any individual host can still override a specific option - the defaults are overridable, not forced.

Look at the full brunnen host config, a production server:

ox = {
  hostName = "brunnen";
  boot = {
    type = "grub";
    efi.enable = false;
  };
  roles = [ "server" ];
  networking.enable = true;
};

That’s it. Eight lines of meaningful configuration plus boilerplate imports. The role system handles the rest - no GUI, no Bluetooth, no printing, no Hyprland, no gaming tools. Clean.

Now look at itw, the gaming desktop:

ox = {
  hostName = "itw";
  boot = {
    efi.dir = "/boot/EFI";
    cryptPart = "crypt";
  };
  roles = [ "desktop" "games" "vr" ];
  monitor.with4k = true;
  monitor.with4kFix = true;
  video = "nvidia";
  wayland.enable = true;
};

environment.etc.crypttab = {
  text = ''
    games /dev/disk/by-uuid/0292b609-...  /etc/luks-keys/games.key luks
  '';
};
fileSystems = {
  "/games" = {
    device = "/dev/mapper/games";
    fsType = "ext4";
  };
  "/mnt/win" = {
    device = "/dev/disk/by-partuuid/3d69d556-...";
    fsType = "ntfs";
    options = [ "defaults" "noauto" ];
  };
};

This is the most complex host in the repo. NVIDIA GPU, 4K display with DPI fixes, encrypted /games partition with a keyfile, dual-boot Windows on an NTFS mount, VR role on top of gaming. And the entire description still fits on a screen. The role system handles enabling Steam, Lutris, Gamescope, and related tools via role.games; the VR tools via role.vr; the Hyprland Wayland compositor via role.enduser. Setting video = "nvidia" injects the correct Wayland environment variables automatically - more on that in a moment.

That’s the full range of the fleet: from a minimal server to a maxed-out gaming and VR workstation, with nothing special needed in between.

home-manager: Your Dotfiles Are Config Too

NixOS manages the system - packages, services, kernel configuration, networking. home-manager extends the same declarative model into the user’s home directory. It manages dotfiles, user services, user packages, and shell configuration, all in the same language and the same git repo.

The modules/home.nix bridges the NixOS config and home-manager with one critical pattern: gconfig. When home-manager modules are evaluated, they need access to the top-level NixOS configuration to ask role questions. gconfig is passed as an extra special argument:

home-manager.extraSpecialArgs = {
  gconfig = config;
  inherit oxpkgs;
};

Now any home-manager module can ask gconfig.ox.role.enduser or gconfig.ox.wayland.enable and get the host-level answer. The home.nix uses this immediately:

ox.home.enable = mkDefault config.ox.role.enduser;

Servers don’t get a user home-manager setup at all. Desktops and laptops do.

The Emacs configuration is the best example of what this buys you. In a single declaration:

home = {
  packages = [ emacs emacsedit ];
  file.".emacs".source = ./dot_emacs;
  activation.emacs-config = lib.hm.dag.entryAfter [ "writeBoundary" ] ''
    [ -d "$HOME/git" ] || run mkdir -p "$HOME/git"
    [ -d "$HOME/git/emacs-config" ] || run ${pkgs.git}/bin/git clone \
      https://github.com/0x17de/emacs-config "$HOME/git/emacs-config"
    [ -f "$HOME/git/emacs-config/custom.el" ] || run touch \
      "$HOME/git/emacs-config/custom.el"
  '';
};
systemd.user.services.emacs-daemon = {
  Unit = {
    Description = "Emacs Daemon";
    After = [ "graphical-session.target" ];
  };
  Service = {
    Type = "notify";
    ExecStart = "${pkgs.emacs-pgtk}/bin/emacs --fg-daemon";
    ExecStop = "${pkgs.emacs-pgtk}/bin/emacsclient --eval '(kill-emacs)'";
    Restart = "on-failure";
    RestartSec = "2s";
  };
  Install = {
    WantedBy = [ "graphical-session.target" ];
  };
};

Four things at once: installs emacs-pgtk and wrapper scripts (emacs and emacsedit), writes a .emacs bootstrap file, registers a systemd user service that starts the daemon after the graphical session is up, and clones the external Emacs configuration repository on first activation. Fresh machine, run the rebuild, open Emacs - everything is there.

Where to draw the line: Nix manages the install, the service lifecycle, and the initial bootstrap. The Emacs config itself (keybindings, packages, themes) lives in its own git repository and manages itself through package.el or use-package. Nix doesn’t try to own what the application is good at managing.

The Desktop Stack

Hyprland and the Overlay Pattern

Hyprland’s Wayland compositor is pinned to v0.54.2 across the whole fleet. This is intentional - the plugin API changes with releases, and the hy3 tiling plugin must match. The overlay from Section 3 means every machine that runs Hyprland gets the same version without any per-host specification.

Monitor configuration is generated per-host from ox.hyprland.monitors. The itw host sets monitor.with4k = true and monitor.with4kFix = true, which triggers the correct scaling and DPI configuration in the Hyprland module. When ox.video = "nvidia", the module automatically injects the Wayland-specific NVIDIA environment variables (GBM_BACKEND, __GLX_VENDOR_LIBRARY_NAME, LIBVA_DRIVER_NAME, and the rest). No hunting through Arch Wiki pages at 2am for the magic env var combination. Declare the GPU, get the right environment.

Rebuilding the Running System

The most-used command in the workflow is nho, defined in modules/nushell/config/nix.nu:

def nho --wrapped [...rest] {
    cd ~/git/nix
    nh os switch . ...$rest
}

nh is a wrapper around nixos-rebuild that adds better output formatting, diff reporting, and some quality-of-life features. This alias changes into the repo directory and invokes nh os switch - rebuild from the current flake, apply immediately. The --wrapped flag in Nushell passes extra arguments through unchanged, so nho --hostname other-machine works for remote deploys.

Make a config change, type nho, the system rebuilds and switches atomically. If something is wrong, the previous generation is one bootloader selection away. That feedback loop changes how you interact with your configuration.

Nushell as Primary Shell

Nushell runs as the primary interactive shell on all enduser machines. Its typed pipeline model, structured data handling, and built-in table rendering make it particularly good for homelab work. The config includes a suite of plugins: polars for dataframe operations, query for SQL-like filtering, net for network diagnostics, formats for parsing structured file formats. These are declared in the nushell module and activated automatically.

Gaming on NixOS

NixOS has a reputation for being difficult for gaming. That reputation is stale.

Steam, Lutris, Heroic Games Launcher, ProtonUp-Qt, MangoHUD, and Gamescope are all declared in the games module, enabled automatically when role.games is true. Steam’s FHS compatibility layer handles most Windows titles without any special configuration. Proton versions are managed via ProtonUp-Qt.

The occasional game that absolutely requires an FHS environment - a proprietary launcher that hardcodes library paths, for instance - can be wrapped in a custom buildFHSEnv expression. That’s about ten lines of Nix and not a deep rabbit hole. nix-alien was tried and found unreliable; the buildFHSEnv approach is more transparent and predictable.

NVIDIA gaming is handled by the video = "nvidia" flag described above. On itw, Gamescope handles frame pacing and fullscreen correctly for both native Linux titles and Windows games via Proton.

The role system means gaming tools are only present on machines that declare role.games. Servers stay lean. Desktops used for work but not gaming skip the Steam runtime and all the associated tooling. Nothing to manage, nothing to drift.

Secrets Without Headaches: SOPS + Age

The natural objection to “everything is in git”: what about passwords?

The answer is SOPS with age keys. Secrets are encrypted in the repository - committed plaintext is never stored - and decrypted at activation time by the sops-nix NixOS module. Each secret is mounted as a file with explicit ownership and permissions.

The PowerDNS module shows this concretely. The primary DNS server needs an API key, and that key must be accessible to the service at runtime but must never appear in plaintext in git or the Nix store:

config = mkIf cfg.enable {
  sops.templates."pdns-env" = lib.mkIf (!cfg.secondary) {
    content = ''
      PDNS_API_KEY=${config.sops.placeholder."powerdns/api-key"}
    '';
    owner = "pdns";
    group = "pdns";
    mode  = "0400";
  };

  services.powerdns = {
    enable = true;
    extraConfig = baseExtraConfig;
    secretFile = lib.mkIf (!cfg.secondary) config.sops.templates."pdns-env".path;
  };
};

config.sops.placeholder."powerdns/api-key" is a reference to an encrypted value in secrets/. During activation, sops-nix decrypts it and writes the rendered template to a path in /run/secrets/, owned by pdns:pdns, mode 0400. services.powerdns.secretFile tells the service to load environment variables from that path. The API key never appears in the Nix store. The secret file in secrets/ is committed encrypted with age - anyone with the host’s age key (generated at first boot, stored in /etc/ssh/ssh_host_ed25519_key) can decrypt it; everyone else sees ciphertext.

This pattern applies uniformly to database passwords, API keys, WireGuard private keys, and anything else that shouldn’t be readable by arbitrary processes. The secrets/ directory lives in the same repo as everything else. The repo truly is the single source of truth.

Deploying a New Machine in One Command

New machine provisioning follows a single script:

nix run github:nix-community/nixos-anywhere -- \
  --flake .#"$NAME" "$REMOTE" \
  --generate-hardware-config nixos-generate-config \
  "./hosts/${NAME}/hardware-configuration.nix"

nixos-anywhere SSHs into a running NixOS installer (booted from a USB stick or network image), generates hardware configuration for the specific machine, partitions disks according to the host’s disko.nix, and installs the full system from the flake. The only prerequisites are a host config in hosts/$NAME/ and SSH access to the target.

Disk layout is declared per-host in disko.nix. LUKS encryption, partition sizes, filesystem types, subvolumes - all expressed as Nix. The itw host has an encrypted system partition, a separate encrypted /games partition with a keyfile, and an NTFS Windows partition, all described in code and applied automatically during provisioning.

After the initial bootstrap, all future updates are nho - atomic, with rollback available from the bootloader. The setup cost is one remote command; the ongoing cost is nearly zero.

The PowerDNS setup illustrates what this means at scale. Two servers - schere (primary) and stein (secondary) - run the DNS infrastructure. Both configs live in the same repo. Adding or changing a DNS-related NixOS option is one file edit, then two deploys: nho --hostname schere and nho --hostname stein. No config management layer to synchronize, no state to reconcile, no “did I update both servers?” uncertainty.

The Hard Parts

In the interest of accuracy: NixOS has rough edges.

The language. Nix is lazy and functional, with an evaluation model that can surprise you. Error messages have improved significantly in recent Nix versions, but a type mismatch deep in a module evaluation can still produce a backtrace that takes time to interpret. The learning curve is front-loaded; it gets easier, but the first weeks are real.

nixos-unstable breakage. This config tracks nixos-unstable, which means nix flake update occasionally introduces a broken package or module API change. The fix is usually in nixpkgs within a day or two, and rollback handles the immediate problem - but it does happen. The stable channels are calmer.

Proprietary FHS binaries. Most software works fine. Occasionally a binary with hardcoded library paths needs a buildFHSEnv wrapper. It’s ten lines of Nix and not scary once you’ve done it once, but it’s a known rough edge.

First-time secure boot setup. Getting lanzaboote (Secure Boot), LUKS encryption, and disko all working correctly on first install has a real setup cost. The pieces fit together well once configured, but the first time through the documentation takes effort.

Honest scope assessment. If you have one machine and enjoy imperative tinkering, the learning investment may not be worth the return. NixOS pays off at scale, for reproducibility across machines, or for users who want the peace of mind that comes from knowing exactly what’s on each system. Single-machine casual users are not the target audience.

Where This Leads

A year ago, provisioning a new server meant: boot the installer, partition manually, install the base system, copy configs from another machine, run Ansible, debug what drifted, repeat. With this setup, it’s ./install-remote.sh newhost root@192.168.1.x and wait. The system that comes up is defined entirely by the config in git, with no variation possible.

That mental shift - from “the system is the result of what I’ve done to it” to “the system is a function of what I’ve declared” - is the core of what NixOS offers.

A few threads worth pulling on from here:

disko and LUKS in depth. The declarative disk setup deserves its own writeup. Combining disko, LUKS, and lanzaboote (Secure Boot) on a single machine, from a declarative base, is a solved problem that’s worth documenting step by step.

Writing your own NixOS module. The ox.* namespace in this repo is a collection of custom modules. The pattern - options, defaults, conditional config - is learnable in an afternoon and immediately useful for your own abstraction layer.

home-manager on non-NixOS. The home-manager configuration described here works on any Linux (or macOS) system, not just NixOS. If NixOS on bare metal is too much commitment, running home-manager standalone on Debian or Ubuntu gets you the dotfile management and reproducible user environment without changing the OS.

Where to start: nixos.org → nix.dev → find a real flake on GitHub and read it → start with one machine.

The whole system is a function. Write better inputs; get a better system. That’s the deal.

The Problem

My document workflow starts with my scanner, which connects to my network via Samba and deposits scanned PDFs into an INBOX folder. I needed a lightweight but effective way to:

1. Navigate through my organized document folders
2. View and examine PDF documents clearly
3. Name files consistently (I prefer the YYmmdd_description format)
4. Move files to their permanent location quickly

After trying several alternatives like ecoDMS, Paperwork, and Paperless-ngx, I found they didn’t quite match my specific workflow needs or added complexity I didn’t want.

DocMan: The Solution

DocMan is a focused Qt application that serves a single purpose extremely well - helping me move documents from my scanner’s output folder to their proper home in my directory structure. The interface is clean and divided into three primary functional areas:

1. Directory Navigation

On the left side, there’s a hierarchical tree view of my document directories. This component:

• Uses a customized file system model that shows only folders (no files)
• Automatically expands folders when directories are loaded
• Displays only the directory name column, keeping the interface clean
• Has my document root folder hardcoded to /home/mh/mnt/user/mh/private/Haushalt

fsModel->setFilter(QDir::AllDirs | QDir::NoDotAndDotDot);
fsModel->setRootPath("/home/mh/mnt/user/mh/private/Haushalt");

2. PDF Viewer

The central component is a PDF viewer built on Poppler-Qt5, offering:

• Clear, high-quality rendering of PDF documents
• Adjustable zoom level via a slider control
• Support for multi-page PDFs with pagination controls
• Quick rendering of each page as you navigate
• Persistence of zoom level when switching between documents

void PdfViewer::renderPage()
{
    scene->clear();
    QImage image = currentDocument->page(currentPage)->renderToImage(currentZoom, currentZoom);
    auto pixmap = new QGraphicsPixmapItem(QPixmap::fromImage(image));
    scene->addItem(pixmap);
}

3. Filename Input

At the top, a simple input field allows me to quickly name each document:

• I enter my preferred YYmmdd_description format
• Pressing Enter commits the name and moves the file
• The field clears automatically, ready for the next document

void MainWindow::moveFile(const QString& name)
{
    if (targetPath != "")
    {
        QDir().rename(currentFilename, targetPath + name + ".pdf");
        nextFile();
    }
}

The Workflow in Action

When I need to process documents (which happens every now and then rather than daily), my routine follows these simple steps:

1. Paper documents go through my scanner, which saves them as PDFs to my INBOX directory on my Samba share
2. I launch DocMan from within the same network
3. The application automatically loads the first document from my INBOX
4. I examine the document using the viewer (zooming or paging through it as needed)
5. I navigate to the appropriate destination folder in the directory tree
6. I type a descriptive filename in my preferred format (e.g., “250508_electric_bill”)
7. Pressing Enter moves the file to the selected location and loads the next document

The efficiency comes from the tight integration of these components. I can process a stack of documents in minutes without context switching between applications.

Technical Implementation

DocMan is quite lean, implemented in C++ with Qt5. Here are some notable technical aspects:

• Uses Poppler-Qt5 for high-quality PDF rendering
• Employs a custom QFileSystemModel to show only directories
• Includes a specialized QLineEdit that captures Enter key presses
• Utilizes signals and slots for component communication
• The file list class automatically populates from my INBOX directory:

PdfList::PdfList(QObject *parent) : QObject(parent)
{
    files = QDir("/home/mh/mnt/user/mh/private/INBOX").entryList(QDir::Files);
}

Why Qt?

I chose Qt for a few practical reasons:

• I was already familiar with the framework, reducing the development time
• Poppler-Qt5 offered the best PDF rendering capabilities I could find
• Qt’s signal/slot mechanism made component communication straightforward
• The cross-platform nature meant I could run it on different systems if needed

Who Needs OCR Anyway?

One notable feature I deliberately omitted was OCR (Optical Character Recognition). While many document management systems emphasize searchable text, I’ve found that my simpler approach works perfectly for my needs. By using consistent naming conventions with dates and brief descriptions, I can easily locate documents later by browsing the logical folder structure or using basic file search tools.

For example, I know roughly when I received a particular utility bill and in which folder it should be, so finding “2504_electric_bill.pdf” is straightforward without needing to search through the document content itself.

Why Build a Custom Tool?

I could have used existing solutions, but creating DocMan gave me several advantages:

1. Perfect workflow fit - It does exactly what I need with no extra features
2. Speed - The dedicated interface makes document processing extremely efficient
3. Control - I can modify it whenever my requirements change
4. Simplicity - No complex setup, cloud dependencies, or maintenance headaches

Conclusion

Sometimes the best tool is one you build yourself. DocMan demonstrates that a focused application solving a specific workflow problem can be more effective than generic solutions. While it’s admittedly simple, it saves me time whenever I need it by streamlining an occasional but necessary task.

What’s been most surprising is its longevity—I built this tool several years ago, and it continues to serve me perfectly without requiring any maintenance or updates. In a world of constantly changing software, there’s something deeply satisfying about a purpose-built tool that just keeps working.

The greatest productivity gains often come not from complex systems but from reducing friction in common workflows. By transforming document management from a chore into a quick, smooth process, DocMan helps me focus on the content of my documents rather than the mechanics of organizing them—which is especially valuable when dealing with the mountain of paperwork that seems to be Germany’s national hobby!

A Final Note on Physical Document Storage

What happens to all those physical papers after scanning? My approach is refreshingly straightforward: they go into cardboard boxes and get archived away. No complex filing cabinets, no elaborate categorization systems—just simple storage.

If I ever need the original paper document, I know roughly where to find it based on when I scanned it. But here’s the thing: I’ve rarely had to retrieve the physical copies. The digital system works so well that the originals become just a backup, safely tucked away without needing to be meticulously organized.

This pragmatic approach speaks to an important principle: optimize the parts of your workflow that matter most. For me, that’s quick processing and reliable digital retrieval—not creating a perfect physical filing system that I’ll rarely use. Sometimes the best solution isn’t the most elaborate one, but the one that solves your actual problems with minimal overhead.

Today I’m dusting off this experiment to share how it works and reflect on how modern C++ features could simplify this approach even further.

The Goal: Declarative XML Handling

The primary goal of this project was to create a system where:

1. You could define an XML structure once, using a declarative syntax
2. The same definition would handle both parsing and serialization
3. Validation would be built-in (required vs. optional elements)
4. The code would be type-safe and leverage compile-time checking

What I wanted to avoid was the typical approach where you write separate code for parsing, validation, and serialization—an approach that often leads to duplicated logic and inconsistencies.

For example, consider this XML document:

<root key="mykey">
  <data id="1">D1</data>
  <data id="2">D2</data>
</root>

In a traditional approach, you might write separate parsing code, validation logic, and serialization functions for this structure. But with this template-based approach, you define the structure once and get all these capabilities automatically.

Here’s a glimpse of what the final API looked like:

auto xml =
    "root"_node(
        Required(),
        "key"_attr(Required()),
        "client_id"_attr(),
        NodeList(
            "data"_node(
                "id"_attr(Required()),
                Text(Required()))));

With this single definition, you could:

• Parse XML strings into a structured NodeData object
• Validate that all required elements exist
• Serialize the data structure back to XML

How It Works: Template Metaprogramming Magic

The implementation relies heavily on several C++ template metaprogramming techniques:

1. The NodeData Structure

The core of the system is a generic NodeData structure that acts as an intermediate representation:

struct NodeData
{
    std::string name;
    std::string text;
    std::map<std::string, std::vector<NodeData>> subnodes;
    std::map<std::string, std::string> attributes;
};

This structure stores everything we need about an XML node: its name, text content, child nodes, and attributes.

2. Node Types and Traits

The system defines several node types that know how to interact with both XML and the NodeData structure:

• Node<name, Args...>: Represents an XML element
• Attribute<name, Args...>: Represents an XML attribute
• Text<Args...>: Represents text content within an element
• NodeList<SubNodeType, Args...>: Represents a list of similar child nodes

Each node type implements several key methods:

• subnode(): Retrieves the relevant part of an XML document
• validate(): Checks if the node meets requirements
• parse(): Extracts data from XML into NodeData
• serialize(): Writes data from NodeData back to XML

3. Type Deduction and User-Defined Literals

One of the key challenges in C++14 was that constructors couldn’t deduce template parameters from arguments. This limitation required a creative workaround using user-defined literals and builder classes:

template<class CharT, CharT... chars> auto operator""_node()
{
    static const char name[] = {chars..., 0};
    return NodeBuilder<name>();
}

This trick allows us to write "root"_node() which creates a NodeBuilder<"root"> that can then build a Node<"root", Args...> with the proper template parameters.

4. Variadic Templates for Complex Structures

Variadic templates allow us to handle arbitrary nesting and combinations of nodes:

template<class... Args>
struct is_required;

template<>
struct is_required<> : std::false_type { };

template<class Arg, class... Args>
struct is_required<Arg, Args...> : 
    std::conditional_t<std::is_same_v<Arg, Required>, 
                      std::true_type, 
                      is_required<Args...>> { };

This recursive template specialization checks if Required is among the arguments passed to a node, enabling us to handle validation elegantly.

Example in Action

Let’s see how the system handles various XML inputs:

auto examples = {
    "<wrong />"s,                                                            // Wrong root element
    "<root />"s,                                                             // Missing required attribute
    "<root key=\"mykey\" />"s,                                               // Valid minimal example
    "<root key=\"mykey\"><data id=\"1\" /></root>"s,                         // Missing required text
    "<root key=\"mykey\"><data id=\"1\">D1</data><data id=\"2\">D2</data></root>"s  // Fully valid
};

The system will:

1. Reject <wrong /> because it has the wrong root element name
2. Reject <root /> because the required “key” attribute is missing
3. Accept <root key="mykey" /> as a minimal valid document
4. Reject <root key="mykey"><data id="1" /></root> because the data node is missing required text
5. Accept the full example with two data nodes, each with their required attributes and text

Modern C++ Improvements

This code was written in the early days of C++17. If I were to revisit it today with C++20, several improvements would be possible:

1. Class Template Argument Deduction (CTAD)

C++17 introduced CTAD, which would eliminate the need for the NodeBuilder approach. With C++20 we could directly write:

// Instead of "root"_node(Required())
Node<"root">(Required())

2. String Literals as Template Parameters

C++20 allows string literals as template parameters, which would greatly simplify the implementation:

template<auto name>
class Node { /* ... */ };

// Usage:
Node<"root">

3. Concepts and Constraints

C++20 concepts would allow for more precise constraints on template parameters, making error messages clearer and improving compile times:

template<NodeConcept... Children>
class Node {
    // Implementation
};

4. std::visit and Variant

Using std::variant from C++17 could provide a more type-safe approach to handling different node types:

using NodeVariant = std::variant<Element, Attribute, Text>;

5. if constexpr

C++17’s if constexpr should eliminate the need for some of the template specializations:

if constexpr (is_required_v<Args...>) {
    // Handle required case
} else {
    // Handle optional case
}

Conclusion

This XML parsing experiment demonstrates the power of template metaprogramming in C++. By combining user-defined literals, variadic templates, and SFINAE, we can create a declarative API for XML handling that provides type safety, validation, and bidirectional conversion.

While the implementation might seem complex, it delivers significant benefits:

1. Write once, use for both parsing and serialization
2. Built-in validation enforced at runtime
3. Declarative syntax that mirrors the structure of XML
4. Type safety that catches errors at compile time

Modern C++ could make this implementation even more elegant and concise, but the core ideas remain valuable. Template metaprogramming allows us to create domain-specific languages within C++ that can dramatically reduce boilerplate and improve code reliability.

The next time you find yourself writing separate code for parsing, validating, and serializing a data format, consider whether a template-based approach might let you define the structure once and get all those operations for free.

See the full code example here

The Problem with Monolithic Configurations

Many Emacs users start with a single .emacs or init.el file. This works fine at first, but as your configuration grows to accommodate different programming languages, workflows, and packages, a single file becomes unwieldy. Common issues include:

• Hard-to-find settings when you need to make changes
• Difficult debugging when something breaks
• Challenging to selectively enable/disable features
• Poor organization making it hard to understand your own setup

A Modular Approach

Let’s explore a modular configuration structure based on my personal setup that addresses these issues.

Root Configuration File

The entry point for my configuration is _0x17de-emacs.el, which defines the base setup and loads other modules:

(setq initial-scratch-message nil
      ring-bell-function #'ignore)

(defvar _0x17de/load-path (file-name-directory (or load-file-name buffer-file-name))
  "Path of the _0x17de emacs config")

(defun _0x17de/load (paths)
  "Load files relative to this emacs config."
  (when (not (listp paths))
    (setq paths (list paths)))
  (dolist (path paths)
    (condition-case err
        (load (expand-file-name path _0x17de/load-path))
      (error (message "Failed to load _0x17de-config sub-library at %S: %S" path err)))))

The key here is the _0x17de/load function that loads modules with error handling, preventing a failure in one module from breaking your entire configuration.

Directory Structure

My configuration is organized into clear directories:

• init/ - Core settings (startup, UI, packages)
• langs/ - Language-specific configurations
• utils/ - General utilities and enhancements
• ext/ - External packages not available in package repositories

This structure makes it immediately clear where to find specific settings.

Core Initialization

The initialization sequence loads modules in logical groups:

(_0x17de/load
 '("./init/custom.el"
   "./init/speedup.el"
   "./init/package.el"
   "./init/encoding.el"
   "./init/gui.el"
   "./init/exwm.el"))

(_0x17de/load
 '("./utils/exec-path-from-shell"
   "./utils/minibuffer"
   "./utils/indention"
   ;; ... more utilities ... 
   ))

(_0x17de/load
 '("./langs/common"
   "./langs/ansible"
   "./langs/plantuml"
   ;; ... more languages ...
   ))

This approach lets you:

1. Control the order of initialization
2. Group related components
3. Easily comment out modules you don’t want to load

Graceful Error Handling

A crucial aspect of this setup is the error handling in the _0x17de/load function. If a module fails to load, it displays a message but continues loading other modules:

(condition-case err
    (load (expand-file-name path _0x17de/load-path))
  (error (message "Failed to load _0x17de-config sub-library at %S: %S" path err)))

This prevents a single module failure from breaking your entire Emacs experience.

Specialized Module Examples

Let’s examine a few module types to see how they encapsulate specific functionality.

Language Support

Language-specific modules handle everything needed for a particular language. For example, my Go configuration:

(use-package go-mode
  :defer t
  :bind
  (:map go-mode-map
        ([tab] . 'company-indent-or-complete-common)
        ([f1] . lsp-describe-thing-at-point)
        ([f12] . lsp-find-definition)
        ("S-<f12>" . lsp-find-references))
  :hook
  (go-mode . (lambda ()
               (setq tab-width 2)
               (setq company-backends '(company-capf
                                        company-files))
               (setq gofmt-command "goimports")
               (add-hook 'before-save-hook 'gofmt-before-save)
               (go-guru-hl-identifier-mode)
               (lsp-deferred)
               ;; ... more settings ...
               )))

This module:

• Loads only when editing Go files (:defer t)
• Sets up keybindings specific to Go files
• Configures language-specific tools and formatting
• Integrates with LSP for auto-completions and advanced IDE features

Utility Modules

Utilities enhance the core editing experience regardless of the file type you’re editing. For example, my multiple-cursors configuration:

(defun _0x17de/add-multiple-cursors-to-non-empty-lines (start end)
  "Add a cursor in each non-empty line of selection"
  (interactive "r")
  ;; ... implementation ...
  )

(use-package multiple-cursors
  :init
  (global-unset-key (kbd "M-<down-mouse-1>"))
  :bind
  (("C-M-z C-c" . mc/edit-lines)
   ("C-M-z C-M-c" . _0x17de/add-multiple-cursors-to-non-empty-lines)
   ("C-M-z >" . mc/mark-next-like-this)
   ;; ... more bindings ...
   ))

This module encapsulates a complete feature set with custom functions and keybindings.

Feature Flags

For optional features, I use custom variables as feature flags:

(defcustom _0x17de/use-exwm nil
  "Non-nil means EXWM will be enabled.
When this option is enabled, EXWM will be loaded and configured
as the window manager for this session."
  :type 'boolean
  :group '_0x17de)

(when _0x17de/use-exwm
  ;; ... EXWM configuration ...
  )

This allows toggling features without commenting out code or editing multiple files.

Benefits of This Approach

This modular approach offers several advantages:

1. Maintainability: Each module has a single responsibility
2. Performance: Lazy loading modules when needed
3. Portability: Easy to share modules between configurations
4. Resilience: Failures in one module don’t break everything
5. Clarity: Clear organization makes finding settings easier
6. Flexibility: Enable/disable features without editing code

Leveraging Built-in Emacs Utilities

Emacs provides several powerful utilities that make modular configurations easier to implement and maintain.

use-package: Package Management Made Simple

While not built into Emacs core (but bundled with Emacs 29+), use-package is essential for modular configurations:

(use-package python
  :ensure nil  ; Don't try to install built-in packages
  :defer t     ; Only load when needed
  :bind (:map python-mode-map
          ([f12] . lsp-find-definition)
          ("S-<f12>" . lsp-find-references))
  :hook ((python-mode . (lambda ()
                          (company-mode t)
                          (flycheck-mode t)
                          (lsp-deferred))))
  :config     ; Executed after the package loads
  (setq python-indent-offset 4))

Key use-package keywords:

• :ensure - Controls package installation
• :defer - Enables lazy loading
• :bind - Sets up keybindings (with automatic lazy loading)
• :hook - Adds mode hooks (with automatic lazy loading)
• :init - Code executed before loading
• :config - Code executed after loading
• :custom - Sets custom variables
• :commands - Creates autoloaded commands (callable via M-x or keybindings)

This declarative approach keeps package configuration concentrated in one place rather than scattered throughout your init file.

defcustom: User-Configurable Variables

defcustom creates proper user options with documentation, custom types, and integration with Emacs’ customization interface:

(defcustom _0x17de/python-global-virtualenv-dir "~/.venv"
  "Default directory for Python virtual environments.
This is used by pyvenv to locate and activate virtual environments."
  :type 'directory
  :group '_0x17de
  :safe #'stringp)

Benefits of defcustom over regular variables:

1. Type checking - The :type property validates values
2. Documentation - Self-documents the option
3. UI integration - Works with customize-* commands
4. Safety - The :safe property helps with security
5. Organization - Options can be grouped logically

define-minor-mode: Creating Toggleable Features

For modular features that can be enabled/disabled, define-minor-mode is perfect:

(define-minor-mode latex-compile-on-save-mode
  "Refresh the preview on save"
  :lighter " LTeXcos"
  :group latex-compile-on-save
  (if latex-compile-on-save-mode
      (add-hook 'after-save-hook 'latex-compile-on-save--compile nil t)
    (remove-hook 'after-save-hook 'latex-compile-on-save--compile t)))

This creates a toggleable mode with:

• A proper minor mode that can be enabled/disabled with M-x
• Automatic hook management when enabled/disabled
• Integration with the mode line via :lighter

eval-after-load: Targeted Configuration

For smaller configurations without use-package, eval-after-load provides similar lazy-loading capabilities:

(eval-after-load 'python-mode
  '(progn
     (define-key python-mode-map (kbd "C-c C-r") 'python-shell-send-region)
     (setq python-indent-offset 4)))

This defers execution until the specified feature is loaded, improving startup times.

with-eval-after-load: Modern Alternative

A more modern, macro-based version of eval-after-load:

(with-eval-after-load 'org
  (setq org-hide-emphasis-markers t)
  (add-hook 'org-mode-hook #'visual-line-mode))

This is cleaner than eval-after-load because it doesn’t require quoting the body.

autoload: Manual Lazy Loading

For maximum control over lazy loading:

;;;###autoload
(defun my/open-config-file ()
  "Open the main configuration file."
  (interactive)
  (find-file (expand-file-name "_0x17de-emacs.el" _0x17de/load-path)))

The ;;;###autoload cookie tells Emacs to make the function available without loading the entire file.

Implementation Tips

If you want to build a similar configuration:

1. Start small: Begin with a single file and add more as needed
2. Use consistent naming: Establish a convention for module files
3. Leverage use-package: For declarative, lazy-loaded package configuration
4. Add error handling: Ensure graceful recovery from failures
5. Document with defcustom: For user-facing configuration options
6. Use minor modes: For toggleable features
7. Embrace lazy loading: With autoload, eval-after-load, and with-eval-after-load

Conclusion

A modular Emacs configuration significantly improves maintainability and flexibility. By separating concerns into discrete modules with clear responsibilities, you can build a robust, personalized environment that evolves with your needs while remaining manageable.

This approach has served me well, allowing my configuration to grow from a few basic settings to a comprehensive development environment without becoming unwieldy. The time invested in proper organization pays dividends in the long term through improved stability and ease of modification.

Feel free to adapt this structure to your own needs or explore my complete configuration for more ideas and inspiration.

Hiera vs. Ansible’s host_vars and group_vars

One of the most significant differences between Puppet and Ansible is how they handle variable hierarchies. Ansible uses host_vars and group_vars directories, which provide a straightforward but somewhat rigid approach to configuration data. In contrast, Puppet’s Hiera offers a much more sophisticated system.

Why Hiera Shines

Hiera’s approach to configuration lookup is fundamentally different and more flexible:

1. Fact-based lookups: Hiera can use any fact about a system to determine which configuration files to load. This goes far beyond Ansible’s group-based approach, allowing for configurations based on operating system, hardware details, custom facts, or any combination thereof.

2. Hierarchical merging: Hiera doesn’t just stop at the first match like Ansible often does. Instead, it can merge data from multiple levels of the hierarchy, allowing for elegant inheritance patterns.

3. Lookup specificity: When Hiera finds a key at one level of the hierarchy, it doesn’t override it with values from less specific levels. This “first match wins” approach ensures that the most specific configuration always takes precedence.

This flexibility makes Hiera an incredibly powerful tool for managing configurations across diverse environments. For instance, you can define a base configuration that applies to all systems, override specific settings for a particular OS, and then further customize for individual hosts—all without repetition.

Hiera in Action: A Real-World Example

Let’s look at an actual Hiera configuration to understand its power (example taken from the official Puppet documentation):

---
version: 5
defaults:  # Used for any hierarchy level that omits these keys.
  datadir: data         # This path is relative to hiera.yaml's directory.
  data_hash: yaml_data  # Use the built-in YAML backend.

hierarchy:
  - name: "Per-node data"                   # Human-readable name.
    path: "nodes/%{trusted.certname}.yaml"  # File path, relative to datadir.
                                   # ^^^ IMPORTANT: include the file extension!

  - name: "Per-datacenter business group data" # Uses custom facts.
    path: "location/%{facts.whereami}/%{facts.group}.yaml"

  - name: "Global business group data"
    path: "groups/%{facts.group}.yaml"

  - name: "Per-datacenter secret data (encrypted)"
    lookup_key: eyaml_lookup_key   # Uses non-default backend.
    path: "secrets/%{facts.whereami}.eyaml"
    options:
      pkcs7_private_key: /etc/puppetlabs/puppet/eyaml/private_key.pkcs7.pem
      pkcs7_public_key:  /etc/puppetlabs/puppet/eyaml/public_key.pkcs7.pem

  - name: "Per-OS defaults"
    path: "os/%{facts.os.family}.yaml"

  - name: "Common data"
    path: "common.yaml"

This configuration demonstrates several key strengths of Hiera:

1. Multiple layers of specificity: The hierarchy starts with node-specific data (nodes/%{trusted.certname}.yaml), then moves through various levels of abstraction (datacenter + business group, business group alone, datacenter-specific secrets), down to OS-specific settings, and finally to common data that applies to all systems.

2. Dynamic path generation: Notice how paths include variables like %{trusted.certname}, %{facts.whereami}, and %{facts.os.family}. These are interpolated with actual system facts at runtime, creating a dynamic configuration system that adapts to each node.

3. Mixed backend support: While most data comes from YAML files, the configuration demonstrates how you can use different backends (like the encrypted eyaml_lookup_key) for sensitive data, with backend-specific options.

4. Clear progression: When Hiera looks up a value, it starts at the top of this hierarchy and works its way down. Once it finds a value, it stops looking (unless you’re using a merge strategy). This means more specific configurations naturally override more general ones.

In Ansible, achieving this level of configuration flexibility would require complex variable precedence rules and possibly custom plugins. Hiera makes it declarative and straightforward.

Puppet Bolt: A Fresh Approach to Remote Execution

Puppet Bolt is another gem in the Puppet ecosystem that offers an interesting alternative to Ansible for ad-hoc tasks and orchestration.

Speed and Flexibility

Bolt takes a different approach to remote execution that brings both advantages and limitations:

• Script-based execution: While Ansible relies heavily on Python modules, Bolt embraces a more universal approach by simply executing scripts on remote systems. This might seem less sophisticated at first, but it brings remarkable flexibility.

• Language agnosticism: Bolt doesn’t care what language your script is written in. Bash, Python, PowerShell, Ruby—it’s all the same to Bolt. This removes the dependency on specific Python versions that can sometimes complicate Ansible deployments.

• Simplicity in execution: Sometimes a simple bash script is the easiest solution to a problem. Bolt acknowledges this reality and makes it trivial to run whatever makes the most sense for a given task.

Limitations and Considerations

Despite its strengths, Bolt isn’t without challenges:

• JSON string conversion: One frustration I’ve encountered is that converting JSON strings to objects isn’t natively supported. You either need to use modules or extend Bolt with custom functions. If you’re not interested in diving into Ruby or managing additional module dependencies, this can be a drawback.

• Less granular modules: We do lose some of the fine-grained task-level module support that Ansible provides. This is the trade-off for Bolt’s flexibility.

Interactive Use and Documentation

Where Bolt really shines is in its usability:

• Standalone operation: Using Bolt without the broader Puppet ecosystem works perfectly fine, making it accessible even if you’re not fully committed to Puppet.

• Interactive experience: Bolt feels designed for more interactive use than Ansible, with features like the prompt::menu function that provides interactive dropdown options, allowing users to make selections directly in the terminal during plan execution.

• Superior plan discovery: The bolt plan show command provides a clean, organized list of all available plans. When viewing a specific plan, Bolt displays detailed documentation that’s easily added through comments in the code.

• Task targeting flexibility: Plans in Bolt are conceptually similar to Ansible playbooks, but with a key difference in execution model. In Bolt, each task within a plan can target a different set of hosts, giving you greater flexibility. This contrasts with Ansible, where a play defines a list of tasks that all run against the same list of hosts. This distinction makes Bolt plans more adaptable when you need different actions on different host groups within the same workflow.

  Here’s a simple example showing how Bolt plans can target different sets of hosts for different tasks:

  plan my_module::my_deployment(
    TargetSpec $web_servers,
    TargetSpec $db_servers,
    String $version
  ) {
    # First, update the database servers
    # Using catch_errors to prevent plan failure on error
    $db_results = run_task('database::update', $db_servers, { 'version' => $version }, _catch_errors => true)
  
    # Check if database update was successful
    if $db_results.ok {
      # Run a simple command on the web servers to verify they're online
      $web_status = run_command('systemctl status nginx', $web_servers)
  
      # Deploy the new application version to web servers
      run_task('webapp::deploy', $web_servers, {
        'version' => $version,
        'db_endpoint' => get_db_endpoint($db_results)
      })
  
      # Send success notification
      run_task('notification::send', 'monitoring.example.com', {
        'message' => "Deployment of version ${version} completed successfully",
        'status' => 'success'
      })
    } else {
      # Send failure notification with details
      $error_msg = $db_results.error_set.first.error.message
      run_task('notification::send', 'monitoring.example.com', {
        'message' => "Deployment of version ${version} failed during database update: ${error_msg}",
        'status' => 'failure'
      })
  
      # Log details for investigation
      out::message("Database update failed: ${error_msg}")
  
      # Raise an exception that the plan failed
      fail_plan("Deployment failed at database stage")
    }
  }
  

  Notice how run_task() and run_command() take a target parameter ($db_servers, $web_servers, or even a specific hostname), allowing each operation to be directed at precisely the right group of hosts. This syntax run_task($task_name, $targets, $args) makes the targeting explicit and flexible.

Plan Functions and Built-in Capabilities

Bolt comes with a rich set of built-in plan functions that extend its capabilities. The Bolt Plan Functions documentation provides a comprehensive reference of these functions, including file manipulation, error handling, and interactive prompts like prompt::menu. These functions make Bolt plans powerful and flexible without requiring external dependencies.

Parameter Handling and Variable Scope

The way Bolt handles parameters and variables feels more like proper programming:

• Function-like parameters: Plans have proper parameters, similar to functions in programming languages, with validation checks at the beginning of execution.

• Cleaner than Ansible’s -e: Compared to the common practice of injecting variables with -e in Ansible, Bolt’s approach feels more correct and maintainable. It asserts that all arguments are defined and used properly.

• Sensible variable scoping: Ansible’s variable precedence can sometimes lead to confusion, with -e variables overriding set_fact values in ways that might be unexpected. Bolt follows traditional programming scopes, where variables have clear context and shouldn’t be redeclared.

Inventory Handling

Like Ansible, Bolt supports dynamic inventories:

• Language flexibility: Again embracing its language-agnostic approach, Bolt allows you to write dynamic inventories in any programming language. The only requirement is that they return properly formatted JSON.

• Task-based inventory sources: Bolt makes it incredibly simple to use a task as an inventory source. As documented in the Bolt inventory reference, you can create a dynamic inventory by simply adding a _plugin: task key at the root level of your inventory file, and then specifying a task key with the name of a module::task to execute. This task returns the inventory data, making it easy to generate target lists from external sources like cloud providers, CMDBs, or custom databases.

  Here’s a minimal example:

  # inventory.yaml
  ---
  _plugin: task
  task: my_module::get_targets
  

  This simple configuration tells Bolt to execute the my_module::get_targets task and use its output as the inventory for the current run.

Conclusion

While Ansible remains a popular and powerful tool, my experience with Puppet’s Hiera and Bolt has revealed significant advantages in certain scenarios. Hiera’s sophisticated data hierarchy provides exceptional flexibility for configuration management, while Bolt offers a refreshing approach to remote execution that balances simplicity with power.

If you’re deep in the Ansible ecosystem, it’s still valuable to understand what benefits other tools can bring. Staying open-minded about different approaches helps broaden your technical perspective, can lead to fresh ideas for solving problems, and ultimately makes you a more versatile infrastructure developer. Even if you don’t switch tools, exploring alternative design philosophies can inspire creative solutions and innovations in your own workflows and automation strategies.

While many configuration management tools like Ansible, Chef, or Puppet offer similar functionality, SaltStack distinguishes itself through its powerful event-driven architecture, efficient minion communication, and exceptional templating capabilities. These features make it particularly well-suited for managing complex, dynamic infrastructure at scale.

In this article, I’ll show you how to make SaltStack more pleasant to use, establish a proper development workflow for modules, and maintain a clean main branch while doing so.

Integrating Git with Salt for Better Version Control

Imagine having your entire infrastructure configuration tracked, versioned, and deployable with the same tools you use for application code. This isn’t just possible with Salt—it’s seamless.

Instead of manually copying files to your Salt master or using rsync jobs, GitFS allows you to use Git repositories directly as your source of truth for states and pillar data. This means automatic versioning, change tracking, and the ability to roll back to any previous state of your infrastructure. For configuration options and credential management, see the official documentation on using GitFS.

# Salt master configuration for GitFS integration
fileserver_backend:
  - gitfs                       # Enable the GitFS backend
gitfs_provider: gitpython       # Use GitPython for Git operations
gitfs_base: main                # Default branch to use
gitfs_update_interval: 60       # Check for updates every 60 seconds
git_pillar_provider: gitpython  # Use GitPython for pillar as well
git_pillar_update_interval: 60  # Check for pillar updates every 60 seconds
git_pillar_base: main           # Default branch for pillar data

gitfs_remotes:
   - ssh://git@gitlab.example.com/username/salt-control.git:
     - root: state              # Root directory for state files in the repo
ext_pillar:
  - git:
    - __env__ ssh://git@gitlab.example.com/username/salt-control.git:
      - root: pillar            # Root directory for pillar data in the repo

With this setup, your entire infrastructure becomes a Git repository. Every change is tracked. Every deployment is versioned. Every rollback is a simple git revert away.

Understanding the __env__ Parameter

The __env__ parameter in the Git pillar configuration serves an important role in environment management. When Salt executes, it replaces __env__ with the current environment name (like “prod”, “dev”, or a Git branch name), automatically ensuring your pillar data matches your state files.

This parameter enables consistent environment isolation by:

• Matching pillar data to the same environment as your state files
• Preventing accidental deployment of development configurations to production
• Enabling branch-specific pillar data that stays synchronized with your state files

For example, when you run salt '*' state.apply saltenv=feature-branch, the __env__ parameter ensures that pillar data from the feature-branch branch is used, maintaining perfect consistency between your states and configuration.

This environment separation is crucial for maintaining a reliable configuration deployment process and preventing environment-specific data from being applied to the wrong targets.

Overcoming GitFS Synchronization Challenges

While GitFS is powerful, it’s important to note that synchronization can be challenging. Salt has a documented issue (Salt Issue #66793) that makes it difficult to update the fileserver from Salt orchestration.

I’ve developed a custom solution that addresses this issue without requiring any patches to Salt’s code. Here’s my implementation, which is derived from Salt’s own fileserver.py update function but modified to work in orchestration:

# /my/salt/repo/states/_runners/fileserver_hotfix.py

import salt.fileserver

def update():
    """
    Update the Salt fileserver cache.
    """
    my_opts = dict(__opts__)
    my_opts.pop('__pub_user', None)  # Remove the problematic key
    fs = salt.fileserver.Fileserver(my_opts)
    fs.update()
    return True

This simple module removes the __pub_user key that causes the issue. After committing and pushing this file, synchronize your GitFS:

salt-run fileserver.update
salt-run saltutil.sync_runners

Now, you can create an orchestration state file that handles the complete synchronization process:

# /my/salt/repo/states/orch/sync_all.sls

update_fileserver:
  salt.runner:
    - name: fileserver_hotfix.update

update_git_pillar:
  salt.runner:
    - name: git_pillar.update
    - require:
      - salt: update_fileserver

sync_all_modules:
  salt.function:
    - name: saltutil.sync_all
    - tgt: '*'
    - tgt_type: glob
    - require:
      - salt: update_git_pillar

refresh_pillar:
  salt.function:
    - name: saltutil.refresh_pillar
    - tgt: '*'
    - tgt_type: glob
    - require:
      - salt: sync_all_modules

update_mine:
  salt.function:
    - name: mine.update
    - tgt: '*'
    - tgt_type: glob
    - require:
      - salt: refresh_pillar

After committing and synchronizing again, you can run this orchestration to update everything:

salt-run state.orchestrate orch.sync_all

This approach ensures that your GitFS, pillar data, and mine information stay in sync, resolving one of the more challenging aspects of working with Git and Salt.

Note that in high-availability setups with multiple Salt masters, you may need to adjust this approach to ensure all masters are synchronized properly. Consider adding a salt-run command that targets all masters in such scenarios.

Repository Organization: Single vs. Multiple

“Should I keep my states and pillars in the same repository or separate them?” (See Salt’s best practices for more guidance.)

This is a common question in the Salt community, and there are compelling arguments for both approaches:

The Single-Repository Approach

When your states and pillars live together:

• Changes to your infrastructure and its configuration happen in one atomic commit
• Your Git history tells a complete story of how your infrastructure evolved
• You can make sweeping changes across your entire system without fear of misalignment
• Testing becomes dramatically simpler since everything moves together

Example scenario: A team managing a consistent application stack across multiple environments (development, staging, production) would benefit from a single repository. When updating the application’s configuration, they can change both the state files (how the application is installed and configured) and the pillar data (environment-specific variables like database credentials) in a single commit, ensuring everything stays in sync.

When to Use Multiple Repositories

• When different teams own the infrastructure code versus the configuration data
• When your security requirements demand stricter access controls for sensitive data
• When your pillar data changes frequently, but your states are relatively stable

Example scenario: An organization where the security team manages credentials and sensitive configuration while the operations team manages infrastructure code. In this case, keeping pillar data in a separate repository with stricter access controls allows the security team to update credentials without requiring changes to the infrastructure code, while still leveraging the same deployment mechanisms.

The beauty of Salt is that it supports both approaches equally well. You can start with a single repository and split later if needed, or vice versa.

Effective Testing with SaltEnv

Here’s where Salt really shines compared to other configuration management tools: the ability to test changes in complete isolation using Git branches.

Before: Traditional Testing Workflow

1. Develop changes locally
2. Push to testing environment
3. Test changes
4. If issues arise, revert changes or fix in place
5. Deploy to production

After: Salt Branch-Based Testing Workflow

1. Create a feature branch in your Git repository
2. Develop and commit changes
3. Push the branch to your Git remote
4. Test using the branch name as the environment:

   salt '*' state.apply saltenv=new-feature
   

5. Iterate on the branch until everything works
6. Merge to main branch only when fully tested

This command tells Salt to use the new-feature branch for both your state files and pillar data, completely isolated from your production environment. It’s like having a parallel universe where you can experiment freely without fear of breaking production.

For even more focused testing, you can apply a single state file instead of running the entire top.sls configuration:

salt '*' state.sls mystate saltenv=new-feature

This allows you to test only the specific component you’re modifying, making the testing process faster and more targeted.

The Test Mode: Simulate Before You Apply

One of Salt’s most valuable features for testing is the test=true parameter. This parameter performs a dry run of your state execution, showing you exactly what would change without actually making any modifications to your systems.

salt '*' state.sls mystate test=true

With test=true, Salt will:

• Connect to your target systems
• Load all state files and pillar data
• Check current system state against desired state
• Report what would change (added, modified, removed)
• Exit without making any actual changes

For more complex state files, you can combine test=true with your branch testing:

salt '*' state.sls mystate saltenv=new-feature test=true

This powerful combination lets you:

1. Test against an isolated Git branch without affecting production code
2. Simulate the execution without making actual changes
3. Verify that your states target the correct systems
4. Confirm that the changes match your expectations

The test mode is particularly valuable when working with destructive operations or when deploying to critical production systems, as it provides an extra layer of verification before committing to changes. Once you’re confident in the changes, you can run the same command without the test=true parameter to apply them for real.

With pillarenv_from_saltenv: True in your configuration, Salt automatically keeps your test data synchronized with your test code, ensuring consistent testing across environments.

This approach allows for thorough testing of complex infrastructure changes before merging to the main branch, significantly reducing the risk of issues in production environments.

Essential Debugging Tools for Salt States

Ever wished you could see exactly what Salt is thinking? You can, with these debugging superpowers:

show_sls - Peek Behind the Curtain

salt '*' state.show_sls my_state

This reveals the raw SLS data structure after Salt has processed it, letting you verify your states are defined correctly.

Example output:

my_host:
    ----------
    nginx:
        ----------
        __env__:
            base
        __sls__:
            nginx
        pkg:
            |_
              ----------
              name:
                  nginx
            - installed
            |_
              ----------
              order:
                  10019

show_low_sls - See the Final Plan

salt '*' state.show_low_sls my_state

The “low state” is Salt’s final internal representation after all preprocessing is done. It’s especially useful when debugging complex states with multiple includes or extensive Jinja templating because it shows exactly what Salt will execute after all rendering and inheritance has been resolved.

Example output:

my_host:
    |_
      ----------
      __env__:
          base
      __id__:
          nginx
      __sls__:
          nginx
      fun:
          installed
      name:
          nginx
      order:
          10019
      state:
          pkg

show_pillar - Expose All Secrets

salt-run pillar.show_pillar my_host

This displays all pillar data available to a minion, which is crucial for tracking down missing or incorrect configuration values.

Example output:

my_host:
    ----------
    nginx:
        ----------
        port: 80
        worker_processes: 4

show_top - Understand the Hierarchy

salt '*' state.show_top

This displays which top files are applied to the minion and in what order, helping you untangle complex state inheritance.

Example output:

my_host:
    ----------
    base:
        - ssh
        - nginx

These commands have saved me days of troubleshooting. When something isn’t working as expected, I don’t have to guess—I can see exactly what Salt sees.

Practical Jinja2 Template Debugging

Jinja2 templates are powerful but can be frustrating when they don’t behave as expected. Here’s my secret weapon for debugging them:

# Interactive Python shell example for debugging complex Jinja2 templates
>>> import jinja2
>>> context = {
...     "users": [
...         {"name": "alice", "groups": ["admin", "dev"]},
...         {"name": "bob", "groups": ["dev"]},
...         {"name": "charlie", "groups": ["ops"]}
...     ]
... }
>>> template = jinja2.Template("""
... {% for user in users %}
... {% if 'admin' in user.groups %}
... {{ user.name }} is an admin
... {% endif %}
... {% endfor %}
... """)
>>> print(template.render(**context))

alice is an admin

Notice the extra blank lines in the output? This is where whitespace control becomes important, as we’ll see in the next section.

This approach lets you test complex templates outside of Salt, iterating quickly until you get exactly what you want. I’ve solved in minutes what would have taken hours of trial and error directly in Salt states.

Optimizing Whitespace in Jinja2 Templates

This might seem trivial, but proper whitespace handling in Jinja2 templates can make your Salt states dramatically more readable and maintainable.

Consider this example:

# Users
{% for user in users %}
{{ user.name }}
{% endfor %}

The rendered output would include an extra blank line before the first user:

# Users
(blank line here)
Alice
Bob
Charlie

However, with {%- syntax (note the dash):

# Users
{%- for user in users %}
{{ user.name }}
{%- endfor %}

The output is cleaner without the extra empty line:

# Users
Alice
Bob
Charlie

This might seem like a small detail, but when you’re working with complex, nested templates, proper whitespace control becomes essential for maintaining sanity.

Conclusion: Making Infrastructure Management More Efficient

After using Salt extensively, these features have transformed infrastructure management from a challenging task into something much more manageable and satisfying. The combination of Git integration, environment isolation, robust debugging tools, and a methodology for testing Jinja templating creates a process that’s not just efficient but actually enjoyable.

The next time you approach an infrastructure change, remember these Salt features. They can significantly improve your configuration management experience.

As DevOps practices continue to evolve toward more GitOps-focused workflows and infrastructure-as-code becomes the standard, mastering these Salt techniques positions you well for the future. The ability to test infrastructure changes in isolation while maintaining strict version control aligns perfectly with modern continuous integration and delivery practices.

Try them out, and you might find that Salt becomes an essential tool in your DevOps toolkit.

In this article, we’ll explore some of the fundamental concepts in Emacs Lisp that often confuse newcomers.

Variables: Global vs Local Scope

Elisp provides several ways to define variables, each with different scoping behaviors.

setq: Global Variables

The setq function is used to define and set global variables:

(setq my-variable "Hello, world!")

These variables are accessible from anywhere in your Emacs session once defined. They’re ideal for configuration settings or values that need to persist.

let and let*: Local Variables

For variables with limited scope, Elisp provides let and let*:

;; Using let for local binding
(let ((x 5)
      (y 10))
  (message "Sum: %d" (+ x y)))

;; x and y are not accessible here

The difference between let and let* is significant:

• let evaluates all variable expressions before binding
• let* evaluates in sequence, allowing later variables to reference earlier ones

;; This won't work with let
(let ((x 5)
      (y (+ x 2)))  ; Error: x is not yet bound when this is evaluated
  (message "y: %d" y))

;; This works with let*
(let* ((x 5)
       (y (+ x 2)))  ; Works because x is already bound
  (message "y: %d" y))  ; Output: "y: 7"

Understanding Quote (')

One of the most distinctive features of Lisp is the quote character ('). It prevents evaluation of an expression, instead treating it as data.

Quoting Variables

When you place a quote in front of a symbol:

'my-symbol

You’re telling Elisp, “Don’t evaluate this symbol, just give me the symbol itself.” This is similar to references in other programming languages.

Without the quote, Elisp would try to look up the value of the variable my-symbol.

List Syntax: '(...) vs (list ...) and (cons ...)

Lists are fundamental data structures in Lisp. There are several ways to create them:

;; These are equivalent:
'(a b c)
(list 'a 'b 'c)

;; These are also equivalent:
'(a . b)
(cons 'a 'b)

The quoted form '(...) is concise but has a limitation: everything inside is automatically quoted. This means you can’t mix evaluated and non-evaluated expressions.

When to use each:

• Use '(...) for literal lists that don’t need evaluation
• Use (list ...) when you need to mix literals and evaluated expressions:

(setq name "John")
(list 'person name 'age 30)  ; => (person "John" age 30)

Understanding Cons Cells: CAR and CDR

At the heart of Lisp’s data structures is the “cons cell” - a simple pair of values. Each cons cell has two parts, traditionally accessed with the functions car and cdr (pronounced “could-er”):

(setq my-pair (cons 'a 'b))

(car my-pair)  ; => a
(cdr my-pair)  ; => b

These peculiarly named functions are historical artifacts from the original Lisp implementation on IBM machines:

• car: Contents of the Address part of Register
• cdr: Contents of the Decrement part of Register

In modern terms, think of them as:

• car: head (first element)
• cdr: tail (rest of the list)

Lists in Lisp are actually chains of cons cells that always terminate with nil:

;; This list:
'(1 2 3)

;; Is actually structured as:
(cons 1 (cons 2 (cons 3 nil)))

Visualized as pairs:

(1 . (2 . (3 . nil)))

This implicit nil at the end is crucial - it’s what defines a “proper list” in Lisp. Without this nil termination, you would have a different data structure. The presence of this nil terminator is what allows functions like length to work properly and what lets Lisp know where a list ends.

You can extract elements from lists using these functions:

(setq numbers '(1 2 3 4))
(car numbers)       ; => 1
(cdr numbers)       ; => (2 3 4)
(car (cdr numbers)) ; => 2

Elisp provides convenient shorthand functions like cadr (car of cdr) for common combinations:

(cadr numbers)      ; => 2 (same as (car (cdr numbers)))
(caddr numbers)     ; => 3 (car of cdr of cdr)

The Dot Syntax in Pairs

In Emacs Lisp, you might encounter constructs with dot notation like:

(add-to-list 'auto-mode-alist '("\\.md\\'" . markdown-mode))

This dot notation represents a “cons cell” or pair. In this example, we’re creating a pair where the car is "\\.md\\'" and the cdr is markdown-mode.

The following expressions are equivalent:

'("\\.md\\'" . markdown-mode)
(cons "\\.md\\'" 'markdown-mode)

The dot creates a direct pair between two values. This syntax is commonly used in alists (like auto-mode-alist), mode hooks, and other paired data structures in Emacs.

When you see a dot in the middle of a list, it means “the rest of this list is the cdr of this cons cell” rather than continuing the list structure with an implicit nil at the end.

To understand the difference clearly:

;; A list with two elements (has an implicit nil at the end)
'(1 2)  ; => equivalent to (cons 1 (cons 2 nil))

;; A single cons cell (a pair) - NOT a proper list
'(1 . 2)  ; => equivalent to (cons 1 2)

The difference is significant:

(length '(1 2))     ; => 2
(length '(1 . 2))   ; => Error: Wrong type argument: listp, (1 . 2)

This distinction is fundamental to understanding how Lisp data structures work.

Function Quoting: #' vs '

In Elisp, you’ll see both 'function-name and #'function-name when referring to functions. The difference is important:

• 'function-name simply quotes the symbol
• #'function-name is shorthand for (function function-name), which specifically tells Elisp that the symbol represents a function

When should you use #'? Best practice is to use it whenever you’re referring to a function as a function, especially with higher-order functions:

;; Good practice
(mapcar #'1+ '(1 2 3))  ; => (2 3 4)

;; Works but less explicit
(mapcar '1+ '(1 2 3))

Using #' helps the byte-compiler optimize your code and makes your intent clearer to other programmers.

Association Lists (alists)

Association lists (alists) are lists of key-value pairs used for simple lookups:

(setq my-alist '((name . "John")
                 (age . 30)
                 (city . "Boston")))

You can access values using functions like assoc:

(cdr (assoc 'age my-alist))  ; => 30

Alists are frequently used in Emacs for configuration options, especially when the list is relatively small and frequently modified. They’re simple but not as efficient for large datasets.

Property Lists (plists)

Property lists are another key-value structure in Elisp, but with a different syntax:

(setq my-plist '(:name "John" :age 30 :city "Boston"))

Notice how the keys are prefixed with colons. These are called keywords, similar to Ruby’s symbols. You can access values using plist-get:

(plist-get my-plist :age)  ; => 30

Other languages with similar concepts include:

• Ruby with its symbols (:symbol)
• Clojure with keywords (:keyword)
• Elixir with atoms (:atom)

Plists are often used for function arguments that have default values or are optional.

Summary

Understanding these fundamental Emacs Lisp concepts will help you:

1. Write more effective Elisp code
2. Understand existing Emacs configurations
3. Create your own Emacs extensions

Emacs Lisp’s power comes from its simplicity and consistency. Once you grasp these basic concepts, you’ll find that the language follows logical patterns that make learning advanced features much easier.

Happy hacking!

Emacs comes with an excellent built-in tutorial that can be accessed with C-h t (Control-h followed by t). I highly recommend all new users complete this tutorial to get comfortable with Emacs’ fundamentals. That said, the tutorial is comprehensive and takes time to complete.

This guide serves as a TL;DR and quick reference of the most essential hotkeys you should memorize. Consider it a short recap of what’s worth knowing by heart to be immediately productive in Emacs, whether you’re new to the editor or just need a refresher.

Understanding Emacs Notation

Before diving into the hotkeys themselves, let’s decode how to read Emacs key notation:

• C- represents the Control key. For example, C-a means “hold Control and press a”.
• M- represents the Meta key, which on modern keyboards is typically the Alt key. So M-f means “hold Alt and press f”.
• C-x followed by another key represents a two-key sequence. Press Control and x together, release, then press the next key.
• M-x similarly starts a command sequence where you type the command name after pressing Alt and x.

Multiple modifier keys can be combined. For example, C-M-f means holding both Control and Alt while pressing f.

With this foundation, let’s explore the essential hotkeys that will dramatically improve your Emacs efficiency.

Essential Emacs Hotkeys by Category

Text Navigation

Efficient text navigation is the foundation of productive editing. These shortcuts let you move through your document with precision:

The navigation keys follow a pattern: f for forward, b for backward, p for previous, and n for next. Once you internalize these patterns, they become second nature.

Text Editing

These shortcuts handle the core editing operations that you’ll use hundreds of times daily:

A unique aspect of Emacs is its “kill ring” - a clipboard system that remembers multiple deleted items. After using C-y to paste the most recent item, you can press M-y repeatedly to cycle through previously killed text.

Region and Selection

Working with selections in Emacs involves understanding the concept of the “mark” and “point”:

The mark is where a selection begins, and the point is your cursor position. Together they define the “region” - Emacs’ term for a selection.

File Operations

These commands handle file management tasks:

The C-x C-f command is particularly powerful as it allows you to create new files by entering paths that don’t exist yet.

Buffer Management

In Emacs, a buffer is essentially an open file or space for editing:

Efficient buffer navigation is crucial when working with multiple files.

Window Management

Emacs allows for sophisticated window layouts:

The ability to split the editor into multiple windows, each showing different buffers, is one of Emacs’ most powerful features.

Search and Replace

Finding and replacing text efficiently is essential for editing:

Emacs’ incremental search updates as you type, making it quick to locate specific text.

Help System

When you inevitably need guidance, Emacs has built-in help:

The self-documenting nature of Emacs means help is always just a few keystrokes away.

Macros

Macros are powerful tools for automating repetitive tasks:

To use macros: press F3, perform the sequence of actions you want to repeat, press F4 to finish recording, then press F4 again each time you want to replay those exact keystrokes. This is incredibly useful for mass operations on text.

Command Execution

Finally, the gateway to thousands of Emacs commands:

This shortcut unlocks Emacs’ full potential, giving you access to any command by name.

Building Your Muscle Memory

Learning these hotkeys will take time, but the productivity payoff is immense. Start by mastering the navigation and basic editing commands, then gradually incorporate the others. Consider printing this list and keeping it near your workspace until the keystrokes become automatic.

Remember that Emacs is highly customizable - once you’re comfortable with these basics, you can create your own keybindings for commands you use frequently.

Conclusion

The beauty of Emacs lies in how it becomes an extension of your thinking process once you’ve internalized these commands. Text manipulation becomes effortless, allowing you to focus on your actual work rather than the mechanics of editing.

While this guide covers the essential hotkeys, it barely scratches the surface of what Emacs can do. As you grow more comfortable with these basics, explore specialized modes for your particular work, whether that’s programming in specific languages, writing prose, managing projects, or organizing your life.

What began as a seemingly steep learning curve will transform into a powerful skillset that follows you throughout your computing life. Happy editing!

The Power of Daily Granularity

One of the most impactful lessons I’ve learned is the importance of splitting journals into smaller, more manageable chunks. The configuration sets up org-journal with daily entries rather than maintaining one massive file for an entire year’s worth of thoughts and tasks. This approach offers several benefits:

• Improved Performance: Smaller files load faster and are less likely to cause Emacs to slow down
• Better Organization: Daily or weekly files create natural boundaries for your thoughts
• Easier Navigation: Finding specific entries becomes simpler when they’re organized by date

My configuration explicitly sets org-journal-dir to “~/org/journal” and uses the format (YYYYMMDD) for consistency and easy sorting.

Task Migration: Never Lose Track Again

A game-changing feature in org-journal is its ability to automatically move unfinished tasks to the next day’s entry. This prevents tasks from being forgotten in past journal entries and keeps your current day’s focus on what still needs attention.

Effective Agenda Management

The configuration reveals a crucial insight: simply journaling isn’t enough—you need to leverage org’s scheduling and deadline functions to create an effective overview of your commitments. The custom function _0x17de/org-highlight-todays-deadlines applies special highlighting to items due today, creating visual urgency for time-sensitive tasks.

Enhanced Task States with Custom Keywords

The standard TODO/DONE paradigm often feels inadequate for complex workflows. This configuration expands the vocabulary of task states:

(setq org-todo-keywords '((sequence "TODO" "WAITING" "DOING" "|" "DONE" "CANCELLED")))

Adding intermediate states like “WAITING” and “DOING” provides more context about where each task stands without requiring additional notes. The vertical bar separates active states from terminal states, a subtle but important distinction.

Visual Differentiation Through Custom Styling

The configuration goes beyond mere keywords by defining custom faces for different task states:

(org-modern-todo-faces '(("TODO" :foreground "white" :background "darkgreen" :weight bold)
                        ("DOING" :foreground "white" :background "orange" :weight bold)
                        ("WAITING" :foreground "white" :background "blue")))

This color-coding creates immediate visual feedback, making it easy to scan a journal page and instantly understand what’s in progress versus what’s waiting on external factors.

Modern Interface Enhancements

The use of org-modern-mode suggests an appreciation for clean, contemporary styling. This package enhances the visual appeal of org files with improved typography and modern design elements, making the journaling experience more pleasant.

Exploring New Frontiers: org-node

The configuration hints at exploration of org-node, a powerful package that enables treating org headings as nodes in a graph. This approach opens up interesting possibilities for knowledge management and connecting ideas across different journal entries and documents.

Convenient Keybindings for Quick Capture

Setting up a global keybinding (C-M-S-j) for creating new journal entries reduces friction in the journaling process. The fewer barriers to capturing thoughts, the more likely you are to maintain the practice consistently.

Integration with the Agenda

The configuration enables agenda integration with org-journal-enable-agenda-integration, ensuring that tasks and events from journal entries appear in your org-agenda views. This creates a unified system where your journal and task management seamlessly work together.

What’s Next?

For those inspired to explore this setup further, my complete Emacs configuration is available at https://github.com/0x17de/emacs-config. It offers additional insights into how these journaling practices fit into a broader productivity system.

As my own journey with org-journal continues, I’m particularly interested in diving deeper into org-node and exploring how graph-based knowledge management might enhance connections between journal entries over time.

Additionally i was looking a little into the programming language rust.

The rust package manager cargo feels well designed. Also the language solves quite common issues in programming which makes classic inheritance unnecessary. The tutorial is easy to read and understand but in advanced topics you can see that things still change. The verbose annotation of object lifetime of references was reduced in recent releases and things that should not compile now finally do without issues. The basic data types quite bring everything you need out of the box, except some language features like drain_filter, quite similar to std::remove_if, are only available as unstable features. Therefore I would not recommend using the language in production, but i can already see its potential in the future. Memory related problems are reduced to ownership problems. It must be clear who is allowed to borrow a variable (keeping a reference) for later writing or read only operations. You can not have two writable references, nor have references that outlive the instance. This might cause a lot of confusion and iterations of code if you try to write code in your usual style. The ownership problem looks even worse when trying to communicate between threads. When a thread stops can be decided at runtime, either it is joined or it could be detached and even outlive the main thread. Therefore the classic thread’s lifetime is considered 'static. Solutions, for having threads with a defined lifetime existed: like std:🧵:scoped, but it had unforseen issues, related to accessing freed memory. To solve the issue, the library crossbeam provides a different approach to threads with a clearly defined lifetime. Still sharing objects as readable between threads is a challenge. One solution to the object being shared across threads could be by using std::sync::Arc<std::sync::Mutex<Foo>>. An Arc is for atomically accessing things from different threads, while the mutex is required to clearly define who is writing the memory. Otherwise for TcpStreams the handles can simply be cloned (see try_clone) without being in need of using a mutex object.

After reading this article you sure feel the complexity of that quite new programming language rust. But even if the language might not be ready for production, I recommend having at least a first look through the entire tutorial or even to learn it. The concepts might be different, but if you do you will be prepared for the future that is likely to come.

The original images which will be preprocessed and passed to a convolutional neural network are scanned A4 letters with a resolution of about 2550x3500 pixels at 300dpi.

At first some letters will be manually converted into text for labeling, which is required for the training process. A self written tool is used to assign a position to each character in the transcription. This process can be seen in the screenshot above. The tool was written with OpenCV and allows a preview of the next two words, highlights the current letter which is to be labeled and shows some bounding box with hints for the later cropping process. (If you want the source just write me some message but i also plan to release the project after some results)

After each letter has some assigned position, the image still needs to be preprocessed/enhanced as the color of the font is quite close to the background color. This step can be done automatically in the future. For each labeled character a 32x32px sub image will be cropped and additionally for each of those images small pixel translations, scaling and sinus wave transformations will be applied to generate a lot more training data for the learning process.

For the CNN i planned to use Tensorflow as it has been proven to be quite flexible for this task.

I will present the network configuration and further results in the next blog posts :)

Current letter counts after labeling two letters:

     16 ,
      2 !
     24 .
      2 0
      4 1
      3 2
      1 3
      2 4
      1 9
     71 a
      3 ä
     22 b
     54 c
     33 d
    162 e
     17 f
     26 g
     76 h
     81 i
      5 j
     13 k
     41 l
     22 m
    100 n
     26 o
      2 ö
      5 p
     65 r
     48 s
      4 ß
     57 t
     22 u
      4 ü
      1 Ü
      7 v
     31 w
     13 z

You can follow the project on the blog, github or via rss with the following urls:

• Blog: https://harpoon.0x17.de/blog/
• RSS: https://harpoon.0x17.de/blog/index.xml
• GitHub Harpoon: https://github.com/HarpoonOrg/Harpoon
• GitHub HarpoonClient: https://github.com/HarpoonOrg/HarpoonClient

Have fun reading! :)

Here are two animations i created with spriter:

The tool i used here is aseprite

And here’s how i’d start with drawing these: I’d use primitive shapes like a filled ellipse/circle and draw some highlights with a contour tool (lasso which fills the selected region). For the next frames i will reduce sections using the lasso or eraser and also move subsections using the lasso tool.

Here are some screenshots of the current state:

Those screenshots need a little description, so here you go:

Harpoon consists of a server program and three different clients: Web/Desktop/Mobile.

In the first screenshot you can see the browser interface (topleft), whereas in the second screenshot you can see the desktop client.

The project is still work in progress as some essential parts are missing. Those are showing the backlog, user login and yet server configuration. In the next few updates those topics will be adressed and i will keep you up to date.

If you have questions you can find me on IRC. Just click the following link: freenode.net channel #harpoon.

Follow me on twitter, watch the project on github, read my RSS or join #harpoon on freenode.net to receieve status updates. :)