Forrest Jacobs

Using Syncthing to sync coding projects

I code on a MacBook and a Windows PC, and I want to keep my coding projects in sync between them. Here are my wishes in decreasing priority:

  1. Code changes on my MacBook should magically update my PC, and vice versa. (Think Dropbox.)
  2. Some files should not sync, like host-specific dependencies and targets. I want to ignore these files via patterns, a la gitignore.
  3. Ideally, this sync extends to both a headless Linux server I use for remote development, and to WSL on my Windows PC.

After experimenting with other solutions (outlined below) I discovered that Syncthing meets every requirement.

What I tried first

1. OneDrive

I use OneDrive to sync most of my files. It’d be nice to just add my coding projects to OneDrive, but it doesn’t work in practice: ignoring files is awkward, and seemingly only works on Windows. Additionally, OneDrive doesn’t run on Linux without some help.

Dropbox looks like a better fit on paper: it can ignore files on any platform (in a different, awkward way) and it has a first party Linux client. But switching to Dropbox would be painful – my partner and I switched away from Dropbox about a year ago because we were getting more storage for less money from Microsoft, and the modern Dropbox app sucks.

2. Remote development

If the issue is syncing files across computers, why don’t I just work on one computer? Well, developing on a remote machine has its own issues:

3. Git

Can’t I just use Git to stay up to date?

No – version control is different than file sync. I don’t want to track personal config files in version control, but I do want to sync them. And I don’t always want to check in work in progress – for example, I don’t want to check in changes that cause builds or tests to fail.

Using Syncthing

Syncthing is amazing. It does everything I outlined at the top – it syncs my projects, it ignores files based on patterns, and it runs everywhere I code (Windows, MacOS, and Linux.)

I resisted using it because of its high barrier to entry. It uses peer-to-peer file syncing, so you need to set it up on a server to ensure each computer sees the latest changes. And its configuration is more involved than something like Dropbox.

But it’s still worth it for me because it solves all my original problems. (And I want to sync these files to a server anyway.) If you’re struggling with the same issues I ran into, and you’re willing to set up a server, give Syncthing a shot.


Addendum: Syncing your ignore patterns

Syncthing does not keep your ignore patterns in sync across hosts, but there’s a way around it:

  1. Create a text file with the patterns you want to ignore.
  2. Save it to your Syncthing folder.
  3. Add #include name-of-that-file to each host’s ignore patterns.

Voilà!

NixOS on WSL

I recently set up a new Windows machine for gaming, but I’m actually using it more to play around with Linux via WSL. I set up NixOS on WSL using the aptly named NixOS on WSL project, but ran into a few issues during setup that bricked my environment1-->. Below are the steps that ended up working for me in the end:

Goals

Step 1: Install WSL 2

Per Microsoft’s documentation:

  1. Open PowerShell as an admin by right-clicking on it and selecting “Run as administrator.”

  2. Run wsl --install

Step 2: Install NixOS

Per NixOS-WSL’s documentation:

  1. Download the installer listed in NixOS-WSL’s 22.05 release.

  2. Move the file where you want your NixOS installation to live. I put it in C:\Users\<my username>\AppData\Local\NixOS, which I think follows Windows’s conventions.

  3. Open PowerShell as a normal user, change to the directory you just made, and run wsl --import NixOS . nixos-wsl-installer.tar.gz --version 2

  4. Run wsl -d NixOS to start NixOS. It’ll run through initial setup, then hang on “Starting systemd…” Press Ctrl + C to get back to PowerShell, and then run wsl --shutdown followed by wsl -d NixOS to get back into NixOS.

Step 3: Configure NixOS

At this point you should have a basic NixOS 22.05 installation! Let’s finish up by setting a username, switching over to flakes, updating to NixOS 22.11, and enabling native systemd:

  1. Edit /etc/nixos/configuration.nix and make the following changes. Use sudo or some such to edit the file as root. You may want to nix-shell -p your favorite editor at this point – maybe Helix?

    • Change wsl.defaultUser to your desired username, and add something like users.users.<your username>.isNormalUser = true;
    • Remove references to ./nixos-wsl since we’ll set NixOS-WSL as a flake input:
        { lib, pkgs, config, modulesPath, ... }:
      -
      - with lib;
      - let
      -   nixos-wsl = import ./nixos-wsl;
      - in
        {
          imports = [
            "${modulesPath}/profiles/minimal.nix"
      -
      -     nixos-wsl.nixosModules.wsl
          ];
      
    • Add wsl.nativeSystemd = true;
    • Add networking.hostName, and set it to your Windows 11 “Device name.” You can find this in the Windows Settings app under System > About.
  2. Add /etc/nixos/flake.nix as root with contents like the following. Replace <host-name> with your actual host name from configuration.nix.

    {
    
      inputs = {
        nixpkgs.url = "github:NixOS/nixpkgs/nixos-22.11";
        NixOS-WSL = {
          url = "github:nix-community/NixOS-WSL";
          inputs.nixpkgs.follows = "nixpkgs";
        };
      };
    
      outputs = { self, nixpkgs, NixOS-WSL }: {
        nixosConfigurations."<host-name>" = nixpkgs.lib.nixosSystem {
          system = "x86_64-linux";
          modules = [
            { nix.registry.nixpkgs.flake = nixpkgs; }
            ./configuration.nix
            NixOS-WSL.nixosModules.wsl
          ];
        };
      };
    
    }
    
  3. Run sudo nixos-rebuild switch

  4. exit out of NixOS, then run wsl --shutdown followed by wsl -d NixOS. You should be logged in as the new default user.

  5. At this point you should have a stable base to make any configuration changes you’d like.

Step 4: Cleaning up

Addendum: Share .ssh directory with Windows

It turns out that file permissions set in WSL are preserved in NTFS2-->. You can store your SSH configuration files in Windows, and still use them from WSL with no permissions issues:

That’s it! You can verify that permissions carried over with ls -al ~/.ssh/

Addendum: Set up fish in NixOS the right way

I use fish as my shell, but when I set it up in WSL I started getting errors like this:

fish: Unknown command: ls
/nix/store/lkf5vmavnxa0s37imb03gv7hs6dh5pll-fish-3.5.1/share/fish/functions/ls.fish (line 64):
    command $__fish_ls_command $__fish_ls_color_opt $opt $argv
    ^
in function 'ls'

Uh oh. Obviously, there are important directories missing from my path. It turns out that I was missing some configuration3-->:

programs.fish.enable = true;

I don’t know why this issue only appeared in WSL, but fish users should probably set programs.fish.enable in NixOS whether the issue appears or not.

See also


  1. I think I changed the default user without actually setting up that user? Or I switched to native systemd before upgrading to a version of NixOS on WSL that supports it?

  2. Thanks to this Stack Exchange answer for pointing me in the right direction. You don’t have to follow the instructions in this post – NixOS on WSL already mounts Windows’s drives with the necessary options.

  3. Thanks to this GitHub issue for alerting me to the problem.