I was sick of reinstalling after every experiment, so I built a desktop that survives distro hopping, dumb package upgrades and the occasional X11/Wayland meltdown. The trick is simple: encrypted LUKS root, Btrfs subvolumes with aggressive zstd compression and snapshotting, plus borg backups mirrored offsite with rclone. I can break everything and be back to a working system in 10 minutes.

Quick recipe you can follow (NVMe primary, small EFI, LUKS encrypted root):

  • Partition: EFI 512M, root as LUKS. Cryptsetup and create Btrfs inside the LUKS container: cryptsetup luksFormat /dev/nvme0n1p2; cryptsetup open /dev/nvme0n1p2 cryptroot; mkfs.btrfs -f /dev/mapper/cryptroot
  • Create subvolumes: mount /dev/mapper/cryptroot /mnt; btrfs subvolume create /mnt/@; btrfs subvolume create /mnt/@home; umount /mnt
  • Mount for install: mount -o subvol=@,noatime,compress=zstd:15,space_cache=v2,ssd /dev/mapper/cryptroot /mnt; mkdir -p /mnt/home; mount -o subvol=@home,compress=zstd:15 /dev/mapper/cryptroot /mnt/home
  • Install your distro onto /mnt normally, put /boot or EFI on the unencrypted partition if your distro needs it.

Make snapshots and backups part of the routine:

  • Local snapshots: use Timeshift (GUI) or snapper for automatic pre/post package snapshots. Snapper is great for root snapshots on Btrfs.
  • Offsite backups: borg init --encryption=repokey-blake2 /path/to/repo; borg create --stats /path/to/repo::‘{now:%Y-%m-%d_%H:%M}’ /home /etc; borg prune --keep-daily=7 --keep-weekly=4 --keep-monthly=6. Mirror the borg repo to cloud with rclone or an inexpensive S3/backblaze bucket.

How to recover when you inevitably mess up:

  1. Boot a live USB, open the LUKS container with cryptsetup open /dev/nvme0n1p2 cryptroot.
  2. Mount the Btrfs and list snapshots: btrfs subvolume list -s /mnt.
  3. Make the snapshot the default root with btrfs subvolume set-default <id> /mnt and reboot.

Hardware notes: buy a second NVMe or a small external SSD for local borg archives and cold snapshots. Avoid weird consumer RAID features on laptops and get a wifi card known to have mainline driver support. This setup saved my butt more than once when a kernel update nuked my X11 config or when I wanted to try a new distro without grief.

If you want, I can post a ready-to-run install script for Ubuntu/Arch with these exact commands and a sample snapper config. Stop treating your root filesystem like a fragile snowflake and start building resilience.

  • Hadriscus@jlai.lu
    link
    fedilink
    arrow-up
    2
    ·
    edit-2
    55 minutes ago

    Two things

    • first off it seems super neat but I understand none of it
    • I’m worried by the fact people with your expertise still fuck up their system

    So I’m not the target audience (for lack of know-how), but at the same time I wish I were so I could gain some peace of mind when using my new Linux install (mostly for work, I probably won’t tinker too much with this one)

    • NightFantom@slrpnk.net
      link
      fedilink
      arrow-up
      1
      ·
      edit-2
      8 minutes ago

      For me the biggest leap was letting go of my local settings. My kubuntu has about everything I want out of the box, then I install zsh with omz and I’m pretty much done.

      So whenever I break something it’s an easy fresh install.

      My data (steam games, code) is in a separate drive, and especially with cloud saves / git everything is available even if I were to break that drive (would just suck to remember which things I need to redownload from where).

      So that helped me release my tinkering spirit as much as I wanted, and while I’m far from a Linux guru, I’ve definitely learned a lot from that.

      Edit: not to say that I don’t try to fix things, just knowing that I can easily restart is the main thing.

  • Ghoelian@piefed.social
    link
    fedilink
    English
    arrow-up
    3
    ·
    2 hours ago

    So all you’re doing is using btrfs snapshots? Don’t most distros that use btrfs by default already do this?

  • harmbugler@piefed.social
    link
    fedilink
    English
    arrow-up
    2
    ·
    3 hours ago

    The solution for me was to install an immutable distro. As someone who even stopped wanting to tweak years ago I’d still bork my system occasionally just through normal use, especially package dependency issues.

  • afk_strats@lemmy.world
    link
    fedilink
    English
    arrow-up
    2
    ·
    3 hours ago

    This is really neat. Thank you. I would love a script or a more newb-friendly guide, not just for me, but for a lot of other users.

    Can I make a suggestion? Post your script on github or similar with a proper (open) liscence so people can make suggestions or versions they find useful.

  • Lonewolfmcquade@lemmy.world
    link
    fedilink
    arrow-up
    2
    ·
    edit-2
    5 hours ago

    Thanks for this! I’m looking forward to trying this. Would you have any recommendations for implementing this on an existing Arch system that’s already LUKS and BTRFS with snapper preconfigured?

    I’ve been looking for something like Macrium Reflect for Linux and this sounds like the solution!

    Currently I’m just using Clonezilla to image the entire drive on occasion. Your solution seems much more elegant

    • Bagels@lemmings.worldOP
      link
      fedilink
      arrow-up
      3
      ·
      5 hours ago

      Just add borg for offsite backups:

      sudo pacman -S borg
      borg init --encryption=repokey-blake2 /path/to/borg-repo
      borg create --stats /path/to/borg-repo::'{now}' /home /etc --exclude '/home/*/.cache'
      

      Then sync the borg repo with rclone to your cloud storage. Since snapper is already handling your root snapshots, you just need the borg layer for full disaster recovery.

      It’s way better than clonezilla imo, you get incremental encrypted and deduplicated backups instead of full disk images every time. Test your recovery, just boot live USB, open LUKS, mount Btrfs, and make sure you can access both snapper snapshots and extract from borg.