I recently implemented a backup workflow for me. I heavily use restic for desktop backup and for a full system backup of my local server. It works amazingly good. I always have a versioned backup without a lot of redundant data. It is fast, encrypted and compressed.
But I wondered, how do you guys do your backups? What software do you use? How often do you do them and what workflow do you use for it?
-
daily important stuff (job stuff, Documents folder, Renoise mods) is kept synced between laptop, desktop and home server via Syncthing. A vimwiki additionally also syncs with the phone. Sync happens only when on home network.
-
the rest of the laptop and desktop I’ll roll into a tar backup every now and then with a quick bash alias. The tar files also get synced onto home server’s big file system (2 TB ssd) via Syncthing. Home server backs itself up on it’s own once a week.
-
clever thing is that the 2 TB ssd replaced an old 2 TB spinning disk. I kept the old disk and set up a systemd thing that keeps it spun down, but starts and mounts it once a week and rsyncs the changes to the ssd over, then unmounts it so that it sleeps again for a week. That old drive is likely to serve for years still with this frugal use.
How do you make sure the disk spins down? Is unmounting enough?
-
My KVM hosts use “virsh backup begin” to make full backups nightly.
All machines, including the KVM hosts and laptops, use rsync with --link-dest to create daily incremental versioned backups on my main backup server.
The main backup server pushes client-side encrypted backups which include the latest daily snapshot for every system to rsync.net via Borg.
I also have 2 DASs with 2 22TB encrypted drives in each. One of these is plugged into the backup server while the other one sits powered off in a drawer in my desk at work. The main backup server pushes all backups to this DAS weekly and I swap the two DASs ~monthly so the one in my desk at work is never more than a month or so out of date.
I’ve found that the easiest and most effective way to backup is with an rsync cron job. It’s super easy to setup (I had no prior experience with either rsync or cron and it took me 10 minutes) and to configure. The only drawback is that it doesn’t create differential backups, but the full task takes less than a minute every day so I don’t consider that a problem. But do note that I only backup my home folder, not the full system.
For reference, this is the full line I use: sync -rau --delete --exclude-from=‘/home/<myusername>/.rsync-exclude’ /home/<myusername> /mnt/Data/Safety/rsync-myhome
“.rsync-exclude” is a file that lists all files and directories I don’t want to backup, such as temp or cache folders.
(Edit: two stupid errors.)
Rsync can do incremental backups with a command-line switch and some symlink jugglery. I’m using it to back up my self-hosted stuff.
You might be interested in “rsnapshot” which uses rsync and manages daily, monthly, etc. snapshots.
only drawback is that it doesn’t create differential backups
This is a big drawback because even if you don’t need to keep old versions of files, you could be replicating silent disk corruption to your backup.
It’s not a drawback because rsync has supported incremental versioned backups for over a decade, you just have to use the --link-dest flag and add a couple lines of code around it for management.
Sure, but that’s not in their answer.
They didn’t provide an rsync example until later in the post, the comment about not supporting differential backups is in reference to using rsync itself, which is incorrect, because rsync does support differential backups.
I agree with you that not doing differential backups is a problem, I’m simply commenting that this is not a drawback of using rsync, it’s an implementation problem on the user’s part. It would be like somebody saying “I like my Rav4, it’s just problematic because I don’t go to the grocery store with it” and someone else saying “that’s a big drawback, the grocery store has a lot of important items and you need to be able to go to it”. While true, it’s based on a faulty premise, because of course a Rav4 can go to the grocery store like any other car, it’s a non-issue to begin with. OP just needs to fix their backup script to start doing differential backups.
My systems are all on btrfs, so I make use of subvolumes and use
brkbkto backup snapshots to other locations.Same! This works really well.
i do backups of my home folder with Vorta, tha uses borg in the backend. I never tried restic, but borg is the first incremental backup utility i tried that doesnt increase the backup size when i move or rename a file. I was using backintime before to backup 500gb on a 750gb drive and if I moved 300gb to a different folder, it would try to copy those 300gb again onto the backup drive and fail for lack of storage, while borg handles it beautifully.
as an offsite solution, i use syncthing to mirror my files to a pc at my fathers house that is turned on just once in a while to save power and disc longevity.
I recently bought a storagebox from Hatzner and set up my server to run borgmatic every day to backup to it.
I’ve also discovered that Pika Backup works really well as a “read only” graphical browser for borg repos.
Do you use some kind of encryption on the VPS?
I have a server with a RAID-1 array, that makes daily, weekly, and monthly read only btrfs snapshots. The whole thing (sans snapshots) is sync’d with syncthing to two rPi’s in two different geographic locations.
I know neither raid nor syncthing are “real” backup solutions, but with so many copies of the files living in so many locations (in addition to my phone, laptop, etc.) I’m reasonably confident its a decent solution.
Since most of the machines I need to backup are VMs, I do it by the means of hypervisor. I’d use borg scheduled in crontab for physical ones.
I’m curious, is there a reason why noone uses deja-dup? I use it with an external SSD on Ubuntu and (receently) Mint, where it comes pre-installed, and did not encounter Problems.
What do you backup with dejadup? Everything under /home?
The restore process takes forever and sometimes fails. Last time I was forced to try every daily backup to several days before last backup to find one that could actually be restored. I have switched to borg (using Pika Backup for desktop and Borgmatic for servers). No restores have failed since.
Backup? What?
Your car.
There’s nothing saved on my system I couldn’t afford to lose. All my work stuff is saved in Google Drive for better or worse. I have a few small files in a personal Proton Drive that I backup manually. I wipe my own system a few times a year and I rarely ever save anything first. Honestly very refreshing to live your life like that. Other than my cat, pretty much all my possessions could disappear tomorrow and I’d get over it pretty quickly.
Borg Backup, whenever I feel like it - usually monthly.
I use external drive for my important data and if my system is borked (which never happen to me) I just reinstall the OS
External drives are more prone to damage and failures, both because they’re more likely to be dropped/bumped/spilled on etc, and because of generally cheaper construction compared to internal drives. In the case of SSDs the difference might be negligible, but I suggest you at least make a copy on another “cold” external drive if the data is actually important
I recently switched to Kopia for my offsite backup solution.
It’s apparently one of the faster options, and it can be set up so that the files of the differential backups are handled by a repository server on the offsite end, so file management doesn’t need to happen over the network at a snails pace.
The result is a way to maintain frequent full backups of my nextcloud instance, with almost no downtime.
Nextcloud only goes into maintenance mode for the duration of a postgres database dump, after which the actual file system backup occurs using a temporary btrfs snapshot, containing a frozen filesystem at the time of the database dump.
My work flow is pretty similar to yours:
For my desktop and laptops: systemd timer and service that backups every 15 minutes using restic to my NAS.
For my NAS : daily backup using restic + ZFS snapshots.
All restic backups are then uploaded daily to Backblaze B2.
Do you create ZFS snapshots and let those be backed up to B2 via restic or do you backup different types of data, one with ZFS snapshots and one with restic?
Only restic snapshots are backed-up to B2. ZFS snapshots are for undoing mistakes, though I enabled them recently and I have yet to use them.
