This is bothering me for years now, my backup script always takes everything with it, taking forever to finish.
I initially used the --exclude
option, but this is rather restrictive, cluttered the script and still had the excluded directories.
Then i discovered -X/--exclude-from
but same result here, weird globbing and still fails.
So i hacked a negative list via fd’s --ignore-file
and tar -T/--files-from
together. But tar still includes files and directories not on the fucking files.tmp.
So i’m not sure if it is a bug in Arch’s GNU tar or if it’s maybe a parameter in the wrong position, tar can be removed there. This is my current code
# tar -cf - -X "$XDG_CONFIG_HOME"/backup/ignore "$INPUT" -P
fd . -Hi --ignore-file "$XDG_CONFIG_HOME"/backup/ignore "${INPUT}" > "$_tmpfile"
tar -cf - --verbatim-files-from --files-from="$_tmpfile" -P \
|pv -tapes "$_fssize" \
|compress >"${OUTPUT}.$_ext"
INPUT is $HOME in this case.
And if anyone has a solution that works on busybox tar as well…
Thanks!
I save it for now, until i work on it again. Possibly the wildcards thing. And that tar includes files of folders given too, from someone else (how to work with that).
I use tar with a an external file for exclude and include files on my custom backup script. There is a
--wildcards
option, which lets you use*
(star) to exclude everything from a directory. The command looks like this:tar --create \ --zstd \ --verbose \ --restrict \ --force-local \ --wildcards \ --exclude-backups \ --exclude-from "${exclude_file}" \ --files-from "${include_file}" \ --file "${archive_file}"
And the include_file is a simple text file with newline separated paths. It looks like this:
/home/tuncay/.var/app/io.gpt4all.gpt4all/cache/* /home/tuncay/.var/app/io.gpt4all.gpt4all/data/*
How to reliably(!) exclude files from a list in tar?
You delete the entire file because the format and tools are hard to deal with for basic operations like this one. :)
Just kidding… but we know there’s some truth to it.
–exclude works reliable for me. can you give us an example of an --exclude and the file name that tar outputs when adding it?
I don’t know what fd does, but at a guess maybe what you’re missing is that tar includes all the files in directories you give it? So if you exclude ‘foo/bar’ but include ‘foo’ then foo/bar will be in your tar file.
What I do is basically
tar cf `ls ~ | grep -v $files_to_exclude`
but if you want to exclude something that isn’t a top-level directory you’d need to get slightly more fancy.