Zobrazit příspěvky
1
Software / Re:Odstranění duplicit a konsolidace dat
« kdy: 15. 08. 2026, 12:42:45 »musim uz ty duplicity nejak resit.
rmlint
https://github.com/sahib/rmlint
Má to veľa možností, ale ja využívam takéto niečo:
rmlint /mnt/kopia1 /mnt/kopia2 // /mnt/archiv --types=df --hidden --progress --paranoid --keep-all-tagged --must-match-tagged --match-basename --match-extension --mtime-window=0
Vygeneruje to json súbor s podrobnosťami, ktoré môžete kvôli kontrole spracovať iným spôsobom a sh súbor s príkazmi na mazanie, takže pri spustení rmlint s uvedenými parametrami nič nezmaže, kým nespustíte ten sh súbor - samozrejme môžete niektoré parametre vynechať - to už je na vás, čo si zvolíte po prečítaní manuálu
/mnt/kopia1 /mnt/kopia2 sú nové "untagged" adresáre, z ktorých chcem vymazať duplicity
// je oddeľovač, za ním sú "tagged" adresáre, v ktorých nechcem nič mazať - t.j. --keep-all-tagged
/mnt/archiv obsahuje súbory, ktoré už mám spracované a nepotrebujem tam hľadať duplicity
dosť často používam aj --size
Kód: [Vybrat]
Usage:
rmlint [OPTION…] [TARGET_DIR_OR_FILES …] [//] [TAGGED_TARGET_DIR_OR_FILES …] [-]
rmlint finds space waste and other broken things on your filesystem and offers to remove it.
It is especially good at finding duplicates and offers a big variety of options to handle them.
Help Options:
-h, --help Show help options
--help-all Show all help options
Application Options:
-d, --max-depth=N Specify max traversal depth
-S, --rank-by=[dlamprxDLAMPRX] Select originals by given criteria
-y, --sort-by=[moansMOANS] Sort rmlint output by given criteria
-T, --types=T Specify lint types
-s, --size=m-M Specify size limits
-a, --algorithm=A Choose hash algorithm
-o, --output=FMT[:PATH] Add output (override default)
-O, --add-output=FMT[:PATH] Add output (add to defaults)
-n, --newer-than-stamp=PATH Newer than stamp file
-N, --newer-than=STAMP Newer than timestamp
-c, --config=FMT:K[=V] Configure a formatter
-C, --xattr= Enable xattr based caching
-g, --progress Enable progressbar
-v, --loud Be more verbose (-vvv for much more)
-V, --quiet Be less verbose (-VVV for much less)
-Y, --replay=path/to/rmlint.json Re-output a json file
--equal=PATHS Test for equality of PATHS
-W, --no-with-color Be not that colorful
-r, --hidden Find hidden files
-f, --followlinks Follow symlinks
-F, --no-followlinks Ignore symlinks
-p, --paranoid Use more paranoid hashing
-x, --no-crossdev Do not cross mountpoints
-k, --keep-all-tagged Keep all tagged files
-K, --keep-all-untagged Keep all untagged files
-m, --must-match-tagged Must have twin in tagged dir
-M, --must-match-untagged Must have twin in untagged dir
-b, --match-basename Only find twins with same basename
-e, --match-extension Only find twins with same extension
-i, --match-without-extension Only find twins with same basename minus extension
-D, --merge-directories Find duplicate directories
-j, --honour-dir-layout Only find directories with same file layout
-z, --perms=[RWX]+ Only use files with certain permissions
-L, --no-hardlinked Ignore hardlink twins
--keep-hardlinked Keep hardlink that are linked to any original
--partial-hidden Find hidden files in duplicate folders only
-Z, --mtime-window=T Consider duplicates only equal when mtime differs at max. T seconds
-0, --stdin0 Read null-separated file list from stdin
--backup Do create backups of previous result files
--dedupe Dedupe matching extents from source to dest (if filesystem supports)
--dedupe-xattr Check extended attributes to see if the file is already deduplicated
--dedupe-readonly (--dedupe option) even dedupe read-only snapshots (needs root)
--is-reflink Test if two files are reflinks (share same data extents)
-H, --show-man Show the manpage
--version Show the version & features
--gui If installed, start the optional gui with all following args
--hash Work like sha1sum for all supported hash algorithms (see also --hash --help)
-q, --clamp-low=P Limit lower reading barrier
-Q, --clamp-top=P Limit upper reading barrier
Only the most important options and options that alter the defaults are shown above.
See the manpage (man 1 rmlint or rmlint --show-man) for far more detailed usage information,
or http://rmlint.rtfd.org/en/latest/rmlint.1.html for the online manpage.
Complementary tutorials can be found at: http://rmlint.rtfd.org
