equivs2 can relatively (or very, depending on options and input size) divide a series of files into equivalence classes.
Version: 3.0equivs2 can relatively (or very, depending on options and input size) divide a series of files into equivalence classes. equivs2 is suitable for use with a very large number of files and/or very large files.
Operating System: Linux
It can perform its task strictly in terms of cryptographic digests, or use cryptographic digests backed up at the end with full byte-for-byte comparisons. It also uses some heuristics like comparing the beginnings of files and the device number and inode number and file sizes.
Further, it knows to only read in the parts that it needs "so far", to avoid a huge inhale at the beginning of the run, instead trying an initial method and falling back to subsequent methods as needed.
$ equivs2 -h
Usage: /Dcs/seki/strombrg/bin/equivs2 [-v] [-s] [-0] [-h] [-f file1 file2 ... filen]
- v says to operate verbosely
- s says to get filenames from stdin instead of from the command line
- 0 says that when getting filenames from stdin, assume null termination, not newline termination
- h says to give this help message
- f file1 file2 ... filen says to use the listed files, not files fromstdin
- p bytes says to cache this many bytes of each file for faster comparisons (may be 0)
- d delim says to use "delim" as the output delimeter character within a line
- c says to not do full comparisions - instead trust hash comparisons.
seki-strombrg:~ i386-redhat-linux-gnu 18834 - above cmd done Sat Apr 22 10:24 PM