I've copied a massive folder from a Windows machine to a Linux machine and due to some files names being too big (and some other errors which I've skipped), some files could not be copied. I'm currently running diff -r between the two folders in order to generate a list of the files that are in the original folder but not the copy. However, so far the only things that it seems to have recognised are missing folders, i.e. it seems to be skipping files. Is there a better way for me to make this comparison? In particularly, I'm worried that Bash simply can't recognise these files with too long file names.
24 Answers
If rsync is a viable option, perhaps the --itemize-changes (-i) and --dry-run options would be of use:
rsync -zaic src_dir/ dest_dir/ --dry-run
-z compresses files during transfer, -a copies in archive mode and -c bases the file comparisons on checksums rather than date modified or size.
-i will list the individual files that are different and --dry-run means that no data will be transferred, just generating a list.
2You might do something not entirely unlike:
(cd some/where; ls -lR) > somewhere.txt
(cd else/where; ls -lR) > elsewhere.txt
diff somewhere.txt elsewhere.txtI haven't tried this, it depends on file metadata (dates etc) being preserved (cp -p ...) and on ls sorting filenames in the same order (which it should).
diff --recursive (-r) does catch file changes, even within in subdirectories.
You might rather want to use diff --unified --recursive, however. It creates a unified diff, which displays changed lines prefixed with (+) for additon and (-) for removal. Conveniently, it also displays surrounding lines (i.e. context), so that you can figure out what's going on there.
diff <(cd /first/path/ && find ./ | sort) <(cd /second/path/ && find ./ | sort)This is similar to this other answer but:
- I'm using
findto generate lists of objects (files, directories); it fits here better thanlsbecause its output contains only paths. sortensures the relative order of objects is preserved, regardless of in what order eachfindlists them.- The
<(…)syntax avoids temporary files inbash. findwill be executed only if the correspondingcdsucceeds, thanks to the&&operator. This will save you from runningfindin current directory if there's a typo in any path.
Additional notes:
- Paths returned by
findwill be relative to directories wecdto. Make sure/first/path/and/second/path/correspond to each other. - Empty output from
diffindicates the two directories are identical; but remember… - … the command operates on paths only, it doesn't check if the contents or metadata match.
- Object names with unusual characters (e.g. with newlines) will break the logic.
