Delete Crawler

[dedndave]

for a long time, i have wanted a little tool for this
it probably exists out there as an app, someplace   :P

sometimes, i want to move to a clean drive....
i have backup images of the OS/installed programs, etc
but, several times, i have wanted to delete files from one drive, if they exist on another

this is a bare-bones program to do that
it recursively steps through all the folders and files of a "destination" drive
if that same file exists on the second "source" drive, it is deleted from the destination

this one has no command-line parser or status display - you can add that stuff, as desired
it does, however, have a Sleep manager - so it does not hog the CPU or thrash the drives
you can add whatever features you like - or put it in a GUI app

i used it yesterday - it deleted about 50 Gb, leaving the desired 5 Gb to go through, manually

[dedndave]

i can see that i need to add a "delete empty folders" feature next time i run it - lol

[jj2007]

Nice idea :t

Good if you have a backup e.g. of fotos, and want to cleanup the original drive.

[dedndave]

it needs some work - but got the job at hand done

i want to re-write it so it will handle seperate paths
i.e. start at non-root paths, and work with different base paths for dest and source

[Gunther]

Dave,

good job. Thank you for providing it.

Gunther

[dedndave]

thanks, Gunther

plenty of room for improvement
but, i thought it might save someone a little coding time if they wanted such a function

[KeepingRealBusy]

Dave,

How do you determine that the file is the same as another file? Can the name change? Can the path change? Can the drive change? Can the date/time change? Obviously, the size cannot change. Do you check or calculate the two CRCs to see if they match?

I ask because my better half has a habit of changing filenames (without a content change), moving directories around to different parent directories, etc. Her backup method is to drag and drop the entire data drive root (E drive) to a backup drive, and create a total new save directory with massive file duplication. This is not necessarily bad because she has a duplicate (or many duplicate) of any file she may loose on her main data drive.

However, she has completely filled a pair of 1 TB drives, I need to get her a pair of 4 TB drives. This is not necessarily a good thing either, as I found out. My new 4 TB drives had some bad space, unbeknown to me, and I discovered that fact when I tried to copy one backup directory to the other drive of the pair. Trying to do a complete surface scan on 4 TB drives is a bitch.

Dave.

[dedndave]

this version only deletes files if they have the same path (except drive letter) and name
it suits my needs when i create an image of a drive, then restore it and wish to remove some duplicates

here is a  scenerio where it makes sense...

a) i create 2 partitions from the same image - they are now identical except drive letter
b) i delete all the "data" files from one of the 2 partitions:
pictures, e-mails, browser bookmarks, asm source files, PDF's and other documents, etc
c) now, i can use the crawler to delete everything but the data files from the other partition
after some clean-up, i have the data files on one partition and the system/program files on the other

in the case of finding match files with different names and locations, that's a whole different app
this one compares the 2 files for exact match, once a location/name/size match has been found
look at the "DoCompare" PROC to see how it's done

[dedndave]

this particular case was a little different than mentioned above

some time ago, i had created a backup of my then current boot drive
so - that partition and the one i was using differed by files added or updated since that time
the one i was using as a boot partition got messed up
so - i switched to the other one, but now i wanted just the files that were added or had been changed   

the messed up partition had something like 55-60 Gb on it
after i ran the crawler - and did some clean-up - i got it down to less than 1 Gb of files  :t
all the stuff that was messed up is now gone
i moved that 1 Gb to another partition - and rebuilt that one as another boot partition