Author Topic: Delete Crawler  (Read 3652 times)

dedndave

  • Member
  • *****
  • Posts: 8829
  • Still using Abacus 2.0
    • DednDave
Delete Crawler
« on: August 03, 2014, 12:04:13 AM »
for a long time, i have wanted a little tool for this
it probably exists out there as an app, someplace   :P

sometimes, i want to move to a clean drive....
i have backup images of the OS/installed programs, etc
but, several times, i have wanted to delete files from one drive, if they exist on another

this is a bare-bones program to do that
it recursively steps through all the folders and files of a "destination" drive
if that same file exists on the second "source" drive, it is deleted from the destination

this one has no command-line parser or status display - you can add that stuff, as desired
it does, however, have a Sleep manager - so it does not hog the CPU or thrash the drives
you can add whatever features you like - or put it in a GUI app

i used it yesterday - it deleted about 50 Gb, leaving the desired 5 Gb to go through, manually

dedndave

  • Member
  • *****
  • Posts: 8829
  • Still using Abacus 2.0
    • DednDave
Re: Delete Crawler
« Reply #1 on: August 03, 2014, 01:21:19 AM »
i can see that i need to add a "delete empty folders" feature next time i run it - lol

jj2007

  • Member
  • *****
  • Posts: 11551
  • Assembler is fun ;-)
    • MasmBasic
Re: Delete Crawler
« Reply #2 on: August 03, 2014, 01:35:57 AM »
Nice idea :t

Good if you have a backup e.g. of fotos, and want to cleanup the original drive.

dedndave

  • Member
  • *****
  • Posts: 8829
  • Still using Abacus 2.0
    • DednDave
Re: Delete Crawler
« Reply #3 on: August 03, 2014, 01:52:36 AM »
it needs some work - but got the job at hand done

i want to re-write it so it will handle seperate paths
i.e. start at non-root paths, and work with different base paths for dest and source

Gunther

  • Member
  • *****
  • Posts: 3723
  • Forgive your enemies, but never forget their names
Re: Delete Crawler
« Reply #4 on: August 03, 2014, 10:42:02 AM »
Dave,

good job. Thank you for providing it.

Gunther
Get your facts first, and then you can distort them.

dedndave

  • Member
  • *****
  • Posts: 8829
  • Still using Abacus 2.0
    • DednDave
Re: Delete Crawler
« Reply #5 on: August 04, 2014, 03:07:22 AM »
thanks, Gunther

plenty of room for improvement
but, i thought it might save someone a little coding time if they wanted such a function

KeepingRealBusy

  • Member
  • ***
  • Posts: 426
Re: Delete Crawler
« Reply #6 on: August 04, 2014, 04:04:35 AM »
Dave,

How do you determine that the file is the same as another file? Can the name change? Can the path change? Can the drive change? Can the date/time change? Obviously, the size cannot change. Do you check or calculate the two CRCs to see if they match?

I ask because my better half has a habit of changing filenames (without a content change), moving directories around to different parent directories, etc. Her backup method is to drag and drop the entire data drive root (E drive) to a backup drive, and create a total new save directory with massive file duplication. This is not necessarily bad because she has a duplicate (or many duplicate) of any file she may loose on her main data drive.

However, she has completely filled a pair of 1 TB drives, I need to get her a pair of 4 TB drives. This is not necessarily a good thing either, as I found out. My new 4 TB drives had some bad space, unbeknown to me, and I discovered that fact when I tried to copy one backup directory to the other drive of the pair. Trying to do a complete surface scan on 4 TB drives is a bitch.

Dave.

dedndave

  • Member
  • *****
  • Posts: 8829
  • Still using Abacus 2.0
    • DednDave
Re: Delete Crawler
« Reply #7 on: August 04, 2014, 04:41:56 AM »
this version only deletes files if they have the same path (except drive letter) and name
it suits my needs when i create an image of a drive, then restore it and wish to remove some duplicates

here is a  scenerio where it makes sense...

a) i create 2 partitions from the same image - they are now identical except drive letter
b) i delete all the "data" files from one of the 2 partitions:
pictures, e-mails, browser bookmarks, asm source files, PDF's and other documents, etc
c) now, i can use the crawler to delete everything but the data files from the other partition
after some clean-up, i have the data files on one partition and the system/program files on the other

in the case of finding match files with different names and locations, that's a whole different app
this one compares the 2 files for exact match, once a location/name/size match has been found
look at the "DoCompare" PROC to see how it's done

dedndave

  • Member
  • *****
  • Posts: 8829
  • Still using Abacus 2.0
    • DednDave
Re: Delete Crawler
« Reply #8 on: August 04, 2014, 04:57:36 AM »
this particular case was a little different than mentioned above

some time ago, i had created a backup of my then current boot drive
so - that partition and the one i was using differed by files added or updated since that time
the one i was using as a boot partition got messed up
so - i switched to the other one, but now i wanted just the files that were added or had been changed   :biggrin:

the messed up partition had something like 55-60 Gb on it
after i ran the crawler - and did some clean-up - i got it down to less than 1 Gb of files  :t
all the stuff that was messed up is now gone
i moved that 1 Gb to another partition - and rebuilt that one as another boot partition