Looking for someone who's delved into the Registry hard core.

[GuruSR]

I'll explain:

I'm under a few NDAs (Non-Disclosure Agreements) and can't really "say" what is what, but I can "mock" the situation to give you a clue.

I'll call the location as having a "lab" (basically it's a room of computers, so I call that a lab, they don't).  In this "lab" are machines that when you log out, "revert" the installed software (only) to their default setup values, thanks to code I wrote ages ago (and I do mean ages ago).  For the longest time they were using XP (32bit) and have gone to some Vista (32bit) machines as the XP ones died, but, recently, they've decided they want another "lab" and now are given Windows 7 (64bit) machines...  Sadly, the code I wrote, mangles under 64bit due to the insanity called "Wow64".

Basically what I did was wrote 2 parts of a program, one that took the original snapshot of Windows, then another snapshot of Windows minus the original stuff after they'd installed their needed software.  So what I wound up with was registry entries and files that supposedly didn't exist in Windows (and apparently some updates got in the way when they did it and, err, yeah, most likely C libraries being included in their software).  Now, I *can't* have their software (it doesn't leave their building, proprietary and all), so what I did was (and this was under XP) did a directory listing of the entire C drive, and dumped a copy of the registry.  Installed 1 program, then repeated the listing and dump.  Then what I did do, was to (painfully) go through the registry and "mark" keys out that were *bad* to import back into Windows, until what I got was a working copy of Windows with all the software's settings intact, so when the program was run to "revert" the changes, it should put back all the original installation information, software settings, registration of libraries, etc.  Basically it did a "Software Restore" on only the software.

As that was done, they (being their IT department) got lazy, and just set the logout script to run my "revert" software as it was to do each logout.  And then when it was logged out fully, they'd simply log in, the software (shortcuts on the desktop, start menu, etc) would all be there, even the uninstall information (so they could actually uninstall the software and log out and back in and it'd be like they never did it).  Now this all worked fine in a 32 bit environment (XP) and oddly worked fine in Vista (apparently, they never asked me to update it).  Except the 64 bit, which blew the situation into the air, dlls are not registered properly, etc.  But, *you can* uninstall the software, go figure, that part still works.  Nothing else does, the shortcuts show the proper program, but fail to run due to SideBySide configuration in some instances and others, libraries not found.

Now, since I've been working on this problem for a few months, I'm at the situation that I really need to get this working, but, I can't share much with anyone (the code is too huge to even send part of it).  Worst part is, the data I'm excluding, I've since added some to trim the registry sections (and I think I shot myself in the foot a lot doing this).

I have a large chunk of registry keys I'm "ignoring", but I'm thinking I am overdoing this and shouldn't be, but again, everything I've tried has led me to think I'm over analyzing the situation and am making the registry stripping rip out stuff I need.  The raw registry package is present, but it holds things in the registry that were changing during the snapshots (since windows loves updating thousands of keys a second).  The other problem is, the data needs to work in the 32bit AND the 64bit environments at the same time, so basically their lazy IT department can just add the same logout task and poof, everything works next login.  Now the software is obviously all 32 bit, so this isn't a big issue.

I *had* backups of the original code waaay back when, but since it's been 2 months, those backups are long gone, so I'm pretty sure I've over done it with the "ignoring" registry key sections.  And before anyone asks, no, "System Restore" is out of the question as it's easily destroyed in all versions of Windows and requires a reboot and undoes key system changes but not always your software, have also seen it damage software (personally).

I'd really love some assistance in this.  I can list the registry keys I have in the filtering here (it's huge) if someone has some insight on it, it'd be really great.

GuruSR.

[anunitu]

Question: if you are a "Guru",then why exactly are you seeking solutions from this forum?

Also from this statement,it seems you are in an "Enterprise" programming group.

"I'm under a few NDAs (Non-Disclosure Agreements) and can't really "say" what is what, but I can "mock" the situation to give you a clue."

So you would seem to want to "HIRE" a programmer to assist in this "Task",right?

[jj2007]

Hi Guru,
Check Running a 32-bit App on WOW64 can be weird, it might be relevant for you.

[hutch--]

Best of luck there Guru but I doubt you will find anyone with much interest as its a tedious task at best with little to gain from it unless they have the same problem that you have. I gather from what you have written that the earlier data is not even backed up so youu cannot go back to scratch to fix it. Personally I avoid the registry like the plague and only produce portable apps that don't use the registry at all.

[mineiro]

I understand that your program do something like "shadowuser" or "deepfreeze" programs, so, your program restore windows to previous point.

But I don't understand where you need help sir GuruSR, can you be more specific?

[rrr314159]

Personally I avoid the registry like the plague and only produce portable apps that don't use the registry at all.

The voice of wisdom! But some people are unfortunately forced to deal with the registry mess. If the advice is applicable to OP, he might consider just getting rid of registry entries. Instead store user data in a file, in his own format, as I (and presumably you) do. However there's probably too much code to change.

[Zen]

GURUSR,
You can't possibly be writing all this code in MASM assembly language,...(I mean, you could, but, you'd already be toast)
I thought that your original post described an interesting project,...conceptually,...but, in real, cyber-life,...I'm fairly certain that actually writing the source code to accommodate all possible scenarios will surely induce some bizarre type of terminal, degenerative mental dysfunction.

...But, if what you stated above is true,...you should definitely be getting hazard pay,...and, diplomatic immunity,...and, at the very least,...all the Sports Illustrated Swimsuit Models you can handle,...

...The level of complexity that you are describing, is causing my nose and ears to bleed,...
...But,...hey,...send me a Non-Disclosure Agreement,...I could really use a near-death experience to brighten up my day,...

If it were me,...I'd be creating a number of different COMPLETE Registry Databases, instead of trying to filter the clusters of registry keys (associated with the current software update),...Although, I'll bet that whatever 32-bit or 64-bit Windows Operating System Version running on an individual computer that you will re-configure with your NDA-enabled 'revert' software won't like it at all,...

[sinsi]

VirtualPC or XP mode?

[GuruSR]

Question: if you are a "Guru",then why exactly are you seeking solutions from this forum?

Also from this statement,it seems you are in an "Enterprise" programming group.

"I'm under a few NDAs (Non-Disclosure Agreements) and can't really "say" what is what, but I can "mock" the situation to give you a clue."

So you would seem to want to "HIRE" a programmer to assist in this "Task",right?

Well, let me elaborate on this, a while back (late 90's) came along a version of Windows called 98 (everyone remembers it, mostly), and the annoying part was "scanreg /restore" wouldn't undo crashes and backup software, well, what was that?  So, for the most part you *could* repair Windows with the disc(s) you had for it *IF* you hadn't ever installed that annoyingly EVIL "Dot Net" (and that same thing applied all the way to XP).  So, to avoid my "fun" of having to reinstall all those programs after all the 1001 updates on DIALUP (yes, did say that), I broke into the cabs, got the Copy data for the installation of Windows out of it, went ahead and dumped the registry to a reg file and proceeded to write the program to rip out everything that was "Windows" from the registry (that I could notice, back then MS had very little in programming stuff and the Internet, well, was less so forthcoming on real answers, much like today).  Then it created a restore package which I could run after reinstalling Windows (a fresh copy, because of the "fantastic Dot Net"ness) and return everything back to the way it was before the refreshed OS install.  It worked great for 98, so I'd been talking to a friend who worked at some facility that was doing trials and such and they have computers in a room (back then they were 98's, went to XP and so forth), doing some testing on software made for them by someone else, but the problem was, people would when they were done, mess around with the computers, typically doing things like uninstalling software, deleting shortcuts, that sort of thing.  System Restore wasn't possible back then (that was later added in Millenium with a fail of it's own), no clue if they ever used any ME version machines, sold one ME machine with multiple 98's prior to it, then XPs, 7's and now 10's.

Anyhow, my friend there told them that he'd heard I'd made a program to put software back to the way it was post a problem with the system, well, apparently, that got them interested and he called me, told me to call a number (forget it now, that was a few years back) and I did.  I had to make the code do more than I did, because it had to form to their needs, so after a few months of hard coding, I got something that worked.  Now, not 100% sure, but I *may* have a ghost backup of that era around somewhere (2000 version), I haven't gone totally digging.

As for being in an "Enterprise" programming group, err, 2 words, "Hell NO!"  Something about frying my brain coding 24/7, err, pass.  The NDAs I had to sign were for legal purposes on their end to prove to whoever they were doing trials for that "I wouldn't say 'anything' about the testing".  Actually was never there to *see* any testing, just a room with computers in it through a small window on a metal fire door.

My hopes (and it was a streeeeetch, as I hate working with the registry on this gritty level, worst ever design for storing data), was someone who may have run into registering a library or any type of object into the registry manually (and not letting the installer do it).  I recall there were programs from long ago that would register items for you if you "asked" the program to, like "install this object" or "add toolbar to menu", etc.

Hi Guru,
Check Running a 32-bit App on WOW64 can be weird, it might be relevant for you.

I've been through the Wow64 stuff MS offers and, well, I'm certainly not Wowed.  Redirected & Reflected (means "copied to both places"), why didn't they say that?  TypeLibs and Interfaces are copied to both 64 and 32 bit versions while straight Classes are always 64 bit, CLSIDs are 32 or 64, but never both (this part is independant), APPIDs are the same as CLSIDS.  I am not worried about running my apps at all on a 64 bit system, it's just that I need to ensure that when my program restores their software, it puts all those pesky registry entries where they belong.  I'm going to do more testing as it failed to do it right again (but this time IE didn't get trashed, apparently my test copied the HTML Class info for IE and it ruined it).

Best of luck there Guru but I doubt you will find anyone with much interest as its a tedious task at best with little to gain from it unless they have the same problem that you have. I gather from what you have written that the earlier data is not even backed up so youu cannot go back to scratch to fix it. Personally I avoid the registry like the plague and only produce portable apps that don't use the registry at all.

Thanks Hutch, I was hoping on I guess a bit of a whim that someone may have delved deep and dark into the mess Microsoft calls the Registry.

I understand that your program do something like "shadowuser" or "deepfreeze" programs, so, your program restore windows to previous point.

But I don't understand where you need help sir GuruSR, can you be more specific?

In a nutshell, run program on a machine without any software, collect the file information (basically a DIR of the entire drive root drive), then with the preset exclusions of the registry "back it up and the non-windows files" after you've installed software.  Then on the flip side, you run it with the restoration and it puts the registry parts back (manually, thanks to keys not being properly written to during my original testing years ago), copies the files that don't exist in the default windows system & user areas and exits.  My issue is it's doing it, but in Windows 7, a ton of new registry entries were added (can you say bloat?) and last thing I wanted to do was to corrupt Windows with me putting stuff back that could cause problems with updates and such, which it currently was doing until I started limiting registry sections (maybe limited too many).  The installer/uninstaller sections are scanned for the keyword MS uses to denote an Update (scan the internet for that one, it's not told anywhere, <SARCASM>love their MSDN</SARCASM>).

Personally I avoid the registry like the plague and only produce portable apps that don't use the registry at all.

The voice of wisdom! But some people are unfortunately forced to deal with the registry mess. If the advice is applicable to OP, he might consider just getting rid of registry entries. Instead store user data in a file, in his own format, as I (and presumably you) do. However there's probably too much code to change.

I normally avoid them too, write my own data type out into a proper user account location, just wish this was that simple...

GURUSR,
You can't possibly be writing all this code in MASM assembly language,...(I mean, you could, but, you'd already be toast)
I thought that your original post described an interesting project,...conceptually,...but, in real, cyber-life,...I'm fairly certain that actually writing the source code to accommodate all possible scenarios will surely induce some bizarre type of terminal, degenerative mental dysfunction.

...But, if what you stated above is true,...you should definitely be getting hazard pay,...and, diplomatic immunity,...and, at the very least,...all the Sports Illustrated Swimsuit Models you can handle,...

...The level of complexity that you are describing, is causing my nose and ears to bleed,...
...But,...hey,...send me a Non-Disclosure Agreement,...I could really use a near-death experience to brighten up my day,...

If it were me,...I'd be creating a number of different COMPLETE Registry Databases, instead of trying to filter the clusters of registry keys (associated with the current software update),...Although, I'll bet that whatever 32-bit or 64-bit Windows Operating System Version running on an individual computer that you will re-configure with your NDA-enabled 'revert' software won't like it at all,...

Thanks Zen, these past 2 months+, that I've been working on this, has been "run the test", "didn't work, WTH!", use my logging library, pop in TONS of logging, last time I saw the log it was 128MB, going to sift through that later, possibly see if I can register one of the libraries it's complaining about with RegShot running before and after, see if I'm missing something and see what changed, so I can track down if the package I made is missing it or if it's actually not in the test.  I have 1 PC with 2 copies of Windows backups, 1 with and 1 without the software I put in (basically a test), problem was, I had to find some heavily involved software, so installed (SWTOR) an online game (never ran it), backed the machine up, made the package with the machine after the install, restored the empty backup and ran my program to restore the software...  The game says libraries are missing, etc, I did install EditPad Lite, but since it uses an ini file, it works fine, so I could read the log file with ease and to ensure the Uninstall has it, because the SWTOR didn't seem to have one.  As for creating different databases, love to, but the problem with that is, they'd be restored on a variety of machines, hardware wouldn't be 100% the same, so there's issues with video cards, sound cards, etc, that would go poof, if I put data back in the wrong places.  Heck, the first time I tried to get the Windows 7 to boot after putting the first test in, gave me a black screen with a mouse pointer, never got to the login screen in or out of Safe Mode!

VirtualPC or XP mode?

VM mode, can't say Real Mode, since I'm not writing a kernel driver.  Looked at doing that once, then shuddered at the insanity.  I'm not interested in going that crazy at writing kernel drivers.

GuruSR.

[mineiro]

From an user point of view with administrative priviledges doing this manually:
User list all files/folders (dir) inside hard disk (partition/mounted volume) and store result into before.txt. After, install some program. List again all files and store result on after.txt . Next step is compare before with after, the difference is just what program inserted on hard disk.
Comparisions can be done by using "windiff" program or another, or just by hashing all files and comparing hashes of files after.
To registry user uses regedit or reg32 and export all keys the same way, before.reg and after.reg, compare both and by exclusion know what that program inserted into registry database.

From an super user point of view:
Monitors files and registry activities by using some of sysinternals tools, exclude a lot of keys that does not be monitored, save these keys like a profile for next sessions, so wasted time is spend only by finding what program is inserting/deleting. Comparisions are done the same way.

The problem with regedit32 family is that it does not export all keys inside registry database. I learned this by hard way. Some keys cannot be restored because windows is using it.
Other problem is when user get some malicious programs that delete keys inside registry database, this way user can't turn on computer on safe mode because that keys have been deleted.
So, is not exclusion only thinking, need have an inclusion too, just this scenario, user need know what keys are need to enable safe mode again.
My opinion say that regedit or reg32 programs are not tools, they are toys.

Other approach is to just copy registry database (.dat) files to after restore/overwrite, but again, it's not possible because Windows is using that files.

I think I understand your point of view about this zoo inside registry.

Well, the best that you can do as a programmer without using drivers is use ntdll functions with correct ACL and permissions/priviledges.
(NtInitializeRegistry,NtRegGetRootPath,RtlInitUnicodeString,NtOpenKey,NtCreateKey,NtNotifyChangeKey,NtSetValueKey,NtQueryKey,NtQueryValueKey,NtQueryMultipleValueKey,NtEnumerateValueKey,NtDeleteKey,NtDeleteValueKey,)
(NtCreateFile,NtSaveKey,NtClose,NtLoadKey2,NtUnloadKey) to backup/load a hive key in a file
InitializeObjectAttributes macro
Keys need be aligned to dwords to access and should be unicode.
Permissions: "SeSecurityPrivilege","SeBackupPrivilege","SeRestorePrivilege","SeManageVolumePrivilege",RtlAdjustPrivilege, LookupPrivilegeValue, AdjustTokenPrivileges, CloseHandle, ....

So, from win95 to win10, keys have been added, other turned obsolete, again, a check of windows version is need because keys changes, and not only this because have device drivers that are machine dependent.
And the same to files/folders: NtQueryAttributesFile,NtCreateFile,NtFileGetFileSize,NtQueryInformationFile,FileTimeToSystemTime,NtReadFile,NtDeleteFile,...

I know is hard do this, I have tried on past a program like yours, just to create a report of machines, like listing drivers, services, files, keys, processes, ..., to help others to remove malicious programs but I simply don't finish that program because WOW.
And look that I'm not talking about "hidden data". On win98 per example, a program insert some data inside unused disk space and setup that hard disk sector as being a 'bad block' by using fat32 manipulation. Today with ntfs have streams.
Other thing is about backup of firmwares like routers, bios, video and sound cards, ... .

I'm out of windows world and I'm happy. I just start a computer with linux, mount a ntfs partition and do all that thing as a normal user. If I need insert or delete keys into windows registry database I use "chntpw" program.
Well, I think sysinternals have done a cd-rom that can been booted to repair windows, have others too that are freeware but I forgot program names. But maybe not your needs because that logoff script.
Well, I only see possibilities without using drivers this way. Maybe a dos version that have a program to manipulate files inside ntfs, but, again, not your need.

Good luck.

[Vortex]

Hi mineiro,

The registry is always causing trouble and a lot of Windows crashes are happening because of the corruption of the registry. I prefer plain text files employed by the UNIX\Linux systems.

[jj2007]

To registry user uses regedit or reg32 and export all keys the same way, before.reg and after.reg, compare both and by exclusion know what that program inserted into registry database.
...
Good luck.

Good luck indeed: Win-R, regedit. Click on Computer, File, Export. On my current modest Win7-64 installation, it's 630MB, and it contains almost 2,000 matches for masm - not bad for a package that doesn't even write to the registry, right?
8)

[mineiro]

I fully agree with you sirs Vortex and jj2007. I prefer the old .ini files to hold configurations. When Microsoft created a heart (registry) inside windows things goes down. I hate that thing too jj2007.

Have other things that I have not said like: When a program deflate/extract a file, execute that file and after delete that created file, so logs from normal user point of view are not seen.
Many programs today that generate text report files to be sent to boards around the world, so a guy analyse that report and answer to user how to proceed to remove ads from browsers to malicious softwares uses "erunt" software to create/restore registry backup, so this can be usefull to GuruSR becauses it's a freeware program, but we are programmers:

Code Select Expand


ERUNT - The Emergency Recovery Utility NT
=========================================

Registry Backup and Restore for Windows NT/2000/2003/XP

v1.1j, 10/20/2005, Freeware
Written by Lars Hederer
e-mail: lars.hederer@t-online.de

Look for the latest version here:
http://www.larshederer.homepage.t-online.de/erunt

To find out what's new in this version, please see the "Version
history" section later in this file.

Introduction
------------

With the invention of Windows 95 Microsoft made the wise decision to
organize all computer- and application-specific data which was spread
over countless INI files before in a centralized Windows database,
called the system "registry". The registry is one of the most
important parts in every Windows system today, without which the OS
would not even boot. And since the registry is quite sensitive to
corruption, it is very advisable to backup its according files from
time to time.

In MS-DOS based Windows versions (95, 98, Me) the registry consists of
the files SYSTEM.DAT and USER.DAT (and CLASSES.DAT in Windows Me). To
backup these files, one can easily go to the Windows folder in
Explorer and copy the files to a safe location, for example another
folder on the hard disk. Microsoft even supplies a utility called ERU
which can be used to backup these and a few other critical system
files to a safe location.

Also, Windows 9x/Me automatically create backups of the registry at
startup, with Windows 95 always backing up the registry from the
previous Windows session, and Windows 98/Me maintaining up to five
registry copies from the last five days where Windows was running.

Unfortunately, this is not the case with Windows versions based on the
NT kernel. In Windows NT and 2000, the registry is never backed up
automatically, and in XP it is backed up only as part of the bloated
and resource hogging System Restore program which cannot even be used
for a "restore" should a corrupted registry prevent Windows from
booting. It has also become impossible to copy the necessary files,
now called "hives" and usually named DEFAULT, SAM, SECURITY, SOFTWARE,
SYSTEM in the SYSTEM32\CONFIG folder, to another location because they
are all in use by the OS. And though the registry in an NT-based
Windows is less likely to become corrupted than in other versions, it
can still happen, and for these cases NT is simply missing an option
for easy registry backup and restore as there is in Windows 9x/Me, to
get the system up and running again in no time.
In 2001, as Windows XP began to come pre-installed on many new home
user PCs and was likely to become the new Windows standard over the
next years, I decided to write a program which offers the ease-of-use
of Windows 9x/Me ERU by Microsoft (hence the name ERUNT) to backup the
registry, as well as providing an auto-backup capability, for example
at Windows startup.

Or, before installing a new program for testing purposes one could
save the registry with ERUNT, install and test the program, uninstall
it and restore the registry to be 100% sure that no debris is left.

Note: The "Export registry" function in Regedit is USELESS (!) for
making a complete backup of the registry. Neither does it export the
whole registry (for example, no information from the "SECURITY" hive
is saved), nor can the exported file be used later to replace the
current registry with the old one. Instead, if you re-import the file,
it is merged with the current registry without deleting anything that
has been added since the export, leaving you with an absolute mess of
old and new entries.


My conclusion about fighting against malicious programs was easy, be a dumb and do dumb things, the most smart guy on earth face that create malicious software will fail. Just create a folder with same program name it's the most dumb solution. Well, this info can be good or bad, just a point of view.

[GuruSR]

Well, at present, I have to restore the machine back to the "installed" backup, as I did a restore test and it still failed, then I installed one of the test programs I had put in with RegShot catching the changes done to the registry during the install and it looks like it's missing values in the class registrations, which I seem to remember being in the original, but am not sure, so I saved the portion of the registry out to verify it, though sadly, my days have been busier than I wanted and I got home about an hour ago and well, about to crash for the night and up early to be out the door again.

Once I get this mess working, I'll be *glad* to avoid the registry again, and move onto other things.  Maybe even get a webspace going for putting up some of my software for others to gawk at (and laugh at my ugly GUIs).

GuruSR.

[rrr314159]

Maybe even get a webspace going for putting up some of my software for others to gawk at (and laugh at my ugly GUIs).

Of course you're welcome to post anything interesting here. Most masm32'ers have short attention spans, but sizable projects have been posted