News:

Masm32 SDK description, downloads and other helpful links
Message to All Guests
NB: Posting URL's See here: Posted URL Change

Main Menu

ExitProcess releases all resources, right?

Started by jj2007, September 17, 2017, 08:47:33 PM

Previous topic - Next topic

jj2007

Raymond Chen once wrote about ExitProcess in The Old New Thing:
QuoteThe building is being demolished. Don't bother sweeping the floor and emptying the trash cans and erasing the whiteboards. And don't line up at the exit to the building so everybody can move their in/out magnet to out. All you're doing is making the demolition team wait for you to finish these pointless housecleaning tasks.

Okay, if you have internal file buffers, you can write them out to the file handle. That's like remembering to take the last pieces of mail from the mailroom out to the mailbox. But don't bother closing the handle or freeing the buffer, in the same way you shouldn't bother updating the "mail last picked up on" sign or resetting the flags on all the mailboxes. And ideally, you would have flushed those buffers as part of your normal wind-down before calling Exit­Process, in the same way mailing those last few letters should have been taken care of before you called in the demolition team.

I regularly use a program that doesn't follow this rule. The program allocates a lot of memory during the course of its life, and when I exit the program, it just sits there for several minutes, sometimes spinning at 100% CPU, sometimes churning the hard drive (sometimes both). When I break in with the debugger to see what's going on, I discover that the program isn't doing anything productive. It's just methodically freeing every last byte of memory it had allocated during its lifetime.

If my computer wasn't under a lot of memory pressure, then most of the memory the program had allocated during its lifetime hasn't yet been paged out, so freeing every last drop of memory is a CPU-bound operation. On the other hand, if I had kicked off a build or done something else memory-intensive, then most of the memory the program had allocated during its lifetime has been paged out, which means that the program pages all that memory back in from the hard drive, just so it could call free on it. Sounds kind of spiteful, actually. "Come here so I can tell you to go away."

All this anal-rententive memory management is pointless. The process is exiting. All that memory will be freed when the address space is destroyed. Stop wasting time and just exit already.

My highlighting for the "good programming practice" fraction ;)

In 99% of all cases, ExitProcess really does the job: It throws away the processes' address space. Bang, gone. No need to free global or heap memory, it's just gone.

The problem are the 1% that do not get released: Apparently, ExitProcess does not release menus created with CreatePopupMenu, as I just found out. Unfortunately, to my knowledge M$ has never clearly documented which kind of "objects" do not get released during ExitProcess. Example: Do DCs get released? Yes and no. Even a precise quoted Google search for "exitprocess" "device context" doesn't reveal anything. My best guess - it's difficult to prove - is that (Chen: What does the CS_OWNDC class style do?) the DCs of windows created with CS_OWNDC will be released by ExitProcess, the others not (which is not a problem normally, because every decent WM_PAINT handler releases it anyway).

Another candidate for the "globally shared resource, must be explicitly released" list are global atoms.

P.S.: We had a thead in 2014 about this, worth reading. SOF also offers a thread on possible for a process to result in leaked memory after it is killed?

jj2007

Quick test whether fonts fall in the category "fully released":

include \masm32\MasmBasic\MasmBasic.inc         ; download
  Init
  xor ecx, ecx
  PrintLine "counter ticks       hit Ctrl C to stop the test"
  .Repeat
        inc ecx
        Print Cr$, Str$(ecx), " "
        Launch "StressTestMakeFont.exe"         ; the exe creates ten fonts
  .Until edx!=123
  deb 1, "test finished", edx, $Err$()
EndOfCode


Here is the source of the exe that creates fonts but does not delete them on ExitProcess:

include \masm32\MasmBasic\MasmBasic.inc         ; download
  Init
  Print Str$(rv(GetTickCount))  ; feedback
  MakeFont hF0, Height:10, "Arial"
  MakeFont hF1, Height:11
  MakeFont hF2, Height:12
  MakeFont hF3, Height:13
  MakeFont hF4, Height:14
  MakeFont hF5, Height:15
  MakeFont hF6, Height:16
  MakeFont hF7, Height:17
  MakeFont hF8, Height:18
  MakeFont hF9, Height:19
  invoke GetLastError
  add eax, 123
  Exit eax                                      ; Launcher will check for !=123
EndOfCode


In a first test on Win7-64, there was no problem with well over 80,000 launches, i.e. 800,000 fonts.

See also this old thread

sinsi

QuoteApparently, ExitProcess does not release menus created with CreatePopupMenu
According to MSDN, the only time a menu is destroyed automatically is if it has been assigned to a window (during CreateWindowEx or with SetMenu)
or as a template with RegisterClassEx.

LiaoMi

Hi,

PspExitProcess(
    IN BOOLEAN TrimAddressSpace,
    IN PEPROCESS Process
    )
{

    ULONG ActualTime;

    PAGED_CODE();

    if (!Process->ExitProcessCalled && PspCreateProcessNotifyRoutineCount != 0) {
        ULONG i;

        for (i=0; i<PSP_MAX_CREATE_PROCESS_NOTIFY; i++) {
            if (PspCreateProcessNotifyRoutine[i] != NULL) {
                (*PspCreateProcessNotifyRoutine[i])( Process->InheritedFromUniqueProcessId,
                                                     Process->UniqueProcessId,
                                                     FALSE
                                                   );
            }
        }
    }

    Process->ExitProcessCalled = TRUE;

    PoRundownProcess(Process);

    //
    // If the process is on the active list, remove it now. Must be done before ObKill
    // due to code in ex\sysinfo that references the process through the handle table
    //

    if ( Process->ActiveProcessLinks.Flink != NULL &&
         Process->ActiveProcessLinks.Blink != NULL ) {

        ExAcquireFastMutex(&PspActiveProcessMutex);
        RemoveEntryList(&Process->ActiveProcessLinks);
        Process->ActiveProcessLinks.Flink = NULL;
        Process->ActiveProcessLinks.Blink = NULL;
        ExReleaseFastMutex(&PspActiveProcessMutex);

    }

    if (Process->SecurityPort) {

        ObDereferenceObject(Process->SecurityPort);

        Process->SecurityPort = NULL ;
    }


    if ( TrimAddressSpace ) {


        //
        // If the current process has previously set the timer resolution,
        // then reset it.
        //

        if (Process->SetTimerResolution != FALSE) {
            ZwSetTimerResolution(KeMaximumIncrement, FALSE, &ActualTime);
        }

        if ( Process->Job
             && Process->Job->CompletionPort
             && !(Process->JobStatus & PS_JOB_STATUS_NOT_REALLY_ACTIVE)
             && !(Process->JobStatus & PS_JOB_STATUS_EXIT_PROCESS_REPORTED)) {

            ULONG_PTR ExitMessageId;

            switch (Process->ExitStatus) {
                case STATUS_GUARD_PAGE_VIOLATION      :
                case STATUS_DATATYPE_MISALIGNMENT     :
                case STATUS_BREAKPOINT                :
                case STATUS_SINGLE_STEP               :
                case STATUS_ACCESS_VIOLATION          :
                case STATUS_IN_PAGE_ERROR             :
                case STATUS_ILLEGAL_INSTRUCTION       :
                case STATUS_NONCONTINUABLE_EXCEPTION  :
                case STATUS_INVALID_DISPOSITION       :
                case STATUS_ARRAY_BOUNDS_EXCEEDED     :
                case STATUS_FLOAT_DENORMAL_OPERAND    :
                case STATUS_FLOAT_DIVIDE_BY_ZERO      :
                case STATUS_FLOAT_INEXACT_RESULT      :
                case STATUS_FLOAT_INVALID_OPERATION   :
                case STATUS_FLOAT_OVERFLOW            :
                case STATUS_FLOAT_STACK_CHECK         :
                case STATUS_FLOAT_UNDERFLOW           :
                case STATUS_INTEGER_DIVIDE_BY_ZERO    :
                case STATUS_INTEGER_OVERFLOW          :
                case STATUS_PRIVILEGED_INSTRUCTION    :
                case STATUS_STACK_OVERFLOW            :
                case STATUS_CONTROL_C_EXIT            :
                case STATUS_FLOAT_MULTIPLE_FAULTS     :
                case STATUS_FLOAT_MULTIPLE_TRAPS      :
                case STATUS_REG_NAT_CONSUMPTION       :
                    ExitMessageId = JOB_OBJECT_MSG_ABNORMAL_EXIT_PROCESS;
                    break;
                default:
                    ExitMessageId = JOB_OBJECT_MSG_EXIT_PROCESS;
                    break;
                }

            PS_SET_CLEAR_BITS (&Process->JobStatus,
                               PS_JOB_STATUS_EXIT_PROCESS_REPORTED,
                               PS_JOB_STATUS_LAST_REPORT_MEMORY);

            ExAcquireFastMutex(&Process->Job->MemoryLimitsLock);

            if (Process->Job->CompletionPort != NULL) {
                IoSetIoCompletion(
                    Process->Job->CompletionPort,
                    Process->Job->CompletionKey,
                    (PVOID)Process->UniqueProcessId,
                    STATUS_SUCCESS,
                    ExitMessageId,
                    FALSE
                    );
            }
            ExReleaseFastMutex(&Process->Job->MemoryLimitsLock);
            }

    } else {
        KeSetProcess(&Process->Pcb,0,FALSE);
        ObKillProcess(FALSE, Process);
        MmCleanProcessAddressSpace();
    }

}


PspCreateProcessNotifyRoutine
Purpose - Array of executive callback objects
Array of callback objects describing the routines to be called on process creation and deletion (maximum of eight)

ExitProcess
Ends a process, and notifies all attached DLLs

TerminateProcess
Ends a process without notifying the DLLs

    } else {
        KeSetProcess(&Process->Pcb,0,FALSE);
        ObKillProcess(FALSE, Process);
        MmCleanProcessAddressSpace();
    }


MmCleanProcessAddressSpace
QuoteRoutine Description:

    This routine cleans an address space by deleting all the
    user and pagable portion of the address space.  At the
    completion of this routine, no page faults may occur within
    the process.

Arguments:

    None.

Return Value:

    None.

Environment:

    Kernel mode, APCs disabled.

Violation in object synchronization, the main cause of memory leaks, incorrect inheritance of processes, violation of process lists, incorrect programming in the system environment and other reasons. In a particular case, you need to know how to open and how to unload the fonts in the system environment. Under normal windows conditions, there should not be any leaks.

Quoteit just sits there for several minutes, sometimes spinning at 100% CPU, sometimes churning the hard drive (sometimes both)

A small amount of RAM actively affects the behavior and the unloading of the program, since the disk cache is involved.

jj2007

Quote from: LiaoMi on September 25, 2017, 08:02:20 PM
PspExitProcess(
    IN BOOLEAN TrimAddressSpace,
    IN PEPROCESS Process
    )
{

    ULONG ActualTime;

    PAGED_CODE();

    if (!Process->ExitProcessCalled && PspCreateProcessNotifyRoutineCount != 0) {
        ULONG i;

That's interesting stuff (see How Advanced Malware Bypasses Process Monitoring), but my question is a different one...

Quote from: sinsi on September 25, 2017, 07:20:28 PM
QuoteApparently, ExitProcess does not release menus created with CreatePopupMenu
According to MSDN, the only time a menu is destroyed automatically is if it has been assigned to a window (during CreateWindowEx or with SetMenu)
or as a template with RegisterClassEx.

The point here is that MSDN is never clear about what "destroyed" really implies. Is it limited to the processes address space, so ExitProcess and wow, it's gone, or does it affect global resources that are NOT "destroyed" or "released" AFTER the ExitProcess. That is why I designed the launcher that creates fonts in an external exe - currently 1.6 Million fonts created, no effect on my system. So that confirms that fonts fall into the category "ExitProcess and forget about it".

Attached the test for CreatePopupMenu, currently at 400,000 menus created, none explicitly destroyed, and no impact on my Win7-64's capacity to provide me with menus.

The interesting question here is indeed "what can cause the OS to run of resources?". Fonts and menus can be created in the Millions, provided they are not kept in memory - and ExitProcess does take care of removing them completely.

But I have seen the OS in trouble in the past, with systemfixedfont replacing all other fonts, in my browser and other applications. It is definitely possible to tear down the system globally, but how exactly? Not releasing or destroying resources at the end of a program cannot be the cause 8)

jimg

How does this work exactly?
when I run the test, I get a command window that says counter ticks 1 1975609
also I get a message box titled test finished, and it says
edx  173
$Err$()  The specified image file did not contain a resource setion.


How does this tell if the menu was released?

jj2007

#6
Quote from: jimg on September 26, 2017, 12:48:04 AMHow does this tell if the menu was released?

Well, the small exe that gets launched reports via ExitProcess, eax whether there was an error (i.e. menu could not be created etc). In the exe, the ten menus created never get "released" in the sense that MSDN uses. So if the OS has no problems to create a Million new menus, then apparently ExitProcess handles the waste disposal of menus 8)

My assumption was that there is no error yet when the exe gets launched, and that is true for Win7-64; but I tested that now for WinXP and yep, there is already an error on initialisation. So I added a SetLastError(0) now to make it compatible with XP - new version attached. You can now drag one of the exes over the launcher to test e.g. the LoadLibrary behaviour.

On a side note, for a Google search on Russinovich ExitProcess, this Masm32 thread pops up at rank 4.