Large buffers

[jj2007]

Couldn't find it either, Dave, but qWord's link is crystal clear:

VirtualSize
The total size of the section when loaded into memory. If this value is greater than SizeOfRawData, the section is zero-padded

SizeOfRawData
The size of the section (for object files) or the size of the initialized data on disk (for image files). For executable images, this must be a multiple of FileAlignment from the optional header. If this is less than VirtualSize, the remainder of the section is zero-filled

Section Data
Initialized data for a section consists of simple blocks of bytes. However, for sections that contain all zeros, the section data need not be included.
The data for each section is located at the file offset that was given by the PointerToRawData field in the section header. The size of this data in the file is indicated by the SizeOfRawData field. If SizeOfRawData is less than VirtualSize, the remainder is padded with zeros

[Tedd]

.bss = "Blank Space Section"

It is necessarily zero initialised.

C programs have always relied on this fact, so even if it wasn't officially, it would have to be de facto.

[dedndave]

bss = block started by symbol

[jj2007]

bss = better save space

Jokes apart, there are sources that pretend that the language C somehow initialises global data. A quick test with Pelles C shows it's the OS, not C.
- Launch Olly with the exe produced by the snippet below
- get the address (for me, it was 4090C4)
- relaunch and poke a nice string into that memory before the code starts running
- hit F9 and let C do its initialisation stuff
- the poked string is still there

#include <stdio.h>

char* buffer[400000];

int main(int argc, char* argv[]) {
  _asm int 3;
  _asm mov ecx, buffer; // get the address of the string here
  printf("The string we poked: [%s]\n", buffer);
}

[sinsi]

BSS = Block Started by Symbol

To be fair, in DOS the C runtime would fill bss with zeros before main().

[Gunther]

Hi sinsi,

BSS = Block Started by Symbol

To be fair, in DOS the C runtime would fill bss with zeros before main().

right. But the root of BSS is the assembly language. It was a pseudo-operation in the United Aircraft Symbolic Assembly Program (UA-SAP), the standard assembler for the IBM 704, developed in the 1950s.

Gunther

[hutch--]

For all of the gravity of this discussion, the bottom line is if you need more than a trivial amount of memory, dynamically allocate it, otherwise your application will sit in memory occupying far more than you may need all of the time. With dynamic allocation you can turn it on and off, with BSS memory you are stuck with it until the app terminates. It is really lousy code design to set big blocks of BSS memory when with dynamic allocation, you can routinely allocate into the gigabyte range.

[Gunther]

It is really lousy code design to set big blocks of BSS memory when with dynamic allocation, you can routinely allocate into the gigabyte range.

Amen to that.  :t

Gunther

[K_F]

It is really lousy code design to set big blocks of BSS memory when with dynamic allocation, you can routinely allocate into the gigabyte range.

Normally with intermittent memory requirments I'd agree with you....

On my lot I need the larger buffers for 'immediate data' which is referenced continously. Doing it via heap allocation, means reloading and recalculating every time a button is pressed.
Had to weigh up between speed and memory allocation and the easiest/fastest method of doing it.
:)

[dedndave]

under DOS, the BSS segment might have been filled with whatever garbage previously filled the area
compiler code may have cleared it for you, but ASM programmers didn't necessarily have that startup code

[hutch--]

Van,

Just step back 1 branch of your call tree, allocate and deallocate in the same scope OR use a GLOBAL handle for the memory and you won't have either problem.

[TWell]

jj2007 and dedndave.

With PellesC it's possible to use it without CRT like this

Code Select Expand

#include <stdio.h>

void __stdcall ExitProcess(int);
#pragma comment(lib, "msvcrt.lib")

char buffer[400000];

//int main(int argc, char* argv[])
void __cdecl mainCRTStartup(void)
{
  printf("The string we poked: [%s]\n", buffer);
  ExitProcess(0);
}and it generates this code

Code Select Expand

mainCRTStartup
[401000] push ebp
[401001] mov ebp,esp
[401003] push buffer
[401008] push 402000
[40100D] call _printf
[401012] add esp,8
[401015] push 0
[401017] call _ExitProcess@4
[40101C] pop ebp
[40101D] ret for x64

Code Select Expand

mainCRTStartup
[40001000] sub rsp,28
[40001004] lea rdx,[buffer]
[4000100B] lea rcx,[0000000140003010]
[40001012] call printf
[40001017] mov ecx,0
[4000101C] call ExitProcess
[40001021] add rsp,28
[40001025] ret

[jj2007]

With PellesC it's possible to use it without CRT

The point is actually that even WITH the usual compiler overheads, nobody fumbles with global buffers except the OS itself:
- compile my code above
- load it into Olly
- run it once until the breakpoint, just to see where that buffer is located
- reload with Ctrl F2
- put the address into eax, e.g. 4090C4
- right-click on eax and click "Follow in dump"
- double-click the cell under Hex dump in the lower left corner
- in the "Edit data" popup, write Assembler is better into the upper edit box and click OK
At this point, your code has not yet started. Only the OS has done its job so far, but the CRT and its overhead have not yet done ANYTHING.
Now you run the code with F9, and you arrive at the point where printf gets its argument. If the theory is correct that the C compiler zeroes the .bss segment, then there should be zerobytes in the buffer. But instead (tested with Pelles C and VC Express), there is the text you poked in before the program even started.

[TWell]

My point was to show that OS gives that zeroed memory as good multiuser OS should do.

[jj2007]

My point was to show that OS gives that zeroed memory as good multiuser OS should do.

We agree on that point, but proving it is difficult. You might find zerobytes in the global buffer because the OS gave you "by accident" a memory area that was zeroed. Maybe one in a Million times the OS gives you garbage - how can you prove that? So I chose the other way: To prove (successfully) that the C compiler does nothing to the memory, i.e. it relies on the memory being zeroed by the OS.