FreeBasic Runtime Library test

[guga]

Hi Guys,

I´m trying to build a runtime library for FreeBasic so we can try to use it on a external dll. This is what i´ve done so far. Didn´t tested yet, but it seems that the functions are working. Still need to build more then 500 of those thing. OUUCH !!!!

(Edited: Updated with more then 400 export now)

[jj2007]

That's an impressive number of exported functions, Guga. Where do they come from, RosAsm? What does _fb_DevFileReadLineWstr@0 do, for example? Read a line from the default file, no parameters needed?

[guga]

That's an impressive number of exported functions, Guga. Where do they come from, RosAsm? What does _fb_DevFileReadLineWstr@0 do, for example? Read a line from the default file, no parameters needed?

Hi JJ

Tks :)

The functions comes from RosAsm, Idapro and Ollydbg. I was trying to create an alternative algorithm for the atan2 function and remembered the LolRemez algorithm we talked about a long time ago. Then, i searched for it and found our old posts on FreeBasic forum and downloaded the Fb version. (I also downloaded the LolRemnez gcc version from github, but didn´t have time to analyse it yet.)

Then, i did this:

1 - Downloaded Fb (Binary and it´s source code)
2 - Compiled LolRemez in Fb with full debug information.
3 - Disassembled it in idapro 7 to analyze their symbols
4 - Disassembled it in RosAsm to make a compilable and working version. (Well..kind of    )
5 - Uses olly to see the cases where Fb fools idapro.
6 - Exported each one of the "fb_xxxx" functions (from inside the compiled LolRemez) and assembled them with masm, taking care to check on the source code if the functions can really be converted to a Runtime Library (Some functions are not the original api, because Fb uses crappy mingwin libraries to compile the files)


About fb_DevFileReadLineWstr, for what i saw so far, it simply read the inputed string when the user opens a file with fb_FileOpenEx and fb_FileOpen, but internally converts it to Unicode and assign the string to a data used by FreeBasic. In fact, the majority of Fbasic seems to be only a huge parser (full of crap inside, btw). Some of the functions, for example "eats" the data that belongs to other ones, like a function called "hRnd_MTWIST32" that was supposed to do some sort oof randomization using a unique buffer, but, in fact it have a internal failure that causes some of the pointers to point inside a data that is being used by the app.

For what i saw so far, Fb is not that bad, but it do have bugs that needs to be fixed, specially in what concern memory allocation. I´ll continue to try to create a dll of it, just to see if the version i made for LolRemez to RosAsm can uses the Fb libraries. I would prefer that the guys who did FB have already made a sort of dll for external usage, but it seems they have 0 intention to do so.

What is a bit hard is the fact that the source code is not totally documented   

Anyway..i´ll give a try to see if i can do it. I tried yesterday to create the dll using the library file generated by Fb, but i couldn´t be able to make it work. The linker (masm linker) was accusing hundreds of unreferenced libraries etc etc. That´s why i decided to do it from scratch last night.

If you have idapro, i can send to you the files i´m working on to try to create this thing.

[jj2007]

If you have idapro, i can send to you the files i´m working on to try to create this thing.

I have neither IdaPro nor the time ;-)

It sounds like an interesting project, though - keep us posted

[guga]

New version released :) Now i´m trying to work with the graphics Api

[guga]

New version. I guess it is more complete now. 408 exported functions. It maybe missing some more, but i didn´t checked yet. I´m releasing it here so we don´t loose the focus and try to make this thing works as it should be. I´m attaching here and updating the 1st post. Later i´ll try to create a app using this dll to see what happens (And perhaps, add some more exports - I guess it may have still 100 more to be build or something - I´ll check the Fb source code later to see exactly how many functions are missing that could be converted to a Runtime Library).

Btw, since the source is a true mess and huge, i was forced to split it in 3 asm files. As i said previously, Fb is not that bad, but internally is a huge piece of crap due to problems with the linker they are using to build this thing. I don´t have time to analyses each function to fix them all, but if someone wants to, then it is worth-full to give a try to improve this. Many functions are simple, so porting them to masm (or RosAsm, fasm etc etc) shouldn´t be that hard at all. Just be carefull with the Table Pointers, because gcc did a complete mess when they compile this. That´s why you will see on thee code i made several functions (procedures) containing internally 2 "::" after a label because the jmp table used by this thing is looooong, masm linker simply don´t find the proper pointers on a regular jmp, so i was forced to use "::" in some of the functions (Specially the graphic one)

[guga]

A Test version more complete (many thanks to Jack for helping compile this thing)

This version was built directly in mingw32 with the good help of Jack guiding me to  build this dll, so we can test it later. It contains more then 1000 Apis (That care that some of them are exported data and not code - See the file FbRtl32.def attached here)

I´m including 2 versions.
1 - With full debug information (and some of the functions contains mangled names) - FbRtl32D.dll
2 - Another version with the debug information stripped and also with the functions without any weird char "_" "@' and friends...so names are unmangled. - FbRtl32.dll

Included also the batch file i used to create these things and the lib file we can use to reassemble with others apps, perhaps.

I strongly advice to you read FreeBasic source code in order to try using these Apis, but get ready for crashes, gdi leaks, memory allocation problems etc (That are a common problems in internal Fb functions, btw - As i said previous, the idea of FB is not bad, but it´s implementation and coding technique approach is terrible)

Tks also to the author of this article that helped me compiling this thing unmangling the names of the function - https://www.transmissionzero.co.uk/computing/advanced-mingw-dll-topics/

[TimoVJL]

In import table additional gnu dlls:

Code Select Expand

libffi-7.dll
libgcc_s_sjlj-1.dll


[jack]

guga, you did ok with the FbRtl32D.dll where you used "-shared -static" but on FbRtl32.dll you used "-shared -s"
you need to use -static
just did some more testing on my end with the dll's I made, when using the dll's you must supply all function/sub parameters whether optional or not
at least the ones builtin FB/FBgfx, have not tested whether this is true also for your own functions/subs

[Vortex]

Hi Guga,

Thanks, impressive work

I tried to build a simple FreeBASIC project with your latest library. The executable reported a missing DLL :

Code Select Expand

D:\FreeBASIC>D:\PellesC\bin\podump.exe /IMPORTS FbRtl32.dll | findstr ".dll"
Dump of FbRtl32.dll
        ADVAPI32.dll
        libffi-7.dll
        libgcc_s_sjlj-1.dll
        GDI32.dll
        KERNEL32.dll
        msvcrt.dll
        USER32.dll

libgcc_s_sjlj-1.dll exists in the folder \FreeBASIC\bin\win32 but libffi-7.dll does not exist on my computer.

Source code sample.bas :

Code Select Expand

Dim As String s="This is a test."

Dim As Integer i

For i=1 To 3

   Print s

Next i


Command to build the sample :

Code Select Expand

D:\FreeBASIC>fbc32 sample.bas FbRtl32.a

Operating system : Windows 7 64-bit
FreeBASIC Version : 1.09

[jack]

Vortex, try FbRtl32D.dll

[Vortex]

Hi Jack,

Thanks. After some renaming the DLLs, the sample worked :

Code Select Expand

D:\FreeBASIC>ren FbRtl32.dll FbRtl32.dll.bak

D:\FreeBASIC>ren FbRtl32D.dll FbRtl32.dll

D:\FreeBASIC>sample.exe
This is a test.
This is a test.
This is a test.

[jack]

Hi Vortex
I found out that when using the dll any FB/FBgfx sub/function that has default/optional parameters you need to supply them otherwise you have crash, it's particularly tricky with the graphic sub/functions

[guga]

Hi Guys...


I´m giving a test trying to port The LolRemez to see if all the functions works correctly. It´s a hell of work trying to find out what exactly are the parameters being used, but i´m succeeding to make some small tests. I did not reached the graphics part yyet, but here is some of the internal code (and mess) that FB does for a simple command line:

Code Select Expand


Proc fb_ctor__LolRemez:
    Local @ZValue, @Order, @NPoints, @NPoints2, @DnValue, @TmpVar3
    Uses ebx

    call 'FbRtl32.fb_Init' 1, D$CommandLine, 0
    call 'FbRtl32.fb_Command' 1
    call 'FbRtl32.fb_StrInit' GValue, 0-1, eax, 0-1, 0
    call 'FbRtl32.fb_StrLcase2' GValue, 0
    call 'FbRtl32.fb_StrAssign' GValue, 0-1, eax, 0-1, 0
    call 'FbRtl32.fb_Cls' 0FFFF0000
    call 'FbRtl32.fb_StrLen' GValue, 0-1
    If eax = 0
        ;call StdOut Sz_InstructionNeeded | C_call 'msvcrt.system' {B$ "cmd", 0} | call 'FbRtl32.fb_End' 0
        call 'FbRtl32.fb_PrintVoid' 0, 1
        call 'FbRtl32.fb_PrintVoid' 0, 1
        call 'FbRtl32.fb_StrAllocTempDescZEx' {B$ "Poly Needs an instruction", 0}, 25
        call 'FbRtl32.fb_PrintString' 0, eax, 1
        call 'FbRtl32.fb_StrAllocTempDescZEx' {B$ "type poly help at the command prompt", 0}, 36
        call 'FbRtl32.fb_PrintString' 0, eax, 1
        call 'FbRtl32.fb_StrAllocTempDescZEx' {B$ "cmd", 0}, 3
        call 'FbRtl32.fb_Shell' eax
        call 'FbRtl32.fb_End' 0
    End_If
    call 'FbRtl32.fb_StrAllocTempDescZEx' { B$ "help", 0}, 4
    call 'FbRtl32.fb_StrLcase2' eax, 0
    call 'FbRtl32.fb_StrCompare' GValue, 0-1, eax, 0-1
    .If eax = 0
        call 'FbRtl32.fb_StrAllocTempDescZEx' {B$ "poly ", 022, "start x value, finish x value, function (in x)", 022, " ", 0}, 54
        call 'FbRtl32.fb_PrintString' 0, eax, 1
        call 'FbRtl32.fb_StrAllocTempDescZEx' {B$ "Everything must be inside quotes", 0}, 32
        call 'FbRtl32.fb_PrintString' 0, eax, 1
        call 'FbRtl32.fb_StrAllocTempDescZEx' {B$ "No spaces allowed, use a comma to separate flags", 0}, 48
        call 'FbRtl32.fb_PrintString' 0, eax, 1
        call 'FbRtl32.fb_StrAllocTempDescZEx' {B$ "operators ^ ( ) / * + - mod ", 0}, 28
        call 'FbRtl32.fb_PrintString' 0, eax, 1
        call 'FbRtl32.fb_StrAllocTempDescZEx' {B$ "available functions :", 0}, 21
        call 'FbRtl32.fb_PrintString' 0, eax, 1
        call LIST
        call 'FbRtl32.fb_StrAllocTempDescZEx' {B$ "Example", 0}, 7
        call 'FbRtl32.fb_PrintString' 0, eax, 1
        call 'FbRtl32.fb_StrAllocTempDescZEx' {B$ "poly ", 022, "-3,3,sin(x)", 022, " ", 0}, 19
        call 'FbRtl32.fb_PrintString' 0, eax, 1
        call 'FbRtl32.fb_End' 0
    .End_If



All of those printvoid, fb_StrAllocTempDescZEx, fb_PrintString and friend can simply be replaced with something like this:


[Sz_InstructionNeeded: B$ "Poly Needs an instruction
type poly help at the command prompt", 0]

call StdOut Sz_InstructionNeeded

Code Select Expand


_____________________________________________________________________________________________

Proc StdOut:
    Arguments @lpszText
    Local @hOutPut, @StringLenght, @BytesWritten
    Uses ecx, edx

    call 'kernel32.GetStdHandle' &STD_OUTPUT_HANDLE
    mov D@hOutPut eax

    call 'kernel32.lstrlenA' D@lpszText
    mov D@StringLenght eax

    lea eax D@BytesWritten
    call 'kernel32.WriteFile' D@hOutPut, D@lpszText, D@StringLenght, eax, &NULL
    mov eax D@BytesWritten

EndP



The main problem of the internal code used by FB is that it is a complete mess. The compiler tries to guess what are the variables to be used and insetad creating tem individually, it seems that they put all together on the stack. As a result we have a stack containing tons of variables that are being reused here and there all along the function. Not to mention that gcc ruins the code flow, putting the parameters of some functions ( push etc) way before the call to the function. In some cases, it tries to save the parameters on a register used lines before the main call, saves it (push), perform a call to another function and just later, use it on the proper call to the function that really uses it. On the Main procedure, for example, we have several structures created by FB that are stored on the stack

Also, on each one of these fb_StrAllocTempDescZEx , fb_PrintString and friends, it tries to allocates or deallocate memory, what is completely unnecessary specially if we are dealing with 1 single byte.


Not to mention that gcc puts hundreds of wweird code on the start of the file. But, all is needed is a simple call to commandline Api followed by the main routine to start. So, this is how actually, LolRemez starts:

Code Select Expand

Main:

    call 'KERNEL32.GetCommandLineA' | mov D$CommandLine eax
    call fb_ctor__LolRemez
    call 'KERNEL32.ExitProcess' 0


This is LolRemez running (on the part i succeeded to convert).

[jj2007]

The main problem of the internal code used by FB is that it is a complete mess. The compiler tries to guess what are the variables to be used and insetad creating tem individually, it seems that they put all together on the stack. As a result we have a stack containing tons of variables that are being reused here and there all along the function. Not to mention that gcc ruins the code flow, putting the parameters of some functions ( push etc) way before the call to the function. In some cases, it tries to save the parameters on a register used lines before the main call, saves it (push), perform a call to another function and just later, use it on the proper call to the function that really uses it. On the Main procedure, for example, we have several structures created by FB that are stored on the stack

Also, on each one of these fb_StrAllocTempDescZEx , fb_PrintString and friends, it tries to allocates or deallocate memory, what is completely unnecessary specially if we are dealing with 1 single byte.

That's what C/C++ compilers do, Guga

PowerBasic and MasmBasic are written in Assembly, that makes a big difference.