Author Topic: Text pre parser.  (Read 272 times)

hutch--

  • Administrator
  • Member
  • ******
  • Posts: 4935
  • Mnemonic Driven API Grinder
    • The MASM32 SDK
Text pre parser.
« on: July 04, 2017, 04:18:03 AM »
This tool contains a number of techniques for preparing text for parsing, its default task is to remove high ascii character (> 126) and to remove extra CRLF pairs. It has additional options to left or right trim each line of text or to do both. An extra option is to set the text to monospacing of spaces (ascii 32). By cleaning up messy text, it becomes a lot easier and faster to parse which is basically what I use algorithms like this for.

The source code contains a couple of extra library modules that were written after the last library version I posted so it cannot be built yet but that will come with the next library release.

The 17k icon blew the size out to 26k but you will find it will not be a real problem in these days of multi-megabyte applications.  :P

Source attached.
hutch at movsd dot com
http://www.masm32.com    :biggrin:  :biggrin:

hutch--

  • Administrator
  • Member
  • ******
  • Posts: 4935
  • Mnemonic Driven API Grinder
    • The MASM32 SDK
Re: Text pre parser.
« Reply #1 on: July 04, 2017, 07:11:16 PM »
I should have added this, one of the ways to get clean Microsoft structures is to load an empty file with as many of the C include files and run the output option, I think from memory its /EP and redirect it to a file. When you open the massive file that results from this there is a massive amount of spaces and CRLFs between the data and by feeding the output through a tool of this type you rip out the dead spacing reducing the file size and as a bonus most of the extra crud that Microsoft add to the include file.
hutch at movsd dot com
http://www.masm32.com    :biggrin:  :biggrin: