Parsing HTML : I would appreciate advice

The problem to solve.


I have to parse millions of html documents, and return just the
plaintext/bytes. Many of the html documents contain Japanese
characters and so it will be necessary to read the codepage in the
html header, so the bytes can be read properly.


Ninety percent of the html documents are well formed, originally
created by code. The rest of the documents are random html documents
from the internet. I will be placing the plaintext in an SQL database
and use full-text search.


What type of parser/lexer is best for parsing html?


Can you offer any links or existing libraries for doing this?


I am most proficient in: C#, C++, C languages.


Thanks
Russell Mangel
Las Vegas, NV
[There's a bazillion HTML parsers available. I write most of my stuff
in perl these days, so I like the HTML::Parser package which could
easily do what you want. -John]

Return to the comp.compilers page.
Search the comp.compilers archives again.