|Expected Token Density in Random Stream firstname.lastname@example.org (Andrew Tomazos) (2011-12-07)|
|Re: Expected Token Density in Random Stream email@example.com (Kaz Kylheku) (2011-12-11)|
|Re: Expected Token Density in Random Stream firstname.lastname@example.org (Andrew Tomazos) (2011-12-13)|
|Re: Expected Token Density in Random Stream email@example.com (Gene) (2011-12-19)|
|From:||Kaz Kylheku <firstname.lastname@example.org>|
|Date:||Sun, 11 Dec 2011 17:56:42 +0000 (UTC)|
|Organization:||A noiseless patient Spider|
|Posted-Date:||11 Dec 2011 16:42:34 EST|
On 2011-12-07, Andrew Tomazos <email@example.com> wrote:
> Summary: We want to find out how often a given token appears in a
> random stream formed by concatenating randomly chosen strings from a
> given set of strings.
> (Note hits can overlap each other)
But tokens do not overlap, so you're not actually extracting tokens. Using C
tokens as an example, the C token >>= is one hit, not four. The longest match
calls for extracting three characters and moving on.
Return to the
Search the comp.compilers archives again.