Package opennlp.tools.tokenize
Class DefaultTokenContextGenerator
- java.lang.Object
-
- opennlp.tools.tokenize.DefaultTokenContextGenerator
-
- All Implemented Interfaces:
TokenContextGenerator
public class DefaultTokenContextGenerator extends java.lang.Object implements TokenContextGenerator
Generate events for maxent decisions for tokenization.
-
-
Constructor Summary
Constructors Constructor Description DefaultTokenContextGenerator()
Creates a default context generator for tokenizer.DefaultTokenContextGenerator(java.util.Set<java.lang.String> inducedAbbreviations)
Creates a default context generator for tokenizer.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description java.lang.String[]
getContext(java.lang.String sentence, int index)
Returns an array of features for the specified sentence string at the specified index.
-
-
-
Constructor Detail
-
DefaultTokenContextGenerator
public DefaultTokenContextGenerator()
Creates a default context generator for tokenizer.
-
DefaultTokenContextGenerator
public DefaultTokenContextGenerator(java.util.Set<java.lang.String> inducedAbbreviations)
Creates a default context generator for tokenizer.- Parameters:
inducedAbbreviations
- the induced abbreviations
-
-
Method Detail
-
getContext
public java.lang.String[] getContext(java.lang.String sentence, int index)
Description copied from interface:TokenContextGenerator
Returns an array of features for the specified sentence string at the specified index.- Specified by:
getContext
in interfaceTokenContextGenerator
- Parameters:
sentence
- The string for a sentence.index
- The index to consider splitting as a token.- Returns:
- an array of features for the specified sentence string at the specified index.
-
-