Class CommonGramsQueryFilter

  • All Implemented Interfaces:
    java.io.Closeable, java.lang.AutoCloseable

    public final class CommonGramsQueryFilter
    extends TokenFilter
    Wrap a CommonGramsFilter optimizing phrase queries by only returning single words when they are not a member of a bigram. Example:
    • query input to CommonGramsFilter: "the rain in spain falls mainly"
    • output of CommomGramsFilter/input to CommonGramsQueryFilter: |"the, "the-rain"|"rain" "rain-in"|"in, "in-spain"|"spain"|"falls"|"mainly"
    • output of CommonGramsQueryFilter:"the-rain", "rain-in" ,"in-spain", "falls", "mainly"
    • Constructor Detail

      • CommonGramsQueryFilter

        public CommonGramsQueryFilter​(CommonGramsFilter input)
        Constructs a new CommonGramsQueryFilter based on the provided CommomGramsFilter
        Parameters:
        input - CommonGramsFilter the QueryFilter will use
    • Method Detail

      • reset

        public void reset()
                   throws java.io.IOException
        This method is called by a consumer before it begins consumption using TokenStream.incrementToken().

        Resets this stream to a clean state. Stateful implementations must implement this method so that they can be reused, just as if they had been created fresh.

        If you override this method, always call super.reset(), otherwise some internal state will not be correctly reset (e.g., Tokenizer will throw IllegalStateException on further usage).

        NOTE: The default implementation chains the call to the input TokenStream, so be sure to call super.reset() when overriding this method.

        Overrides:
        reset in class TokenFilter
        Throws:
        java.io.IOException
      • incrementToken

        public boolean incrementToken()
                               throws java.io.IOException
        Output bigrams whenever possible to optimize queries. Only output unigrams when they are not a member of a bigram. Example:
        • input: "the rain in spain falls mainly"
        • output:"the-rain", "rain-in" ,"in-spain", "falls", "mainly"
        Specified by:
        incrementToken in class TokenStream
        Returns:
        false for end of stream; true otherwise
        Throws:
        java.io.IOException
      • isGramType

        public boolean isGramType()
        Convenience method to check if the current type is a gram type
        Returns:
        true if the current type is a gram type, false otherwise