org.apache.lucene.analysis.query

Class QueryAutoStopWordAnalyzer

java.lang.Object

org.apache.lucene.analysis.Analyzer

org.apache.lucene.analysis.query.QueryAutoStopWordAnalyzer

public class QueryAutoStopWordAnalyzer
extends Analyzer

Field Summary

Fields

Modifier and Type	Field and Description
static float	defaultMaxDocFreqPercent

Constructor Summary

Constructors

Constructor and Description
QueryAutoStopWordAnalyzer(Analyzer delegate) Initializes this analyzer with the Analyzer object that actual produces the tokens

Method Summary

Methods

Modifier and Type	Method and Description
int	addStopWords(IndexReader reader) Automatically adds stop words for all fields with terms exceeding the defaultMaxDocFreqPercent
int	addStopWords(IndexReader reader, float maxPercentDocs) Automatically adds stop words for all fields with terms exceeding the maxDocFreqPercent
int	addStopWords(IndexReader reader, int maxDocFreq) Automatically adds stop words for all fields with terms exceeding the maxDocFreqPercent
int	addStopWords(IndexReader reader, java.lang.String fieldName, float maxPercentDocs) Automatically adds stop words for the given field with terms exceeding the maxPercentDocs
int	addStopWords(IndexReader reader, java.lang.String fieldName, int maxDocFreq) Automatically adds stop words for the given field with terms exceeding the maxPercentDocs
Term[]	getStopWords() Provides information on which stop words have been identified for all fields
java.lang.String[]	getStopWords(java.lang.String fieldName) Provides information on which stop words have been identified for a field
TokenStream	tokenStream(java.lang.String fieldName, java.io.Reader reader) Creates a TokenStream which tokenizes all the text in the provided Reader.

Methods inherited from class org.apache.lucene.analysis.Analyzer

close, getPositionIncrementGap, getPreviousTokenStream, reusableTokenStream, setPreviousTokenStream

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

defaultMaxDocFreqPercent

public static final float defaultMaxDocFreqPercent

See Also:

Constant Field Values

Constructor Detail

QueryAutoStopWordAnalyzer

public QueryAutoStopWordAnalyzer(Analyzer delegate)

Initializes this analyzer with the Analyzer object that actual produces the tokens

Parameters:

delegate - The choice of analyzer that is used to produce the token stream which needs filtering

Method Detail

addStopWords

public int addStopWords(IndexReader reader)
                 throws java.io.IOException

Automatically adds stop words for all fields with terms exceeding the defaultMaxDocFreqPercent

Parameters:

reader - The IndexReader class which will be consulted to identify potential stop words that exceed the required document frequency

Returns:

The number of stop words identified.

Throws:

java.io.IOException

addStopWords

public int addStopWords(IndexReader reader,
               int maxDocFreq)
                 throws java.io.IOException

Automatically adds stop words for all fields with terms exceeding the maxDocFreqPercent

Parameters:

reader - The IndexReader class which will be consulted to identify potential stop words that exceed the required document frequency

maxDocFreq - The maximum number of index documents which can contain a term, after which the term is considered to be a stop word

Returns:

The number of stop words identified.

Throws:

java.io.IOException

addStopWords

public int addStopWords(IndexReader reader,
               float maxPercentDocs)
                 throws java.io.IOException

Automatically adds stop words for all fields with terms exceeding the maxDocFreqPercent

Parameters:

reader - The IndexReader class which will be consulted to identify potential stop words that exceed the required document frequency

maxPercentDocs - The maximum percentage (between 0.0 and 1.0) of index documents which contain a term, after which the word is considered to be a stop word.

Returns:

The number of stop words identified.

Throws:

java.io.IOException

addStopWords

public int addStopWords(IndexReader reader,
               java.lang.String fieldName,
               float maxPercentDocs)
                 throws java.io.IOException

Automatically adds stop words for the given field with terms exceeding the maxPercentDocs

Parameters:

reader - The IndexReader class which will be consulted to identify potential stop words that exceed the required document frequency

fieldName - The field for which stopwords will be added

maxPercentDocs - The maximum percentage (between 0.0 and 1.0) of index documents which contain a term, after which the word is considered to be a stop word.

Returns:

The number of stop words identified.

Throws:

java.io.IOException

addStopWords

public int addStopWords(IndexReader reader,
               java.lang.String fieldName,
               int maxDocFreq)
                 throws java.io.IOException

Automatically adds stop words for the given field with terms exceeding the maxPercentDocs

Parameters:

reader - The IndexReader class which will be consulted to identify potential stop words that exceed the required document frequency

fieldName - The field for which stopwords will be added

maxDocFreq - The maximum number of index documents which can contain a term, after which the term is considered to be a stop word.

Returns:

The number of stop words identified.

Throws:

java.io.IOException

tokenStream

public TokenStream tokenStream(java.lang.String fieldName,
                      java.io.Reader reader)

Description copied from class: Analyzer

Creates a TokenStream which tokenizes all the text in the provided Reader. Must be able to handle null field name for backward compatibility.

Specified by:

tokenStream in class Analyzer

getStopWords

public java.lang.String[] getStopWords(java.lang.String fieldName)

Provides information on which stop words have been identified for a field

Parameters:

fieldName - The field for which stop words identified in "addStopWords" method calls will be returned

Returns:

the stop words identified for a field

getStopWords

public Term[] getStopWords()

Provides information on which stop words have been identified for all fields

Returns:

the stop words (as terms)