Lucene++ - a full-featured, c++ search engine
API Documentation
This class converts alphabetic, numeric, and symbolic Unicode characters which are not in the first 127 ASCII characters (the "Basic Latin" Unicode block) into their ASCII equivalents, if one exists. More...
#include <ASCIIFoldingFilter.h>
Public Member Functions | |
ASCIIFoldingFilter (const TokenStreamPtr &input) | |
virtual | ~ASCIIFoldingFilter () |
virtual String | getClassName () |
boost::shared_ptr< ASCIIFoldingFilter > | shared_from_this () |
virtual bool | incrementToken () |
Consumers (ie., IndexWriter ) use this method to advance the stream to the next token. Implementing classes must implement this method and update the appropriate Attribute s with the attributes of the next token. | |
void | foldToASCII (const wchar_t *input, int32_t length) |
Converts characters above ASCII to their ASCII equivalents. For example, accents are removed from accented characters. | |
![]() | |
virtual | ~TokenFilter () |
boost::shared_ptr< TokenFilter > | shared_from_this () |
virtual void | end () |
Performs end-of-stream operations, if any, and calls then end() on the input TokenStream. NOTE: Be sure to call TokenFilter::end() first when overriding this method. | |
virtual void | close () |
Close the input TokenStream. | |
virtual void | reset () |
Reset the filter as well as the input TokenStream. | |
![]() | |
virtual | ~TokenStream () |
boost::shared_ptr< TokenStream > | shared_from_this () |
![]() | |
AttributeSource () | |
An AttributeSource using the default attribute factory DefaultAttributeFactory . | |
AttributeSource (const AttributeSourcePtr &input) | |
An AttributeSource that uses the same attributes as the supplied one. | |
AttributeSource (const AttributeFactoryPtr &factory) | |
An AttributeSource using the supplied AttributeFactory for creating new Attribute instances. | |
virtual | ~AttributeSource () |
boost::shared_ptr< AttributeSource > | shared_from_this () |
AttributeFactoryPtr | getAttributeFactory () |
returns the used AttributeFactory. | |
template<class ATTR > | |
boost::shared_ptr< ATTR > | addAttribute () |
This method first checks if an instance of that class is already in this AttributeSource and returns it. Otherwise a new instance is created, added to this AttributeSource and returned. | |
void | addAttribute (const String &className, const AttributePtr &attrImpl) |
Adds a custom Attribute instance. | |
bool | hasAttributes () |
Returns true if this AttributeSource has any attributes. | |
template<class ATTR > | |
bool | hasAttribute () |
Returns true, if this AttributeSource contains the passed-in Attribute. | |
template<class ATTR > | |
boost::shared_ptr< ATTR > | getAttribute () |
Returns the instance of the passed in Attribute contained in this AttributeSource. | |
void | clearAttributes () |
Resets all Attributes in this AttributeSource by calling AttributeImpl#clear() on each Attribute implementation. | |
AttributeSourceStatePtr | captureState () |
Captures the state of all Attributes. The return value can be passed to restoreState to restore the state of this or another AttributeSource. | |
void | restoreState (const AttributeSourceStatePtr &state) |
Restores this state by copying the values of all attribute implementations that this state contains into the attributes implementations of the targetStream. The targetStream must contain a corresponding instance for each argument contained in this state (eg. it is not possible to restore the state of an AttributeSource containing a TermAttribute into a AttributeSource using a Token instance as implementation). | |
virtual int32_t | hashCode () |
Return hash code for this object. | |
virtual bool | equals (const LuceneObjectPtr &other) |
Return whether two objects are equal. | |
virtual String | toString () |
Returns a string representation of the object. | |
AttributeSourcePtr | cloneAttributes () |
Performs a clone of all AttributeImpl instances returned in a new AttributeSource instance. This method can be used to eg. create another TokenStream with exactly the same attributes (using AttributeSource(AttributeSource) ) | |
Collection< AttributePtr > | getAttributes () |
Return a vector of attributes based on currentState. | |
![]() | |
virtual | ~LuceneObject () |
virtual void | initialize () |
Called directly after instantiation to create objects that depend on this object being fully constructed. | |
virtual LuceneObjectPtr | clone (const LuceneObjectPtr &other=LuceneObjectPtr()) |
Return clone of this object. | |
virtual int32_t | compareTo (const LuceneObjectPtr &other) |
Compare two objects. | |
![]() | |
virtual | ~LuceneSync () |
virtual SynchronizePtr | getSync () |
Return this object synchronize lock. | |
virtual LuceneSignalPtr | getSignal () |
Return this object signal. | |
virtual void | lock (int32_t timeout=0) |
Lock this object using an optional timeout. | |
virtual void | unlock () |
Unlock this object. | |
virtual bool | holdsLock () |
Returns true if this object is currently locked by current thread. | |
virtual void | wait (int32_t timeout=0) |
Wait for signal using an optional timeout. | |
virtual void | notifyAll () |
Notify all threads waiting for signal. | |
Static Public Member Functions | |
static String | _getClassName () |
![]() | |
static String | _getClassName () |
![]() | |
static String | _getClassName () |
![]() | |
static String | _getClassName () |
Protected Attributes | |
CharArray | output |
int32_t | outputPos |
TermAttributePtr | termAtt |
![]() | |
TokenStreamPtr | input |
The source of tokens for this filter. | |
![]() | |
AttributeFactoryPtr | factory |
MapStringAttribute | attributes |
AttributeSourceStatePtr | currentState |
![]() | |
SynchronizePtr | objectLock |
LuceneSignalPtr | objectSignal |
Additional Inherited Members | |
![]() | |
TokenFilter (const TokenStreamPtr &input) | |
Construct a token stream filtering the given input. | |
![]() | |
TokenStream () | |
A TokenStream using the default attribute factory. | |
TokenStream (const AttributeSourcePtr &input) | |
A TokenStream that uses the same attributes as the supplied one. | |
TokenStream (const AttributeFactoryPtr &factory) | |
A TokenStream using the supplied AttributeFactory for creating new Attribute instances. | |
![]() | |
AttributePtr | getAttribute (const String &className) |
The caller must pass in a className value. This method checks if an instance of that class is already in this AttributeSource and returns it. | |
bool | hasAttribute (const String &className) |
Returns true, if this AttributeSource contains the passed-in Attribute. | |
void | computeCurrentState () |
![]() | |
LuceneObject () | |
This class converts alphabetic, numeric, and symbolic Unicode characters which are not in the first 127 ASCII characters (the "Basic Latin" Unicode block) into their ASCII equivalents, if one exists.
Characters from the following Unicode blocks are converted; however, only those characters with reasonable ASCII alternatives are converted:
C1 Controls and Latin-1 Supplement: http://www.unicode.org/charts/PDF/U0080.pdf Latin Extended-A: http://www.unicode.org/charts/PDF/U0100.pdf Latin Extended-B: http://www.unicode.org/charts/PDF/U0180.pdf Latin Extended Additional: http://www.unicode.org/charts/PDF/U1E00.pdf Latin Extended-C: http://www.unicode.org/charts/PDF/U2C60.pdf Latin Extended-D: http://www.unicode.org/charts/PDF/UA720.pdf IPA Extensions: http://www.unicode.org/charts/PDF/U0250.pdf Phonetic Extensions: http://www.unicode.org/charts/PDF/U1D00.pdf Phonetic Extensions Supplement: http://www.unicode.org/charts/PDF/U1D80.pdf General Punctuation: http://www.unicode.org/charts/PDF/U2000.pdf Superscripts and Subscripts: http://www.unicode.org/charts/PDF/U2070.pdf Enclosed Alphanumerics: http://www.unicode.org/charts/PDF/U2460.pdf Dingbats: http://www.unicode.org/charts/PDF/U2700.pdf Supplemental Punctuation: http://www.unicode.org/charts/PDF/U2E00.pdf Alphabetic Presentation Forms: http://www.unicode.org/charts/PDF/UFB00.pdf Halfwidth and Fullwidth Forms: http://www.unicode.org/charts/PDF/UFF00.pdf
See: http://en.wikipedia.org/wiki/Latin_characters_in_Unicode
The set of character conversions supported by this class is a superset of those supported by Lucene's ISOLatin1AccentFilter
which strips accents from Latin1 characters. For example, 'à' will be replaced by 'a'.
Lucene::ASCIIFoldingFilter::ASCIIFoldingFilter | ( | const TokenStreamPtr & | input | ) |
|
virtual |
|
inlinestatic |
void Lucene::ASCIIFoldingFilter::foldToASCII | ( | const wchar_t * | input, |
int32_t | length | ||
) |
Converts characters above ASCII to their ASCII equivalents. For example, accents are removed from accented characters.
input | The string to fold |
length | The number of characters in the input string |
|
inlinevirtual |
Reimplemented from Lucene::TokenFilter.
|
virtual |
Consumers (ie., IndexWriter
) use this method to advance the stream to the next token. Implementing classes must implement this method and update the appropriate Attribute
s with the attributes of the next token.
The producer must make no assumptions about the attributes after the method has been returned: the caller may arbitrarily change it. If the producer needs to preserve the state for subsequent calls, it can use captureState
to create a copy of the current attribute state.
This method is called for every token of a document, so an efficient implementation is crucial for good performance. To avoid calls to addAttribute(Class)
and getAttribute(Class)
, references to all Attribute
s that this stream uses should be retrieved during instantiation.
To ensure that filters and consumers know which attributes are available, the attributes must be added during instantiation. Filters and consumers are not required to check for availability of attributes in incrementToken()
.
Implements Lucene::TokenStream.
|
inline |
|
protected |
|
protected |
|
protected |