Home

Awesome

FuzzyMatcher

A C++ class extension of the RE/flex Matcher class for efficient fuzzy matching and fuzzy search with regex patterns. Regex patterns are of the POSIX ERE type, but also support Unicode matching, lazy quantifiers, word boundaries and lookaheads.

Requires

RE-Flex version 4.0 or greater, because of regex pattern analysis and translation updates to RE/flex 4.0 that are also used by ugrep 5.0.

Examples

patternmaxfuzzy find() matchesbut not
abc1abc, ab, ac, axc, axbca, axx, axbxc, bc
año1año, ano, aoanno, ño
ab_cd2ab_cd, ab-cd, ab Cd, abCdab\ncd, Ab_cd, Abcd
a[0-9]+z1a1z, a123z, az, axzaxxz, A123z, 123z

Note that the first character of the pattern must match when searching a corpus with the find() method. By contrast, the matches() method to match a corpus from start to end does not impose this requirement:

patternmaxfuzzy matches() matchesbut not
abc1abc, ab, ac, Abc, xbc bc, axc, axbca, axx, Ab, axbxc
año1año, Año, ano, ao, ñoanno
ab_cd2ab_cd, Ab_Cd, ab-cd, ab Cd, Ab_cd, abCdab\ncd, AbCd
a[0-9]+z1a1z, A1z, a123z, az, Az, axz, 123zaxxz

Optimizations

Fuzzy find() and split() make a second pass over a fuzzy-matched pattern when the match has a nonzero error. This second pass checks if an exact match exists or if a better match exists that overlaps with the first pattern found. For example, the pattern abc is found to fuzzy match all of the text aabc with one error (an extra a). The second pass of find() detects an exact match after skipping the first a. Likewise, the pattern abc is found to fuzzy match ababc with a match for aba with one error (substitution of c by an a). The second pass of find() detects an exact match after skipping ab in the text. This approach is faster than minimizing the edit distance when searching text, while returning exact matches when possible.

Usage

Fuzzy searching

#include "fuzzymatcher.h"

// MAX:   optional maximum edit distance, default is 1, up to 255
// INPUT: a string, wide string, FILE*, or std::istream object
reflex::FuzzyMatcher matcher("PATTERN", [MAX,] INPUT);

// find all pattern matches in the input
while (matcher.find())
{
  std::cout << matcher.text() << '\n'  // show each fuzzy match
  std::cout << matcher.edits() << '\n' // edit distance (when > 0 not guaranteed minimal)
}

See the RE/flex user guide for the full list of Matcher class methods available to extract match info. The edits() method is a FuzzyMatcher extension of the Matcher class.

Fuzzy matching

#include "fuzzymatcher.h"

// match the whole input (here in one go with a temporary fuzzy matcher object)
if (reflex::FuzzyMatcher("PATTERN", [MAX,] INPUT).matches())
{
  std::cout << "fuzzy pattern matched\n";
}

Fuzzy splitting (text between matches)

#include "fuzzymatcher.h"

reflex::FuzzyMatcher matcher("PATTERN", [MAX,] INPUT);

// split the input into parts separated by pattern matches
while (matcher.split())
{
  std::cout << matcher.text() << '\n' // show text between fuzzy matches
}

Character insertion, deletion and substitution

The MAX parameter may be combined with one or more of the following flags:

For example, to allow approximate pattern matches to include up to three character insertions, but no deletions or substitutions (allowing insertions only is actually the most efficient fuzzy matching possible):

reflex::FuzzyMatcher matcher(regex, 3 | reflex::FuzzyMatcher::INS, INPUT);

To allow up to three insertions or deletions (note that a substitution counts as two edits: one insertion and one deletion):

reflex::FuzzyMatcher matcher(regex, 3 | reflex::FuzzyMatcher::INS | reflex::FuzzyMatcher::DEL, INPUT);

When no flags are specified with MAX, fuzzy matching is performed with insertions, deletions, and substitutions, each counting as one edit.

Full Unicode support

To support full Unicode pattern matching, such as \p Unicode character classes, convert the regex pattern before using it as follows:

std::string regex(reflex::Matcher::convert("PATTERN", reflex::convert_flag::unicode));
reflex::FuzzyMatcher matcher(regex, [MAX,] INPUT);

Static regex patterns

Fixed patterns should be constructed (and optionally Unicode converted) just once statically to avoid repeated construction, e.g. in the body of loops and in frequently executed functions:

static const reflex::Pattern pattern(reflex::Matcher::convert("PATTERN", reflex::convert_flag::unicode));
reflex::FuzzyMatcher matcher(pattern, [MAX,] INPUT);

Compiling

Assuming reflex dir with RE/flex source code is locally built:

c++ -o myapp myapp.cpp -Ireflex/include reflex/lib/libreflex.a

When the libreflex library is built and installed:

c++ -o myapp myapp.cpp -lreflex

Testing

$ make ftest
$ ./ftest 'ab_cd' 'abCd' 2
matches(): match (2 edits)
find():    'abCd' at 0 (2 edits)
split():   '' at 0 (2 edits)
split():   '' at 4 (0 edits)

License

BSD-3