[Feature] use the fancy-regex crate in colgrep

[dlight]

The fancy-regex crate builds upon regex to provide more matching features like lookarounds. It uses backtracking like PCRE, so it has some performance pitfalls, but this only applies in the regex parts where the user actually uses fancy features. If they don't use it anywhere, the regex is run straight by regex, with no performance penalty.

Those additional features are occasionally useful when searching for code, and since fancy-regex use a familiar syntax I think that agents would make use of it when needed (maybe with some adjustments to SKILL.md). For example (from the readme)

For example, the following regex captures a single-line Rust raw string:

r(#*)".*?"\1

There is no NFA that can express this simple and useful pattern. Yet, a backtracking implementation handles it efficiently.

Since fancy-regex reverts to regex when you don't use its additional features, it wouldn't be a breaking change to simply change the regex engine to it. But if one wants to be careful, this could be controlled by a fancy_regex feature flag (initially it could be disabled by default, and at a later date enabled by default).

[raphaelsty]

Hi @dlight right now our regex system rely on sqlite which is quite powerful for filtering and ease of life, is there any alternatives using sqlite while keeping backward compatibility ?

[dlight]

I'm confused. If colgrep uses sqlite's builtin regex support, what is the regex dependency for in colgrep and in next-plaid?

In any case, you can use alternative regex engines in sqlite, like this to use the Rust regex crate in sqlite for better performance (a small modification on that could enable using fancy-regex instead). This SO answer gives some context (it's about using PCRE, which is about as powerful as fancy-regex, albeit slower)