I've not had much luck with the versions query in SEARCH.

It works fine for individual topics with QUERY, but SEARCH is just too slow unless you only have a few dozen topics.

I think we need a strategy before we ship this. At a minimum, disable versions in SEARCH with RCS stores, or alternatively we introduce a new feature to make SEARCHes abort long-running queries in a graceful manner (can we just return a shorter-than-normal resultset, and log/emit an error?).

The query I ran on trunk.foswiki.org was something like:

%SEARCH{
  "versions[author='PaulHarvey']"
  web="Development"
  limit="50"
}%

Which ran for several minutes before Koen killed the process for me.

-- PaulHarvey - 11 Dec 2010

Yeah, well, TBH I'm not that surprised. A versions query has to load a shitload of information just to query, and that's not efficient. As you say, the RCS store just isn't built for this kind of query.

Should this be a fix just for a versions query, or is there a more general problem, that a user should be able to put a limit on the amount of time spent on a query? A general mechanism would work for other types of bad query.

-- CrawfordCurrie - 11 Apr 2011

I don't have any good ideas on how to control a hypothetical "search timeout" feature. Configure setting? Macro param? URL param? What should be the default?

On my own wiki, I have some reports that just take 10s of seconds, and a dot graph that takes minutes - I need to be able to run those from a cron job, where I save the output back into "cache" topic which is refreshed every hour.

Let's say we have a default SEARCH timeout of 10s - I need a way for my cron-job scenario to override it so that it will run to completion from CLI.

Additionally: I don't know about general Foswiki practice, but the biggest troubles on my wiki are nested searches. If there's 200 SEARCHes as a result of an outer SEARCH, how do we apply a "timeout" in that case, if each individual SEARCH is still on the order of ~2s?

Hmm, so at first glance it seems that a general solution is a can of worms.

If I get time to invest in this, I think I'd rather try to ship a SearchAlgorithm which continues to work with RCS store but perhaps caches just the %META part of every topic version in working somehow (reproducing the data/ directory layout, but Topic.txt,v would be a directory)

-- PaulHarvey - 11 Apr 2011