Item11326: SubscribePlugin is extremely slow with large complex WebNotify topics.
Priority: Urgent
Current State: Closed
Released In: 2.0.0
Target Release: major
Topic view for logged-in users on Tasks web is extremely slow.
- View any task or non-search topic as WikiGuest ~ 1 second to return the main document
- Log in - same view takes 13-18 seconds to retrieve the base document.
Other factors:
cover=print |
Fast |
skin=print |
Fast |
skin=plain |
Fast |
skin=default |
SLOW |
raw=on or raw=debug |
SLOW |
raw=text or raw=all |
Fast |
- Running
./view -user GeorgeClark -topic Tasks.Item11051
is Fast
- Checked with the release01x00 vhost - seemed faster, but still very slow
- Cloned Tasks web to another server. Seems fast in all conditions.
- Logged in as admin superuser. Still slow.
--
GeorgeClark - 04 Dec 2011
The problem is the
SubscribePlugin. It is adding 10-20 seconds to the rendering time. Disabled for now.
--
GeorgeClark - 04 Dec 2011
I feel as if this is a recent thing. And trunk is much faster. I've reproduced locally.
--
PaulHarvey - 04 Dec 2011
It happens on trunk code, something to do with Main web, I think.
--
PaulHarvey - 04 Dec 2011
I've tried versions of
MailerContrib back to
distro:9c88747740c3, but it's all still terribly slow..
--
PaulHarvey - 04 Dec 2011
Support web recommends that users should subscribe to their own question topics, but we've got
SubscribePlugin turned off, so they can't.
Added
WebSiteTasks to reflect this
I changed target to patch, given that we need to fix this for foswiki.org
--
PaulHarvey - 03 Mar 2012
The workaround for this is to set DISABLEDPLUGINS =
SubscribePlugin in Tasks.WebPreferences (and in any other webs with exceedingly large
WebNotify topics.
I added a config setting -
{SubscribePlugin}{AvailableWebs}
, and planned to handle the issue that way, but just disabling the plugin is just as effective, provided that it's not overridden in a topic. It might be best to add this feature anyway, as it gives a bit more control to the administrator.
I'm opening a separate task to add that feature, and will re-title this to deal with the performance issue directly.
Setting this task to be a release blocker for 1.2. The performance issues are too extreme to enable this as a default plugin.
--
GeorgeClark - 03 Mar 2012
the problem is due to the fact that the code that adds entries to a subscriber set attempts to optimise the set each time an entry is added. This is an O(N^2) process, but guarantees a fast lookup when searching for matches in the set. So there's a comromise here; by skipping the optimisation on the read we accept there may be some suboptimal entries. Optimising when a new webnotify is written means we get the best of both worlds.
Not quite finished here yet; the load is still too slow, need to take a closer look at that.
--
CrawfordCurrie - 22 Jan 2013
OK, I'm done. It should be useable on f.o again now. No changes to the
SubscribePlugin, all changes were in
MailerContrib.
--
CrawfordCurrie - 23 Jan 2013
- nytprof.tar.gz: NYTProf output using trunk code and foswiki.org's
data