Feature Proposal: Need better mechanism than {PluginsOrder} to control plugin execution order
Motivation
Sometimes plugins have dependencies on other plugins, or need to be executed last. individual handlers may have different constraints.
Description and Documentation
Requirements:
Need to be able to express the following:
- Plugin A handler H needs to be executed before Plugin B handler H
- Plugin A handler H needs to be executed after Plugin B handler H
- Plugin A handler H needs to be executed before Plugin B handler H and again after Plugin B handler H
Perhaps:
- Plugin A handler H needs to be executed after context identifier xyz is set
- Plugin A handler H needs to be executed as early as possible
- Plugin A handler H needs to be executed as late as possible
"handler H" could be black (all handlers) or a specific handler.
So, like zone dependencies, the plugins order is an acyclic graph.
Note that this execution order will only apply to handlers (like
commonTagsHandler
,
beforeEditHandler
). Registered tags already have a well defined execution order (left-right-inside-out).
Approach 1
As described in
PluginOrderShouldSpecifyLastPlugins. Split the cfg{PluginOrder} variable into a First and Last section. Current setting is list of plugins, comma separated. Break the list into
FirstPlugins;LastPlugins, delimited by the semi-colon. If the ;Last section is present, those plugins will be run after the alphabetical processing of un-named plugins.
Example:
AnnotateAttachmentPlugin,SpreadSheetPlugin;DBCachePlugin,SpreadSheetPlugin
Approach 2
Finer control over the relationships between specific plugins using a more complex syntax to express them.
constraints ::= constraint ( ',' constraints )? ;
constraint ::= ( 'REPEAT' )? handler constraintname ( plugin | '*' ) ;
handler ::= plugin ( '::' handlername )? ;
constraintname ::= 'BEFORE' | 'AFTER' ;
plugin ::= <name> ;
handlername ::= <name> ;
Thus:
CommentPlugin::commonTagsHandler BEFORE WorkflowPlugin,
SpreadSheetPlugin BEFORE *,
AnnotateAttachmentPlugin::beforeSaveHandler BEFORE *,
REPEAT SpreadSheetPlugin::commonTagsHandler AFTER *
Note also that Approach 1's
AnnotateAttachmentPlugin,SpreadSheetPlugin;DBCachePlugin,SpreadSheetPlugin
is:
AnnotateAttachmentPlugin BEFORE *,SpreadSheetPlugin BEFORE *;DBCachePlugin AFTER *,REPEAT SpreadSheetPlugin AFTER *
Approach 3
Finer control as in (2), but removing the need for admins to maintain, or even know about our mess:
Each Plugin would contain its constraints
within its shipped Config.spec, and by default they would mix together just as desired.
eg.
$Foswiki::cfg{Plugins}{Order}{CommentPlugin::commonTagsHandler}{BEFORE} = 'WorkflowPlugin';
$Foswiki::cfg{Plugins}{Order}{SpreadSheetPlugin::*}{BEFORE} = '*';
$Foswiki::cfg{Plugins}{Order}{AnnotateAttachmentPlugin::beforeSaveHandler}{BEFORE} = '*';
$Foswiki::cfg{Plugins}{Order}{SpreadSheetPlugin::commonTagsHandler}{REPEAT AFTER} = '*';
Note also that Approach 1's
AnnotateAttachmentPlugin,SpreadSheetPlugin;DBCachePlugin,SpreadSheetPlugin
is:
$Foswiki::cfg{Plugins}{Order}{AnnotateAttachmentPlugin}:BEFORE} = '*';
$Foswiki::cfg{Plugins}{Order}{SpreadSheetPlugin}{BEFORE} = '*';
$Foswiki::cfg{Plugins}{Order}{DBCachePlugin}{AFTER} = '*';
$Foswiki::cfg{Plugins}{Order}{SpreadSheetPlugin}{REPEAT AFTER} = '*';
--
Contributors: GeorgeClark - 3 Jul 2010,
CrawfordCurrie - 18 Jul 2010, --
SvenDowideit - 19 Jul 2010
Discussion
See also
Tasks.Item1580 and
PluginOrderShouldSpecifyLastPlugins. It proposes a simple change to the
{PluginsOrder}
to allow specification of the "Last to execute" as well as the first to execute plugins.
Are you proposing that the user needs to configure the the plugin / handler order using a syntax shown in your verbatim blocks above? Would the typical admin have a chance of understanding how to do this? I'm concerned that it's too complicated.
--
GeorgeClark - 18 Jul 2010
For
Tasks.Item1943, it would be nice if there was a way the
InterwikiPlugin could execute a
commonTagsHandler
before the expansion of registered macros.
--
AndrewJones - 18 Jul 2010
George, sorry for missing your
PluginOrderShouldSpecifyLastPlugins proposal - I didn't see it at the time, and didn't find it when searching. What you describe is of course a way to express a subset of the possible relationships.
Plugins execution order bites down hard with a specific group of plugins;
SpreadSheetPlugin,
TablePlugin,
CommentPlugin,
EditTablePlugin,
ActionTrackerPlugin,
DBCachePlugin,
FormQueryPlugin - and just about anything else that uses a
commonTagsHandler
to process topic content. We have been eliminating it steadily with the introduction of registered tag handlers, so that the fine control is only required in a small number of cases. I'd prefer for plugins to themselves declare their dependencies, but to do that the plugins have to know about each other, which is impractical (many plugins are private to users).
In some cases - I'm thinking specifically of
SpreadSheetPlugin and
DBCachePlugin - are very delicate and difficult to get right. The first-last order just doesn't cut it, which was the reason for proposing the more complex relationships.
Andrew, the
beforeCommonTagsHandler
and
commonTagsHandler
are both called immediately before any macros are expanded. In effect, if plugin A declares
beforeCommonTagsHandler
and plugin B declares
commonTagsHandler
, this is equivalent to an execution order of
A::commonTagsHandler BEFORE *
, allowing
A::beforeCommonTagsHandler
to be moved to
A::commonTagsHandler
i.e. aspects of the execution order control are already hard-coded into the handler names (albeit rather inflexibly).
--
CrawfordCurrie - 19 Jul 2010
i prefer the dependency like approach in (2) more, but suggest
Approach 3
, as it can be implemented in a way that should require the installer not to know about it, or change anything unless there's a surprising wrinkle.
by this, I mean that rather than it being an ordered list, or string that has to be manually edited, it can be a hash of settings that each Plugin's Config.spec can set
eg.
$Foswiki::cfg{Plugins}{Order}{CommentPlugin::commonTagsHandler}{BEFORE} = 'WorkflowPlugin';
$Foswiki::cfg{Plugins}{Order}{SpreadSheetPlugin::*}{BEFORE} = '*';
$Foswiki::cfg{Plugins}{Order}{AnnotateAttachmentPlugin::beforeSaveHandler}{BEFORE} = '*';
$Foswiki::cfg{Plugins}{Order}{SpreadSheetPlugin::commonTagsHandler}{REPEAT AFTER} = '*';
where we use
*
as wildcard, and consider
REPEAT
as a modifier for
BEFORE
and
AFTER
.
the details of the example are only a first stab - the point is more that we should strive to make an already parsed, normalised form, so that foswiki admins don't need to make changes
most of the time.
Note: in either Approach 2 or 3, a Checker would need to be implemented to help admins resolve, fix and disambiguate (and report) issues with the constraints
--
SvenDowideit - 19 Jul 2010
OK. Well, that's pretty much just a flattening out of the syntax I proposed, which I don't have a problem with. It might be cleaner to take a slightly different approachj:
$Foswiki::cfg{Plugins}{CommentPlugin}{Order}{commonTagsHandler}{BEFORE} = 'WorkflowPlugin';
$Foswiki::cfg{Plugins}{AnnotateAttachmentPlugin}{Order}{beforeSaveHandler}{BEFORE} = '*';
$Foswiki::cfg{Plugins}{SpreadSheetPlugin}{Order}{'*'}{BEFORE} = '*';
$Foswiki::cfg{Plugins}{SpreadSheetPlugin}{Order}{commonTagsHandler}{AFTER} = '*';
So the {Order} is in the same place as the {Module} and {Enabled}. REPEAT can be eliminated as BEFORE and AFTER are relative to another handler call (or set of handler calls), so we can have as many BEFORE and AFTERs as we want, as long as the dependency graph remains acyclic.
--
CrawfordCurrie - 19 Jul 2010
I like the syntax you proposed in your last comment, Crawford.
In any case, tinkering with plugins order - no matter how sofisticated the means are to specify it - is calling for trouble and very delicate to get right. Worst case would be that
some TML works with one plugin order while other TML requires a different order on the same site. So that's not solvable using any fixed plugins order. I just mention it as
this all is trying to lower the symptoms rather than fixing
SpreadSheetPlugin (or
TablePlugin or
EditTablePlugin just to name another
commonTagsHandler
affine plugin).
So getting away from having a
commonTagsHandler
in these plugins is most probably better than
any plugin order (by rationalizing parsing tables rather than reparsing tables again and again in separate plugins).
There are some handlers other than
commonTagsHandler
, i.e. those around saving a topic, that are less obvious when they get it wrong and thus even harder to fix. These plugins may tinker with the
meta
object about to be saved. While
meta
is passed around it might be enriched with additional data or analyzed. Obviously, a different plugins order will unveil different results. From the docu alone it might not be obvious what happens to
meta
during save. Nor is it obvious or foreseeable who should specify which
handler comes first: in plugin A's Config.spec or in plugin B's?
So this proposal might most probably not fix this general problem.
Well, I'd welcome any means to be able to tinker with plugins order
as a last resort . But be assured that getting it right in a specific constellation of plugins is very hard and might need a detailed
code audit of the involved plugins ... a worst case of its own.
--
MichaelDaum - 20 Jul 2010
Note that if you want to pre-declare a plugins order in a Config.spec you can do it:
# **PERL**
# Define where handlers in this plugin need to be called, relative to the handlers in other plugins
$Foswiki::cfg{Plugins}{SpreadSheetPlugin}{Order} = {
'*' => { BEFORE => '*' },
commonTagsHandler => { AFTER => '*' }
};
--
CrawfordCurrie - 20 Jul 2010
What should happen if I have two plugins, each of which specify in their Config.spec that their handlers should be called first:
# **PERL**
# Define where handlers in this plugin need to be called, relative to the handlers in other plugins
$Foswiki::cfg{Plugins}{PushToTheFrontOfTheQueuePlugin}{Order} = {
'*' => { BEFORE => '*' },
};
and
# **PERL**
# Define where handlers in this plugin need to be called, relative to the handlers in other plugins
$Foswiki::cfg{Plugins}{ThereShallBeNoOtherPluginsBeforeMePlugin}{Order} = {
'*' => { BEFORE => '*' },
};
--
MichaelTempest - 20 Jul 2010
The ultimate sanction - alphanumeric ordering (alphabetti spaghetti)
--
CrawfordCurrie - 20 Jul 2010
like i said, a Checker would
need to be written to help the admin resolve issues like clashes - and more importantly, impossible rules. You don't get to add this kind of complexity without writing helpers to fix the corners.
--
SvenDowideit - 20 Jul 2010
If I (1) install two plugins and then (2) enable the two plugins at once, and then (3) don't check for warnings in configure, I could be running my wiki with impossible rules. Sure, (1-2-3) is bad practice, but it happens. Foswiki must do something about clashes and impossible rules - infinite loops and "die" should be avoided :). That may be as simple as disabling one of the plugins and reporting the error on
InstalledPlugins.
InstalledPlugins should also report the actual plugin order for each handler.
--
MichaelTempest - 22 Jul 2010