Item6085: FilterPlugin documentation - examples reworked

I had no idea how much easier table extracts were with TWiki:Plugins.FilterPlugin until I saw TWiki:Blog.BlogEntry200808x7. In some ways, the blog described the plugin better than its home page. As KennethLavrsen indicated on 19 Jan 2007 in TWiki:Plugins.FilterPluginDev, it was hard to see what the plugin does. In part, this is because no simulated output was provided. But even after the plugin is installed, I had to switch between the raw and rendered views to understand what was going on.

Changes:

• Added sections like "You Type:", "Expected result (simulated)", and "Actual Result (this site)", stealing that idea from TWiki:Plugins.RenderListPlugin and TWiki:Plugins.DirectedGraphPlugin, (among others). That way the plugin page is still a useful reference after installation (and it's also easier to compare the expected and actual rendering, for installation test cases).
• The headings now clearly indicate what feature is being demonstrated (and arranged them alphabetically)
• I found the "Safe HTML" example to be particularly terse. I added a few words as to what was done, and why it's useful.
• I replaced the MAKEINDEX example with a shorter list, and only two columns instead of 3 to try fit in both the simulation and rendered versions (the image attached to the dev topic is required for that example).

Other comments:

• FORMATLIST has no example, and I'm still not clear on what you could with it. A little help?
• I probably broke whatever the %STARTINCLUDE%/%STOPINCLUDE% was being used for in the current version, so I simply removed it from this revision.
• By the way, in "EXTRACT example 2", header needs $n at the end: is that a bug?

---

---

Examples

EXTRACT Example 1: "Convert Table into Text"

One of the uses of this plugin is to extract data from tables, which is useful for creating "database-like" TWiki applications. While it is certainly possible to do that without this plugin (see "Pattern 1: Extract values from a table" in SearchPatternCookbook), the plugin makes these requests easier to create and maintain. For another example, see TWiki:Blog.BlogEntry200808x7.

The table:

Pos	Description	Hours
1	onsite troubleshooting	3
2	normalizing data to new format	10
3	testing server performance	5

You type:

%EXTRACT{topic="%TOPIC%" expand="off" 
  pattern="^\|\s\s(.*?)\s*\|\s*(.*?)\s*\|\s*(.*?)\s*\|" 
  format="   * it took $3 hours $2$n"
  skip="1"
}% 

Expected result (simulated):	Actual result (this site):
it took 3 hours onsite troubleshooting it took 10 hours normalizing data to new format it took 5 hours testing server performance	it took 3 hours onsite troubleshooting it took 10 hours normalizing data to new format it took 5 hours testing server performance

EXTRACT Example 2: "Convert Text into Table"

Use CSS tags to format text comments as a tabular data (e.g., to allow sorting).

The comments:

You type:

%EXTRACT{
   topic="%TOPIC%" expand="off"
   pattern=".div class=\"text\">.*?[\r\n]+(.*?)[\r\n]+(?:.*?[\r\n]+)+?-- (.*?) on (.*?)[\r\n]+"
   format="| $3 | $2 | $1 ... |$n" header="|*Date*|*Author*|*Headline*|$n"
}%

Expected result (simulated):	Actual result (this site):
Date Author Headline 22 Aug 2005 MichaelDaum This is the first comment. ... 22 Aug 2005 MichaelDaum This is the second comment. ...	Date Author Headline 22 Aug 2005 MichaelDaum This is the first comment. ... 22 Aug 2005 MichaelDaum This is the second comment. ...

FORMATLIST example

(example needed. Maybe use TWiki:Support.FormattingFieldValues).

MAKEINDEX example

Create an index of TWiki web (for brevity, this demonstration uses only a small subset of the web):

You type:

%MAKEINDEX{
   "%SEARCH{".*" web="%SYSTEMWEB%" scope="topic" type="regex" limit="30" nonoise="on" format="$topic" separator=", " excludetopic="CGI*,*Plugin"}%" 
   cols="2"
   format="[[TWiki/$item][$item]]"
}% 

Expected result (simulated):

Actual result (this site):

A ACRONYM AccessControl AdminDocumentationCategory AdminSkillsAssumptions AdminToggle AdminToolsCategory AppendixCascadingStyleSheets B BeginnersStartHere BookView BulkRegistration C CacheContrib Category ChangeEmailAddress ClassMethod CommandAndCGIScripts CommentPluginTemplate CompareRevisionsAddOn CompleteDocumentation Contribs ContributedAddOns CountryList D DataForms DefaultPreferences DefaultPreferencesForm DefaultUserRegistration DefaultWebStatistics DeveloperDocumentationCategory DevelopingPlugins DocumentGraphics E EditRowPluginExamples

SUBST example: "Safe HTML"

Use %STARTSUBST%/%STOPSUBST% to disable potentially annoying or dangerous Javascript in user provided content.

You type:

%STARTSUBST{pattern="<(a href=\"javascript:.*?)>(.*?)</a>" format="<$1>$2</a>"}%
<a href="javascript:window.alert('Pop me up');">Pop me up</a>
%STOPSUBST% 

If not installed (simulated):

Pop me up

Expected result (simulated):	Actual result (this site):
<a href="javascript:window.alert('Pop me up');">Pop me up</a>	<a href="javascript:window.alert('Pop me up');">Pop me up</a>

Looks great! Have a checkin.

-- MichaelDaum - 23 Oct 2008

Another example I have solved with FilterPlugin is when using select+values. The topic is autoinc so thats easy to save but the text the user want to see sometimes have commas, and sometime I dont own the text so that I can just remove them. For example when we import the data from external source. Then filterplugin saves the day, that is: strips away commas from the formfield so it doesn't interfer with dataform handling of the select.

| Owner | select+values | 5 | %DBQUERY{ "MyForm.Active = 'Yes'" separator=", " sort="Nr" format="$formfield(Nr) $percntSUBST{ text=\"$formfield(Name)\" pattern=\",\"  }$percnt=$topic" }% | Who is the owner? |  |

Not to say I didn't wish there was an easier way.

-- LarsEik - 09 Nov 2009

Wow this was lost long time.

-- MichaelDaum - 07 Jun 2010