Feature Proposal: DataForm inheritance

Motivation

In modelling data, generalisations are common. In object oriented programming it is called subclassing, or extending a class. However, generalisations can not be expressed with DataForms. Each topic can only have one form, and forms can not include other forms.

Description and Documentation

The implementation interprets lines of the form:

BASE=Foo.BarForm

On save, this will add:

%!META:BASE{value="Foo.BarForm"}%

There can be multiple BASE forms defined for a given form, resulting in a comma-seperated list in the value of META:BASE.

All of the fields of the BASE forms will be added to the Foswiki::Form object, so when rendered for viewing or editting, those fields will be part of the form as if they were defined in the form itself.

We can search for forms of a given 'type' in various ways:

Classic search

...

Query search

%!SEARCH{"type ~ '*DataDefinition.VideoForm*'" type="query"}%

Would give me all video topics.

DBCacheContrib

Using DBCacheContrib, from within a plugin i could:

        my $db=Foswiki::Plugins::DBCacheCache::getDb( $web );

        my $query="(type 'VideoForm')";

        my $search=new Foswiki::Contrib::DBCacheContrib::Search($query);

        my @results=$db->getKeys();

and get all Video topic names in the @results array.

Furthermore, given a DBCacheContrib topic 'object' (map is the proper term), one can get the list of baseforms by doing:

  my $topic=$db->get('SomeVideo');
  my $base=$topic->get('base');

It works

The implementation works (attached patches against 4.2-RC2), i'm using it currently and it adheres to the KISS principle. It does not incur a performance hit as far as i can oversee right now, which is also quite important.

Examples

An example would be a database with persons, say: customers and employees. Both share a lot of fields, but you have to define two seperate forms to accomodate each of those. With inheritance, one could define a PersonForm, and let CustomerForm and EmployeeForm inherit fields of that form.

Another example is in the context of the application i did recently (http://www.biografievanamsterdam.nl/). The site contains media, so it has a MediaForm with the fields title, author and common attributes like that. Media can be audio, video, text or picture, so there are AudioForm, VideoForm, TextForm and PictureForm. Going even deeper, there are several types of possible videos: youtube, stage6, ... So there are YoutubeVideoForm, Stage6Form, with fields such as YoutubeID.

I can now loop over all media, or maybe over all video's, regardless of the specific type it is.

This is the MediaForm:

| *Name* | *Type* | *Size* | *Values* | *Tooltip message* | *Attributes* |
| 
 naam 
 | 
 text 
 | 
 64 
 |
   
 |
   
 | 
 M 
 |
| 
 abstract 
 | 
 textarea 
 | 
 80x6 
 |
   
 |
   
 | 
 M 
 |
| 
 maker 
 | 
 link[Main.User]+multi 
 | 
 64 
 |
   
 |
   
 |
   
 |
| 
 publicatiedatum 
 | 
 Date 
 | 
 20 
 |
   
 |
   
 |
   
 |
| 
 bron 
 | 
 link[Bron] 
 | 
 3 
 |
   
 |
   
 |
   
 |
| 
 licentie 
 | 
 link[Licentie] 
 | 
 3 
 |
   
 |
   
 | 
 M 
 |
| 
 kanaal 
 | 
 link[Kanaal]+multi 
 | 
 6 
 |
   
 |
   
 |
   
 |

And here is my VideoForm:

BASE=DataDefinition.MediaForm




| *Name* | *Type* | *Size* | *Values* | *Tooltip message* | *Attributes* |
| 
 duur 
 | 
 duration 
 | 
 20 
 |
   
 |
   
 |
   
 |
| 
 xres 
 | 
 int 
 | 
 20 
 |
   
 |
   
 |
   
 |
| 
 yres 
 | 
 int 
 | 
 20 
 |
   
 |
   
 |
   
 |
| 
 format 
 | 
 select 
 | 
 3 
 | 
 super8,16mm,swf,mpeg,anders 
 |
   
 |
   
 |

And finally, the YoutubeVideoForm:

BASE=DataDefinition.VideoForm




| *Name* | *Type* | *Size* | *Values* | *Tooltip message* | *Attributes* |
| 
 youtubeID 
 | 
 text 
 | 
 64 
 |
   
 |
   
 | 
 M 
 |

Impact

WhatDoesItAffect:

%WHATDOESITAFFECT%

edit

Implementation

The attached patches are against 4.2-RC2, and are quite raw. I need to update this project to 4.2-RELEASE and polish it all up.

-- Contributors: KoenMartens - 19 Sep 2007

Discussion

mmm, we've been here before Personally I'd rather use mashup than OO to do this.

What I mean, is that both the customers and employees form are fully self contained, and instead of defining some inheritance (or other technical thing) you can add more than one form to a topic - where fields with the same name are then common to both.

Mind you, there's a issue to be resolved - if the form definitions are in seperate webs, and you specify the fields in the definition using webname too (a very useful practice) then mashing up the fields needs some other mechanism.

I guess we're looking at different purposes - I'd like not-co-developed wiki apps to be able to share linkages, whereas you're looking towards reuse inside a wiki app..

worth re-reading the multiple forms topics (they're around here somewhere)

-- SvenDowideit - 19 Sep 2007

We discussed this before, though probably only on IRC. When you mashup, TopicOne contains:

%META:FORM{name="Webzero.MyForm"}%
%META:FIELD{name="Glumph" ...}%
%META:FIELD{form="Webone.MyForm" name="Glumph"...}%
%META:FIELD{form="Webtwo.MyForm" name="Glumph"...}%

should all work in the same topic, and should present three different fields all called 'Glumph'. The problem with this is content access. When I write %IF{"TopicOne.Glumph='...'"}% which field am I referring to? Do I have to write =TopicName.Webone.MyForm.Glumph to disambiguate it? If I only allow one 'Glumph' in a topic, then I have to have some mechanism for disambiguating which 'Glumph' takes precedence (which is the advantage of an inheritance based approach, in that it does that disambiguation for you). One approach is to simply take the last field called 'Glumph' as the 'master Glumph' and throw the others away, which is OK but requires us to carefully manage the order in which forms are attached to topics.

BTW my preference is to:

1. Support mashup rather than inheritance
2. Attach a form anywhere in a topic (not just at the end)
3. Disambiguate the ContentAccessSyntax using an optional form specifier

So my TopicOne would contain:

Some text
%META:FORM{name="Webzero.MyForm"}%
some more text
%META:FORM{web="Webone" topic="MyForm" name="Chatter"}%
some more text
%META:FORM{web="Webtwo" topic="MyForm" name="Natter"}%
some concluding text.
%META:FIELD{name="Glumph" ...}%
%META:FIELD{form="Natter" name="Glumph"...}%
%META:FIELD{form="Chatter" name="Glumph"...}%

and my IF statement: %IF{"Glumph='...' AND Natter.Glumph='...' AND TopicOne.Chatter.Glumph='...'" ...}%

-- CrawfordCurrie - 20 Sep 2007

Most of the mentionend features can be implemented with the means at hand already, given you replace inheritance with mixins.

I don't think that we need to render forms in different places of the text. The standard appearance of a form is much too static and ugly. You can do much better with a TopicFunction, like this %INCLUDE{"RenderPartOftheForm"}%, instead of relying on the way the appearance of form data is hardcoded to the core engine.

The only inconvenience is that you have to create a new DataForm whenever you want to store the data of two different applications in one topic. What you lose then is the ability to search for topics that contain a certain data model, or part of it. So when there are two applications A and B, one defines FormA and the other FormB, and both applications have different controls for both forms, then a combined form that holds both sets of data, a FormC, will not be controllable by the applications A and B anymore .... if they relied on the form name. For example a SEARCH for FormA will only find topics that have a FormA, but not FormC, even though FormC implements all of the data model that FormA did. The solution to this problem is to properly type topics by having a formfield TopicType in all of your forms. To continue the example,all topics that implement the data model as defined in Form will be typed as TypeA. Similarly for topics that have FormB that is typed as TypeB. The new form that implements FormA + FormB, called FormC, will be typed TypeA, TypeB, TypeC. Thus applications A and B can still control topics of the respective type, and cover any derived topic automatically. This, infact, is the basic principle of the type system in the upcoming WikiWorkbenchContrib.

-- MichaelDaum - 21 Sep 2007

Ok, seems i have some reading to (preceded by a long search around the Codev maze probably). Mashup sounds like an ugly concept though. What i want to do is write a base class (base form really), and then derive more specific instances of such a class. I do not want to have to define the common fields twice, or have multiple instances of the same field in a single topic. It sounds silly to me.

I like Michaels aproach, and am looking forward to what Michael has implemented. I do, however, need this inheritance for a customer project right now, so regardless of this discussion i will be implementing it as i originally proposed. My only hope is that sometime inheritance becomes part of core so i won't have to maintain a seperate code branch.

-- KoenMartens - 22 Sep 2007

I created some wiki applications in the past with "poor-man's-inheritance-without-inheritance-forms": Create a GlossaryForm as the base form and a EmployeeHandbookForm that has all field of the base form, plus a few other ones. This allows one to run reports across similar types of forms, while doing some special reports. Granted, the duplication of form fields can get out of sync, so a true inheritance is a better solution.

By implementation, we could add a parameter to the form:
%META:FORM{name="EmployeeHandbookForm" base="GlossaryForm"}%.

Fields that are defined in the child form can overloading the ones in the base form (if present).

As for user interface, when you select a form you can optionally select a base form. I do not think we need to worry about multiple inheritance, so just one base form should be sufficient. A base form could inherit from another form, so multiple levels of inheritance could could be possible.

Once a form with base form has been selected, users see and edit a form that is composed of the form and the base form. That is, when a user edits a topic (s)he does not care where the fields are defined, the only concern is to see the fields needed (don't make me think.)

-- PeterThoeny - 24 Sep 2007

At one stage I also had form definition inheritance working using searcional INCLUDE - each form definition topic would have a STARTSECTION, ENDSECTION around the definition table (without the HEADER bit), and then i just INCLUDEd them together

-- SvenDowideit - 25 Sep 2007

This discussion seems to have stalled. And I do not feel we have either a concensus or a particular clear spec to vote on. The customer advocates and users that may want to vote needs a spec. They cannot read the code and figure out how the feature incl syntax will work. So I will not take the proposal to a release meeting without more meat (10-20 lines of text kind of meat will do).

I would also like to hear if Crawford, Sven and Michael still have concern about the feature or just wanted to advice for alternative ways to do similar.

The scope of the proposal will be GeorgetownRelease and not Freetown (4.2.0).

-- KennethLavrsen - 09 Oct 2007

From what I can see above, all 4 of us are leaning towards using mixins (mashup is really the same thing), but I agree with Kenneth, a more meaty example of how a user would use it will help allot.

-- SvenDowideit - 29 Dec 2007

Ok, i expanded a bit on the topic. Patches are attached. Could we discuss this on the next release meeting? I don't think mashups or mixins are what i'm after, but i lack a bit of time to dive into it. I'm willing to, but only if this proposal has a chance of making it. If not, i'm not going to waste my time of course, i've got tons of other things on my todo list

-- KoenMartens - 14 Feb 2008

http://www.biografievanamsterdam.nl/ looks great! Can't believe it is done via Foswiki. Excellent work.

-- FranzJosefGigler - 15 Feb 2008

Franz, thanks Most of the credit for its appearance must go to the designer though (Auke Touwslager, http://www.informationlab.org/, i merely implemented his drawings

-- KoenMartens - 15 Feb 2008

I'd very much like to continue this discussion as I do form inheritance as well on standard TWiki-4.(1|2).x. Instead of inventing a new META tag, have you considered to implement inheritance using a normal FORMFIELD with a special name. For example I use a TopicType formfield to mark its type. I once had another exta "Inheritance" formfield to make it even more explicit, but found out I don't need it really. Inheritance does not come with just marking the TopicType in terms of actually reusing structural properties from other form definitions. However, I can simulate this using mixins and appropriate type markers, that is used as a kind of "contract" to wiki apps that are then apple to pick up those topics. Which means: one topic can participate in different applications. I only have to type it accordingly. This is very light-weighted, no extra processing is needed. The only downside - and I think that is what you are looking for - is modelling the DataForms. This has to be done with care so that the "contract" expressed by the TopicType marker is actually fulfilled. Note however, that using inheritance for "code reuse" is considered a weak argument for inheritance.

For instance, I have created a basic set of TopicTypes and DataForms that describes all aspects of a wiki application:

Types

1. TopicType: the base of the type system
2. WikiTopic: a normal topic
3. ApplicationTopic: a WikiTopic that is part of a WikiApplication
4. WikiApplication: description of a WikiApplication, similar to a plugin description, used to list all of its ApplicationTopics
5. TopicFunction: an ApplicationTopic used to implement reusable functions, that are called using a parametrized %INCLUDE{}% or %DBCALL{}
6. TopicView: an ApplicationTopic to implement VIEW_TEMPLATEs or EDIT_TEMPLATEs
7. TopicTemplate: an ApplicationTopic used as a blueprint during creation of a WikiTopic
8. DataForm: an ApplicationTopic used to defined a form
9. DataFormAttribute: an ApplicationTopic listing defined values in a DataForm definition
10. TopicStub: an ApplicationTopic used as a placeholder for another topic being targeted by the stub. This a convenient way to deploy WikiApplications into another web, that is a primitive topic that only does one transclusion of the real implementation that stub is standing for. TopicStubs have an installer to ease that process.

Note that one WikiTopic can have multiple types. For example, you end up defining new TopicTypes that correlate with a DataForm definition one on one. So you can implement both on the same topic and mark its of type "TopicType, DataForm".

(Hehe, once you groked "topic of type TopicType", you've got it .)

Anyway, this is the basic set.

There are some more that I use regularly, e.g. DocuTopic and the like. But the above outline the things that come together in a WikiApplication on a regular base. You might have noticed that there is already some kind of inheritance going on defining those types. This becomes more visible if you consider the predefined DataForms that might be used together with the above types:

Forms:

1. WikiTopic: every topic of type WikiTopic shall have a form with at least three formfields:
   1. TopicType: every topic is typed
   2. TopicTitle: a free-form description of the title of this topic, making things independent from the actual topic name; DBCachePlugin implements a %TOPICTITLE% that expands to TopicTitle, if defined, and defaults to the topic name if not.
   3. Summary: a one-line description what this topic is about, used in various dynamic lists, reused as a holder of taglines in other applications, etc.
2. ApplicationTopic: inherits all formfields from WikiTopic and adds only one other formfield:
   1. TopicType
   2. TopicTitle
   3. Summary
   4. WikiApplication: points back to the application this topic is part of
3. TopicStub: inherits all of the formfields of ApplicationTopic plus
   1. TopicType
   2. TopicTitle
   3. Summary
   4. WikiApplication
   5. Target: full qualified topic name of the topic this topic is a stub for
   6. Section: named section to be transcluded from Target

As you see there are more types of topics in Foswiki than you'd need forms for. Forms are reused by various TopicTypes depending on the function this topic has within the application.

This is actually the basic idea on which I build up all of my WikiApplications. All of them are build the same way, using inheritance and types. etc.

Does that make sense for you?

-- MichaelDaum - 15 Feb 2008

Yes, i am interested in automatically have the fields of base forms (included forms, parent forms, whatever we call them) automatically appear in whatever instance of the form anywhere. I'm not sure about mashups/mixins, as i really don't understand what those are and how they are implemented currently.

Can you maybe elaborate a bit more, how your inheritance works? How does your solution work for the example I gave, with media <- video <- youtubevideo? In my setup, i create a topic and attach the YoutubeVideo form. That topic can then automatically also be treated as a VideoForm topic or a MediaForm topic, without the need of a change in the WikiApplication or plugin.

May I ask, apart from Occam's razor, why you do not like (interpretation is mine) the new META tag?

Simulating something is nice, but it does obfuscate how things work i think. Explicating such things make for easier use.

-- KoenMartens - 15 Feb 2008

I am not opposing of expanding Foswiki's power! However, I am first looking at what I have and if it is expressive enuf and how far I can stretch it without tearing it appart obviously. I do think that using formfields and having a bit of convention is not that much of a stretch, for now.

In your example I'd add three topic types to the specific MyNewYoutubeGem topic

TopicType: "MediaTopic, VideoTopic, YoutubeVideo" 

so that it can participate in all media, video and youtube apps. Note, that this does not necessarily impose a specific form. A normal MediaForm or VideoForm, might be enuf for YoutubeVideos already. But once you want to want to store specific youtub params, params to the flash player and the like, I'd copy-paste the VideoForm into a YoutubeVideoForm and add a couple of lines to define the additional formfields needed. In your approach, you'd make it an extra META:BASE entry in the YoutubeVideForm pointing back to the VideoForm definition, right?

I was considering type inference so that I wouldn't have to use three type markers but only one - YoutubeVideo - and infer the other types automatically. But I am afraid that this is having a real impact on performance...

-- MichaelDaum - 15 Feb 2008

I like your system of building wiki applications, but i really think the ability to have this kind of inheritance makes sense. In my example use case a youtube video always has a required YoutubeID, without it you can not locate the actual video on youtube. And this is also the only field that is added relative to a generic video.

I would not want to have to maintain copies of the base forms in each of the descendant forms. That would mean, that if i add a field to, say, the MediaForm, i need to track down all forms that are descendants of it and add the field there too. What a pain!

I haven't so far noticed any problem with performance, though i haven't done any thorough testing. Since i'm using DBCacheContrib for all searches anyway, i'm not parsing those forms for each operation. I'm not sure of the internals, does wiki do any caching of parsed form definitions? If so, the perfomance hit would be virtually none. If not, it might be something worth considering as a seperate future request anyway (tangential with the XML discussion i guess).

(On a side note, how well does DBCacheContrib scale with hundreds of thousands of topics?? Not very well i would guess.)

I've been thinking a bit more about this mashup/mixin stuff, and i was wondering. Would that mean i need to address fields as 'YoutubeForm.YoutubeID' instead of just 'YoutubeID' as i can do now? Or more to the point 'MediaForm.Title' instead of just 'Title'. Ie. do i have to know, when i have a topic, what form the field i am interested in belongs too?

-- KoenMartens - 16 Feb 2008

does foswiki do any caching of parsed form definitions - within a session, yes. Otherwise, no.

how well does DBCacheContrib scale with hundreds of thousands of topics - not well. Technical limits are arounf 50K topics.

-- CrawfordCurrie - 17 Feb 2008

Koen, you are right. Maintenance using inheritance to reuse code is eased. Having worked with lots of oo libraries in different languages, I have seen developers fail to grok this kind of inheritance (inheritance for code reuse) when inheritance hierarchies become deeper than a level of three. Infact, inheritance isn't used that much as one expects approaching oo libraries. More often, you see "delegation" (objects delegate specific properties and methods to sub-objects). Such a class will then by typed accordinginly (interfaces in java), to flag capabilities. Most of the time code of inheritted base classes that gets reworked falls on your feet in derived classes easily, and you'd have to touch, at least look at and take care of them anyway, while expanding the base class. You might think "hey, but redundancy is evil". I agree in general, but system designed with minimal redundancy often develop a lot of crossdependencies in the code that get out of hand.

Anyway, that all said, making form inheritance possible in Foswiki might still be a good idea. It is then up to the application developer which means he makes use of. So I am nearly convinced actually.

-- MichaelDaum - 17 Feb 2008

I was not considering my proposal in the perspective of code, merely of being able to specify a hierarchy of data types. That is probably an all-together different subject, given that (afaik) there is no such thing as a Foswiki function/method, apart in various frameworks such as your own (of which i saw a demo you gave, it is great stuff). I do think there is a seperation between data and code currently in Foswiki. Obviously, object orientation is not a part of it, you can not call method Foo on topic BarBaz. You can 'call' a static function in a WikiApplication, by using a parametrized include, and pass the topic as an argument.

I think (and i probably could substantiate this by collecting Codev topics), that this feature would fill a need many have. My example is one of many where this kind of sub-typing would be very convenient.

Given that i have not seen much discussion on this topic anymore, might i conclude that (in the sense of WikiRelease04x01Process), we have reached a consensus? Should this be pushed to a release meeting? Crawford, Sven, Michael, you are listed under 'concerns raised'. Are your concerns addresses?

(ps: i meant this proposal for freetown / 4.2.x, so i changed the ProposedFor field, is that ok? i am still not comfortable with city names for releases, numbers work better for me

-- KoenMartens - 21 Feb 2008

I am removing my concerns.

-- MichaelDaum - 21 Feb 2008

Sorry, I hadn't realised I was listed as having a concern. I don't.

-- CrawfordCurrie - 21 Feb 2008

I have no concern for the feature as such.

I am a little concerned about the syntax seen from the users point of view. Will a real estate agent, a dentist, a nurse etc using Foswiki and trying to use this feature be able to understand the syntax? I must admit that I am not myself 100% sure I understand it. I only have the example to go by and not much documentation. Can you elaborate a little more on this Koen and also do a self evaluation against the NerdoMeter. And if needed propose improvements in syntax.

-- KennethLavrsen - 21 Feb 2008

Hehe, if a nurse tried using this feature I'd pay her more money and give her a new job ... next to a coffee machine.

-- MichaelDaum - 22 Feb 2008

Kenneth, will a nurse be able to use the SpreadsheetPlugin? No, it's convoluted, buggy but extremely powerful and the basis of many great wiki applications. I think the syntax for this proposal is sufficiently simple that most moderately advanced wiki users (those that actually develop wiki applications) can understand and use it.

Sven, you're the last one listed as having a concern, however your last comment was from before i gave the topic some more meat. Care to comment on your concerns a bit more, or have they been adressed?

-- KoenMartens - 02 Mar 2008

ok, lets begin with the definition syntax:

BASE=DataDefinition.VideoForm

| *Name* | *Type* | *Size* | *Values* | *Tooltip message* | *Attributes* |
| youtubeID | text | 64 | | | M |

I would suggest that seperating the definition like this is fraught with dangers

better to do somethign more consistent

| *Name* | *Type* | *Size* | *Values* | *Tooltip message* | *Attributes* |
| DataDefinition.VideoForm | BASE |  | | | |
| youtubeID | text | 64 | | | M |

next, we need to work out howto change the defaults for members of a base

and then, there's the issue of what happens when several base's define similar fields, but perhaps use different webs due to history - do we have formfield aliasing to coaless or disabiguate fields?

which leads me to ponder

| *Name* | *Type* | *Size* | *Values* | *Tooltip message* | *Attributes* |
| BASE1 | BASE | | DataDefinition.VideoForm | | coaless=(*),disabiguate=() |
| BASE2 | BASE | | Sandbox.RatingForm | | coaless=(Rating),disabiguate=(Sandbox.Author, Main.Date) |
| youtubeID | text | 64 | | | M |

i guess I'm looking at this, and thinking that alot more needs to be defined before its ready to be given to everyone.

Another thing that always bothered me, is that if the form definition topic changes, the topics that use that form have no idea what version of the form definition they came from, and somehow, adding multiple form topics, and mashing them together is scary.

-- SvenDowideit - 18 Mar 2008

Wait, why not keep it lean and just include the base form definition at the point it is inserted. That's how Koen did it, right? Do we really need to cope with name clashes? How about simply ignore this problem for the sake of simplicity?

A change in a form definition has always been a probable cause of data loss on the next action, nothing new here...

-- MichaelDaum - 18 Mar 2008

my point is that just include the base form definition at the point it is inserted. is too vague a specification, and that this is less about just implementing something, and more about not making Foswiki even harder for users to understand and use (ie, its not about the 12 of us that are intimatly aware of Foswiki's innards, its about everyone else)

-- SvenDowideit - 18 Mar 2008

lol

-- MichaelDaum - 18 Mar 2008

I agree witch Michael, let's keep it simple. We can add ways of dealing with clashes or overriding defaults later on.

Anyway, i'm about to give up on this. I've wasted too much time on getting this into core already, and i'm happy with it as a private patch. I'll happily modify the feature code if we can reach some concensus, but not if it means implementing lots of extra code to complicate things in ways i don't see the current use of.

So Sven, can you live with this as a 'growing' feature? To which we can later add complications such as name-clash resolution and default overriding? Or will you veto this? Let me know, so i know where i stand and whether i should invest any more time in this. I believe it is a much requested feature, which is why i went through the trouble of this bureacracy anyway, but not at all costs.

-- KoenMartens - 26 Mar 2008

Koen, have you read RequirementsForMultipleFormSupport?

-- RafaelAlvarez - 27 Mar 2008

Is there any docs on howto mashup/mixin today (cant' find any)? I think this sounds like a good feature.

-- LarsEik - 27 Mar 2008

Changed the ProposedFor to GeorgetownRelease (5.0). No new features will be added to patch releases. Only bugfixes. This is the general rule for patch releases and has always been. The target for any new feature is 5.0 or later.

Koen said: "Kenneth, will a nurse be able to use the SpreadsheetPlugin?" - The answer is "No". And so what? Just because we have horrible syntax in an old feature does not mean we have to add more of it. Nurses should be able to use Foswiki. Foswiki should not be a wiki just for software developer geeks and if all we need is to spend another hour on getting a better syntax then why not do it? Even I do not yet understand the syntax suggested in this proposal and I am a Foswiki geek.

-- KennethLavrsen - 30 Mar 2008

Just a few clarifying words on which users will benefit from this.

• It is not for nurses.
• It is not for wiki content readers, lurkers, page hoppers.
• It is not for content contributors, authors.
• It is uninteresting for web designers.
• It is not for marketing guys.
• The sales department wont be interested.
• There's no screenshot that would thrill a usability guru.
• It is not even for plugin authors, perl coders.

However, this is targetted for a certain subset of WikiApplication programmers and knowledge architects that like to structure their work in an object oriented fashion as this is the way some of us are used to think about it. At least you'd like to have the choice to have DataFormInheritance. And it definitely is a missing feature in Foswiki that has been mused about earlier.

Contrary to common sayings about siturational wiki applications, WikiApplications are all geekiness pure. A single hour thinking about this will not change this. Agreed, the overall geekiness of Foswiki's syntax (its application part) is already extraordinary. The proposed syntax here is comparably harmless.

-- MichaelDaum - 01 Apr 2008

Why not spend that hour and get this feature defined so for example I understand the syntax???

Why does it have to be geek? Why is it not for nurses? (the nurse with a little interest in computers who can make a formular in Excel etc).

Naturally this feature can be defined a little more user friendly!

-- KennethLavrsen - 01 Apr 2008

My issue is simple - I'd like to avoid making a quick syntax change here that leads to years of supporting what we realise could have been avoided if only we'd bothered discussing and designing a full solution.

Inheritance is one of those wonderful things that if you don't get it right, you're lumbered with cornercases and unexpected side effects.

One thing I'm reminded of, is that in Cairo days I implemented form definition inheritance using INCLUDE and other TML syntax - in essence, by pre-evaluating a form definition topic before its used. That mechanism required no added syntax, just named INCLUDE sections (i think)

-- SvenDowideit - 02 Apr 2008

Did anybody try? I doubt it works out with a simple INCLUDE right now already. I do remember Sven proposed this some time ago during the 4.2 development cycle, but nothing happend in that respect. Maybe I missed it.

-- MichaelDaum - 02 Apr 2008

Michael, or anybody, can you make an example with a PersonForm, CustomerForm and EmployeeForm?

-- LarsEik - 02 Apr 2008

This is untested, expected not to work in released Foswiki's, but is a reconstruction of my memory of the sort of thing I did ~2003

This is the MediaForm:

| *Name* | *Type* | *Size* | *Values* | *Tooltip message* | *Attributes* |


%STARTSECTION{MediaFields}%



| 
 naam 
 | 
 text 
 | 
 64 
 |
   
 |
   
 | 
 M 
 |
| 
 abstract 
 | 
 textarea 
 | 
 80x6 
 |
   
 |
   
 | 
 M 
 |
| 
 maker 
 | 
 link[Main.User]+multi 
 | 
 64 
 |
   
 |
   
 |
   
 |
| 
 publicatiedatum 
 | 
 Date 
 | 
 20 
 |
   
 |
   
 |
   
 |
| 
 bron 
 | 
 link[Bron] 
 | 
 3 
 |
   
 |
   
 |
   
 |
| 
 licentie 
 | 
 link[Licentie] 
 | 
 3 
 |
   
 |
   
 | 
 M 
 |
| 
 kanaal 
 | 
 link[Kanaal]+multi 
 | 
 6 
 |
   
 |
   
 |
   
 |


%STOPSECTION{MediaFields}%

And here is my VideoForm:

| *Name* | *Type* | *Size* | *Values* | *Tooltip message* | *Attributes* |


%STARTSECTION{VideoForm}%
%INCLUDE{"Forms.MediaFields" section="MediaFields"}%



| 
 duur 
 | 
 duration 
 | 
 20 
 |
   
 |
   
 |
   
 |
| 
 xres 
 | 
 int 
 | 
 20 
 |
   
 |
   
 |
   
 |
| 
 yres 
 | 
 int 
 | 
 20 
 |
   
 |
   
 |
   
 |
| 
 format 
 | 
 select 
 | 
 3 
 | 
 super8,16mm,swf,mpeg,anders 
 |
   
 |
   
 |


%STARTSECTION{VideoForm}%

And finally, the YoutubeVideoForm:

| *Name* | *Type* | *Size* | *Values* | *Tooltip message* | *Attributes* |


%INCLUDE{"Forms.VideoForm" section="VideoForm"}%



| 
 youtubeID 
 | 
 text 
 | 
 64 
 |
   
 |
   
 | 
 M 
 |

It doesn't take much to imagine a way to have an INCLUDE like (or INCLUDE type) that would bring in the fields from another form definition, without the need for the cluncky named SECTIONS, but my goal at the time was to use the existing power of TML.

IIRC the code change needed to make it work, was to add one line - a getRenderedVersion before the form definition topic was parsed - the net result being that I was able to integrate that companies TasksContrib with their external timesheet & billing system, by INCLUDEing an external URL that contained a list of fields & field values.

However, this solution ignored the side effects of multiple inheritance, issues with multiple instances of base members, and reflections and queries on the base form types - all of which ought to be considered if we are going to create a new syntax for this sort of thing.

-- SvenDowideit - 03 Apr 2008

This proposal seems stalled. So I will bring it up at next release meeting for debate.

I would not want to vote on it yet, unless the original proposal develops towards something that more then 2-3 people in the community understands.

-- KennethLavrsen - 14 Apr 2008

Another important point: a plugin may need access to the DataForm definition, e.g. to find out which formfields are of type foo. Do we actually have a perl interface in place to do that?

-- MichaelDaum - 16 Apr 2008

yes. Foswiki::Form.

-- SvenDowideit - 16 Apr 2008

an interesting related feature idea: HowToUseFormattedSearchWithForms

-- SvenDowideit - 01 Jul 2008

Sorry, it does not seem likely that i will be developing on Foswii in the near future. So unless there is someone who wants to be put down as a committed developer, let's close this.

-- KoenMartens - 07 Jul 2008

Moved over from TWikiFormInheritance as there still is interest in such a feature. (Note, the attachments didn't make it while copying this topic over here due to a bug in the foswiki engine used on foswiki.org ... a bug fixed in the upcoming release of foswiki).

-- MichaelDaum - 01 Oct 2010

I like the idea of using %INCLUDE a lot, with the caveat that whoever implements it has to be careful to order the columns in the form definition tables consistently (yes, I know they are almost always done the same way, but there is potential to vary the column ordering).

A step further would be to support a %FORMREF{}% in the form def as well, that would allow you to embed a separate form, making the forms hierarchical rather than flat as at present. However this would be a lot more work - I only mention it so it can be considered during the design.

-- CrawfordCurrie - 01 Oct 2010

Well, I really appreciate this discussion. I agonised over this missing feature for a long time; the complexity seemed intractable (power + complexity of inheritence vs limitations + elegance of mixins).

In the end I was forced to build up data models using metadata on the form rather than the topic.

Actually, this turns out to simplify a lot of problems that you would otherwise get with form inheritance.

It basically works with a lot of custom template hackery, and something called NestedFormsPlugin.

This defines a couple of new formfield types: hasone and hasmany. These are just placeholders and don't store any important data that can be queried on the topic which uses this field.

In the Values column you specify the DataForm(s) the user may choose at topic-creation/edit time. As a side-effect, users may conveniently choose a 'type' of thing they want to populate this placeholder with.

The fact that the hasone=/=hasmany topics are actually separate to the containing topic, is largely hidden from the user. As a side-effect, we can sort of get repeating form elements too.

There is an ugly network of barely-working plugins I'm half-way through refactoring into something that can be used for production use.

The fact this nearly works without any core changes is good, but clearly now I've got a tree (or graph/network) of topics and our query language doesn't do this very well (closest functionality is in DBRECURSE, part of DBCachePlugin).

I'm using this hierarchy-of-topics (instead of hierarchy-of-forms) to leverage some other patterns, too: putting special meta on the DataForm topics enables us to do tricky templating hacks that allow multiple views of the data as necessary: so far XML and JSON - RDF to come. Actually, for the RDF to work (among other features), my pattern needs topic-per-field for every row of the DataForm, too. So creating a DataForm (especially one that uses 'nested' fields) actually creates many 'schema' topics - a wiki-app helps with setting these up, and defining semantics so that the interesting descriptive/mappable RDF can be emitted.

I think this is a powerful approach for complex data modeling in Foswiki. For what it's worth, after reading the discussions here, I feel that mixins would probably benefit more users than a complex form inheritance system that properly addresses the corner cases, unless we get a new/pluggable schema representation (see BrainstormingDataFormDefinitions)

-- PaulHarvey - 01 Oct 2010

This valuable little subject seems to have gone quiet so I've attached my stab at it. It's a patch to Form.pm @ Revision 11967 of trunk. It introduces two new field types, isa and include, but doesn't make any attempt at mixins.

Multiple include fields are allowed and are directly replaced by the form defined in the value (no check is made of duplicate fields, this is left to the App developer to control).

Only one inheritance ( isa ) field is allowed (if there are multiple ones, the last one is taken) but multiple forms can be declared in a comma separated list. Left-to-right defines the structure of the resulting form and right-to-left defines the priority of the entries (i.e. the left-most form has its entries overwritten or appended to, and so on).

-- DavidPatterson - 12 Aug 2011

David, that sounds interesting - is there docco in the patch, or unit tests?

-- SvenDowideit - 12 Aug 2011

Commenting is disabled while not logged in