This manual process describes the autoBuildFoswiki.pl script

Reminder to Kenneth and Sven: Need to change the sequence below so that the Patch branch is made at feature freeze and releases always created from the Patch branch. See SubversionBranchingAndTagging. (Sven will modify the script as we nail down the details of the process to be able to release from the appropriate places.)

WARNING: since forking a lot of this infrastructure isn't in place yet, or names have changed to remove the old brand. THIS PROCESS REQUIRES PICKING OVER WITH A FINE-TOOTHED COMB.

How to Build a Release

It is an important principle that anyone can build a release using the automatic tools, given nothing more than some standard F/OSS tools and an SVN checkout area. The following sections describe the stages that need to be gone through to correctly build a Beta or Final release.

The contents of a release are specified by the contents of the master lib/MANIFEST file, that defines the release as core plus default extensions. Core files are listed at lib/Foswiki/Contrib/core/MANIFEST. !include statements are used to make it possible to have a release composed by the lists in many MANIFEST files.

The instructions are for building a standard release using the tools/build.pl build script, on Linux. It should be possible to generate a release on Windows, but it's untested.

Current nightly build process (using Release01x01) - Runs nightly at 4am Sydney time. (GMT+10)

Note: when there is an official beta release, it will be uploaded

This will create tgz and zip files named using the version number in Foswiki.pm, and appending the SVN repository number to get files named like Foswiki-1.0.0-beta1-auto14551.zip.

Note: The script will abort if there are unit test failures. (sending an email to the developer events mailing list)

The builds will be uploaded to the Release01x01 nightly build area

Current nightly build process (from trunk) - Runs nightly at 1am Sydney time. (GMT+10)

curl http://svn.foswiki.org/trunk/core/tools/autoBuildFoswiki.pl > autoBuildFoswiki.pl
perl autoBuildFoswiki.pl

This will create tgz and zip files named using the version number in Foswiki.pm, and appending the svn repository number to get files named like Foswiki-5.0.0-auto14551.zip.

The builds will be uploaded to trunk Nightly build area.

Note: The script will abort if there are unit test failures. (sending an email to the developer events mailing list) and no build will be generated.

---

Build procedure

Note: at this time, this document and the build script is heavily biased towards creating official releases

Before generating a release package:

1. Get an SVN checkout (see SvnRepository)
   • Checkout core to a clean build area
     • svn co http://svn.foswiki.org/branches/Release01x01
   • once you have a clean checkout area you can re-use it for subsequent builds. just
     • svn up
     • svn diff should show no results
       • git: git status
     • svn status | grep '?' | sed 's/?/rm -rf/' | sh
       • git: git clean -f -d (removes untracked files and directories. Add -x option to remove "ignored" files)
     • # or similar, gets rid of local files
2. Install default plugins (symbolic-link)
   • perl -wT pseudo-install.pl -A developer to install the plugins specified in MANIFEST and the developer tools
3. Run the /bin/configure script from the browser to make your system basically functional
   • CPAN prerequisites: CPAN:CGI::Session
   • ensure that your apache has sufficient file and directory permissions for data and pub
4. Run the unit tests. DO NOT RUN TESTS AS THE SUPER-USER. Some tests will be designed to test boundary conditions where the current user is denied write access to files. Since the superuser cannot be denied write access, you may get false failures.

   export FOSWIKI_HOME=`pwd`  # the core directory
   export FOSWIKI_LIBS=$FOSWIKI_HOME/lib:$FOSWIKI_HOME/lib/CPAN/lib:your_own_perl_lib_directories_here
   cd $FOSWIKI_HOME/test/unit
   perl ../bin/TestRunner.pl FoswikiSuite.pm

5. Run the TestCaseAmISane sanity check manual testcase
6. Install the TestFixturePlugin: perl -wT pseudo-install.pl TestFixturePlugin
   • Do not enable TestFixturePlugin in configure
   • CPAN prerequisites: CPAN:HTML::Tagset CPAN:HTML::Parser
7. Run all the automatic testcases in the TestCases web (use firefox, and open each 'run' link in a new tab)
   • LOG IN FIRST (admin with configure pwd?)
   • Make sure there is at least one user registered on the site.
   • Enable $Foswiki::cfg{INCLUDE}{AllowURLs} = 1; in the Security settings | Miscellaneouss area of configure in order to run Develop:TestCases.TestCaseAutoIncludeAttachment
   • You should see ALL TESTS PASSED on all test pages. If you see the test itself, something is not configured correctly (check CPAN dependencies).
8. check for memory leaks by running perl -Ibin tools/MemoryCycleTests.pl
   • CPAN prerequisites: CPAN:Devel::Monitor
9. Check the manifest: cd lib/Foswiki/Contrib/core; perl ../../../../tools/check_manifest.pl
   • Note: Does not currently work for a git checkout, only svn.
   • Some files (top level .html, *DotPm.txt in the system web, and AUTHORS are automatically generated by the build process in the staging area, and do not appear in the MANIFEST.
   • Expect to see the following warnings; they are OK:

     The following files were found in subversion, but are not in MANIFEST
     bin/update-develop-links
     lib/Foswiki/Contrib/core/DEPENDENCIES
     lib/Foswiki/Contrib/core/MANIFEST
     lib/Foswiki/Contrib/core/build.pl
     lib/LocalSite.cfg  # created when running /bin/configure from the browser
     

10. Check that all doc topic have correct TOPICINFOs
    • cd data; grep '%META:TOPICINFO{' */*.txt | grep -v TestCases | grep -v 'author="ProjectContributor".*version="1"'
      • if some topics are not 'valid' edit the TOPICINFO by hand
      • Note: The only topic with version="2" will be the CompareRevisionsAddOnDemo topic in Sandbox
11. Fix up release notes with new changelogs - see
    • http://foswiki.org/Tasks/ReleaseNotesTml?type=patch
    • Note that the release note is edited by editing the topic data/System/ReleaseNotes01x00. The build script creates a file in the root of the zip called ReleaseNotes??x??.html, and the build script needs your installation to be running to look up the release note topic and show it with the simple text skin.
12. Add the new release to the System.ReleaseHistory topic.

Build a test release package

Do this from a freshly cleaned checkout - see step one above

1. Install default plugins (hard copy)
   • perl -wT pseudo-install.pl -copy default to install the plugins specified in MANIFEST
2. use the configure script to make your system basically functional
   • ensure that your web server has sufficient file and directory permissions for data and pub
3. from command line (using /var/www/Release01x01 as example)
   • export FOSWIKI_LIBS=/var/www/Release01x01/core/lib:/var/www/Release01x01/core/lib/CPAN/lib
   • export FOSWIKI_HOME=/var/www/Release01x01/core
   • cd /var/www/Release01x01/core/lib
   • perl ../tools/build.pl release
   • When asked if you want to name version, answer No
4. The release tgz and zip is now in /var/www/Release01x01/core/ with the name Foswiki.tgz and Foswiki.zip

After the package is built, but before it is uploaded

1. unarchive the .tgz in a clean area
2. make sure that the generated files are all correct
3. configure the web server and run configure, make sure it works
   • you may have to change the permissions or ownerships of the files to match your web server setup ( chown -R www-data.www-data * works on my debian system)
4. Verify that all expected plugins are installed and functioning

Build a final release package and upload to Download web

1. Clean the build area again (see above)
2. Install default plugins
   • perl -wT pseudo-install.pl -link default to install the plugins specified in MANIFEST
3. use the configure script to make your system basically functional
4. from command line (using /var/www/Release01x01 as example)
   • export FOSWIKI_LIBS=/var/www/Release01x01/core/lib:/var/www/Release01x01/core/lib/CPAN/lib
   • export FOSWIKI_HOME=/var/www/Release01x01/core
   • cd /var/www/Release01x01/core/lib
   • perl ../tools/build.pl release
   • When asked if you want to name version, answer yes and give the version in the format X.X.X
5. The release tgz and zip is now in /var/www/Release01x01/core/ with the name Foswiki-X.X.X.tgz and Foswiki-X.X.X.zip

Beta and release candidates are uploaded

• ..to foswiki.org to Download/FoswikiReleaseXXxXXxXXBetaX or Download/FoswikiReleaseXXxXXxXXRCX

Make upgrade packages

• Upgrade package is a full release package with a few files removed.
• We do not make upgrade packages for new major and minor releases. And not for beta or RC releases either.
• Unpack the release tgz to a fresh directory. Rename the directory to Foswiki-upgrade-X.X.X
• The following files are not distributed in the upgrade package

Filename	Manually excluded from Upgrade Package	Skipped by foswiki-upgrade-check
data/_default/WebHome.txt	X	X
data/_default/WebLeftBar.txt	X
data/_default/WebPreferences.txt	X	X
data/Main/AdminGroup.txt	X	X
data/Main/AdminUser.txt	X
data/Main/NobodyGroup.txt	X
data/Main/PatternSkinUserViewTemplate.txt	X
data/Main/ProjectContributor.txt	X
data/Main/RegistrationAgent.txt	X
data/Main/SitePreferences.txt	X	X
data/Main/UnknownUser.txt	X
data/Main/UserHomepageHeader.txt	X
data/Main/UserListByDateJoined.txt	X
data/Main/UserListByLocation.txt	X
data/Main/UserListHeader.txt	X
data/Main/UserList.txt	X
data/Main/WebAtom.txt	X
data/Main/WebChanges.txt	X
data/Main/WebCreateNewTopic.txt	X
data/Main/WebHome.txt	X	X
data/Main/WebIndex.txt	X
data/Main/WebLeftBarExample.txt	X
data/Main/WebNotify.txt	X	X
data/Main/WebPreferences.txt	X	X
data/Main/WebRss.txt	X
data/Main/WebSearchAdvanced.txt	X
data/Main/WebSearch.txt	X
data/Main/WebStatistics.txt	X	X
data/Main/WebTopicList.txt	X
data/Main/WikiGuest.txt	X
data/Sandbox/CommentPluginExampleComments.txt	X
data/Sandbox/CommentPluginExamples.txt	X
data/Sandbox/CommentPluginTemplateExample.txt	X
data/Sandbox/CompareRevisionsAddOnDemoTopic.txt	X
data/Sandbox/CompareRevisionsAddOnDemoTopic.txt,v	X
data/Sandbox/PluginTestEmptyPlugin.txt	X
data/Sandbox/WebAtom.txt	X
data/Sandbox/WebChanges.txt	X
data/Sandbox/WebCreateNewTopic.txt	X
data/Sandbox/WebHome.txt	X	X
data/Sandbox/WebIndex.txt	X
data/Sandbox/WebLeftBarExample.txt	X
data/Sandbox/WebNotify.txt	X	X
data/Sandbox/WebPreferences.txt	X	X
data/Sandbox/WebRss.txt	X
data/Sandbox/WebSearchAdvanced.txt	X
data/Sandbox/WebSearch.txt	X
data/Sandbox/WebStatistics.txt	X	X
data/Sandbox/WebTopicList.txt	X
data/System/InterWikis.txt	X
data/System/NewUserTemplate.txt	X
data/System/WebPreferences.txt	X	X
data/System/WebTopicEditTemplate.txt	X
data/Trash/TrashAttachment.txt	X	X
data/Trash/WebPreferences.txt	X	X
data/TWiki/WebPreferences.txt	X	X
index.html	X

• Manually remove the files listed above.- See attached file: clean-upgrade.sh: Shell script removes files not distributed in upgrade package
• Create the upgrade zip zip -r Foswiki-upgrade-1.1.1.zip Foswiki-upgrade-1.1.1
• Create the upgrade tgz tar -zcf Foswiki-upgrade-1.1.1.tgz Foswiki-upgrade-1.1.1
• Generate md5 sums of the two and add them to the already generated md5 file.

GPG sign the files

• Release manager signs the 4 files (2 for new minor/major) with his private GPG signature
• Example: gpg -a -u email-address -b Foswiki-1.1.0.zip
• Result is 4 (2) .asc files.

Final releases are uploaded to Sourceforge

• http://sourceforge/projects/foswiki
• Project Admin / File Manager
• Create new folder in Foswiki folder. Use name X.X.X
• Upload both tgz and zip and md5 and the .asc files.
• Make the zip defailt for Windows and Others
• Make the tgz default for Unix/Linix/Solaris/BSD etc
• OBS! Do not mark any other uploads like VMs or debs or windows installers as default for anything. Only ONE file can be the default for a given OS and it should always be .zip or .tgz. It is this setting that defines which download a visitor sees on the Foswiki Sourceforge landing page.

Create a release tag, and if it's a new major release, a release branch

1. create an svn tag from the released version (you NEED to know the revision number that was commited when creating the final release package)
   svn copy -r *REV* http://svn.foswiki.org/branches/Release01x01/ http://svn.foswiki.org/tags/FoswikiRelease*RELEASENAME*

Note that unlike the old project we do not build on the old ,v files. Each release is shipped with a clean System web where all topics are rev 1. The old system made no sense in practical and the ,v files grew and grew and just contained garbage. The release tag is now only used as a reference point in case we want to go back.

New release maintenance branch

1. create an svn branch from the released version (you NEED to know the revision number that was commited when creating the final release package)
   svn copy -r *REV* http://svn.foswiki.org/trunk/core http://svn.foswiki.org/branches/Release*RELEASENAME_WITHOUTPATCHNUMBER*/core
2. repeat with all the default plugins and the build, and test extensions. Do not copy all of trunk or you will have a major clean up deleting all the none default plugins.

Build and upload all default extensions that have changed version since last Foswiki release

• Make a normal release with perl build.pl upload from the lib/Foswiki/Plugins|Contrib/NameOfExtention directory.
• If you are in doubt anything has changed, re-upload. It cannot harm uploading the same again.
• You can check in a Foswiki installation in configure if the version installed maches the version on foswiki.org.

Announce the release

• If System.ReleaseHistory is out of date - need to update and rebuild!
• foswiki-announce mailing list
• Update the Download.DownloadFoswiki include macro
• Update Download.GetFoswikiTrigger
• Update About.ProjectUpdate
• Update Extensions.ExtensionNews
• Update Tasks.ReleasedIn and Tasks.Codebase to adjust older, current and future releases.
• Update Tasks.ReleaseBlocker
• Update Support.QuestionForm
• update irc /topic
• http://freecode.com/
• blog about it ?? (how/where)
• Social Networks
  • LinkedIn?
  • twitter
  • facebook
  • wikimatrix
• Should we upload to github too?
• Press Releases
  • http://www.newswiretoday.com/