Infrastructure Upgrade Web 112
Upgrade planning.
Outstanding Questions
- Timing - When can/should this be done?
- GeorgeClark can do the upgrade Sunday Feb. 20, but the client cache expires was changed from 17 days to 1 day on Friday the 18th, so all the client caches will be refreshed by Monday March 7th. If we delay the upgrade to Sunday the 6th, we'll probably have a lot less issues with stale javascript, css and other static files.
- New
compare
script. Any authentication, apache or mod_perl considerations
- Any recommendations on how to make web unavailable during the upgrade process,
- Help develop a list of things to check after upgrade.
- PiwikPlugin not in svn or documented?
- might be a quick and dirty thingy i did when people wanted piwik...
- I just found an issue with the foswiki-upgrade-check bash script. It was not handling the case correctly when a file was missing from the old distribution, but was installed, for example by installation of a plugin that was added to core. The file was silently ignored. It needs to be replaced by the new version. I've got a local fix that appears to work and will commit after a bit more testing. -- GeorgeClark - 19 Feb 2011
- UsePathForRedirect - enabled pharvey on IRC. Should be fixed in 1.1, caused ?redirect=// instead of ?redirect=/ to test, visit http://foswiki.org/ and login. IIRC the failure mode is that you get redirected to localhost; and/or there are '//' instead of '/' at the root of the URL path. See Tasks.Item2083 for background - PH
- Sessions ExpireAfter -21600
- Minimum password length 1?
-
compare
is not set to be logged - needs to be added to the logging settings
Dry run
- Built /home/gac/foswiki/ directory for upgrade practice
- prod - copy of the live install excluding non-distribution directories
- Foswiki-1.1.2 - untar of full 1.1.2 package
- Foswiki-1.0.9 - untar of full 1.0.9 package
- "live" test
- Copy entire
foswiki.org/public_html
to foswiki.org/test_html
- To refresh:
sudo rsync -av --delete public_html/ test_html/
- Update LSC for new paths.
- Set up test vhost in apache
- Reload server and test
- run
foswiki-upgrade-check
-
./foswiki-upgrade-check -v -r -b backup foswiki.org/test_html Foswiki-1.0.9/ Foswiki-1.1.2
- remove / update plugins
- migrate log files
Before Upgrade
- Change expiration time for
mod_expires
- Build/Test UserRegistration - will become active after 1.1.2 installed.
- Create a new LocalSite.cfg
Upgrade steps
- Verify backups have run successfully
-
cp -a public-html public_html-1.0.9
(Saves current install as a backup - 1-2 hours) Completed to test_html
- Stop web server
/usr/local/etc/rc.d/apache22 stop
-
mv test_html public_html-1.0.9
rename the snapshot
- Disable www crontab -
crontab -e [-u ...],
and comment them out
-
rsync -av --delete public_html/ public_html-1.0.9/
(sync over any last minute changes)
- Run
foswiki-upgrade-check
against prod-html
- See below for keep/use new regarding conflict files. None require merge.
- run uninstall for ZonePlugin
- Install the new LocalSite.cfg - allowed foswiki to migrate
- Migrate the logs Not done yet - running in Compatibility mode
- Start server and verify basic functionality
/usr/local/etc/rc.d/apache22 start
- Upgrade backlevel extensions and re-test.
- Need to change jquery version to 1.4.3 or new Twisty plugin doesn't work.
- ImagePlugin re-installs the ZonePlugin
- Re-enable www crontab
- Change from FoswikiSiteSkin to FatwillyTheme
- uninstall FoswikiSiteSkin
- uninstall other unused extensions? ( KinoSearchContrib )
Logging changes.
Foswiki 1.0.x logs to the
data
directory, 1.1.x logs to working/logs.
Foswiki 1.0.x |
Foswiki 1.1.x |
|
Current Month |
Prior Months |
foswiki-root/data/logYYYYMM |
foswiki-root/working/logs/events.log |
foswiki-root/working/logs/events.YYYYMM |
foswiki-root/data/warnYYYYMM |
foswiki-root/data/error.log |
foswiki-root/working/logs/error.log |
foswiki-root/working/logs/error.YYYYMM |
foswiki-root/data/debug.txt |
foswiki-root/working/logs/debug.log |
foswiki-root/working/logs/debug.YYYYMM |
foswiki-root/data/configurationlog.txt |
foswiki-root/working/logs/configure.log |
|
- Copy / rename logs from
data
to working/logs
directory
-
for i in log*.txt; do f1=${i/log/events.}; echo mv $i ../working/logs/${f1/.txt/}; done
-
for i in warn*.txt; do f1=${i/warn/warn.}; echo mv $i ../working/logs/${f1/.txt/}; done
- rename current month to events.log or warn.log
Other settled questions / issues
- Do we use the old FoswikiSiteSkin or the new PatternSkin FatWilly theme?
- Fatwilly theme as that will be better supported.
- It needs tweaking: Tasks.Item10355 but that can be done anytime.
- Structure of the foswiki install directories:
- install into a html-foswiki112 directory and symlink into the data webs? no
- update the html directory "in place" after saving a copy (This appears to be what was done for the 1.0.7 -> 1.0.9 update
- Install a new version or use the foswiki-upgrade-check script?
- Update the mod_expires times to a very low number prior to starting the upgrade?
- Apache stop/start procedures - need advice on FreeBSD (where is the init script to run?)
- /usr/local/etc/rc.d/apache22 stop / start
- Where / how are the backups done? We do have them, right?
- nightly rsync to a machine with zfs volume, snapshotted daily, retention policy is daily for the past week, weekly for the past month, monthly for the past year
- Mod_Perl considerations?
- Is mod_perl even active? It's enabled in the vhost configuration but is commented out in the main apache configuration. Unsure if it's being enabled somewhere else.
- had problems setting up mod_perl w/ apache22 back in the days when apache22 was cutting edge, foswiki did not like mod_perl w/ apache22 back then, things might have changed
- What is best way to stop / suspend www cron jobs during the upgrade.
- crontab -e [-u ...], and comment them out
- There are two symbolic links in the Sandbox web. ??? (They were links to missing files - removed)
Things to check post-upgrade
- Register a user
- <please add>
Status of current extensions
- AutoViewTemplatePlugin - core - will be updated
- BehaviourContrib - ok
- BlogAddOn - ok
- CommentPlugin - Core extension - core backlevel needs update
- DebugLogPlugin - ok
- EditTablePlugin - Core extension - core backlevel needs update
- ExtensionsRepositoryContrib - needs upgrade
- ExternalLinkPlugin - needs upgrade
- FamFamFamContrib - new Core - core backlevel needs update
- FilterPlugin - needs upgrade
- FindElsewherePlugin - ok
- FoswikiSiteSkin - Remove?
- GoogleAnalyticsPlugin - not enabled. Remove?
- HistoryPlulgin - new Core - core backlevel needs update
- IfDefinedPlugin - ok
- ImagePlugin - needs upgrade
- InterwikiPlugin - Core extension
- JQueryPlugin - ok - core backlevel verify after upgrade
- JSCalendarContrib - core version ok
- KinoSearchContrib - not used - Remove
- MailerContrib - core version ok
- ModPerlEngineContrib - not used - Remove for now?
- MultiTopicSavePlugin - ok
- PatternSkin - Core version ok
- PiwikPlugin - Unknown extension, not in svn or released?
- PreferencesPlugin - ok
- RatingContrib - ok
- RedirectPlugin - needs upgrade
- RenderListPlugin - ok
- SlideShowPlugin - Core, needs upgrade
- SmilesPlugin - ok
- SpreadSheetPlugin - Core, needs upgrade
- SubscribePlugin - ok
- TWikiCompatibilityPlugin - Core, needs upgrade
- TablePlugin - Core, needs upgrade
- TinyMCEPlugin - Core ok
- TipsContrib - Core ok
- TopicUserMappingContrib - Core
- TwistyPlugin - Core - core version backlevel verify after upgrade
- VarCachePlugin - ok
- VotePlugin - needs upgrade
- WysiwygPlugin - Core, needs upgrade
- ZonePlugin - remove
Upgrade script results
Conflicts requiring possible merge
-
data/System/ChangePassword.txt
: Parent change - Use new
-
data/System/UpgradeGuide.txt
: Use new
-
data/System/PatternSkin.txt
: Upgraded to 4.0.7.2 - Use new
-
data/System/SkinBrowser.txt
: Upgraded. Changes in 1.1.2, Use new. Remove TWiki web from search
-
data/System/HierarchicalNavigation.txt
: WebLeftBar escaped to avoid linking. Use new
-
data/System/TablePlugin.txt
: Upgraded to 1.046. Use new
-
data/System/SiteChanges.txt
: Use new. Foswiki specific exists in FoswikiSiteChanges.
-
data/System/WebPreferences.txt
: Keep old
-
data/System/ReleaseNotes01x00.txt
: Use new
-
data/System/FrequentlyAskedQuestions.txt
: Local changes - manually merge or figure alternative Seeing issues on this page
-
data/System/InterWikis.txt
: Use new. Customizations Copied to InterWikis, and settings updated to use Main version.
-
data/System/IfStatements.txt
: Change "String Comparsion" example - but example removed from the 1.1.2 version. Use new.
-
data/System/SiteTools.txt
: Change to Wikipedia link for Crontab. Not in 1.1.2, but 1.1.2 version works fine. Use new.
-
data/System/UserRegistration.txt
: In 1.1.2, includes DefaultUserRegistration or Main/UserRegistration if the topic exists. Migrate to System.UserRegistration CSS changes (color -> class), Twk1Country set to Twk0Country. Added country codes, changes to instructions.
-
data/Sandbox/WebPreferences.txt
: Keep old
-
data/Sandbox/WebNotify.txt
: Keep old
-
data/Sandbox/WebHome.txt
: Keep old
-
data/Main/GroupTemplate.txt
: Why is VIEW_TEMPLATE set to PersonalInfoUserView? Revert. Use new
-
data/Main/WebPreferences.txt
: Keep old
-
data/Main/NobodyGroup.txt
: ? Minor update. Use new
-
data/Main/WikiGroups.txt
: Had update ACL added after vandalism. Use new
-
data/Main/WebNotify.txt
: Keep old
-
data/Main/SitePreferences.txt
: Keep old
-
data/Main/AdminGroup.txt
: New Group API will upgrade. Keep old
-
data/Main/WebHome.txt
: Keep old
-
data/Trash/WebStatistics.txt
: Keep old
-
data/Trash/WebPreferences.txt
: Keep old
-
data/Trash/TrashAttachment.txt
: Keep old
-
lib/Foswiki.pm
: Use new. (change to email regex already in 1.1.2)
-
lib/Foswiki/Plugins/TablePlugin.pm
: f.o was updated to Release 1.046. Use new
-
lib/Foswiki/Plugins/TablePlugin/Core.pm
: f.o was updated to Release 1.046. Use new
-
lib/Foswiki/Contrib/PatternSkin.pm
: f.o was updated to Release 4.0.7.2. Use new
-
pub/System/PatternSkin/print.css
: Change to float: setting incorporated in new version. Use new
Files to be removed by upgrade
ReleaseNotes01x00.html
bin/logos/foswiki-logo-slogan.png
bin/logos/warning.gif
bin/logos/info.gif
bin/logos/favicon.ico
data/System/PatternSkinHeaderArt.txt
data/System/WhatIsWikiWiki.txt
data/System/SimultaneousEdits.txt
data/System/RegistrationApprovals.txt
data/System/EditDoesNotIncreaseTheRevision.txt
data/System/MoveTopic.txt
data/System/SearchDoesNotWork.txt
data/System/DownloadSources.txt
data/System/FaqTemplate.txt
data/System/AnApplicationWithWikiForm.txt
data/System/BehaviourContrib.txt
data/System/GnuGeneralPublicLicense.txt
data/System/CategoryCategory.txt
data/System/DeleteOrRenameATopic.txt
data/System/CascadingStyleSheets.txt
data/System/CommentPluginExamples.txt
data/System/StartingPoints.txt
data/System/DeleteOrRenameAnAttachment.txt
lib/Foswiki/Prefs/PrefsCache.pm
lib/Foswiki/Contrib/BehaviourContrib.pm
lib/Foswiki/Contrib/MailerContrib/CompatibilityHacks.pm
lib/Foswiki/Contrib/MailerContrib/TopicContext.pm
lib/Foswiki/Configure/PLUGINS.pm
lib/Foswiki/Configure/FINDEXTENSIONS.pm
lib/Foswiki/Configure/JS.pm
lib/Foswiki/Configure/CSS.pm
lib/Foswiki/Configure/Checkers/RCS/FgrepCmd.pm
lib/Foswiki/Configure/Checkers/RCS/SearchAlgorithm.pm
lib/Foswiki/Configure/Checkers/RCS/EgrepCmd.pm
lib/Foswiki/Configure/Checkers/Environment.pm
lib/Foswiki/Configure/Checkers/DebugFileName.pm
lib/Foswiki/Configure/Checkers/WarningFileName.pm
lib/Foswiki/Configure/Checkers/ConfigurationLogName.pm
lib/Foswiki/Configure/Checkers/LogFileName.pm
lib/Foswiki/Configure/Checkers/StoreImpl.pm
lib/Foswiki/Configure/LANGUAGES.pm
lib/Foswiki/Configure/UIs/TAGS.pm
lib/Foswiki/Configure/UIs/PasswordProtected.pm
lib/Foswiki/Access.pm
lib/Foswiki/Store/RcsFile.pm
pub/System/JavascriptFiles/foswikiArray.js
pub/System/JavascriptFiles/foswikiStyles.js
pub/System/JavascriptFiles/foswikiEvent.js
pub/System/JavascriptFiles/foswikiHTML.js
pub/System/JavascriptFiles/foswiki_renamebase.js
pub/System/JavascriptFiles/foswikiCSS.js
pub/System/PatternSkinGraphics/bg_button_16x16_generic.psd
pub/System/PatternSkinGraphics/button_arrow.psd
pub/System/PatternSkinGraphics/bg_button_gradient.psd
pub/System/PatternSkinGraphics/top_bar_gradient.psd
pub/System/PatternSkinGraphics/button_font_selector.psd
pub/System/PatternSkinGraphics/bg_submit_gradient.psd
pub/System/PatternSkinGraphics/bg_buttoncancel_gradient.psd
pub/System/DocumentGraphics/proportional.gif
pub/System/DocumentGraphics/monospace.gif
pub/System/DocumentGraphics/empty.gif
pub/System/BehaviourContrib/behaviour_src.js
pub/System/BehaviourContrib/behaviour.compressed.js
pub/System/BehaviourContrib/behaviour.js
pub/System/BehaviourContrib/behaviour.compressed_src.js
pub/System/ProjectLogos/foswiki-logo-slogan.png
pub/System/ProjectLogos/foswiki-logo.xcf
pub/System/ProjectLogos/foswiki-poweredby.gif
pub/System/TwistyPlugin/twisty.jquery.css
pub/System/TwistyPlugin/twisty.dojo.js
pub/System/TwistyPlugin/twisty.jquery.js
pub/System/TinyMCEQuickHelp/Format_Drop_Down.gif
pub/System/TinyMCEPlugin/verbatim.gif
pub/System/TinyMCEPlugin/unlink.gif
pub/System/TinyMCEPlugin/indent.gif
pub/System/TinyMCEPlugin/anchor.gif
pub/System/TinyMCEPlugin/outdent.gif
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/plugins/compat2x/editor_plugin_src.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/plugins/compat2x/editor_plugin.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/plugins/safari/editor_plugin_src.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/plugins/safari/editor_plugin.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/plugins/safari/blank.htm
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/plugins/paste/css/pasteword.css
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/plugins/paste/css/blank.css
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/plugins/paste/blank.htm
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/util/JSONRequest.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/util/URI.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/util/Cookie.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/util/JSONP.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/util/XHR.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/util/JSON.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/util/Dispatcher.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/UndoManager.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/WindowManager.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/ControlManager.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/EditorCommands.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/EditorManager.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/ui/ColorSplitButton.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/ui/Menu.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/ui/MenuItem.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/ui/DropMenu.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/ui/Container.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/ui/NativeListBox.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/ui/SplitButton.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/ui/Control.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/ui/MenuButton.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/ui/Toolbar.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/ui/ListBox.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/ui/Separator.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/ui/Button.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/ForceBlocks.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/xml/Parser.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/Editor.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/AddOnManager.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/commands/CutCopyPaste.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/commands/RemoveFormat.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/commands/BlockQuote.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/commands/UndoRedo.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/commands/InsertHorizontalRule.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/Developer.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/dom/StringWriter.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/dom/EventUtils.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/dom/Event.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/dom/Element.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/dom/XMLWriter.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/dom/Selection.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/dom/ScriptLoader.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/dom/TridentSelection.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/dom/Range.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/dom/Sizzle.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/dom/Serializer.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/dom/DOMUtils.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/tinymce.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/firebug/firebug-lite.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/CommandManager.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/Popup.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/adapter/jquery/jquery.tinymce.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/adapter/jquery/adapter.js
pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/classes/adapter/prototype/adapter.js
pub/System/TinyMCEPlugin/tinymce/build.xml
pub/System/TinyMCEPlugin/tinymce/build.bat
pub/System/TinyMCEPlugin/link.gif
pub/System/PatternSkin/bg_buttoncancel_gradient.gif
pub/System/PatternSkin/button_arrow.gif
pub/System/PatternSkin/bullet-lock.gif
pub/System/PatternSkin/background_alert.gif
pub/System/PatternSkin/column_right.css
pub/System/PatternSkin/bullet-toc.gif
pub/System/PatternSkin/top_bar_gradient.png
pub/System/PatternSkin/theme_twiki.css
pub/System/PatternSkin/TWiki_header.gif
pub/System/PatternSkin/column_twiki.css
pub/System/PatternSkin/style.css
pub/System/PatternSkin/form_gradient.gif
pub/System/PatternSkin/column_none.css
pub/System/PatternSkin/bg_submit_gradient.gif
pub/System/PatternSkin/theme_foswiki.css
pub/System/PatternSkin/bg_button_gradient.gif
pub/System/PatternSkin/column_left.css
pub/System/PatternSkin/button_font_selector.gif
pub/System/PatternSkin/patternskin_screenshot.jpg
pub/System/PatternSkin/colors.css
pub/System/PatternSkin/theme_foswiki_noframe.css
pub/System/PatternSkin/layout.css
pub/System/PatternSkin/bullet-personal_sidebar.gif
templates/oopsmore.tmpl
templates/twistyplugin.dojo.tmpl
templates/twistyplugin.jquery.tmpl
templates/twistyplugin.pattern.tmpl
tools/rewriteshbang.pl
Manually applied hotfix for =~ which modified these files:
-
lib/Foswiki/Query/Node.pm
-
lib/Foswiki/Query/HoistREs.pm
-
lib/Foswiki/Store/QueryAlgorithms/BruteForce.pm
--
PaulHarvey - 09 Mar 2011