Feature Proposal: It would be cool to ease Perl module installation or at least to simplify figuring out what is missing
A first time installer may for each extension carefully check related dependencies of Perl modules. But if one already knows what to install and quickly does it for quite some plugins we are loosing track of what is in the dependencies or have to re-visit all the pages and then check the Perl installation manually if all modules are available.
Description and Documentation
Configure already provides an overview of required Perl modules but AFAIK this is only for the well maintained parts (default installation and extensions). Also from there installation cannot be done automatically (at least currently no button to say please download from CPAN and install for me).
I wrote a script which could be placed in the tools directory and tries to:
- identify Foswiki related Perl modules and lists them
- offers to install anything missing via CPANPLUS
The script is already available and seems to do its job for me. So I am here merely asking if it may be added to the tools directory, so others could use it too.
There are probably good arguments to better do parts of this functionality via configure but that's at least currently something I cannot commit myself to. This script therefore may also be considered as a prototype of features potentially a nice to have in configure.
-- Contributors: IngoKappler
- 17 Nov 2010
Code is already available and could be put into trunk quickly.
Interesting to look more carefully at.
And what I say now is not in any way to be taken as a negative input.
When you run with a certain Linux distribution many of the CPAN libraries are available was packages. As debs for Debian/Ubuntu family, or RPMs for the Redhat/Mandriva/Suse family. And when you install your packages you automatically have your system in an automatic security update cycle. And you also have some automation around making sure that the versions are compatible with other packages.
If you install CPAN libraries via CPAN you have to keep yourself up to date manually.
So many of us try to install CPAN libraries using standard packages for our distribution as the first choice. And when it is not available, we go for CPAN.
Whatever we introduce has to take this into consideration. At least for the major distrinbutions.
I have installed 95% of my CPAN libraries via RPM repositities and not via CPAN.
- 11 Nov 2010
I fully agree and handled it on my production system in he same way. But sometimes I found it pretty tiring to figure out what distribution package actually contained the module. It is much easier and quicker to find it via CPAN but has the downside of being solely responsible.
CPAN so far was just easier to implement and I like to keep it as an option.
But I've been wondering if there could be a way to find out automatically e.g. what Debian package contains a certain module by maybe using the definitions Sven or the Debian maintainers of the Foswiki package define in their text files. I didn't have a chance to dive into that yet, so guiding feedback is welcome here. Maybe it can be parsed out from these files...
- 12 Nov 2010
I am currently working on handling Debian packages as well. I am not sure I will remain the right one to handle e.g. RPM too.
- 02 Dec 2010
I'm all in favor of having a script to check the dependencies, and maybe making this script RPM / apt-get / ... aware. RPM should be easy, as it provides
, apt-get might be a bit more cumbersome, but it should be doable, worst case, with a hashing table.
I'm, on the other side, totally against putting any of this into configure. We (Ok, I did it) removed this feature from configure because we agreed that the webserver user shouldn't be playing around with CPAN modules, and on many cases this won't work. So, configure could say something like:
- You are missing these perl modules: Algorithm::Diff, Encode, LWP. Please logon to the machine with a user allowed to sudo, or root, and run:
- This will download the missing dependencies from CPAN, and install them (using sudo if possible).
Last point, there are many CPAN-like thingies: CPAN, CPANPLUS, CPANMINUS, each having its advantages and drawbacks. Not sure it's worth making the code CPAN-backend-pluggable, but it could be.
Or better, maybe we could integrate CPAN:App::cpanminus
to our release, or we could even use cpanminus to install itself, as explained in the documentation:
curl -L http://cpanmin.us | perl - --sudo App::cpanminus
- 02 Dec 2010
I decided against CPAN and use already CPANPLUS while not knowing CPANMINUS. CPANPLUS is part of recent Perl installations (CPANPLUS: 5.009005) and in my environment I didn't tweak or configure it at all to see how it goes with it.
packages already works using
while only testing it on Debian so far. I am not clear about Ubuntu. On Debian aptitude does everything required. I am using a hash table and don't see another way so far. There is no place in Foswiki (yet) where we could retrieve the mapping from. Well, having it in some topic and maintaining it there could be an option but for now I am happy with the hash.
Actually, nice to hear that there is already some experience regarding configure. So looks that's not an option and I don't have an issue with it.
- 02 Dec 2010
Would the name
be good enough?
Current version works for me but there might be lots of other features possible including e.g.
, certain command line switches and RPM support. What I should do is some more documentation.
is also available on Ubuntu, so the script should run there too (aptitude on Ubuntu
- 03 Dec 2010
Ubuntu removed aptitude from the standard install in 10.10. Thankfully, one can still install it via apt-get. So please do list aptitude explicitly as a dependency.
- 03 Dec 2010
It seems to me that this type of script is only useful for people having shell access and root privilege. That is a nice tool anyway, and if it is added to Foswiki, the installation documentation for this type of installation (having shell access and root privilege) should be updated to mention it.
Now, I believe that such a script could be highly useful for people who do not have shell access and root privilege, or who want an extremely simplifed installation process (they don't know Perl, don't use Perl, and don't care to have a system-wide installation of the modules even though they may have root access). For those people, it would be nice if the script could also be run by the Webserver, and could suggest installing the missing dependencies in a local repository used by Foswiki only.
The script could be run when configure is first started and warn the users that, if they have shell and root access, they have the option to run the script from shell if they want a centralised installation of the missing modules usable by the whole system, and then restart configure when they are done.
That would be a huge step towards an installation of Foswiki as simple as Dotclear (a Blog engine http://dotclear.org/
- where you just download an installer and copy it in a Web server, which asks you a few simple questions, and then what is needed is downloaded and installed for you).
For the name, maybe
is a better choice, since
already has an
- 03 Dec 2010
Thanks for the feedback.
For some of the stuff e.g. to maybe call it from configure probably someone else may need to do it. If the script is run by a non root user it can install modules locally but I didn't fully test that yet.
However, I will try to take some care of the Ubuntu situation (installing aptitude via apt-get).
sounds good to me, so this is what it will become unless others raise objections.
Updating related docu will also be done by me, once I made it available and probably got some feedback which hopefully won't kill me.
BTW, it does currently:
- I also believe it should work on Windows by using CPANPLUS but didn't test that yet, so don't know what's lurking around this corner.
- 03 Dec 2010
Note that the current Extension installer already deals with CPAN dependencies when run from the Shell environment. It does not do it from the web invocation.
It would be nice if whatever you do could be plugged into the shell version of the extension installer, so at least from the command line, the installation of an extension can be a complete package. Enhancing this to support deb, rpm, and other packages would be nice. The CPAN code is currently in
, in subroutine
Also note that there is another proposal, UseEnhanceAndMoveSpecFiles
If the extension specifications were better organized, it would be easier for configure to report on all required dependencies.
Oh.. and after a quick look at your attached run log, it would be nice if you identified which plugin was triggering the dependency in the list of extension dependencies. This way it might be possible to decide not to install a dependency for example, or attribute some failure to a missing dependency.
I keep thinking of more.. You should probably leverage Configure::Package::checkDependencies http://trunk.foswiki.org/System/PerlDoc?module=Foswiki::Configure::Package#ObjectMethod_checkDependencies_40_41
instead of parsing the packages and DEPENDENCIES files yourself. There is quite a bit of logic in comparing revisions and dealing with conditional dependencies (triggers and ONLYIF) embedded in there, and the module Configure::Dependency http://trunk.foswiki.org/System/PerlDoc?module=Foswiki::Configure::Dependency
If you can stick with these routines, the restructuring of the Spec files in Babar's proposal will not break your work.
- 03 Dec 2010
Gosh! When I started I thought I stay relatively small. Now I have several requests coming down, so I need a table and set expectations. Generally all this sounds nice and thanks again for the input. I already have some working code and I am interested in getting it out. I am not stopping anyone from taking it further as wished or to abondon it in favor of something else.
Status and Feature Overview (being updated)
No, doesn't mean it couldn't be done later. Following is what I see on my personal horizon considering the time I can spend for it.
- 07 Dec 2010
Ingo, thanks for clarifying this. About 10., my understanding is that the local directory should be
[foswiki base dir]/lib/CPAN/lib
as indicated in http://foswiki.org/Support/HowToInstallCpanModules#Using_your_private_CPAN_reposito
otherwise you have to modify
setting the value of
) if it is set elsewhere.
I have limited knowledge in this respect, so maybe you would like to cross-check all this with CrawfordCurrie
- 08 Dec 2010
No problem, I am aware of these details and plan to cover them by related messages. Maybe it is good to provide a default suggestion pointing at /lib/CPAN/lib/ which can be overwritten by user input and then will remind to also address it in
- 08 Dec 2010
I spent some time looking into local installation via
and finally concluded the script itself shouldn't start changing it on its own. What I did, is to warn a user and provide hints about related
configurations, if the script detects that a user wants to install locally (not being root or using sudo). I expect that a user capable of doing local installations, should better know what he is doing, hence tweaking
configuration manually is good to understand how it works and what was changed.
- 09 Dec 2010
Changed to AcceptedProposal as the 14 day rule applies by now.
- 16 Dec 2010