CPAN::Search::Lite::Index - set up or update database tables. |
CPAN::Search::Lite::Index - set up or update database tables.
my $index = CPAN::Search::Lite::Index->new(config => 'cpan.conf', setup => 1); $index->index();
This is the main module used to set up or update the database tables used to store information from the CPAN and ppm indices. The creation of the object
my $index = CPAN::Search::Lite::Index->new(%args);
accepts three arguments:
CSL_CONFIG_FILE
pointing
to the configuration file may be specified.
Most of the options used to control the behaviour of the indexing are contained in a configuration file. An example of the format of such a file is
[CPAN]
CPAN = /var/ftp/pub/CPAN pod_root = /usr/local/POD html_root = /usr/local/httpd/htdocs/CPAN
[DB]
db = pause user = sarah passwd = lianne
[WWW]
css = cpan.css up_img = up.gif tt2 = /usr/local/tt2 geoip = /usr/local/share/geoip/cpan.txt
This consists of 3 sections.
This is associated with various things related to CPAN.
no_mirror
option is specified.
dist_name
under this directory
will be created corresponding to the name of the distribution.
Perl::Tidy
. For a module
such as Foo::Bar
, the documentation will be saved as a
file Foo/Bar.html, while the sources will be saved
as Foo/Bar.pm.html.
dist_name
under this directory
will be created corresponding to the name of the distribution.
ignore = <<EOL Module-CPANTS-asHash CORBA-IDL EOL
This array of values (which may include regular expressions) is joined together as
$pat = join '|', @ignore_dists
and if the distribution name matches
$dist_name =~ /^($pat)$/
the distribution is ignored.
$repositories
of the CPAN::Search::Lite::Util manpage.
no_mirror
is specified, the value of remote_mirror
will
be used to fetch the CPAN indices. If not given, http://www.cpan.org
will be used.
This is used to store connection information to the database used to populate the tables.
This is used for various information related to a web interface.
html_root
of the CPAN
section.
html_root
of the CPAN
section.
Geo::IP
or Apache::GeoIP
is used to
provide a redirection service to a nearby CPAN mirror
based on the location of origin, this file will be
created to provide the necessary country of origins of the
CPAN mirrors.
Calling
$index->index();
will start the indexing procedure. Various messages
detailing the progress will written to STDOUT,
which by default will be captured into a file
cpan_search_log.dddddddddd, where the extension
is the time
that the method was invoked. Passing
index
an argument of log => log_file
will
save these messages into log_file. Error messages
are not captured, and will appear in STDERR.
The steps of the indexing procedure are as follows.
no_mirror
option is specified, the
necessary CPAN index files $CPAN/MIRRORED.BY,
$CPAN/indices/ls-lR.gz, $CPAN/authors/01mailrc.txt.gz,
$CPAN/modules/02packages.details.txt.gz, and
$CPAN/modules/03modlist.data.gz will be fetched
from the CPAN mirror specified by the $cpan
variable
at the beginning of the CPAN::Search::Lite::Index manpage. If you are
using this option, it is recommended to use the
same CPAN mirror with subsequent updates, to ensure consistency
of the database. As well, the information on the locations
of the CPAN mirrors used for Template-Toolkit and GeoIP
is written.
setup
argument within the new
method of the CPAN::Search::Lite::Index manpage is specified,
this will get information on the state of the database
through the CPAN::Search::Lite::State manpage.
A comparision is then made between this information
and that gathered from the CPAN indices, and if there's
a discrepency in some items, those items are marked
for either insertion, updating, or deletion, as appropriate.
no_mirror
option is specified, this
will extract, through the CPAN::Search::Lite::Extract manpage,
the available pod sections of files of
a distribution, placing them under a subdirectory
dist_name
(corrsponding to the name of the distribution)
under the specified pod_root
in the configuration file.
pod2html
is then run on them, with the results placed
under dist_name
of html_root
. Also, a README, Changes,
INSTALL, and META.yml file, if present, will be copied over
into dist_name
under pod_root
. Finally, information on
the prerequisites of the distribution, and distribution and
module descriptions, if available and needed, is extracted.
As well, unless the no_mirror
option is specified, the
html files created under html_root
will be edited to
adjust the links to module files. This is necessary because
when the html files are created no cache is used (in order to
maintain consistency between updates), and consequently links
to packages outside of a given package may be incorrect.
This is fixed by querying the database to see what module
documentation is actually present, and then adjusting the links in
the html files accordingly (or removing a link if the
indicated documentation is missing).
the CPAN::Search::Lite::Info manpage, the CPAN::Search::Lite::PPM manpage, the CPAN::Search::Lite::State manpage, the CPAN::Search::Lite::Extract manpage, the CPAN::Search::Lite::Populate manpage, and the CPAN::Search::Lite::Util manpage. Development takes place on the CPAN-Search-Lite project at http://sourceforge.net/projects/cpan-search/.
This software is copyright 2004 by Randy Kobes <randy@theoryx5.uwinnipeg.ca>. Use and redistribution are under the same terms as Perl itself.
CPAN::Search::Lite::Index - set up or update database tables. |