1 DEBIAN GFORGE MAINTAINER HOWTO
2 ------------------------------
4 Here is a short HOWTO explaining a few of the tricks that are used by
5 the Debian Gforge packages.
9 If you intend to NMU this package, *please* read it thoroughly. While
10 you can make it up as you do it as far as the helpers are organised,
11 the database handling is, to say the least, very fragile. And errors
12 are fatal. So go read the section entitled "THE DATABASE". Do it
13 now. Don't skip it. Really.
15 HELPERS (DEBHELPER & DEBCONF)
16 -----------------------------
17 The package uses Debhelper and Debconf. While not all features of are
18 used (no Emacsen modules, no shared libraries, no rollback features in
19 Debconf, etc.), some of them are. Where they are, it is in a fairly
20 straightforward way. No black magic or advanced features are used.
24 Historically, the package was monolithic: there was only one (rather
25 large) package, called "sourceforge". This package went and changed
26 things all over the system, configuring a database, a web server, the
27 mail transfer agent, etc. In order to keep a semblance of order in
28 all the actions involved, we separated them by "subsystem". There was
29 the database subsystem, the web server subsystem, the LDAP subsystem,
30 etc. These subsystems were each represented by a script handling most
31 of it. These scripts are called deb-specific/install-*.sh. The main
32 maintainer scripts (postinst, prerm, cron jobs and suchlike) called
33 these scripts in turn. The install-*.sh scripts handle different
34 parameters: "configure", "purge" and "update" have rather explicit
35 names (I hope); "configure-files" and "purge-files" are special
36 targets. They are used as a way for the postinst to delegate the task
37 of computing a proposed change in a configuration file to the
38 subsystem. The postinst then uses Debconf to ask whether the proposed
39 file should be used, takes appropriate action, then lets the subsystem
40 finish its configuration. Similar things happen for prerm scripts.
42 The package is now split into several pkgname-* packages. Some
43 of them still contain some install-*.sh scripts (usually at most one).
44 Each pkgname-* package installs its corresponding subsystem, or a
45 semblance thereof. For instance, it is planned that the database can
46 be hosted on a different server than the website. But the website
47 still needs to know where the database server is. In this case, the
48 database host will have to install pkgname-db-local (or whatever
49 the package is named), and the web server will have to install
50 pkgname-db-remote. The -db-local package will still install the
51 database (thus invoking install-db.sh), but -db-remote will only
52 install what is needed by the database *clients* (basically, the host
53 where the database is installed and the appropriate password).
55 There are therefore some areas where packages overlap: the database
56 password is a variable "provided" by both the -db-local and -db-remote
57 packages. Debconf is intelligent enough not to ask the corresponding
58 question twice, but it is interesting nevertheless to keep the
59 appropriate variables where they belong, and only there. Hence the
60 use of a Debhelper-like trick, as described below.
64 This is the most tricky part of the source package. The maintainer
65 scripts and Debconf templates are not used "as is", but they are
66 instead generated from templates. In much the same way as Debhelper
67 replaces #DEBHELPER# lines in maintainer scripts by appropriate chunks
68 of code to add the needed functionality to packages, this package
69 builds the maintainer scripts (and Debconf templates) from templates
70 by inserting bits of text in them. These bits can be either simple
71 text (like for Debconf templates), or bits of code (like the
72 appropriate code to handle one particular Debconf variable in a
73 .config file, or a function to repeatedly ask for a password until two
74 consecutive answers match).
76 I call this trick DSF-Helper (for "Debian Sourceforge helper").
77 It's largely inspired from Debhelper (particularly dh_installdeb) in
78 both its concepts and implementation, and it might result in a patch
79 submitted against Debhelper proper when I'm confident it works and is
80 useful. It is currently implemented in Perl.
82 The "bits of stuff" are grouped by identifiers. For each
83 identifier, you can have one chunk of text for each family of
84 generated files (currently the families are .templates, .config,
85 .preinst, .postinst, .prerm and .postrm). For instance, a Debconf
86 variable shared between several packages will have one chunk for the
87 .templates file (containing the Debconf template), one for the .config
88 file (containing the appropriate Debconf call), and one for the
89 .postinst file (containing code to turn this Debconf variable into a
90 line in a configuration file).
92 Each subpackage can then use some of these "bits of stuff" in its
93 files. To do so, the files must be named *.dsfh-in and include lines
94 like #DSFHELPER:identifier#. These files will be processed by
95 dsf-helper.pl and turned into the appropriate files, with the keywords
96 replaced by the appropriate text.
98 Now for a few examples.
100 - get-debconf-password: this is a simple shell function looping until
101 the user types the same passwrd twice. This function is mostly
102 useful in .config files, hence the "group" consists of the sole
103 debian/dsf-helper/get-debconf-password.config file. To use it in a
104 <blah>.config file, just rename that <blah>.config file as
105 <blah>.config.dsfh-in and include #DSFHELPER:get-debconf-password#
108 - ldap-variables: this one involves both Debconf templates and .config
109 code. Just include #DSFHELPER:ldap-variables# in both the
110 <blah>.templates.dsfh-in and <blah>.config.dsfh-in, and DSF-Helper
111 will insert the appropriate chunk of text in the appropriate file.
113 The rationale behind DSF-Helper is that the code handling, say, one
114 particular Debconf variable is likely to change from time to time, and
115 to be added to one subpackage and removed from another. It can become
116 a big hassle just to maintain the code in different files and keep it
117 consistent, and creating a new subpackage is also a tedious task.
118 DSF-Helper makes these tasks a bit more automated. Each bit of code
119 is only maintained in one file, and it's propagated into every package
120 at package building time. Creating a new package can be "just" a
121 matter of picking the appropriate bits of code, and DSF-Helper will
122 put them where needed.
124 To draw a comparison with compiled C code, DSF-Helper is separate
125 compilation (each function in its own file) made into static binaries.
126 The "static" part is this: I could of course have put all the bits of
127 code into one external file, and source it at run time, but that
128 cannot work for .config, .preinst and .postrm scripts since they are
129 executed when the package is not installed (not yet unpacked or
134 It is vitally important that extreme care be taken for changes in the
135 database. A smooth upgrade path has been provided so far by carefully
136 sequencing the changes in the database and making sure they will not
137 conflict with each other or some other trick. This section is
138 targetted at people who touch deb-specific/db-upgrade.pl. Please read
139 it thoroughly, and don't skip paragraphs. A single mistake can be a
140 nightmare to fix (and believe me, I know that).
142 The database, as created by the package, has a lot of tables for the
143 software proper, plus one especially used to store a version of the
144 database scheme. That one table is named debian_meta_data, and
145 contains two fields (named 'key' and 'value'). Two rows are currently
146 used: the one for which the key is 'db-version', and the one for which
147 the key is 'current-path'. Together they store the current status of
150 The value of 'current-path' is only used during the first
151 installation of Sourceforge 2.6. If that installation is done on a
152 fresh machine, the value will be 'scratch-to-2.6'. If the
153 installation is done as an upgrade on Sourceforge 2.5, the value will
154 be '2.5-to-2.6'. In any case, when that first installation is
155 completed, the row is deleted. There's currently no other use for
158 The value of 'db-version' is a string encoding a version. The
159 ordering method for these strings is the one provided by dpkg
160 --compare-versions. The values currently used more or less match the
161 package versions, but you shouldn't depend on it. Special procedures
162 (upgrading from 2.5, or installing a fresh 2.6, or future cases maybe)
163 involve names not corresponding to a package version. They still must
164 be ordered according to dpkg.
166 The database upgrader (db-upgrade.pl) can be seen as walking along a
167 path. Either that path is an explicit one (during first install or
168 upgrade from 2.5), or it is the default one (upgrading betwen versions
169 of 2.6). In any case, the walking is made in steps. Each step has a
170 target version (where it leads) and a series of actions to perform to
171 reach that target. If the current version of the database is lesser
172 than the target version, the actions are performed, and the current
173 version is updated to the target version. If not, that step is
174 skipped. It is very important that steps are attempted in the correct
175 order (ascending order of target versions), otherwise steps will be
176 skipped. It is also very important that the actions are dependable.
177 Generally, when executing the actions for step n, you can depend on
178 the database being compliant with the last step before n.
180 Tip #1: don't assume things unless you are certain they are true.
181 Don't assume some value is missing from a table simply because it's
182 missing from yours. Don't assume some value is available.
184 Tip #2: test your patch. Install Sourceforge 2.6 from scratch.
185 Install it over a freshly installed Sourceforge 2.5. Install it over
186 a previous version of Sourceforge 2.6. If at all possible, install it
187 over a non-empty database coming from 2.5, then do it again over a
188 non-empty database of 2.6. If *any* of these break, *don't upload*.
191 Tip #3: remember, there's no way back. Once your package has been
192 installed by some user, the database version has irreversibly been
193 bumped up. While you can sometimes revert changes by hand on your
194 local database, you can't provide a rollback for potential thousands
195 of users (even it the actual number is more in the tens than in the
198 Tip #4: the db-upgrade.pl script has evolved over time to involve a
199 few functions, such as get_db_version and update_db_version. Use
200 them. Your best bet would be to cut and paste a block, change the
201 $target, and change the actions.
203 Tip #5: pay attention to the ordering of the blocks. You can't go
204 back in time, so all your changes must be at the end of the series of
207 There. Thanks for reading. I know it sounds boring, but I can
208 guarantee you will avoid problems if you understand this. Maybe not
209 all problems, but definitely some of them.
214 The source is maintained in bzr (see debian/control's Vcs-Bzr:
215 field). A copy can be checked-out using : debcheckout gforge.