Articles Home
List articles
Wiki Books
Orphan pages
Backup
Page Contents
From 0 to RPM in 2.2 Seconds...
Creating a new package from a downloaded tarball involves a series of basic steps that goes something like this:
- Download tarball.
- Unarchive tarball.
- Scan package for clues as to how it builds (i.e., configure script, Makefile, Imakefile, etc.).
- Look for documentation files which will need to be specially flagged.
- Create the spec file, populating the header fields and build instructions based on previous inspections.
- Perform a test build to verify the spec file's functionality.
- Correct any errors found in the previous step. Lather, rinse, repeat.
Sometimes the last step is completely unnecessary, and sometimes it can be a real doozy. Unfortunately, there's not much I can do about that. But the other steps are exactly why specgen exists: it gets you as far down the road as possible so that you can focus your attention solely on the parts that actually require skill and thought.
A Simple Example
ARPAlert is an ARP table monitoring utility which can notify/alert you when an unauthorized connection is detected on your network. It just so happens that it uses GNU autotools and builds properly, so specgen has no problem with it:
$ specgen http://www.arpalert.org/src/arpalert-1.1.3.tar.gz Downloading http://www.arpalert.org/src/arpalert-1.1.3.tar.gz................................................................................done. Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.18294 sh-trace: umask 022 sh-trace: cd /var/tmp/mezzanine.temp.build.15458.477c/arpalert/BUILD sh-trace: cd /var/tmp/mezzanine.temp.build.15458.477c/arpalert/BUILD sh-trace: rm -rf arpalert-1.1.3 sh-trace: /bin/gzip -dc /var/tmp/mezzanine.temp.build.15458.477c/arpalert/SOURCES/arpalert-1.1.3.tar.gz ... Checking for unpackaged file(s): /usr/lib/rpm/check-files /var/tmp/mezzanine.temp.build.15458.477c/arpalert/buildroot Wrote: /var/tmp/mezzanine.temp.build.15458.477c/arpalert/SRPMS/arpalert-1.1.3-1.caos.src.rpm Wrote: /var/tmp/mezzanine.temp.build.15458.477c/arpalert/RPMS/i386/arpalert-1.1.3-1.caos.i386.rpm Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.61585 sh-trace: umask 022 sh-trace: cd /var/tmp/mezzanine.temp.build.15458.477c/arpalert/BUILD sh-trace: cd arpalert-1.1.3 sh-trace: test x/var/tmp/mezzanine.temp.build.15458.477c/arpalert/buildroot '!=' x/ sh-trace: rm -rf /var/tmp/mezzanine.temp.build.15458.477c/arpalert/buildroot sh-trace: exit 0 $
As we can see from the output of the test build, the build worked. This means that specgen has successfully created a package directory for us (in PDR format) using the automatically-detected package name (arpalert in this case). All that's left is to tweak the 3 bits of data in the spec file that specgen can't automatically populate: "Summary," "Description," and "Group."
$ cd arpalert $ ls -Fla total 116 drwxr-xr-x 2 mej mej 4096 Oct 17 12:02 ./ drwxr-xr-x 15 mej mej 4096 Oct 17 12:02 ../ -rw-r--r-- 1 mej mej 102200 Oct 17 12:02 arpalert-1.1.3.tar.gz -rw-r--r-- 1 mej mej 1247 Oct 17 12:02 arpalert.spec $
Upon opening arpalert.spec in your favorite editor, you'll find that the string FIXME occurs in exactly the 3 spots mentioned above. These 3 fields need to be filled in with appropriate information; I usually get the %description text from either the project home page or the Freshmeat.Net project page. The value for the "Group" tag should be one of those listed in the /usr/share/doc/rpm-4*/GROUPS file.
Once those 3 fields are filled in, exit your editor, and run mzbuild to generate a new set of packages with the replacements you just made. You can then upload these packages, or mzimport them into a repository, or whatever you need to do...You're done!
Now see how easy that was?
In Case of Emergency, Pull Lever
Unfortunately, only about half of the packages out there will actually build successfully right off the bat like that last one did. More often than not, the test build attempt that specgen performs will fail. However, even if this happens, the directory for the package has still been created for you! And if you cd into it and run mzbuild, you'll get the exact same failure specgen got. This gives you a starting point to work from, so even though it's not a complete success right away, it's still better than nothing.
So let's look at something in the "close, but no cigar" category. A C++ version of the old Acquire board game does like so:
$ specgen http://superb-east.dl.sourceforge.net/sourceforge/acquiregame/acquire-0.4.zip
Downloading http://superb-east.dl.sourceforge.net/sourceforge/acquiregame/acquire-0.4.zip...........................................done.
extract: Archive: /home/mej/local/acquire-0.4.zip
extract: inflating: INSTRUCTIONS
extract: inflating: Makefile
extract: inflating: bag.h
...
Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.29890
sh-trace: umask 022
sh-trace: cd /var/tmp/mezzanine.temp.build.13851.1c64/acquire/BUILD
sh-trace: cd acquire-0.4
sh-trace: make prefix=/var/tmp/mezzanine.temp.build.13851.1c64/acquire/buildroot/usr exec_prefix=/var/tmp/mezzanine.temp.build.13851.1c64/acquire/buildroot/usr bindir=/var/tmp/mezzanine.temp.build.13851.1c64/acquire/buildroot/usr/bin sbindir=/var/tmp/mezzanine.temp.build.13851.1c64/acquire/buildroot/usr/sbin sysconfdir=/var/tmp/mezzanine.temp.build.13851.1c64/acquire/buildroot/etc datadir=/var/tmp/mezzanine.temp.build.13851.1c64/acquire/buildroot/usr/share includedir=/var/tmp/mezzanine.temp.build.13851.1c64/acquire/buildroot/usr/include libdir=/var/tmp/mezzanine.temp.build.13851.1c64/acquire/buildroot/usr/lib libexecdir=/var/tmp/mezzanine.temp.build.13851.1c64/acquire/buildroot/usr/libexec localstatedir=/var/tmp/mezzanine.temp.build.13851.1c64/acquire/buildroot/var sharedstatedir=/var/tmp/mezzanine.temp.build.13851.1c64/acquire/buildroot/usr/com mandir=/var/tmp/mezzanine.temp.build.13851.1c64/acquire/buildroot/usr/share/man infodir=/var/tmp/mezzanine.temp.build.13851.1c64/acquire/buildroot/usr/share/info install
make: *** No rule to make target `install'. Stop.
error: Bad exit status from /var/tmp/rpm-tmp.29890 (%install)
RPM build errors:
Bad exit status from /var/tmp/rpm-tmp.29890 (%install)
specgen: Error: Unable to run test build -- The RPM %install stage exited abnormallyAs we can see from the test build, the program compiled successfully but does not have an "install" target in the Makefile. This kind of thing can happen with programs that are small and self-contained (i.e., only an executable, no data files). Chances are, if we do the install of the binary by hand instead of trying to run make install, it will work fine. So let's try that.
$ cd acquire $ edit acquire.spec
The old %install section should be replaced with the below:
%install
%{__mkdir_p} $RPM_BUILD_ROOT%{_bindir}
%{__install} -m 0755 %{name} $RPM_BUILD_ROOT%{_bindir}/While you're at it, don't forget to replace the FIXME's in the spec file just like we did before!
Now try running mzbuild again.
$ mzbuild Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.81180 sh-trace: umask 022 sh-trace: cd /home/mej/local/acquire/build.mezz/BUILD sh-trace: cd /home/mej/local/acquire/build.mezz/BUILD sh-trace: rm -rf acquire-0.4 ... Wrote: /home/mej/local/acquire/build.mezz/SRPMS/acquire-0.4-1.caos.src.rpm Wrote: /home/mej/local/acquire/build.mezz/RPMS/i386/acquire-0.4-1.caos.i386.rpm Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.84282 sh-trace: umask 022 sh-trace: cd /home/mej/local/acquire/build.mezz/BUILD sh-trace: cd acquire-0.4 sh-trace: test x/var/tmp/mezzanine-buildroot.23449 '!=' x sh-trace: rm -rf /var/tmp/mezzanine-buildroot.23449 sh-trace: exit 0 Package build succeeded. Output files are: acquire-0.4-1.caos.src.rpm acquire-0.4-1.caos.i386.rpm $
See? Now that wasn't so bad, was it? Just one more thing to fix. When the test build fails, specgen can't figure out the file list needed for the %files section of the spec file, so it simply puts /* to capture everything that gets installed. While this technically works, it's not the best idea. Before considering the package final, you should update the %files section to something more respectable. This command can help you do that:
rpm -qlp *6.rpm *noarch.rpm | sed 's!^/usr/bin!%{_bindir}!g;
s!^/usr/libexec!%{_libexecdir}!g; s!^/usr/lib!%{_libdir}!g;
s!^/usr/sbin!%{_sbindir}!g; s!^/usr/include!%{_includedir}!g;
s!^/usr/share/man!%{_mandir}!g; s!^/usr/share/info!%{_infodir}!g;
s!^/usr/share!%{_datadir}!g; s!^/etc!%{_sysconfdir}!g;
s!^/usr!%{_prefix}!g'Total Meltdown
Then there are packages that just don't obey any of the standard rules. These are the toughest packages of all, and they take a good bit of knowledge and experience (as well as, in many cases, trial and error) to tackle. There's no way any single document could encompass all the possible lunacy that can occur when novice developers create ridiculous build mechanisms. As previously mentioned, specgen is a "best effort" tool and can only do so much. Sometimes significant human intervention is inevitable.
I ran across a program called Cultivation which sounded neat, kinda like "The Sims" meets genetic food engineering. So I attempted to use specgen to package it.
$ specgen http://superb-east.dl.sourceforge.net/sourceforge/cultivation/Cultivation_5_UnixSource.tar.gz
Downloading http://superb-east.dl.sourceforge.net/sourceforge/cultivation/Cultivation_5_UnixSource.tar.gz............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................done.
specgen: Warning: Unable to parse package name/version from base name Cultivation_5_UnixSource.
Please enter package name (default "Cultivation_5_UnixSource"): cultivation
Please enter package version (default "1.0"): 5
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.14197
sh-trace: umask 022
sh-trace: cd /var/tmp/mezzanine.temp.build.8040.7b7c/cultivation/BUILD
...
sh-trace: ./configure --host=i686-caos-linux-gnu --build=i686-caos-linux-gnu --target=i386-caos-linux --program-prefix= --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/usr/com --mandir=/usr/share/man --infodir=/usr/share/info
/var/tmp/rpm-tmp.74872: line 35: ./configure: No such file or directory
error: Bad exit status from /var/tmp/rpm-tmp.74872 (%build)
RPM build errors:
Bad exit status from /var/tmp/rpm-tmp.74872 (%build)
specgen: Error: Unable to run test build -- The RPM %build stage exited abnormallyIf specgen can't find a Makefile, configure script, or any other clues on how to build a package, it gives up and generates a standard GNU autotools-based spec file. That's exactly what happened here. So let's take a look at what the tarball contents look like:
$ cd cultivation $ mzprep Creating working directory /home/mej/local/cultivation/work.... You may now chdir to work to make changes. Use "mzpatch -n " to generate a patch when done. $ ls -Fla work/Cultivation_5_UnixSource/ total 20 drwxr-xr-x 4 mej mej 4096 Oct 13 13:31 ./ drwxr-xr-x 3 mej mej 4096 Oct 19 10:30 ../ drwxr-xr-x 5 mej mej 4096 Oct 13 13:31 game2/ drwxr-xr-x 14 mej mej 4096 Oct 13 13:48 minorGems/ -rwxr-xr-x 1 mej mej 489 Oct 13 13:29 runToBuild* $
No wonder specgen couldn't figure out what to do — who ever heard of a "runToBuild" script?! This kind of thing often happens with novice programmers or fledgling projects that aren't yet robust and portable, and it's a royal pain in the butt.
The first item of business is to edit the spec file and fix the FIXME's. Then the %build section needs to be changed to execute the runToBuild script. As it turns out, runToBuild expects a choice on stdin for which platform to build on, so the %build section becomes echo 1 | ./runToBuild
Running mzbuild again after saving the spec file results in more, but not total, success:
$ mzbuild
...
In file included from ../../minorGems/graphics/openGL/ScreenGL.cpp:45:
../../minorGems/graphics/openGL/ScreenGL.h:43:21: GL/glut.h: No such file or directory
make: *** [Makefile.minorGems_dependencies] Error 1
cp: cannot stat `game2/gameSource/Cultivation': No such file or directory
Run Cultivation to play.
In file included from ../../minorGems/graphics/openGL/ScreenGL.cpp:45:
../../minorGems/graphics/openGL/ScreenGL.h:43:21: GL/glut.h: No such file or directory
make: *** [Makefile.minorGems_dependencies] Error 1
cp: cannot stat `game2/gameSource/Cultivation': No such file or directory
Run Cultivation to play.
sh-trace: exit 0
Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.1933
sh-trace: umask 022
sh-trace: cd /home/mej/local/cultivation/build.mezz/BUILD
sh-trace: cd Cultivation_5_UnixSource
sh-trace: /usr/bin/make install DESTDIR=/var/tmp/mezzanine-buildroot.7009
make: *** No rule to make target `install'. Stop.
error: Bad exit status from /var/tmp/rpm-tmp.1933 (%install)
RPM build errors:
Bad exit status from /var/tmp/rpm-tmp.1933 (%install)
pkgtool: Error: Package build failed: The RPM %install stage exited abnormallyCareful examination of the output reveals that not only did the program fail to build properly, but the home-brew script failed to detect the failure! And of course there appears to be no "install" target either, not surprisingly.
So we need to add a BuildRequires for freeglut-devel to our spec file to fix the build problem. After that's done, another mzbuild shows that all's well with the build now. Fixing up the %install and %files sections as we did in the previous example finishes this package off.
Trust me...it could've been far worse!
Templates and Types
Apart from looking into the package tarball to try and figure out various things (doc files, build mechanism, etc.), the bulk of the work done by specgen is simply substituting values for variables in a spec file template. The templates, usually found in /usr/lib/perl5/vendor_perl/*/Mezzanine/templates/, support a variety of variables and build methodologies, and new templates are continually being added to widen the scope of packages specgen can handle with little or no intervention.
One particularly difficult type of package to auto-detect is a web-based application (e.g., PHP, CGI, etc.). Fortunately, specgen allows you to specify a particular template to use (which is analogous to specifying a particular package build/install mechanism) as a command line option. So, for example, if you know a particular package is a web-based application, you can pass this knowledge on to specgen to improve its chances of Doing The Right Thing:
$ specgen -t web http://www.cacti.net/downloads/cacti-0.8.6i.tar.gz Downloading http://www.cacti.net/downloads/cacti-0.8.6i.tar.gz................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................done. Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.42187 sh-trace: umask 022 sh-trace: cd /var/tmp/mezzanine.temp.build.22828.5f3e/cacti/BUILD ... Wrote: /var/tmp/mezzanine.temp.build.22828.5f3e/cacti/SRPMS/cacti-0.8.6i-1.caos.src.rpm Wrote: /var/tmp/mezzanine.temp.build.22828.5f3e/cacti/RPMS/noarch/cacti-0.8.6i-1.caos.noarch.rpm ... sh-trace: exit 0
One word of caution, however: the web template simply copies the tarball contents into the proper directory for a web application. Since there's no build involved, and the spec file itself manages the install, chances are the web template will appear to "work" (i.e., create packages) regardless of what's actually in the package. Subsequent testing is a must, and package tweaks may be required to get the application fully operational on the web. (In particular, a package.conf file for /etc/httpd/conf.d/ will probably be desirable.)