Ports - Porting Guide
- Overview
- Porting Checklist
- Handling Complex Situations
- Updating Ports
- OpenBSD Porting Policy
- Security Recommendations
- Generic Porting Hints
- Other Helpful Hints
- Additional Information
Overview
So you've just compiled your favorite software package on your OpenBSD machine and you want to share your effort by turning it into a standard port. What to do?The most important thing to do is to communicate. Ask people on ports@openbsd.org if they are working on the same port. Tell the original software authors about it, including problems you may find. If licensing information appears incorrect, tell them. If you had to jump through hoops to make the port build, tell them what can be fixed. If they are only developing on Linux and feel like ignoring the rest of the Unix world, try to make them change their view.
Communication makes the difference between a successful port and a port that will slowly be abandoned by everyone.
First, look at the porting information on this page. Test, then re-test, and finally test again!
OpenBSD fully supports updates. This means that some issues must be taken into account.
Submit the port. Create a gzipped tarball of the port directory. You can then either place it on a public HTTP server, sending its URL to ports@openbsd.org, or send the port MIME encoded to the same address.
Porting some new software takes time. Maintaining it over time is harder. It is quite okay to port software and let other people handle it afterwards. It is also okay to help other people update and maintain other ports, as long as you communicate to avoid doing the same things twice.
In OpenBSD culture, MAINTAINERship is not a status item, but a
responsibility.
We have CVS and comments to give credit to the person who did the work.
A port MAINTAINER is something else: a person who assumes
responsibility for the working of the port, and is willing to spend some
time ensuring it works as best as can be.
Porting Checklist
The list below is a useful reminder of things to do. This is neither totally accurate nor perfect. Direct comments and questions to ports@openbsd.org.-
If you want to be a maintainer, subscribe to
ports@openbsd.org.
- This is where all ports discussions take place.
- Reading this list is important since many announcements are made there.
- You will find a lot of porting-savvy people there. They can often give you good advice or test ports for you.
-
Being a maintainer means more than just submitting ports.
It also means trying to keep them up to date, and answering questions
about them.
-
Check out a copy of the ports tree from CVS.
You can find instructions on how to do this at the
AnonCVS page.
As a porter, you should keep your base OS, ports tree, and installed
packages up to date.
-
From the names of the first level subdirectories of
/usr/ports/, pick a primary category for your port. Create a new directory below/usr/ports/<category>/or/usr/ports/mystuff/<category>/and create the basic infrastructure there. Copy the templateMakefilefrom/usr/ports/infrastructure/templates/Makefile.template. Fill inCATEGORIESwith the primary category you have chosen. -
Add the fetch portions of the Makefile.
-
Fill in
EXTRACT_SUFXif it's anything besides .tar.gz. Other examples are .tar.Z, or .tgz. -
Fill in
DISTNAMEwith the name of the file minus the extract suffix. If you havefoo-1.0.tar.gz,DISTNAMEisfoo-1.0. -
Fill in
SITESwith a list of URLs to the locations where the distfile is kept, https://ftp.openbsd.org/pub/OpenBSD/distfiles/ for example. Don't forget the trailing slash. Try to have at least three distinct sites as well. Place the most easily accessible first as they are traversed in order. -
Keep in mind that fetch references the file as
${SITES}${DISTNAME}${EXTRACT_SUFX}. All three are used. Don't setDISTNAMEto point to the file directly. -
You can check to see if you have filled these values in correctly by typing
make makesum.
For more complex ports, you have more options and tools available to you:
-
You also have the variable
PATCHFILESavailable. This is a list of vendor (not OpenBSD) patches to the port. Common uses are things like security or reliability fixes which are occasionally distributed as a patch rather than a full release. -
If your ports are available over large public mirrors such as GNU,
Sourceforge, or CPAN, we have already provided a list of sites for
your use in
/usr/ports/infrastructure/db/network.conf. SetSITESto${SITE_GNU}, or${SITE_SOURCEFORGE}, etc. To simplify this process, use the construct${SITE_FOO:=subdir/}to append the distribution subdirectory. -
For ports with source distributed from GitHub there are several possibilities.
-
In some cases, for example icinga or darktable, a distribution tar file has
been prepared and is available in a /releases/ directory.
Where available, use these in preference to other options as they usually
include the proper build infrastructure (configure scripts, etc) which are
often missing from "tagged" releases.
Specify as normal, using something like
SITES=https://github.com/acct/project/releases/download/relname/ - In other cases, files have been tagged but they rely on GitHub's
on-the-fly archive creation.
These can be specified using the following
bsd.port.mk(5) variables:
GH_ACCOUNT,GH_PROJECT,GH_TAGNAME. - Occasionally there is no alternative but to have a port reference only a
commit id.
In this case,
GH_COMMITshould be used instead ofGH_TAGNAME.
-
In some cases, for example icinga or darktable, a distribution tar file has
been prepared and is available in a /releases/ directory.
Where available, use these in preference to other options as they usually
include the proper build infrastructure (configure scripts, etc) which are
often missing from "tagged" releases.
Specify as normal, using something like
-
Ports normally correspond to given versions of software.
Once they are retrieved, files are checksummed and compared to the recorded
checksum(s) in distinfo.
So, to avoid confusion,
DISTFILESandPATCHFILESshould have clearly visible version numbers: don't retrievefoo-latest.tar.gzif it is a link tofoo-1.0.5.tar.gz. File names should not contain just the version number (1.0.5.tar.gz); files can be renamed during fetching if necessary, see bsd.port.mk(5). If something is only available in an unversioned file, gently ask the original program author to make such distinctions clear. In the meantime, if you must use a file like this, setDIST_SUBDIRto what a reasonableDISTNAMEwould be, likefoo-1.0.5, such that different, identically named versions of the distfile do not clash on the distribution file mirrors or build machines. -
If a given port needs more than about five
DISTFILESandPATCHFILESto work, useDIST_SUBDIRto avoid clutteringDISTDIR(/usr/ports/distfilesby default) too much. In this case,DIST_SUBDIRmust not include version numbers. When the port is updated to a later version, some distfiles may not change, but will be refetched ifDIST_SUBDIRis changed. Even if all distfiles change, it is easier for the user to track cruft. -
All
DISTFILESandPATCHFILESdon't necessarily come from the same set ofSITES. Supplementary sites can be defined using suffixes:DISTFILES.alt = foo-1.0.5.tar.gzwill come fromSITES.alt. -
Some ports don't always need to retrieve all files in all circumstances.
For instance, some ports may have some compilation options, and associated
files which are only required in such a case.
Or they may need some files for some architectures only.
In such a case, those supplementary optional files must be mentioned in the
SUPDISTFILESvariable. Targets such asmakesumorfetch-allwill fetch those supplementary files that the casual user doesn't need.
-
Fill in
-
Create a checksum in
distinfoby typingmake makesum. -
All files in
DISTFILES*are usually processed duringmake extract.EXTRACT_ONLYmay be used to limit extraction to a subset of files (possibly empty). The customary use of this variable is to customize extraction: for instance, if someDISTFILESneed some special treatment, they will be removed fromEXTRACT_ONLYand handled manually atpost-extractstage. For historic reasons,make extractdoes set up the working directory first along with extracting files. Thus, providing apre-extractordo-extracttarget is highly unusual (and fairly suspicious behavior, indicative of a high degree of obfuscation in the port). -
Patches that need specific treatment should be mentioned in
DISTFILES, and removed fromEXTRACT_ONLY, for historic reasons. -
Extract the port with
make extract. Pay attention to where the base of the sources are. Usually, it's/usr/ports/pobj/${PKGNAME}${FLAVOR_EXT}/${DISTNAME}. You may need to modify theMakefile'sWRKDISTvariable if it is different. -
Read the installation documentation and note what you have to do to build
the port and any special options that might be needed.
-
Now is also a good time to figure out what kind of licensing restrictions
apply to your port.
Many are freely redistributable, but quite a few are not.
We need two questions answered to distribute ports properly.
These are the
PERMIT_*values in theMakefile.PERMIT_PACKAGEtells us whether we can put the package on the mirrors.
PERMIT_DISTFILEStells us whether we can mirror the distfiles.If both are permitted, simply set
PERMIT_PACKAGE=Yes. Otherwise, set each variable to eitherYesor a comment string indicating the reason distribution is not allowed. Pay attention to any special conditions you may need to fulfill later on. For example, some ports require installing a copy of the license. We recommend you place the license in/usr/local/share/doc/<name>/.In addition to the
PERMIT_*values, put a license marker like# Licenseabove them as a comment, this way we know why thePERMIT_*values are set the way they are. For GPL, specify which version is applicable e.g. "GPLv2 only", "GPLv2+" (meaning "v2 or newer"), "GPLv3 only", etc. -
Add configuration options to
Makefileand/or create the configuration script.-
If GNU configure is used, run
./configure --helpto see what options are available. In particular, look for optional dependencies which may not be present on your system, but might be picked up on another system or in a bulk build. -
Anything that you may want to override can be changed by adding the
--optionflags to theCONFIGURE_ARGSparameter in theMakefile. -
Use
CONFIGURE_ARGS+=to append to the variable.CONFIGURE_ARGS=will overwrite it.
-
If GNU configure is used, run
-
Try building the port with
make build.- If you're lucky, the port will go all the way through without errors.
- If it exits with an error, you will need to generate patches for your port. Figure out what needs to be changed and make a patch for it.
-
the sequence to make a patch for a file is usually:
-
cd `make show=WRKSRC` ; cp foo/bar.c{,.orig.port} -
edit
foo/bar.cto fix the error. -
cd - ; make update-patches -
this will create
patches/patch-foo_bar_cwith your modifications.
-
-
The easiest way to reset the port and test your patches is
make clean patch. This will delete the work directory, re-extract, and patch your port.
-
Begin a cycle of
make build, edit source code, generate a patch usingmake update-patchesandmake clean patch.-
Patches go in the directory
patches/and should be named patch-* with * being something meaningful. The preferred naming ispatch-FILENAME, whereFILENAMEis the name of the file it is patching. (make update-patchesdoes this automatically for you.) -
Applying
PATCHFILESis the first half of themake patchstage. It can be invoked separately asmake distpatch, which is a convenient target for porters. Ignore this if you haven't set it. - Only patch one source file per patch file, please.
-
Use
make update-patchesto generate patches. -
All patches MUST be relative to
${WRKDIST}. -
Check that patches DON'T contain tags that cvs will replace
(update-patches(1)
will warn you about it).
If they do, your patches won't apply after you check them in.
You can check in your changes with
-kkto avoid this. - Write a short explanation at the beginning of the patch file about its purpose (not mandatory, but for example often done for security patches).
- Where your patches make sense for use outside of the ports tree, please feed them back to the author of that software.
-
Patches go in the directory
-
Try setting
SEPARATE_BUILD.-
If the port can build with object files outside its source tree,
this is cleaner (many programs using
CONFIGURE_STYLE=gnucan). - This can also spare you some effort, as you will possibly be able to restart the cycle at configure most of the time.
-
If the port can build with object files outside its source tree,
this is cleaner (many programs using
-
Peruse the output (if any) and tweak any options in the
Makefile. To repeat issue the commandmake clean configure.Configuration files will usually be installed somewhere under
/usr/localand handled by pkg_add(1)@sampleannotations. After a package has been installed the contents ofpkg/MESSAGEwill be displayed if it exists.The OpenBSD file locations are:
user executables: /usr/local/bin system admin executables: /usr/local/sbin program executables: /usr/local/libexec libraries: /usr/local/lib architecture dependent data: /usr/local/lib/<name> installed include files: /usr/local/include or /usr/local/include/<name> single-machine data: /etc or /etc/<name> local state: /var/run games score files: /var/games GNU info files: /usr/local/info man pages: /usr/local/man/... read-only architecture-independent: /usr/local/share/<name> misc documentation: /usr/local/share/doc/<name> example files: /usr/local/share/examples/<name>
-
Begin a cycle of makes until the port is ready.
Patch (see above) clean, and make until the port is generated.
Get rid of all warnings if possible, especially security-related warnings.
-
Add
COMMENTinMakefile.COMMENTis a SHORT one-line description of the port (max. 60 characters). Do NOT include the package name (or version number of the software) in the comment. Do NOT start with an uppercase letter unless semantically significant, and do NOT end with a period. DON'T EVER START WITH AN INDEFINITE ARTICLE SUCH AS "a" or "an" - remove the article altogether. The main usage ofCOMMENTis the one-liner from pkg_info(1):py3-cairo-1.24.0 cairo bindings for Python
-
Put a longer description of the port into
pkg/DESCR. One to a few paragraphs concisely explaining what the port does is sufficient. Lines should be no longer than 80 characters. This can be done by first editing theDESCRfile and then running it throughfmt. -
If the application needs to create a user or a group, choose the lowest free
id from
/usr/ports/infrastructure/db/user.listfor your port to use and don't forget to include that file in your commit. -
Install the application with
make fake. Libraries should never be stripped. Executables are stripped by default, unless you setDEBUG_PACKAGES; this is governed by${INSTALL_STRIP}.${INSTALL_PROGRAM}honors this automatically and is preferable to unconditional stripping (e.g., by aninstall-striptarget or by runningstripfrom theMakefile). You can useobjdump --symsto determine if a binary is stripped or not. Stripped files have no symbols in theSYMBOL TABLE. -
Check port for security holes again.
This is especially important for network-facing programs.
See our security recommendations for that.
-
Make sure your
/etc/mtreedirectory is up to date. (The next step uses the mtree lists to remove existing directories from generated packing-lists). Remember that the OpenBSD(U)pdatedoes not touch/etc... For automatic updating of/etc, sysmerge(8) may help. -
Create
pkg/PLIST. After the installation withmake fakeis complete, use the developer's commandmake update-plist, which creates or updates the filePLISTin thepkgdirectory. This file is a candidate packing list.Peruse
PLISTand verify that everything was installed and that it was installed in the proper locations. Anything not installed can be added to a portMakefilepost-installrule. Note thatPLISTannotations are documented in the pkg_create(1) manual, and that update-plist(1) is also documented. -
It is possible some directories do not need to be in the
PLISTas they've been installed by a dependency; if you added toLIB_DEPENDSorRUN_DEPENDS, runmake update-plistto remove these. -
Test the packaging with
make package, test installation of the resulting package withmake install, and test its removal withmake uninstall. When dealing with multi-packages, it may instead be convenient to use pkg_add(1) and pkg_delete(1) directly, settingTRUSTED_PKG_PATHto/usr/ports/packages/%a/all/in the environment. -
Verify dependencies.
Peruse your logs to verify the port did detect what is mentioned in
*DEPENDS, and nothing more. Check names, particularly in themake configurestage, for hidden dependencies (stuff that exists elsewhere in the ports tree and might be detected if the user installs some other ports first). -
Verify shared library dependencies.
Run
make port-lib-depends-checkand add everyLIB_DEPENDSorWANTLIBannotation that is needed until it runs cleanly. You may want to read the update guidelines to understand why this is so important. -
Check for regression tests, and whether they run cleanly.
Set
NO_TEST=Yesif a port has no test infrastructure. If dependencies are required to run the tests, but not to build the port, list them inTEST_DEPENDS. Please note: do not setNO_TESTif a port has an empty regression test infrastructure. -
Test that
make cleansucceeds. Sometimes themake fakestage creates files in the build directory which will cause this to fail. -
Run the
/usr/ports/infrastructure/bin/portcheckutility in your port directory and take care of problems it finds, if any. -
Mail ports@openbsd.org with a
description, the homepage (if any), and a short note asking for comments
and testing.
Make sure to attach the port/patch to this email (or include a URL where
it can be found) and send it out.
Try to get others to test it on a variety of platforms for you.
-
The AMD64 systems are good because they are fast, and because
sizeof(int) != sizeof(long)on this platform. - sparc64 systems are good because they are fairly common, can be quite fast, and their byte order is the opposite of i386; if you developed on sparc64, of course, you'd want it tested on i386.
-
The AMD64 systems are good because they are fast, and because
-
Incorporate any useful feedback you get. Test it again on your platform.
Get those who gave you feedback to test it again from your new port.
-
Get the port committed to CVS.
If you are not a developer with CVS access, then you will have to find an OpenBSD developer to do the following steps for you (ask on ports@openbsd.org).
Before you import anything, get at least one "OK" from another ports developer (the more the better).
If using
@newuseror@newgroupin the PLIST files, check that no users were added to/usr/ports/infrastructure/db/user.listsince the port was created.For new ports we use
cvs import, rather than adding new files individually. For example, to import a newlang/kaffe1port, first do a dry-run import:$ cd /usr/ports/lang/kaffe1 $ cvs -ndcvs.openbsd.org:/cvs import ports/lang/kaffe1 username username_yyyymmdd
Whereusernameis your account username, andyyyymmddis the current date. If this succeeds, then you can remove-nto import for real. Your editor will be invoked so that you can enter a commit message. In the commit message, at least state what you are importing and which developers provided OKs. Make sure the import path is correct; it always starts withports/.Alternatively, you can use
ports/infrastructure/bin/portimportto import new ports. -
Last but not least, add a one-line entry for the new port in its parent
directory's
Makefile, e.g., forports/lang/kaffe1, add it toports/lang/Makefile. Don't forget to commit any changes made to/usr/ports/infrastructure/db/user.list. -
Maintain the port!
As time goes by, problems may arise, or new versions of the software may be
released.
You should strive to keep your port up to date.
In other words - iterate, test, test, iterate...
-
When updating a port, remember to handle dependencies!
You shouldn't break any port that depends on yours.
The
sqlportspackage includesshow-reverse-deps, making it trivial to know the full tree of ports that depend directly or indirectly on a given port. In case of problems, communicate with the maintainers of such ports. Likewise, be alert for dependency updates, and check that the maintainer did their job.
Handling Complex Situations
Assume you've managed to build the software, provide required patches, and you want to finish the port.Know the Software
- Identify options
- The first step is usually to identify build options. You will often have to read the configuration log, see what stuff your port auto-detects. Read the configure script options. Read the port documentation for extra stuff.
- Make options work
- Recompile your port with various options. Install extra dependencies. Make sure your port detects them correctly. Add supplementary patches to ensure compilation. Test the result, and verify extra stuff does work.
- Identify missing software
- Some dependencies won't be fulfilled because the missing software has not yet been ported. It is highly recommended to explicitly disable those options. Failure to do that breaks bulk builds all the time: people port new software and import it, and soon after, old ports stop building because they detect the dependency, try to use it, and fail to build or package.
- Check run-time dependencies versus build-dependencies
-
Update your packing-list with
make update-plist. Usemake port-lib-depends-checkto see what libraries your software needs (that will end up inLIB_DEPENDSorWANTLIB, usually). Identify various files and binaries in the dependencies that have to be present for the port to work.
By this point, you should have a fair understanding of your port's working.
Figure Out Important Options
You won't care about some options. It makes no sense to disable some stuff if it always works, and if the dependencies are quite small. Take special notes of licences on dependencies, especially thePERMIT* stuff.
As a rule, even if a dependency is very small, if it affects the licensing of
the resulting package, you will have to explicitly take care of it.
Considering all possible options, you should be left with a much smaller set of options for your port, mostly depending on what packages are needed to run the software. For now, do not worry about build dependencies. Remember that the OpenBSD ports system is focused on the end user, and the end user will want to install binary packages, so it doesn't matter if you need a huge piece of software to build your port if it doesn't show up as a library or runtime dependency.
The Ideal Case: MULTI_PACKAGES and
PSEUDO_FLAVORS
By now, you should have a fairly good idea of:
- which new files show up when you activate each option
- which libraries/runtime files are needed when you activate each option
In this case, try setting the MULTI_PACKAGES variable to a list of
-sub packages, add corresponding COMMENTS,
and look at your packaging.
Basically, MULTI_PACKAGES only affects the packaging: if you have
MULTI_PACKAGES=-s1 -s2 all stuff relevant to the package will exist
in two variants:
COMMENT-s1 for the first package,
COMMENT-s2 for the second package,
PLIST-s1, PLIST-s2,
DESCR-s1, DESCR-s2.
You need to write those COMMENT-s1 and COMMENT-s2
in the Makefile, split your PLIST into two parts,
and create DESCR-s1/DESCR-s2.
It is a good idea to start with the minimal framework work required:
just copy the existing description and comments, because you will have
to fiddle with MULTI_PACKAGES and stuff before you polish this.
Once you've separated the files properly, you will need to check dependencies:
LIB_DEPENDS, WANTLIB, and RUN_DEPENDS
will be split for each subpackage.
It is usually time to check that your multi-packaging "works," and that
those nasty dependencies you don't want to force on the user are indeed
relegated to a specific subpackage.
Assuming everything works, you're mostly done. Just pick reasonable names for the various packages, and fill in the comments and descriptions. The end-user will be able to just install the package(s) they want.
But wait.
What about the build, you say?
Well, having a lot of dependencies during build is not a problem.
Most packages are built by the OpenBSD team using special build runs (known
as bulk builds) where a developer just builds all possible packages on a
dedicated machine (or several, for slow architectures).
Since everything will get built, having big dependencies is not an issue.
Building the same thing several times is an issue, though, which is why
MULTI_PACKAGES are the best way to handle options (when possible):
one build, one set of packages to test, better quality overall...
If you also want to help people who build packages themselves, you may consider
adding PSEUDO_FLAVORS.
A pseudo-flavor is a way to tweak an option (say, disable the graphical
interface) that's very similar to actual flavors.
In fact, the biggest difference is a functional difference: a pseudo flavor
should only affect the set of packages that get built, but it is never
allowed to modify the actual package contents.
For instance, assuming you separated the graphical interface into a separate
subpackage (MULTI_PACKAGES=-main -x11), you could create a pseudo
flavor no_x11 that avoids building the -x11 subpackage.
The crucial point is that this flavor should NOT affect the -main package
in any way.
You would end up with a Makefile that looks something like this:
COMMENT-main = foo core application
COMMENT-x11 = foo graphical interface
V = 1.0
DISTNAME = foo-$V
CATEGORIES = app
MULTI_PACKAGES = -main -x11
PSEUDO_FLAVORS = no_x11
FLAVOR ?=
WANTLIB = c m crypto ssl
WANTLIB-x11 = ${WANTLIB} X11 Xt
RUN_DEPENDS-x11 = ${BASE_PKGPATH},-main>=$V
CONFIGURE_STYLE = gnu
.include <bsd.port.arch.mk>
.if !${BUILD_PACKAGES:M-x11}
CONFIGURE_ARGS += --disable-x11
.endif
.include <bsd.port.mk>
Notice that you only have to write a very small conditional section in the
Makefile:
the system doesn't care at all that you define extra variables.
Interdependencies Between Subpackages
MULTI_PACKAGES setups used to be asymmetric,
with a -main subpackage and other subpackages,
with the -main subpackage always built,
and other subpackages possibly depending upon it.
The current situation is totally symmetric: any subpackage can depend on any
other.
The infrastructure has specific provisions to avoid looping indefinitely.
The infrastructure takes special care of library inter-dependencies:
it can detect which WANTLIB come from external dependencies,
and which come from inter-dependencies.
While external LIB_DEPENDS and WANTLIB are checked at
the start of build, LIB_DEPENDS and WANTLIB that
refer to one of the subpackages currently being built
will only be checked at packaging time
(and thus packages may be created in a specific order to satisfy
interdependencies).
The infrastructure provides specific variables to help in writing
inter-dependencies:
BUILD_PKGPATH contains the PKGPATH used
during building the current packages, taking flavors and pseudo-flavors into
account.
It is highly recommended to use this variable instead of rolling your own:
failure to do so will often trigger rebuilds in interesting flavors situations.
For instance:
... FLAVORS = a b FLAVOR ?= MULTI_PACKAGES = -p1 -p2 WANTLIB-p1 = foo LIB_DEPENDS-p1 = some/pkgpath,-p2 ...If you go on and build in some/pkgpath with
FLAVOR=a,
then creating the subpackage for -p1
will trigger a rebuild with FLAVOR=''.
You would instead write:
LIB_DEPENDS-p1 = ${BUILD_PKGPATH},-p2
There is also a BASE_PKGPATH variable, which does not take
pseudo-flavors into account.
This variable has limited applicability: it corresponds to a transition between
old MULTI_PACKAGES and newer ones, where the old main
subpackage did not have any marker in its pkgpath, and thus the corresponding
package needs a @pkgpath ${BASE_PKGPATH} in its packing-list.
(In general, pseudo-flavors are build information, and should not make their
way into packages and packing-lists).
True FLAVORS and PKGNAMES
There are some cases where configuration options are too invasive, and you
will have to add true flavors to the Makefile: those flavors will
command some configuration options,
and usually additions to various dependencies.
Note that package naming is mostly automatic: the PKGNAME will have
an extension built by appending the specified flavors to its name.
So, if
PKGNAME = foo-1.0 FLAVORS = f1 f2 f3and you build the port with
FLAVOR='f3 f1', then
FULLPKGNAME=foo-1.0-f1-f3 (FLAVORS is used to reorder
specified flavors in a canonical way).
There are sometimes mixed situations, where some packages do depend on the
FLAVOR, and some don't.
For instance, some ports include a large set of documentation that does not
depend on the FLAVOR, and some actual programs that depend on the
FLAVOR.
In such cases,
you can specify the FULLPKGNAME for the documentation
subpackage explicitly.
Something like this:
CATEGORIES = app
COMMENT-core = foo application
COMMENT-doc = foo documentation
V = 1.0
DISTNAME = foo-1.0
PKGNAME-core = foo-$V
FULLPKGNAME-doc = foo-doc-$V
FLAVORS = crypto
MULTI_PACKAGES = -core -doc
WANTLIB-core = c m
.if ${FLAVOR:L:Mcrypto}
WANTLIB-core += ssl crypto
CONFIGURE_ARGS += --enable-crypto
.endif
As mentioned in the documentation, all package names have the same structure:
stem-version-flavor_extension.
By default, packages with the same stem do conflict, and update paths will
look at candidates with the same stem.
The right package will be the one coming from the exact same
PKGPATH,
or matching @pkgpath annotation in the packing-list.
Usually, MULTI_PACKAGES should not conflict, so they must have
different names (and the infrastructure has no way to build those names).
On the other hand, flavors should conflict, and thus have the same name.
The flavor information should end at the end of the package name, except for
pseudo-flavors, which do not change the way a package is built.
As far as dependencies go, by default,
specifying a PKGPATH will just
create a stem-* dependency, meaning any package with the right stem
will match the dependency.
By default, any flavor will match.
If only specific flavors are desired, you must include them in your
specification, e.g., stem-*-flavor.
If some flavors are unwanted, you can remove them from matching packages,
e.g., stem-*-!flavor.
Since OpenBSD 4.9, asking for at least some version of a dependency can be
directly tacked to the PKGPATH, e.g., dir/foo>=2.0.
Updating Ports
The package tools can do updates, so maintainers have to be aware of one simple fact: update is not instantaneous. Even if a user just goes from release to release, each time they runpkg_add -u,
the system will replace each package with a new version,
one package at a time.
So the user will run a mixed system, even if it is for just a few minutes.
There are basically two update models of which maintainers must be aware.
- Some users follow current on a loose basis. They update their packages every few days/every few weeks. Either all packages, or only some selected packages. The update mechanism should work for them: it can force them to update some specific packages, or to install some new stuff, but it should not fail silently. Micro-updates must be tested. These users will most often be able to cope with small changes, like program name changes, or configuration adjustments.
- Some users update every six months with a new release. They also will download stable updates (security and critical updates). During six months, a large part of the ports tree changes. These users do not care about individual software changes. They just want a system that works. Changing binary names is completely unfriendly to these users. Tweaking hundreds of configuration files will be a major pain for them. Maintainers can help by providing smooth update paths, and major hints whenever something important changes.
You should note that part of the update process, especially the macroscopic changes for users who update every six months, is not yet automated. The global update mechanism is still a work in progress, and pkg_add will be able to cope with more issues in the future. As of now, you should focus on making sure the system updates correctly, one port at a time, and that one port's update takes other ports into account, as far as conflicts and other issues are concerned.
Update Checklist
Part of the work is to be done when making the port itself.- Make sure the software authors are aware of changes that are required to make it work on OpenBSD. Some changes only relate to ports policies and these don't usually make sense to feed back, but other than this, having changes included upstream makes it easier for the author to test on OpenBSD (they will usually want to do this outside of the ports framework), and reduces the need to merge conflicting patches later yourself. When updating, check that the patches still make sense. Perhaps the same problem was fixed in a different way, in which case a patch may still apply but should be removed.
- Make sure the software authors understand version numbering. Sometimes the names of the distfiles do not contain version numbers, or reference a branch - that is, they're not a fixed point in the code, but a moving target. If this happens, communicate with the authors and try to get permanent URLs.
- Document work-arounds. This will help you remember to check whether they are still necessary when you try to update a port.
- Document dependencies, especially the stuff you don't use. Some ports can use external software that may not be available at the time of porting. Make sure you do not pick it up, and document it, so that you can update your port when this software becomes available later.
-
If the port uses libtool, copy its log from the file
${WRKBUILD}/shared_libs.logverbatim as a basis for yourSHARED_LIBSsetup. This will help during updates. -
PLIST_REPOSITORYis set by default and will build a database of packing-lists as packages are built on your system. This is useful to find out about forgotten package name bumps, and also to check for conflicts with older packages. -
Make sure dependencies stay as loose as they can be.
By default,
RUN_andLIB_DEPENDSwill be satisfied by any version of a package. Do not insist on a given version unless you have to. Use minimal versions whenever possible.
-
Each port update needs a package name bump.
Otherwise, the update mechanism for binary packages won't work.
Anything that affects the binary package implies a bump.
This includes
HOMEPAGE,MAINTAINERor description changes, changes to patches or build flags. If the upstream version has not changed, the package name bump is done by incrementingREVISIONif already present, otherwise addingREVISION = 0towards the top of the Makefile. IfMULTI_PACKAGESare used, incrementREVISIONto set the default revision for all subpackages, or increment the individualREVISION-subpkgas necessary (this overridesREVISION). If in doubt, usemake show=PKGNAMESto check before and after making changes. -
Version numbers always go forward.
If something unexpected happens and you have to revert an update, or if the
upstream numbering changes completely so that the version number appears
to go backwards, you must use
EPOCHto make sure pkg_add(1) sees the package as a newer package. This will addv${EPOCH}toFULLPKGNAMEto form a full package-name conforming to packages-specs(7). - If the package never built with a particular version, no bump is needed.
- Changes to make sure a port does or does not pick up an external dependency require a bump.
- Changes in existing dependencies generally do not affect a port version number. The package system uses a signature mechanism such that a binary package is fully identified by its package names, plus the dependencies against which it was built, plus the version numbers of all shared libraries it contains. So, if your port depends on a library, a change in that library's version number will trigger an update for packages built from your port.
-
The version comparison ordering can be summarized as:
0.1 < 0.1p0 < 0.2 < 0.1v0 < 0.1p0v0 < 0.2v0 < 0.1v1. Also additional words includingbeta,rcandpreare handled. See packages-specs(7) for more detailed information, and use pkg_check-version(1) to check the ordering of version numbers.
-
Run
make patchto have an initial copy of the port before the update.
-
Edit the port's
Makefileto grab the new version, runmake makesumandmake patchas a starting basis. - Once you've fixed patches that failed, run a full diff between the old and the new version to figure out exactly what changed. Take notes, be prepared to revisit things later.
- Do whatever porting work is necessary to get the new version to run under OpenBSD: adjust dependencies, change package contents, etc.
-
Double check shared library versions.
For libtool-based ports, you have the
shared_libs.logto check whether the software authors made significant changes. Note well that this is not enough. Many software authors do not really understand shared library issues. Runmake check-shlib-symsto see whether dynamic export changes require aSHARED_LIBSversion bump. You still have to read the full diff between the old and the new version, because added exports do not prove that a minor bump is sufficient, and an empty result only means that no dynamic export change was reported. When in doubt, bump the major version. - Be aware of conflicts with already-built packages. For simple updates, nothing needs to be done. If you split a package into several subpackages, make sure the subpackages have proper conflict annotations: they should usually conflict with the "old" single package.
-
Help the user.
If some specific steps must be taken beyond
pkg_add -u, make sure they are visible in the package. When your packaging policy changes, document it in the package description. For instance, if you move the documentation into a separate package for space constraints, the main package description should mention that the documentation is now available as a separate package.
Committing Port Updates
When a port update is ready, get it in CVS. If you do not have a CVS account, then you will need to find a developer to do the following for you (ask on ports@openbsd.org).- Get at least one "OK" from another ports developer.
- Use
cvs addto add any new files. - Use
cvs rmto remove files that are no longer needed (e.g. upstreamed patches). - Check the output of
cvs -n up -d. New files should be markedA, deleted files should be markedD, and changed files should be markedM. Look for files marked?- did you mean tocvs addthem? - If all is well, commit the new/deleted/changed files using
cvs commit. When invokingcvs commit, you can either list the files individually, or if you provide no filenames, CVS will recursively commit (be careful with this feature). You will be asked to enter a commit message, in which you should state which port is being updated, and who provided an OK.
OpenBSD Porting Policy
-
OpenBSD does NOT use
/usr/local/etc/rc.d./usr/localis often shared between several machines, thanks to NFS. For this reason, configuration files that are specific to a given machine can't be stored under/usr/local./etcis the central repository for per machine configuration files. Moreover, OpenBSD policy is to never update files under/etcautomatically. Ports that need some specific boot setup should advise the administrator about what to do instead of blindly installing files. - OpenBSD does NOT compress man pages.
-
OpenBSD does NOT require
-lcrypt,-ldl, or-lrt. The functions provided by these libraries are part oflibc. Thecrypt()function does not support DES, only bcrypt. -
OpenBSD has a separate namespace for users and groups created by ports.
See
/usr/ports/infrastructure/db/user.listfor details. - OpenBSD is strongly security-oriented. You should read and understand this page's security section.
- The goal is to get all ported applications to support OpenBSD. To achieve this goal, feed patches to support running on OpenBSD back to the application maintainer. (If you are not the port maintainer, check with them first. There may be a reason why they have deliberately not done this).
Security Recommendations
There are many security problems to worry about. If you are not absolutely sure of what you are doing please request help from the ports mailing list.- Do not use alpha or beta code when preparing a port. Use the latest regular or patch release.
-
Any software to be installed as a server should be scanned for buffer
overflows, especially unsafe use of
strcat/strcpy/strcmp/sprintf. In general,sprintfshould be replaced withsnprintf. -
Never use filenames instead of true security.
There are numerous race conditions where you don't have proper control.
For instance, an attacker who already has user privileges on your machines
may replace files in
/tmpwith symbolic links to more strategic files, such as/etc/master.passwd. -
For instance, both
fopenandfreopencreate a new file or open an existing file for writing. An attacker may create a symbolic link from/etc/master.passwdto/tmp/addrpool_dump. The instant you open it, your password file is hosed. Yes, even with anunlinkright before. You only narrow the window of opportunity. UseopenwithO_CREAT|O_EXCLandfdopeninstead. -
Another very common problem is the
mktempfunction. Heed the warnings of the bsd linker about its uses. These must be fixed. This is not quite as simple ass/mktemp/mkstemp/g. Refer to mktemp(3) for more information. Correct code usingmkstempincludes the source toedormail. A rare instance of code that usesmktempcorrectly can be found in thersyncport. -
Just because you can read it doesn't mean you should.
A well-known hole of this nature was the
startxproblem. As a setuid program, you could launch startx with any file as a script. If the file was not a valid shell script, a syntax error message would follow, along with the first line of the offending file, without any further permission check. Pretty handy to grab the first line of a shadow passwd file, considering these often start with root entry. Do not open your file, and then do anfstaton the open descriptor to check if you should have been able to open it (or the attacker will play with/dev/rst0and rewind your tape) -- open it with the correct uid/gid/grouplist set. -
Don't use anything that forks a shell in setuid programs before dropping
your privileges.
This includes
popenandsystem. Usefork,pipeandexecveinstead. -
Pass open descriptors instead of filenames to other programs to avoid race
conditions.
Even if a program does not accept file descriptors, you can still use
/dev/fd/0. - Access rights are attached to file descriptors. If you need setuid rights to open a file, open that file, then drop your privileges. You can still access the open descriptor, but you have less to worry about. This is double-edged: even after dropping privileges, you should still watch out for those descriptors.
- Avoid root setuid as much as you can. Basically, root can do anything, but root rights are very rarely needed, except maybe to create socket ports with a number under 1024. You must know the appropriate magic for writing daemons to achieve that. It could be argued that you have no business writing setuid programs if you don't know how to do that.
-
Use setgid instead of setuid.
Apart from those specific files needed by setgid programs, most files are not
group-writable.
Hence, a security problem in a setgid program won't compromise your system as
much: with only group rights, the worst an intruder will be able to do is hack
a games score table or some such.
See the
xkoboport for an instance of such a change. - Don't trust group-writable files. Even though they are audited, setgid programs are not perceived as important potential security holes. Hence stuff they can tamper with shouldn't be considered sensitive information.
-
Don't trust your environment!
This involves simple things such as your
PATH(never usesystemwith an unqualified name, avoidexecvp), but also more subtle items such as your locale, timezone, termcap, and so on. Be aware of transitivity: even though you're taking full precautions, programs you call directly won't necessarily. Never usesystemin privileged programs, build your command line, a controlled environment, and callexecvedirectly. Theperlsecman page is a good tutorial on such problems. - Never use setuid shell-scripts. These are inherently insecure. Wrap them into some C code that ensures a proper environment. On the other hand, OpenBSD features secure perl scripts.
- Beware the dynamic loader. If you are running setuid, it will only use trusted libraries that were scanned with ldconfig. Setgid is not enough.
-
Dynamic libraries are tricky.
C++ code sets a similar problem.
Basically, libraries may take some action based upon your environment before
your main program even gets to check its setuid status.
OpenBSD
issetugidaddresses this problem, from the library writer point of view. Don't try to port libraries unless you understand this issue thoroughly. -
On OpenBSD, rand(3),
random(3), and the
drand48(3) family return
non-deterministic random numbers by default.
Any specified seed value is ignored by the associated seed function, and
arc4randomis used instead. If the deterministic (i.e: repeatable) behavior must be preserved, use the OpenBSD extensions:srand_deterministic,srandom_deterministic,srand48_deterministic,seed48_deterministicandlcong48_deterministic.
Generic Porting Hints
-
__OpenBSD__should be used sparingly, if at all. Constructs that look like#if defined(__NetBSD__) || defined(__FreeBSD__)
are often inappropriate. Don't blindly add__OpenBSD__to it. Instead, try to figure out what's going on, and what actual feature is needed. Manual pages are often useful, as they include historic comments, stating when a particular feature was incorporated into BSD. Checking the numeric value ofBSDagainst known releases is often the right way. See the NetBSD pkgsrc guide for more information. -
Defining
BSDis a bad idea. Try to includesys/param.h. This not only definesBSD, it also gives it a proper value. The right code fragment should look like:#if (defined(__unix__) || defined(unix)) && !defined(USG) #include <sys/param.h> #endif
-
Test for features, not for specific OSes.
In the long run, it is much better to test whether
tcgetattrworks than whether you're running under BSD 4.3 or later, or SystemVR4. These kinds of tests just confuse the issue. The way to go about it is, for instance, to test for one particular system, set up a slew ofHAVE_TCGETATTRdefines, then proceed to the next system. This technique separates feature tests from specific OSes. In a hurry, another porter can just add the whole set of-DHAVE_XXXdefines to the Makefile. One may also write or add to a configure script to check for that feature and add it automatically. As an example not to follow, check nethack 3.2.2 source: it assumes loads of things based on the system type. Most of these assumptions are obsolete and no longer reflect reality: POSIX functions are more useful than older BSD versus SystemV differences, to the point that some traditional bsd functions are now only supported through compatibility libraries. - Avoid include files that include other includes that... First, because this is inefficient. Your project will end up including a file that includes everything. Also, because it makes some problems difficult to fix. It becomes harder to not include one particular file at a given point.
-
Avoid bizarre macro tricks.
Undefining a macro that was defined by a header file is a bad idea.
Defining macros to get some specific behavior from an include file is also
a bad idea: compilation modes should be global.
If you want POSIX behavior, say so, and
#define POSIX_C_SOURCEthroughout the whole project, not when you feel like it. -
Don't ever write system function prototypes.
All modern systems, OpenBSD included, have a standard location for these
prototypes.
Likely places include
unistd.h,fcntl.hortermios.h. The man page frequently states where the prototype can be found. You might need another slew ofHAVE_XXXmacros to procure the right file. Don't worry about including the same file twice; include files have guards that prevent all kinds of nastiness. If some broken system needs you to write the prototype, don't impose on all other systems. Roll-your-own prototypes will break because they may use types that are equivalent on your system, but are not portable to other systems (unsigned longinstead ofsize_t), or get someconststatus wrong. Also, some compilers, such as OpenBSD's own gcc, are able to do a better job with some very frequent functions such asstrlenif you include the right header file. -
Carefully check the build log for any compiler warnings.
-
implicit declaration of function foo()indicates that a function prototype is missing. This means that the compiler will assume the return type to be an integer. Where the function actually returns a pointer, on 64-bit platforms it will be truncated, usually causing a segmentation fault.
-
-
Don't use the name of a standard system function for anything else.
If you want to write your own function, give it its own name, and call that
function everywhere.
If you wish to revert to the default system function, you just need to comment
your code out, and define your own name to the system function.
Don't do it the other way around.
Code should look like this:
/* prototype part */ #ifdef USE_OWN_GCVT char *foo_gcvt(double number, size_t ndigit, char *buf); #else /* include correct file */ #include <stdlib.h> /* use system function */ #define foo_gcvt gcvt #endif /* definition part */ #ifdef USE_OWN_GCVT char *foo_gcvt(double number, size_t ndigit, char *buf) { /* proper definition */ } /* typical use */ s = foo_gcvt(n, 15, b);
Other Helpful Hints
-
Recent versions of
bsd.port.mkset the installer's path. Specifically, they set/usr/binand/binto be searched before/usr/local/binand/usr/X11R6/bin. -
It is OK to rely on a feature that appeared in a recent version of
bsd.port.mk, as people are supposed to update their whole ports tree, includingbsd.port.mk. -
Prefer using
update-plistto generate and update packing-lists instead of doing things manually. You can comment unwanted lines out with@comment.update-plistcan detect most file types and copy most extra annotations correctly. It is not infallible; review the changes it makes. Check they make sense, and check if any new example configuration files should be followed by@sampleannotations. -
In OpenBSD,
curses.h/libcurses/libtermlibis the "new curses." Change:
ncurses.h ==> curses.h
"old (BSD) curses" is available by defining_USE_OLD_CURSES_before includingcurses.h(usually in a Makefile) and linking with-locurses. -
In OpenBSD, terminal discipline has been upgraded from the older BSD
sgttyto the newer POSIXtcgetattrfamily. Avoid the older style in new code. Some code may definetcgetattrto be a synonym for the oldersgtty, but this is at best a stopgap measure on OpenBSD. Thextermsource code is a very good example of what not to do. Try to get your system functionality right: you want a type that remembers the state of your terminal (possible typedef), you want a function that extracts the current state, and a function that sets the new state. Functions that modify this state are more difficult, as they tend to vary depending upon the system. Also, don't forget that you will have to handle cases where you are not connected to a terminal, and that you need to handle signals: not only termination, but also background (SIGTSTP). You should always leave the terminal in a sane state. Do your tests under an older shell, such as sh, which does not reset the terminal in any way at program's termination. -
The newer termcap/terminfo and curses, as included with OpenBSD, include
standard sequences for controlling color terminals.
You should use these if possible, reverting to standard ANSI color sequences
on other systems.
However, some terminal descriptions have not been updated yet, and you may
need to be able to specify these sequences manually.
This is the way vim handles it.
This is not strictly necessary.
Except for privileged programs, it is generally possible to override a termcap
definition through the
TERMCAPvariable and get it to work properly. -
Signal semantics are tricky, and vary from one system to another.
Use
sigactionto ensure a specific semantics, along with other system calls referenced in the corresponding man page.
Additional Information
- The man page bsd.port.mk(5). This documents the ports infrastructure makefile that is included at the end of each individual port makefile. There are still a few comments at the start of the file itself, but most of the useful information is now documented.
- The man pages packages-specs(7) and library-specs(7). These explain the exact syntax of package and library dependencies, respectively.
- The man page port-modules(5). This documents the different port modules which are used throughout the ports infrastructure, e.g. cpan, java, python, and so on.