Upgrade Guide: 4.5 to 4.6

[FAQ Index] | [4.4 -> 4.5] | [4.6 -> 4.7]

Note: Upgrades are only supported from one release to the release immediately following it. Do not skip releases.

It is highly recommended that you read through and fully understand this process before attempting it. If you are doing it on a critical or physically remote machine, it is recommended that you test this process on an identical, local system to verify its success before attempting on a critical or remote computer.

Upgrading is a convenient way to bring your OpenBSD system up to the most recent version. However, the results are not intended to precisely match the results of a wipe-and-reload installation. Old library files in particular are not removed in the upgrade process, as they may be required by older applications that may or may not be upgraded at this time. If you REALLY wish to get rid of all these old files, you are probably better off reinstalling from scratch.

Table of Contents:

Before upgrading
The upgrade process
Final steps

Before upgrading: things to think about and be aware of

This is not a complete list of the changes that took place between 4.5 and 4.6, but rather some of the important things that will impact a large number of users in the upgrade process. For a more complete list of changes, see plus46.html and the CVS change logs.

NEW INSTALLER!
OpenBSD's traditionally lean and easy to use installer has been improved and made even leaner and easier to use! This is not really an upgrade issue, but experienced OpenBSD users will probably want to look through the new Installation Guide.
A number of changes have been made to PF.
Your existing pf.conf file will probably not work with 4.6 without modifications! If you reboot your system without a usable pf.conf file in place, your pf rules will not be loaded, and you will end up using the default rule set, which will block all traffic EXCEPT for ssh over the standard port 22. This means that if you do not fix your pf.conf rules before rebooting, you may be greeted by a box that does not even respond to pings. Do not panic, as you can still ssh to the box, assuming you have sshd(8) listening on the usual port.
- pf is now enabled by default in rc.conf.
  The default pf.conf file is still quite permissive, but if that file does not load (and again, it probably won't without modifications!) a very restrictive rule set will be activated in place of the one in your pf.conf file, which will permit nothing other than SSH to the system. This upgrade process will activate pf by default. If you don't wish this to happen, add "pf=NO" to your /etc/rc.conf.local
- Packet normalization ("scrubbing") has been integrated with the main ruleset.
  This is now specified by adding the "scrub (options)" parameter to rules. For example:
```
scrub in all no-df max-mss 1440
```
  can be replaced with a rule using the new "match" action:
```
match in all scrub (no-df max-mss 1440)
```
- Packet reassembly has been simplified.
  "Crop" and "drop-ovl" have been removed; only full reassembly remains. It is now on by default, manually controlled like this:
```
set reassemble yes|no [no-df]
```
  If "no-df" is given, fragments (and only fragments!) with the df bit set have it cleared before entering the fragment cache, and thus the reassembled packet doesn't have df set either. Non-fragmented packets are unaffected by no-df.
- pfctl(8) was changed so that "set require-order" is now disabled by default.
  This is advisory only, no configuration change is necessary.
Changes to daily(8), weekly, monthly, and security(8) scripts
When updating your periodic maintenance scripts to the latest versions, pay attention to the following changes:
- The scripts daily, weekly, and monthly now all use the PATH variable from the root crontab(5), which is /bin:/sbin:/usr/bin:/usr/sbin by default. The daily(8) PATH no longer includes /usr/local/bin, the weekly(8) PATH no longer includes /usr/libexec. In daily.local, weekly.local and monthly.local, you can still add additional directories to the PATH.
- All four scripts now suppress section headers when there is no content to follow. When a script produces no output whatsoever, it does not send mail to root any more. This may require adjustment of your parser scripts.
- The security(8) script now uses some shell functions from daily(8), so it cannot run stand-alone. Use "sudo /bin/sh /etc/daily" in case you need to run it manually.
tmux(1) has moved from being a port to being part of the base system.
For most users, tmux(1) is a functional replacement for the commonly installed package, "screen". It also provides the functionality of the rarely used window(1) command, which has been removed. Users of "screen" (and window) are highly encouraged to check it out, more here.
lynx(1) Has been updated to 2.8.6rel.5.
This upgrade process will overwrite your existing /etc/lynx.conf file, if you have local modifications to it, you will have to reapply them to the new file.
New file install/reboot sequence (non-install kernel process)
Previous upgrade processes used a process of install kernel, reboot, install userland. We now suggest a different process -- install everything, THEN reboot. This must be done very carefully, otherwise one could end up with a system where the reboot(1) command is inoperable. Understand and follow the directions below before doing your upgrade remotely
sysmerge(8) utility:
OpenBSD now includes the sysmerge(8) utility, which helps administrators update configuration files after upgrading their system. Sysmerge(8) compares the current files on your system with the files that would have been installed with a new install, and gives you the option of keeping the old file, installing the new file, or assisting you in the manual merging of the old and new files, using sdiff. For past upgrades, we've presented a list of files that are usually copied over "as-is", and a list of files which should be changed, and a patch file that applies those changes to what might be in those files on your system. You may opt to use sysmerge to make the changes, or you may wish to use the patch file first, and then follow up with a sysmerge session to clean up any loose ends.
Who should use sysmerge(8):
People running highly modified systems or systems that didn't start out at the previous release (for example, a snapshot partway between releases), who are upgrading to a snapshot or who have not carefully upgraded their system in the past will find sysmerge vastly superior to using the patches, as it works with what is actually on your system, instead of what we expected was on your system. It will also give you much greater control over your upgrade process, and will involve you in it more closely.
Who may wish to NOT use sysmerge(8):
People who have a lot of machines to upgrade that were kept fairly simple and at the previous release/stable point will probably find the old patch file system much faster.
Note that while sysmerge can handle ALL the changes of /etc, /dev, /root and /var, we highly recommend that you do some steps manually before hand, as it will save time and reduce the possibility of user error. In particular, it is highly recommended that you do not use sysmerge to update your user and group accounts, as it is very easy to chose the wrong option leading to erasing your entire user base and setting the root password to an empty value, preventing remote login to fix the problem.
rc.conf:
It is assumed that /etc/rc.conf is not a user-altered file. If you have made changes to your /etc/rc.conf file, merge those changes into /etc/rc.conf.local. If you have NO /etc/rc.conf.local, simply copy your existing /etc/rc.conf file to /etc/rc.conf.local and delete the last line of the script! Otherwise, pull your existing rc.conf into the top of your existing rc.conf.local file and remove the last line before doing the rest of this process.
Modified kernel:
Check whether you have made any modifications to your kernel. For example, you might have modified your network device to use a non-default setting using config(8). Note your changes, so you can repeat them for the new 4.6 kernel.

The upgrade process

Upgrading by install kernel

If you have access to the system's console, the easiest and safest way to upgrade is to boot from install media or bsd.rd and follow the upgrade steps, which are very similar to the install process.

Afterwards, complete the upgrade by following the final steps as detailed below.

One easy way to boot from the install kernel is to place the 4.6 version of bsd.rd in the root of your boot drive, then instruct the boot loader to boot using this new bsd.rd file. On amd64 and i386, you do this by entering "boot bsd.rd" at the initial boot> prompt.

Upgrading without install kernel

This is NOT the recommended process. Use the install kernel method if at all possible!

Sometimes, one needs to do an upgrade of a machine when one can't easily use the normal upgrade process. The most common case is when the machine is in a remote location and you don't have easy access to the system console. One can usually do this by carefully following this process:

Place install files in a "good" location. Make sure you have sufficient space!
Stop any appropriate applications: During this process, all the userland applications will be replaced but may not be runnable, and strange things may happen as a result. If this is a concern to you, shut down any applications that may be impacted. There may be other applications which you wish to keep from running immediately after the upgrade, stop and disable them as well.
Check the kernel: Although most people can skip this step, if you had a modified kernel in 4.5, it is likely you will need to modify the stock kernel of 4.6. Especially when you are performing the upgrade process remotely, now is the time to make sure the new kernel will work upon rebooting the machine. If any changes must be made to the kernel, the safest thing to do is to make those changes on a local 4.6 system. This can be as simple as modifying a specific device using config(8), or it can involve a recompilation if the option you need is not included in the GENERIC kernel. Please consult FAQ 5 - Building the system from source before deciding to recompile your kernel.
OpenBSD/sparc users only:
The kernel has grown enough that the sparc boot loader has to be updated with the new 4.6 boot loader before the system is rebooted, using installboot(8). These steps should be used at this point to install the new boot code:
```
export RELEASEPATH=/usr/rel   # where you put the files
tar -C / -xzphf ${RELEASEPATH}/base46.tgz ./usr/mdec
cp /usr/mdec/boot /
/usr/mdec/installboot -v /boot /usr/mdec/bootxx /dev/rsd0c
```
Note: there is no need to do this if running the install kernel, this only is needed for upgrading a running system remotely.
Install new kernel(s)
```
export RELEASEPATH=/usr/rel   # where you put the files
cd ${RELEASEPATH}
rm /obsd ; ln /bsd /obsd && cp bsd /nbsd && mv /nbsd /bsd
cp bsd.rd bsd.mp /
```
(if you are using the multiprocessor kernel, follow up with
cp /bsd /bsd.sp ; mv /bsd.mp /bsd
afterwards)
Note the extra steps for copying over the primary kernel: those are done to ensure that there is always a valid copy of the kernel on the disk that the system can boot from should there be a really badly timed power outage or system crash.
Save yourself a copy of the old reboot(1) command: You are still running the old kernel, it is possible the new reboot command will not run on the old kernel.
```
cp /sbin/reboot /sbin/oreboot
```
Install new userland applications. Do NOT install etc46.tgz and xetc46.tgz now, because that will overwrite your current configuration files! Note that we are installing base46.tgz LAST, because it will include a new tar(1) utility, which may or may not run on the old kernel. We reboot immediately, as the system is probably barely runnable now.
```
tar -C / -xzphf xserv46.tgz
tar -C / -xzphf xfont46.tgz
tar -C / -xzphf xshare46.tgz
tar -C / -xzphf xbase46.tgz
tar -C / -xzphf game46.tgz
tar -C / -xzphf misc46.tgz
tar -C / -xzphf comp46.tgz
tar -C / -xzphf man46.tgz
tar -C / -xzphf base46.tgz # Install last!
/sbin/oreboot
```
Not all file sets will need to be installed for all applications, however if you installed a file set originally, you should certainly upgrade it with the new file set now.
Again, the files in /etc are handled separately below, so etc46.tgz and xetc46.tgz are NOT unpacked here.
After reboot completes, upgrade /dev. The new MAKEDEV file was copied to /dev by the installation of base46.tgz, so you simply need to do the following:
```
cd /dev
./MAKEDEV all
```

Final steps

Whether you upgrade by using an install kernel and doing a formal "upgrade" process, or do a "in-place" binary upgrade, there are certain manual steps that have to be performed.

1. New Users and Groups

Users need to be added for smtpd(8) and rpc.rwalld(8):

useradd -u 95 -g =uid -c "SMTP Daemon" -d /var/empty -s /sbin/nologin _smtpd
useradd -u 96 -g =uid -c "rpc.rwalld" -d /var/empty -s /sbin/nologin _rwalld

2. Upgrading `/etc`

You will want to extract the etc46.tgz files to a temporary location:

export RELEASEPATH=/usr/rel
tar -C /tmp -xzphf ${RELEASEPATH}/etc46.tgz

Files that can probably be copied from etc46.tgz "as is":

etc/daily
etc/weekly
etc/monthly
etc/lynx.cfg
etc/rc
etc/rc.conf
etc/security
etc/skel/.Xdefaults
etc/mail/Makefile
etc/mail/smtpd.conf
etc/mailer.conf
etc/mtree/4.4BSD.dist
etc/mtree/special
root/.Xdefaults

Note that it IS possible to locally modify these files, if this has been done, do NOT copy over those files, and use the sysmerge(8) process instead. Here are copy/paste lines for copying these files, assuming you unpacked etc46.tgz in the above recommended place:

cd /tmp/etc 
cp daily weekly monthly lynx.cfg mailer.conf rc rc.conf security /etc
cp skel/.Xdefaults /etc/skel
cp mtree/4.4BSD.dist mtree/special /etc/mtree
cp ../root/.Xdefaults /root
cp mail/Makefile mail/smtpd.conf /etc/mail

3a. Merging locally changed files via a patch file

These files likely have local changes, but should be updated for 4.6. IF you have not altered these files, you can copy over the new version, otherwise the changes should be merged with your files:

etc/changelist
etc/ftpusers
etc/login.conf
etc/sudoers
etc/sysctl.conf
etc/mail/aliases
etc/skel/.login
etc/skel/.mailrc
root/.login
root/.profile
var/cron/tabs/root
var/www/conf/httpd.conf

The changes to these files are in this patch file. You can attempt to use this by executing the following as root:

cd /
patch -C -p0 < upgrade46.patch

This will test the patch to see how well it will apply to YOUR system; to actually apply it, leave off the "-C" option. Note that it is likely that if you have customized files or not kept them closely updated, or are upgrading from a snapshot of 4.5, they may not accept the patch cleanly. In those cases, you will need to manually apply the changes. Please test this process before relying on it for a machine you can not easily get to.

The following files have had changes which should be looked at, but it is unlikely they should be directly copied or merged (i.e., if you are using pf.conf, look at the suggested change of strategy, and decide if it is appropriate for your use).

etc/ntpd.conf
etc/pf.conf

Finally, use newaliases(8) to update the aliases database, mtree(8) create any new directories:

newaliases
mtree -qdef /etc/mtree/4.4BSD.dist -p / -u

3b. Merging locally changed files via sysmerge(8)

The new sysmerge(8) utility will compare the files that are actually on your system with those that would be installed to a fresh install, and assist you in merging the changes into your system. Note that unlike the patch file, there are no assumptions made about what is actually on your system, so you can use sysmerge(8) to move between more arbitrary points in the development process, such as from an earlier -current to 4.6-release or from one -current to a later one.

Please read the sysmerge(8) manual page before using it on your system. You are also advised to read the diff(1), sdiff(1) and even review more(1) manual pages before continuing.

Assuming the etc46.tgz and xetc46.tgz files exists in your $RELEASEPATH, run it with:

sysmerge -as $RELEASEPATH/etc46.tgz -x $RELEASEPATH/xetc46.tgz

Sysmerge(8) will show you a unified diff(1), run through your favorite $PAGER (i.e., more(1)) and ask you, for most changed files, if you wish to:

  Use 'd' to delete the temporary ./var/www/htdocs/index.html
  Use 'i' to install the temporary ./var/www/htdocs/index.html
  Use 'm' to merge the temporary and installed versions
  Use 'v' to view the diff results again

  Default is to leave the temporary file to deal with by hand

If you wish to retain your existing file, delete the temporary file, if you wish to replace your existing file with the new version, install the temporary file. If you wish to merge the two together, choosing 'm' will put you into sdiff(1), where you can manually merge the file. The default is to come back and deal with the file later, manually.

While it can work, we do not recommend you use sysmerge to integrate new users into the system, but rather use the useradd(8) line above. We believe it is much less error prone. (hint: do not install the temporary master.passwd file over your existing one!).

Sysmerge(8) saves all your replaced files into a temporary directory, similar to /var/tmp/sysmerge.24959/backups, so if you accidentally clobber something that was probably not such a good idea, you have a chance to recover it. Note that daily(8) cleans old files from this directory.

4. Checking the kernel

Note: most people can skip this step!

If you followed the instructions for the upgrade process without install kernel, you have already completed this step. However, if you used the install kernel, and if you had a modified kernel in 4.5, it is likely you will need to modify the stock kernel of 4.6. This can be as simple as modifying a specific device using config(8), or it can involve a recompilation if the option you need is not included in the GENERIC kernel. Please consult FAQ 5 - Building the system from source before considering to recompile your kernel.

5. Upgrading packages

If you installed any packages on your system, you should upgrade them after completing the upgrade of the base system. Be aware, however, many packages will require further setup before and/or after upgrading the package. Check with the application's upgrade guide for details.

The following packages are known to have significant upgrade issues that will impact a large number of users. The fact that a package is not on this list doesn't mean it will have a trivial upgrade. You must do some homework on the applications YOU use.

py-Django: Django 1.0 breaks compatibility with 0.96 in some areas, as described here.
kqemu: Must be kept in sync with the kernel. pkg_delete prior to the upgrade, then pkg_add the new package once the upgrade has been completed.

The package tools support in-place updating using pkg_add -u. For instance, to update all your packages, make sure PKG_PATH is pointing to the 4.6 packages directory on your CD or nearest FTP mirror, and use something like

pkg_add -ui -F update -F updatedepends

where the -u indicates update mode, and -i specifies interactive mode, so pkg_add will prompt you for input when it encounters some ambiguity. Read the pkg_add(1) manual page and the package management chapter of the FAQ for more information.