Building FreeBSD for Kirkwood

Prepare a drive
Obtain the source
Patching
Build world
Build the Kernel
Install the kernel
Install world
Configure some niceties

Prepare a drive

I’m using a slow, generic 2GB flash drive, but anything with at least a GiB or so of space should do.

Delete any existing partitions.

gpart delete -i 1 da0
gpart destroy da0

Create a 32MiB FAT partition and fill the rest of the drive with a FreeBSD slice.

gpart create -s MBR da0
gpart add -s 32M -t freebsd da0
gpart add -t freebsd da0

Format the FAT partition.

newfs_msdos /dev/da0s1

Label the UFS slice, create the filesystem, and add a filesystem volume label so the kernel can find it later.

bsdlabel -w /dev/da0s2
newfs -n da0s2a
tunefs -L kirkwoodroot /dev/da0s2a

Obtain the source

Use csup to grab the source for FreeBSD if you don’t already have it. Check here for the available release tags.

*default host=cvsup8.freebsd.org
*default base=/var/db
*default prefix=/usr
*default release=cvs tag=RELENG_8_1
*default delete use-rel-suffix
*default compress src-all

Patching

FreeBSD requires a few patches to run comfortably and properly on these devices. The patches are available from my FreeBSD-Kirkwood repo on GitHub.

A patch to vfs_mount.c is only required until [pr]usb/138798[/pr] is fixed. Until then, we need to insert a delay before mounting the root filesystem, since our USB drive might not be enumerated.

--- /usr/src/sys/kern/vfs_mount.c.orig 2010-07-04 22:50:00.613726077 -0400
+++ /usr/src/sys/kern/vfs_mount.c 2010-07-05 12:11:09.986561693 -0400
@@ -1651,6 +1651,9 @@
  int error, i, asked = 0;

  options = NULL;
+
+ /* NASTY HACK: wait for USB sticks to appear */
+ pause("usbhack", hz * 10);

  root_mount_prepare();

This patch, shamelessly stolen from the mailing list, will give us the option to force fsck on the root filesystem. Otherwise, an unclean shutdown would lock us out of FreeBSD unless we had serial console access.

--- /usr/src/etc/rc.d/fsck.orig	2010-07-07 13:02:41.765255856 -0400
+++ /usr/src/etc/rc.d/fsck	2010-07-07 13:02:46.286575144 -0400
@@ -27,7 +27,16 @@
 		if checkyesno background_fsck; then
 			fsck -F -p
 		else
-			fsck -p
+			if checkyesno force_fsck; then
+				echo "Force fsck enabled"
+				for filesystem in ${force_fsck_list}
+				do
+					echo "Force check $filesystem"
+					fsck -y $filesystem
+				done
+			else
+				fsck -p
+			fi
 		fi

 		case $? in

Patch master.passwd to give root a default password of “root”. There’s probably a better way to do this.

--- /usr/src/etc/master.passwd.orig	2010-06-13 22:09:06.000000000 -0400
+++ /usr/src/etc/master.passwd	2010-07-08 02:27:52.148993197 -0400
@@ -1,6 +1,6 @@
 # $FreeBSD: src/etc/master.passwd,v 1.40.22.1.4.1 2010/06/14 02:09:06 kensmith Exp $
 #
-root::0:0::0:0:Charlie &:/root:/bin/csh
+root:$1$jr4u13xy$GivPnOZ5kslfsJL4LmLK41:0:0::0:0:Charlie &:/root:/bin/csh
 toor:*:0:0::0:0:Bourne-again Superuser:/root:
 daemon:*:1:1::0:0:Owner of many system processes:/root:/usr/sbin/nologin
 operator:*:2:5::0:0:System &:/:/usr/sbin/nologin

Patch atomic.h to make BIND in the base system not segfault, stolen from this mailing list thread.

--- /usr/src/contrib/bind9/lib/isc/arm/include/isc/atomic.h.orig	2010-08-04 02:02:01.194401084 -0400
+++ /usr/src/contrib/bind9/lib/isc/arm/include/isc/atomic.h	2010-08-04 02:04:53.462379414 -0400
@@ -49,26 +49,22 @@
 static inline isc_int32_t
 isc_atomic_cmpxchg(isc_int32_t *p, isc_int32_t cmpval, isc_int32_t val)
 {
-	register int done, ras_start;
+	register int done, ras_start  = 0xffff1004;

 	__asm __volatile("1:\n"
 	    "adr	%1, 1b\n"
-	    "mov	%0, #0xe0000004\n"
 	    "str	%1, [%0]\n"
-	    "mov	%0, #0xe0000008\n"
 	    "adr	%1, 2f\n"
-	    "str	%1, [%0]\n"
+	    "str	%1, [%0, #4]\n"
 	    "ldr	%1, [%2]\n"
 	    "cmp	%1, %3\n"
 	    "streq	%4, [%2]\n"
 	    "2:\n"
 	    "mov	%3, #0\n"
-	    "mov	%0, #0xe0000004\n"
 	    "str	%3, [%0]\n"
 	    "mov	%3, #0xffffffff\n"
-	    "mov	%0, #0xe0000008\n"
-	    "str	%3, [%0]\n"
-	    : "=r" (ras_start), "=r" (done)
+	    "str	%3, [%0, #4]\n"
+	    : "+r" (ras_start), "=r" (done)
 	    ,"+r" (p), "+r" (cmpval), "+r" (val) : : "memory");
 	return (done);

Patch config.h to fix liblzma on ARM (as reported by Crest!)

--- /usr/src/lib/liblzma/config.h.orig	2010-09-24 13:42:10.346344850 -0400
+++ /usr/src/lib/liblzma/config.h	2010-09-24 13:44:29.568683203 -0400
@@ -66,7 +66,7 @@
 #define SIZEOF_SIZE_T 8
 #define STDC_HEADERS 1
 #define TUKLIB_CPUCORES_SYSCONF 1
-#define TUKLIB_FAST_UNALIGNED_ACCESS 1
+/* #define TUKLIB_FAST_UNALIGNED_ACCESS 1 */
 #define TUKLIB_PHYSMEM_SYSCONF 1
 #ifndef _ALL_SOURCE
 # define _ALL_SOURCE 1

This patch disables a warning about unsupported TCP offload for IPv6 packets in if_mge.

--- if_mge.c.orig	2011-05-11 14:27:51.692628521 -0400
+++ if_mge.c	2011-05-11 14:28:20.950873599 -0400
@@ -1607,9 +1607,9 @@
 		}
 
 		if (etype != ETHERTYPE_IP) {
-			if_printf(sc->ifp,
+			/*if_printf(sc->ifp,
 			    "TCP/IP Offload enabled for unsupported "
-			    "protocol!\n");
+			    "protocol!\n");*/
 			return;
 		}

Patch conf/files to allow us to build runfw into the kernel for if_run wireless controllers. Normally it’s only loadable as a module via loader, but the way we load the kernel bypasses that.

--- /usr/src/sys/conf/files.orig	2011-01-02 17:33:24.330796637 -0500
+++ /usr/src/sys/conf/files	2011-01-02 17:33:52.730749215 -0500
@@ -1759,6 +1759,22 @@
 #
 dev/usb/wlan/if_rum.c		optional rum
 dev/usb/wlan/if_run.c		optional run
+
+runfw.c				optional runfw				\
+	compile-with	"${AWK} -f $S/tools/fw_stub.awk runfw:runfw -mrunfw -c${.TARGET}"	\
+	no-implicit-rule before-depend local					\
+	clean		"runfw.c"
+runfw.fwo			optional runfw					\
+	dependency	"runfw"							\
+	compile-with	"${LD} -b binary -d -warn-common -r -d -o ${.TARGET} runfw"		\
+	no-implicit-rule							\
+	clean		"runfw.fwo"
+runfw				optional runfw					\
+	dependency	"$S/contrib/dev/run/rt2870.fw.uu"			\
+	compile-with	"uudecode -o ${.TARGET} $S/contrib/dev/run/rt2870.fw.uu"\
+	no-obj no-implicit-rule							\
+	clean		"runfw"
+
 dev/usb/wlan/if_uath.c		optional uath
 dev/usb/wlan/if_upgt.c		optional upgt
 dev/usb/wlan/if_ural.c		optional ural

Finally, if you’re using RELENG_8_2, pull in post-8.2 revision 1.12.2.19 of if_axe.c and revision 1.3.2.5 of if_axereg.h to fix a frequent problem where if_axe usb ethernet controllers would stop working until re-initialized. Discussion in this thread and usb/140883.

Build world

Build world for ARM, but don’t install anything.

make -j 8 buildworld TARGET_ARCH=arm

Build the Kernel

My kernel configuration file is much more fully-featured than the stock SHEEVAPLUG config file. It includes support for GEOM, filesystems, USB ethernet controllers, USB audio hardware, and PF/ALTQ.

#
# Custom kernel for DockStar devices.
# http://cooltrainer.org 20100705
#
# $FreeBSD: src/sys/arm/conf/DOCKSTAR,v 1.1.2.3.2.1 2010/06/14 02:09:06 kensmith Exp $
#

ident		DOCKSTAR-COOLTRAINER
include		"../mv/kirkwood/std.sheevaplug"

options 	SOC_MV_KIRKWOOD
makeoptions	MODULES_OVERRIDE=""

#DockStar has 128MiB of memory, not 512 like the reference design
#(512 is defined in std.sheevaplug and should be commented out)
options		PHYSMEM_SIZE=0x8000000

#makeoptions	DEBUG=-g		#Build kernel with gdb(1) debug symbols
makeoptions	WERROR="-Werror"

options 	SCHED_4BSD		#4BSD scheduler
options 	INET			#InterNETworking
options 	INET6			#IPv6 communications protocols
options 	FFS			#Berkeley Fast Filesystem
options 	NFSCLIENT		#Network Filesystem Client
options 	NFSLOCKD		#Network Lock Manager
options 	NFS_ROOT		#NFS usable as /, requires NFSCLIENT
#options 	BOOTP
#options 	BOOTP_NFSROOT
#options 	BOOTP_NFSV3
#options 	BOOTP_WIRED_TO=mge0

options 	GEOM_PART_BSD
options 	GEOM_PART_GPT
options 	GEOM_PART_MBR
options 	GEOM_LABEL	# Provides labelization

options 	SOFTUPDATES		# Enable FFS soft updates support
options 	MSDOSFS			# MSDOS Filesystem
options 	PROCFS		# Process filesystem (/proc)
options 	PSEUDOFS		# Pseudo-filesystem framework

# Root fs on USB device
options 	ROOTDEVNAME=\"ufs:/dev/ufs/root\"

options 	SYSVSHM			#SYSV-style shared memory
options 	SYSVMSG			#SYSV-style message queues
options 	SYSVSEM			#SYSV-style semaphores
options 	_KPOSIX_PRIORITY_SCHEDULING #Posix P1003_1B real-time extensions
options 	MUTEX_NOINLINE
options 	RWLOCK_NOINLINE
options 	NO_FFS_SNAPSHOT
options 	NO_SWAPPING

# Debugging
#options 	ALT_BREAK_TO_DEBUGGER
#options 	DDB
#options 	KDB

# Pseudo devices
device		random
device		pty
device		loop
device		firmware	# firmware assist module (for wlan)

# Serial ports
device		uart

# Networking
device		ether
device		mge			# Marvell Gigabit Ethernet controller
device		mii
device		e1000phy
device		bpf
options		HZ=1000
options		DEVICE_POLLING
device		vlan

# USB
options 	USB_DEBUG	# enable debug msgs
device		usb
device		uhci
device		ohci
device		ehci
device		umass
device		scbus
device		pass
device		da

#USB Ethernet
device		miibus
device		aue			# ADMtek USB Ethernet
device		axe			# ASIX Electronics USB Ethernet
device		cdce		# Generic USB over Ethernet
device		cue			# CATC USB Ethernet
device		kue			# Kawasaki LSI USB Ethernet
device		rue			# RealTek RTL8150 USB Ethernet
device		udav		# Davicom DM9601E USB

#OpenBSD Packet Filter
device		pf
device		pflog
device		pfsync

#ALTQ (QoS)
options		ALTQ
options		ALTQ_CBQ	# Class Bases Queuing (CBQ)
options		ALTQ_RED	# Random Early Detection (RED)
options		ALTQ_RIO	# RED In/Out
options		ALTQ_HFSC	# Hierarchical Packet Scheduler (HFSC)
options		ALTQ_PRIQ	# Priority Queuing (PRIQ)
#options	ALTQ_NOPCC	# Required for SMP build

#Swap space (Yes, just in case)
device		md			# Memory "disks"

#USB audio
device		sound
device		snd_uaudio

#USB Wireless
# Wireless NIC cards
device		wlan		# 802.11 support
options 	IEEE80211_DEBUG	# enable debug msgs
options 	IEEE80211_AMPDU_AGE # age frames in AMPDU reorder q's
options 	IEEE80211_SUPPORT_MESH	# enable 802.11s draft support
device		wlan_wep	# 802.11 WEP support
device		wlan_ccmp	# 802.11 CCMP support
device		wlan_tkip	# 802.11 TKIP support
device		rum		# Ralink Technology RT2501USB wireless NICs
device		uath		# Atheros AR5523 wireless NICs
device		ural		# Ralink Technology RT2500USB wireless NICs
device		zyd		# ZyDAS zb1211/zb1211b wireless NICs
device		urtw	# Realtek RTL8187B/L USB
device		upgt	# Conexant/Intersil PrismGT SoftMAC USB

Comment out PHYSMEM_SIZE in /usr/src/sys/arm/mv/kirkwood/std.sheevaplug, or your kernel might only boot as far as the copyright declaration on the DockStar.

Then build the kernel.

make buildkernel TARGET_ARCH=arm KERNCONF=DOCKSTAR-COOLTRAINER

Install the kernel

mount -t msdosfs /dev/da0s1 /root/usb
cp /usr/obj/arm/usr/src/sys/DOCKSTAR-COOLTRAINER/kernel.bin /root/usb
umount /root/usb

Install world

mount /dev/da0s2a /root/usb
export DESTDIR=/root/usb
make installworld distrib-dirs distribution TARGET_ARCH=arm

Configure some niceties

Set the hostname, turn on DHCP, and enable SSHd.

echo 'hostname="pochan"' > $DESTDIR/etc/rc.conf
echo 'ifconfig_mge0="DHCP"' >> $DESTDIR/etc/rc.conf
echo 'sshd_enable="YES"' >> $DESTDIR/etc/rc.conf

Enable automatic fsck, so we can get back into the OS in the event of power loss or an unclean shutdown.

echo 'fsck_y_enable="YES"' >> $DESTDIR/etc/rc.conf
echo 'background_fsck="NO"' >> $DESTDIR/etc/rc.conf
echo 'force_fsck="YES"' >> $DESTDIR/etc/rc.conf
echo 'force_fsck_list="/"' >> $DESTDIR/etc/rc.conf

Turn on NTPd. The DockStar has no real time clock, so this is required to set the time and date on boot. The -g flag makes NTPd ignore the first huge jump in time. If you prefer, OpenBSD’s openntpd is available through Ports.

echo 'ntpd_enable="YES"' >> $DESTDIR/etc/rc.conf
echo 'ntpd_sync_on_start="YES"' >> $DESTDIR/etc/rc.conf

Set a better motd ( :3 ) and tell Ports not to enable some ports’ X11 dependencies.

echo 'Heya cutie <3' > $DESTDIR/etc/motd
echo 'WITHOUT_X11=yes' >> $DESTDIR/etc/make.conf

Set some mount options for our root filesystem. Disabling clustered reads and writes is reccommended on ARM.

echo '# Device Mountpoint FStype Options Dump Pass#' > $DESTDIR/etc/fstab
echo '/dev/ufs/kirkwoodroot / ufs rw,noclusterr,noclusterw 0 0' >> $DESTDIR/etc/fstab

Add your device’s hostname to the IPv4 and IPv6 loopback addresses in /etc/hosts. This example uses “pochan”.

/etc/hosts

::1 localhost localhost.my.domain pochan
127.0.0.1 localhost localhost.my.domain pochan

Finally, edit $DESTDIR/etc/ssh/sshd_config and change PermitRootLogin to yes, since there will be no user accounts on first boot.