Building FreeBSD for Kirkwood

• Prepare a drive
• Obtain the source
• Patching
• Build world
• Build the Kernel
• Padding the kernel
• Install world
• Configure some niceties

Prepare a drive

I’m using a slow, generic 8GB flash drive, but anything with at least a GiB or so of space should do.

Delete any existing partitions.

[root@Emi#/root]gpart delete -i 1 da0
da0s1 deleted
[root@Emi#/root]gpart destroy da0
da0 destroyed

Then create a 32MiB FAT partition and fill the rest of the drive with a FreeBSD slice.

[root@Emi#/root]gpart create -s MBR da0
da0 created
[root@Emi#/root]gpart add -s 32M -t freebsd da0
da0s1 added
[root@Emi#/root]gpart add -t freebsd da0
da0s2 added

Format the FAT partition.

[root@Emi#/root]newfs_msdos /dev/da0s1
/dev/da0s1: 65416 sectors in 8177 FAT16 clusters (4096 bytes/cluster)
BytesPerSec=512 SecPerClust=8 ResSectors=1 FATs=2 RootDirEnts=512 Sectors=65520 Media=0xf0 FATsecs=32 SecPerTrack=63 Heads=255 HiddenSecs=0

And the UFS slice.

[root@Emi#/root]bsdlabel -w /dev/da0s2
[root@Emi#/root]newfs -n da0s2a
/dev/da0s2a: 7680.0MB (15728564 sectors) block size 16384, fragment size 2048
	using 42 cylinder groups of 183.72MB, 11758 blks, 23552 inodes.
super-block backups (for fsck -b #) at:
 160, 376416, 752672, 1128928, 1505184, 1881440, 2257696, 2633952, 3010208,
 3386464, 3762720, 4138976, 4515232, 4891488, 5267744, 5644000, 6020256,
 6396512, 6772768, 7149024, 7525280, 7901536, 8277792, 8654048, 9030304,
 9406560, 9782816, 10159072, 10535328, 10911584, 11287840, 11664096, 12040352,
 12416608, 12792864, 13169120, 13545376, 13921632, 14297888, 14674144,
 15050400, 15426656

The DockStar’s crippled bootloader can’t read FAT or use USB drives, but creating a FAT partition gives us an alternate storage location for the kernel and the ability to use fatload on boards with a fully-capable U-boot.

Obtain the source

Use csup to grab the source for FreeBSD if you don’t already have it. I used FreeBSD 8.1 (RELENG_8_1), but there may be something newer.

*default host=cvsup8.freebsd.org
*default base=/var/db
*default prefix=/usr
*default release=cvs tag=RELENG_8_1
*default delete use-rel-suffix
*default compress
src-all

Patching

This patch is only required until usb/138798 is fixed. Until then, we need to insert a delay before mounting the root filesystem, since our USB drive might not be enumerated.

--- /usr/src/sys/kern/vfs_mount.c.orig	2010-07-04 22:50:00.613726077 -0400
+++ /usr/src/sys/kern/vfs_mount.c	2010-07-05 12:11:09.986561693 -0400
@@ -1651,6 +1651,9 @@
 	int error, i, asked = 0;

 	options = NULL;
+
+	/* NASTY HACK: wait for USB sticks to appear */
+	pause("usbhack", hz * 10);

 	root_mount_prepare();

This patch, shamelessly stolen from the mailing list, will give us the option to force fsck on the root filesystem. Otherwise, an unclean shutdown would lock us out of FreeBSD unless we had serial console access.

--- /usr/src/etc/rc.d/fsck.orig	2010-07-07 13:02:41.765255856 -0400
+++ /usr/src/etc/rc.d/fsck	2010-07-07 13:02:46.286575144 -0400
@@ -27,7 +27,16 @@
 		if checkyesno background_fsck; then
 			fsck -F -p
 		else
-			fsck -p
+			if checkyesno force_fsck; then
+				echo "Force fsck enabled"
+				for filesystem in ${force_fsck_list}
+				do
+					echo "Force check $filesystem"
+					fsck -y $filesystem
+				done
+			else
+				fsck -p
+			fi
 		fi

 		case $? in

Patch master.passwd to give root a default password of “root”. There’s probably a better way to do this.

--- /usr/src/etc/master.passwd	2010-06-13 22:09:06.000000000 -0400
+++ /home/nicole/master.passwd	2010-07-08 02:27:52.148993197 -0400
@@ -1,6 +1,6 @@
 # $FreeBSD: src/etc/master.passwd,v 1.40.22.1.4.1 2010/06/14 02:09:06 kensmith Exp $
 #
-root::0:0::0:0:Charlie &:/root:/bin/csh
+root:$1$jr4u13xy$GivPnOZ5kslfsJL4LmLK41:0:0::0:0:Charlie &:/root:/bin/csh
 toor:*:0:0::0:0:Bourne-again Superuser:/root:
 daemon:*:1:1::0:0:Owner of many system processes:/root:/usr/sbin/nologin
 operator:*:2:5::0:0:System &:/:/usr/sbin/nologin

Patch atomic.h to make BIND in the base system not segfault. (Stolen from this mailing list thread.)

--- /usr/src/contrib/bind9/lib/isc/arm/include/isc/atomic.h.orig	2010-08-04 02:02:01.194401084 -0400
+++ /usr/src/contrib/bind9/lib/isc/arm/include/isc/atomic.h	2010-08-04 02:04:53.462379414 -0400
@@ -49,26 +49,22 @@
 static inline isc_int32_t
 isc_atomic_cmpxchg(isc_int32_t *p, isc_int32_t cmpval, isc_int32_t val)
 {
-	register int done, ras_start;
+	register int done, ras_start  = 0xffff1004;

 	__asm __volatile("1:\n"
 	    "adr	%1, 1b\n"
-	    "mov	%0, #0xe0000004\n"
 	    "str	%1, [%0]\n"
-	    "mov	%0, #0xe0000008\n"
 	    "adr	%1, 2f\n"
-	    "str	%1, [%0]\n"
+	    "str	%1, [%0, #4]\n"
 	    "ldr	%1, [%2]\n"
 	    "cmp	%1, %3\n"
 	    "streq	%4, [%2]\n"
 	    "2:\n"
 	    "mov	%3, #0\n"
-	    "mov	%0, #0xe0000004\n"
 	    "str	%3, [%0]\n"
 	    "mov	%3, #0xffffffff\n"
-	    "mov	%0, #0xe0000008\n"
-	    "str	%3, [%0]\n"
-	    : "=r" (ras_start), "=r" (done)
+	    "str	%3, [%0, #4]\n"
+	    : "+r" (ras_start), "=r" (done)
 	    ,"+r" (p), "+r" (cmpval), "+r" (val) : : "memory");
 	return (done);

Build world

Build world for ARM, but don’t install anything.

[root@Emi#/usr/src]make -j 8 buildworld TARGET_ARCH=arm

Build the Kernel

My kernel configuration file is much more fully-featured than the stock SHEEVAPLUG config file. It includes support for GEOM, filesystems, USB ethernet controllers, USB audio hardware, and pf/ALTQ.

#
# Custom kernel for DockStar devices.
# http://cooltrainer.org 20100705
#
# $FreeBSD: src/sys/arm/conf/DOCKSTAR,v 1.1.2.3.2.1 2010/06/14 02:09:06 kensmith Exp $
#

ident		DOCKSTAR
include		"../mv/kirkwood/std.sheevaplug"

options 	SOC_MV_KIRKWOOD
makeoptions	MODULES_OVERRIDE=""

#DockStar has 128MiB of memory, not 512 like the reference design
#(512 is defined in std.sheevaplug and should be commented out)
options		PHYSMEM_SIZE=0x8000000

#makeoptions	DEBUG=-g		#Build kernel with gdb(1) debug symbols
makeoptions	WERROR="-Werror"

options 	SCHED_4BSD		#4BSD scheduler
options 	INET			#InterNETworking
options 	INET6			#IPv6 communications protocols
options 	FFS			#Berkeley Fast Filesystem
options 	NFSCLIENT		#Network Filesystem Client
options 	NFSLOCKD		#Network Lock Manager
options 	NFS_ROOT		#NFS usable as /, requires NFSCLIENT
#options 	BOOTP
#options 	BOOTP_NFSROOT
#options 	BOOTP_NFSV3
#options 	BOOTP_WIRED_TO=mge0

options 	GEOM_PART_BSD
options 	GEOM_PART_GPT
options 	GEOM_PART_MBR
options 	GEOM_LABEL	# Provides labelization

options 	SOFTUPDATES		# Enable FFS soft updates support
options 	MSDOSFS			# MSDOS Filesystem
options 	PROCFS		# Process filesystem (/proc)
options 	PSEUDOFS		# Pseudo-filesystem framework

# Root fs on USB device
options 	ROOTDEVNAME=\"ufs:/dev/da0s2a\"

options 	SYSVSHM			#SYSV-style shared memory
options 	SYSVMSG			#SYSV-style message queues
options 	SYSVSEM			#SYSV-style semaphores
options 	_KPOSIX_PRIORITY_SCHEDULING #Posix P1003_1B real-time extensions
options 	MUTEX_NOINLINE
options 	RWLOCK_NOINLINE
options 	NO_FFS_SNAPSHOT
options 	NO_SWAPPING

# Debugging
#options 	ALT_BREAK_TO_DEBUGGER
#options 	DDB
#options 	KDB

# Pseudo devices
device		random
device		pty
device		loop

# Serial ports
device		uart

# Networking
device		ether
device		mge			# Marvell Gigabit Ethernet controller
device		mii
device		e1000phy
device		bpf
options		HZ=1000
options		DEVICE_POLLING
device		vlan

# USB
#options 	USB_DEBUG	# enable debug msgs
device		usb
device		ehci
device		umass
device		scbus
device		pass
device		da

#USB Ethernet
device		miibus
device		aue			# ADMtek USB Ethernet
device		axe			# ASIX Electronics USB Ethernet
device		cdce		# Generic USB over Ethernet
device		cue			# CATC USB Ethernet
device		kue			# Kawasaki LSI USB Ethernet
device		rue			# RealTek RTL8150 USB Ethernet
device		udav		# Davicom DM9601E USB

#OpenBSD Packet Filter
device		pf
device		pflog
device		pfsync

#ALTQ (QoS)
options		ALTQ
options		ALTQ_CBQ	# Class Bases Queuing (CBQ)
options		ALTQ_RED	# Random Early Detection (RED)
options		ALTQ_RIO	# RED In/Out
options		ALTQ_HFSC	# Hierarchical Packet Scheduler (HFSC)
options		ALTQ_PRIQ	# Priority Queuing (PRIQ)
#options	ALTQ_NOPCC	# Required for SMP build

#Swap space (Yes, just in case)
device		md			# Memory "disks"

#USB audio
device		sound
device		snd_uaudio

Comment out PHYSMEM_SIZE in /usr/src/sys/arm/mv/kirkwood/std.sheevaplug, or your kernel might only boot as far as the copyright declaration on the DockStar.

Then build the kernel.

[root@Emi#/usr/src]make buildkernel TARGET_ARCH=arm KERNCONF=DOCKSTAR

Padding the kernel

Copy the built kernel from /usr/obj/arm/usr/src/sys/DOCKSTAR/kernel.bin to your working directory, then run this shell script. It will pad the file out to a multiple of 4096 bytes to match the pages of NAND flash and will report the size of your padded kernel in hex for use later on.

nandwrite does have a pad option ( -p ), but it just gave me trouble.

#!/bin/sh
# Cooltrainer.org kernel padder - 20100704

if [ ! -r kernel.bin ];
then
	echo "kernel.bin not found."
	exit
fi

if ! type perl >/dev/null 2>&1; then
    echo "Perl was not found."
    exit
fi

cp kernel.bin kernel.bin.page

bytes=$(ls -lrt kernel.bin | awk '{print $5}')
fillbytes=$((4096 - ($bytes % 4096)))
echo "Padding kernel.bin to 4096 byte pages as kernel.bin.page"

while [ $fillbytes -gt 0 ]; do
	#printf "\xff" >> only seemed to work in ZSH
	echo "\xff" | perl -ne 's/([0-9a-f]{2})/print chr hex $1/gie' >> kernel.bin.page
	fillbytes=$(($fillbytes - 1))
done

hexcvt ()
{
	if [ -z "$1" ]
	then
		echo 0
		return
	fi

	echo ""$1" "16" o p" | dc
	return
}

paddedbytes=$(ls -lrt kernel.bin.page | awk '{print $5}')

echo -n "Padded kernel size: 0x" && hexcvt $paddedbytes

[root@Emi#/root]sh pad.sh
Padding kernel.bin to 4096 byte pages as kernel.bin.page
Padded kernel size: 30E000

Write down the size of your padded kernel. You’ll need it when configuring the bootloader.

Install world

[root@Emi#/root]mount /dev/da0s2a /root/usb
[root@Emi#/usr/src]export DESTDIR=/root/usb
[root@Emi#/usr/src]make installworld distrib-dirs distribution TARGET_ARCH=arm

Configure some niceties

Set the hostname, turn on DHCP, and enable SSHd.

[root@Emi#/root]echo 'hostname="pochan"' > $DESTDIR/etc/rc.conf
[root@Emi#/root]echo 'ifconfig_mge0="DHCP"' >> $DESTDIR/etc/rc.conf
[root@Emi#/root]echo 'sshd_enable="YES"' >> $DESTDIR/etc/rc.conf

Enable automatic fsck, so we can get back into the OS in the event of power loss or an unclean shutdown.

[root@Emi#/root]echo 'fsck_y_enable="YES"' >> $DESTDIR/etc/rc.conf
[root@Emi#/root]echo 'background_fsck="NO"' >> $DESTDIR/etc/rc.conf
[root@Emi#/root]echo 'force_fsck="YES"' >> $DESTDIR/etc/rc.conf
[root@Emi#/root]echo 'force_fsck_list="/"' >> $DESTDIR/etc/rc.conf

Turn on NTPd. The DockStar has no real time clock, so this is required to set the time and date on boot. The -g flag makes NTPd ignore the first huge jump in time.

[root@Emi#/root]echo 'ntpd_enable="YES"' >> $DESTDIR/etc/rc.conf
[root@Emi#/root]echo 'ntpd_flags="-g"' >> $DESTDIR/etc/rc.conf

Set a better motd ( :3 ) and tell Ports not to build X11.

[root@Emi#/root]echo 'Heya cutie <3' > $DESTDIR/etc/motd
[root@Emi#/root]echo 'WITHOUT_X11=yes' >> $DESTDIR/etc/make.conf

Set some mount options for our root filesystem. Disabling clustered reads and writes is Reccommended on ARM

[root@Emi#/root]echo '# Device	Mountpoint	FStype	Options					Dump	Pass#' > $DESTDIR/etc/fstab
[root@Emi#/root]echo '/dev/da0s2a	/			ufs		rw,noclusterr,noclusterw	0		0' >> $DESTDIR/etc/fstab

Add your device’s hostname to /etc/hosts. This example uses “pochan”.

pochan# ee $DESTDIR/etc/hosts
::1                     localhost localhost.my.domain pochan
127.0.0.1               localhost localhost.my.domain pochan

Finally, edit $DESTDIR/etc/ssh/sshd_config and change PermitRootLogin to yes, since there will be no user accounts on first boot.

Installation on the Device »