User Tools

Site Tools



Installing on USB Thumbdrive

The main thing with using a USB thumbdrive is to decrease the number of writes. Compared to SSD and platter based block storage devices (disk drives, called “spinner” in this document), USB has a very limited number of writes available.

As soon as possible after doing the initial install, begin decreasing writes by modifying the atime parameter on anything stored on the USB, then moving active partitions (/tmp, /var/log, /var/run, etc…) off of the USB, either into a Memory Device (aka md, RAMDisk), and SSD or, preferably, a “spinner”.

Quick and Dirty

  • Install your base system using UFS. I do a manual creation and simply use the entire space for /, then began tweaking
    • When given the opportunity to create the partition, do manual partitioning and set the following options in the Configure section:
      • SUJ off (turn off journaling, since this generates a lot of writes)
      • TRIM on (let TRIM work to balance where the writes go, SSD only)
      • SU on
    • The most important thing to decrease writes is to set that noatime parameter. This greatly reduces writes (we'll handle the other constant writes later).
  • Put /tmp in md and point /var/tmp to it. If you have plenty of memory, this is very simple and really, really fast.
  • Build your disk set of spinners. In my setup, I had a ZFS Z-RAID on some 10k drives, and an SSD card for the ZFS cache (I needed speed since it was planned to be a iSCSI server).
  • put the following onto separate partitions on the spinners:
    • /var/log
    • /var/run
    • swap

More Detail

When I am done, my fstab looks like this.

# Device	Mountpoint	FStype	Options	Dump	        Pass#
/dev/da4s1a	/		ufs	rw,noatime	        1	1
md11            /tmp            mfs     rw,-s256M,-M,nosuid,noatime,noexec 0	0
md              /var/run        mfs     rw,-s4M,nosuid,noatime  0	0
/dev/ada0p2     none            swap    sw                      0	0
/dev/ada1p2     none            swap    sw                      0       0

The above is a little different, since I already had an SSD in the system as the L2ARC for ZFS. FreeBSD does not require L2ARC have an entire device, so I was able to partition this out to give myself a block for swap. This particular card is actually two small devices which can be RAID'd, but I broke the RAID set and simply set up each device on its own, thus creating the two swap devices above (ada0p2 and ada1p2). ada0p1 and ada1p1 were for the L2ARC.

I also created a small mfs for /tmp and /var/run. /tmp really needs 256M (128 will work) if you are using pkg to install. If you use ports, most of the action takes place in the /usr/ports and /usr/src directories, and you should create partitions for that on the spinners (I would use ZFS since I'm already set up for it).

One thing is to point /var/tmp to /tmp. That is as simple as making sure nothing is using /var/tmp (do an ls, some files are ok), then run the following:

rm -fRv /var/tmp
ln -s /tmp /var/tmp

I set up /var/log on the zfs platter I created after the fact, so it doesn't show up in fstab. My pool is called Storage in this example:

zfs create storage/varlog
mv /var/log/* /storage/varlog
zfs set mountpoint=/var/log storage/varlog
service syslogd restart

The output of the above command will show you the mounted file system

/dev/da4s1a on / (ufs, local, noatime, journaled soft-updates)
devfs on /dev (devfs, local, multilabel)
/dev/md0 on /tmp (ufs, local, noatime, nosuid, soft-updates)
/dev/md1 on /var/run (ufs, local, noatime, nosuid, soft-updates)
storage on /storage (zfs, local, nfsv4acls)
storage/varlog on /var/log (zfs, local, nfsv4acls)

Other Options

There are some really great options available when booting from USB that I ran into. Manual installs, installations with two copies of the boot image on the flash ( The nice thing is, you can quickly make a copy of the low cost device and swap them out when/if needed.

The best one is, however, the ability to store an image very nicely and recreate the boot device if you ever need to. To do this.

  1. Remove USB from drive
  2. Plug into a second system
  3. Use the dd command to make a copy.
    1. Determine the USB drive name
    2. Determine an output file on a device which has sufficient space
    3. dd if=/dev/ad0 of=/my/path/to/image/ad0.img && sync
  4. You can now create a second device by simply using dd to copy from the file back to it.
    1. Determine the USB drive name (assume ad0 for this)
    2. dd if=/my/path/to/image/ad0.img of=/dev/ad0 && sync

The && sync in the above command ensures all data is written before the cursor returns. Note that the target device (the second USB) must be as large or larger than the original one. I tend to make it the exact make and model.

Note I have found that booting from an image on some enterprise servers requires you to reconfigure the BIOS, telling it to boot off the new “drive.” It appears the BIOS tracks which device it is to boot off of, not the particular slot. If you image a flash, be prepared to go into BIOS and reconfigure your boot device.


unix/freebsd/installing_to_usb_thumbdrive.txt · Last modified: 2018/04/08 17:43 by rodolico