• Our Debian Mirror Is Now Private To Our Network

    Mon, 01 Jun 2020 20:30:00 -0700 - Chris Brannon

    For a number of years, we have operated a private mirror of part of Debian’s repository for use by our customers. Though it was never publicized, it was usable from outside of our network.

    As of Monday June 1st, 2020 we have limited access to the Debian portion of our mirror to our internal network. The maintenance overhead of this mirror has become too high and there are already an adequate number of Debian mirrors available in our geographic location.

    Our public mirrors of EPEL and Fedora will continue to be available.

    If you have any questions or concerns, please write us at support@prgmr.com.

  • OpenBSD 6.7 Installer Available

    Tue, 19 May 2020 11:30:00 -0700 - Chris Brannon

    The OpenBSD installer has been updated to OpenBSD 6.7. Notable changes include:

    • Implemented “strip” option in httpd.conf(5) for fastcgi to be able to have multiple chroots under /var/www for FastCGI servers.
    • Changed httpd(8) to send a 408 response when a timeout happens while headers are being received, but close the connection if no request is received.

    Have a look at the OpenBSD 6.7 release notes for more details.

    If you experience a crash on a prgmr VPS running a previous version of OpenBSD, upgrading to 6.7 may help. Please refer to the OpenBSD Upgrade Guide.

    If you’re interested in running OpenBSD on prgmr.com, please see our wiki page on OpenBSD.

    Note that the OpenBSD installer is only available for VPSes with HVM virtualization. The virtualization type can be checked from the ‘system details’ option of the management console. If your VPS uses PV and you are interested in a conversion to HVM, please write support@prgmr.com.

  • Scheduled Maintenance for Billing System

    Mon, 18 May 2020 11:00:00 -0700 - Brandon McGinty-Carroll

    We will perform a software upgrade for our billing system, billing.prgmr.com, during a two-hour maintenance window starting Saturday May 23 18:00:00 UTC. Downtime is expected to be less than 20 minutes.

    If you have any questions or concerns, please write us at support@prgmr.com.

  • Full Disk Encryption with OpenBSD and a Serial Console

    Fri, 08 May 2020 10:00:00 -0700 - Chris Brannon

    We recently went looking for a method of installing OpenBSD with full disk encryption that would work with a serial console, because that is what we provide to our VPS customers. We already provided instructions for installing OpenBSD on our VPSes and installing OpenBSD with full disk encryption is itself well-documented. Unfortunately, when we tried the standard procedures, we did not get a passphrase prompt from the bootloader when booting the fresh installation. There appears to be no obvious way to boot from an OpenBSD softraid – such as an encrypted disk – and have it use the serial console. Here is a description of how we kludged it. Some familiarity with OpenBSD terminology is expected.


    We make two OpenBSD slices. One contains a tiny filesystem with a bootloader configuration file, and the other contains the encrypted part of the disk. At boot, the OpenBSD bootloader looks for /etc/boot.conf in the first slice, sd0a. That file contains 3 instructions: set serial baud rate, use a serial console, and set the real boot device. The bootloader then boots from the real encrypted boot device, with output going to the serial console.

    A Tutorial Explanation

    First, boot the OpenBSD installer, but select the shell option at the prompt, rather than the install option. We are going to need to do some pre-configuration before performing an install. Note that any line starting with a # is a shell prompt, and others are output.

    Welcome to the OpenBSD/amd64 6.6 installation program.
    (I)nstall, (U)pgrade, (A)utoinstall or (S)hell? s

    First, we’ll make the devices that we will need for the hard disk and the softraid. We’ll rewrite the master boot record for the hard disk with fdisk.

    # cd /dev
    # sh MAKEDEV sd0 sd1
    #   fdisk -iy sd0
    Writing MBR at offset 0.

    We can optionally overwrite the disk with random data to hide which part of the disk has been written to:

    # dd if=/dev/urandom of=/dev/rsd0c bs=1M

    Now, using disklabel, we will make two slices on the raw disk. The first, sd0a, is going to contain a really tiny filesystem with just one file in it: /etc/boot.conf. The second slice will be the softraid slice for the encrypted disk. In order to keep things simple, we don’t add a swap slice. If you want one, you should add it as another slice to sd0, rather than to the softraid. OpenBSD already encrypts swap, so there is no point in doubly-encrypting it.

    # disklabel -E sd0
    Label editor (enter '?' for help at any prompt)
    sd0> a
    partition: [a] a
    offset: [64] 8192
    size: [62902348] 8192
    FS type: [4.2BSD] 
    sd0*> a
    partition: [b] b
    offset: [16384] 
    size: [62894156] 
    FS type: [swap] raid
    sd0*> w
    sd0> q
    No label changes.

    We use an offset of 8192 for the first partition because we want partitions aligned on a 4 MiB boundary, in order to reduce the possibility of write amplification when using solid state storage.

    Next, we initialize the softraid. There are other options for the bioctl command that might be interesting or useful here, including -r for specifying the number of rounds of the key derivation function. However, we’ll keep it simple.

    # bioctl -c C -l /dev/sd0b softraid0
    New passphrase: 
    Re-type passphrase: 
    sd1 at scsibus3 targ 1 lun 0: <OPENBSD, SR CRYPTO, 006>
    sd1: 30709MB, 512 bytes/sector, 62893628 sectors
    softraid0: CRYPTO volume attached as sd1

    We need to make the filesystem on sd0a and put /etc/boot.conf there:

    # newfs /dev/rsd0a
    /dev/rsd0a: 4.0MB in 8192 sectors of 512 bytes
    4 cylinder groups of 1.00MB, 64 blocks, 128 inodes each
    super-block backups (for fsck -b #) at:
      32, 2080, 4128, 6176,
    # mount /dev/sd0a /mnt
    mkdir /mnt/etc
    echo '
    stty com0 115200
    set tty com0
    set device sr0a' > /mnt/boot.conf
    # umount /dev/sd0a

    That is all of the pre-configuration work, and you can now start the OpenBSD installer by invoking /install at the shell prompt. From this point forward, you can just do a fairly standard installation process. Use the disk sd1 for the install. The installer should leave sd0 alone, since we set it up manually. You also probably do not want to use the auto disk layout with a swap slice, since your system would be swapping to something on sd1. Here are the relevant bits from our interactive session with the installer; yours will likely be a bit different.

    Available disks are: sd0 sd1.
    Which disk is the root disk? ('?' for details) [sd0] sd1
    No valid MBR or GPT.
    Use (W)hole disk MBR, whole disk (G)PT or (E)dit? [whole] w
    Setting OpenBSD MBR partition to whole sd1...done.
    The auto-allocated layout for sd1 is:
    #                size           offset  fstype [fsize bsize   cpg]
      a:          1024.0M               64  4.2BSD   2048 16384     1 # /
      b:          1264.0M          2097216    swap                    
      c:         30709.8M                0  unused                    
      d:          1713.6M          4685888  4.2BSD   2048 16384     1 # /tmp
      e:          2669.6M          8195392  4.2BSD   2048 16384     1 # /var
      f:          2496.0M         13662816  4.2BSD   2048 16384     1 # /usr
      g:           981.6M         18774656  4.2BSD   2048 16384     1 # /usr/X11R6
      h:          4012.0M         20784992  4.2BSD   2048 16384     1 # /usr/local
      i:          1698.4M         29001664  4.2BSD   2048 16384     1 # /usr/src
      j:          5916.8M         32480000  4.2BSD   2048 16384     1 # /usr/obj
      k:          8926.2M         44597632  4.2BSD   2048 16384     1 # /home
    Use (A)uto layout, (E)dit auto layout, or create (C)ustom layout? [a] c
    Label editor (enter '?' for help at any prompt)
    sd1> a
    partition: [a] a
    offset: [64] 8192
    size: [62870218] 
    FS type: [4.2BSD] 
    mount point: [none] /
    sd1*> w
    sd1> q
    No label changes.
    /dev/rsd1a: 30698.3MB in 62870208 sectors of 512 bytes
    152 cylinder groups of 202.47MB, 12958 blocks, 25984 inodes each
    Available disks are: sd0.
    Which disk do you wish to initialize? (or 'done') [done] done
    /dev/sd1a (cd70684e3d78d3b3.a) on /mnt type ffs (rw, asynchronous, local)

    Once you’re done with the installer, that’s it. You should have an encrypted installation of OpenBSD that prompts for the password on the serial console.

    Final Notes

    There are two areas of your disk that are not encrypted. One is the slice containing /etc/boot.conf, and the other is the unencrypted area of the softraid, where the bootloader is stored.

    This blog post came into existence due to a customer request for help installing OpenBSD. If you encounter any problems performing an install on a VPS with prgmr.com, please feel free to each out to support.

  • Distributions Update: Alpine Linux, CentOS 7, Fedora, NetBSD, NixOS, Ubuntu

    Wed, 29 Apr 2020 18:30:00 -0700 - Chris Brannon

    We made the following additions and updates to our distribution images and netboot installers:

    Here are some highlights from each.

    Alpine Linux

    This includes an important security fix for openssl (CVE-2020-1967).

    CentOS 7

    • Python 3 is now available. Installing the python3 package gives you the Python 3.6 interpreter.
    • bind has been rebased to version 9.11.
    • chrony has been rebased to 3.4.
    • Since release 1503 (abrt>= 2.1.11-19.el7.centos.0.1) CentOS-7 can report bugs directly to bugs.centos.org.

    Fedora 32

    Updated packages include: python 3.8, ruby 2.7, and gcc 10. Most Pythron 2 packages have been removed from Fedora, since Python 2 is end of life. A legacy python27 package is provided for those who need it.

    NetBSD 8.x

    • httpd(8): fixed various security issues
    • named(8): stop using obsolete dnssec-lookaside.
    • Various kernel memory info leaks fixes.
    • Updated expat to 2.2.8.
    • Accepts root device specification as NAME=label.


    Support is planned until the end of October 2020. Among numerous added and updated packages, the following version updates were made to core packages:

    • gcc: 8.3.0 -> 9.2.0
    • glibc: 2.27 -> 2.30
    • linux: 4.19 -> 5.4
    • mesa: 19.1.5 -> 19.3.3
    • openssl: 1.0.2u -> 1.1.1d

    Ubuntu 20.04

    Ubuntu 20.04 offers zfs version 0.8.3. A few new zfs features include:

    • Native Encryption (with hardware acceleration enabled in Focal)
    • Device removal
    • Pool TRIM
    • Sequential scrub and resilver (performance)

    In 20.04 LTS, the python included in the base system is Python 3.8. Python 2.7 has been moved to universe and is not included by default in any new installs. The /usr/bin/python executable is not available in new installs by default.

    It doesn’t seem to be mentioned anywhere on the net, but the value of the kernel.pid_max sysctl was bumped from 32768 to 4194304. We mention it in case someone is surprised by very large process IDs in their ps output.


    Our distribution images and netboot installers are available from the management console of any Prgmr.com VPS.

    Note that all of the updates other than CentOS 7 and NetBSD are only available to customers with a VPS using HVM virtualization. NetBSD is only available to customers with a VPS using PV virtualization. To check the virtualization mode of existing services, log in to the management console, select “system details”, and check the virtualization mode.