Pxe on Backend Engineering Strategy Tools

PXE Booting with OPNSense + iPXE

Thu, 14 May 2026 00:00:00 +0000

How to configure OPNSense as a PXE boot server using its built-in DHCP and TFTP services, and how to write an iPXE boot menu that can chainload Talos Linux (or anything else).

OPNSense DHCP — Network Boot Fields

Services → ISC DHCPv4 → [LAN] → Network Booting

Field	Value
Enable network booting	✓
Next-server IP	`192.168.1.1` (OPNSense LAN address)
Default BIOS filename	`undionly.kpxe`
x86 UEFI (32-bit) filename	`ipxe.efi`
x64 UEFI/EBC (64-bit) filename	`ipxe.efi`
iPXE boot filename	`default.ipxe`

The DHCP server serves the correct boot file based on client architecture. BIOS clients get undionly.kpxe; UEFI clients get ipxe.efi. Both then chainload default.ipxe.

TFTP — Downloading the Boot Files

OPNSense runs a TFTP server rooted at /usr/local/tftp. SSH in and fetch the iPXE binaries:

fetch -o /usr/local/tftp/undionly.kpxe https://boot.ipxe.org/undionly.kpxe
fetch -o /usr/local/tftp/ipxe.efi https://boot.ipxe.org/ipxe.efi

iPXE Boot Script

Save as /usr/local/tftp/default.ipxe. This example has a boot menu with options for netboot.xyz, a Talos Omni boot, and a debug shell.

#!ipxe

dhcp
set menu-timeout 5000

:start
menu PXE Boot Menu
item --gap -- ----------------------------
item netboot Boot netboot.xyz
item talos Boot Talos (Omni)
item shell iPXE Shell
item --gap -- ----------------------------
choose target && goto ${target}

:netboot
chain http://boot.netboot.xyz || goto failed
goto start

:talos
echo Booting Talos via Omni...

set api https://<your-omni-instance>.omni.siderolabs.io
set token <join-token>
set event [<siderolink-ipv6>]:8090
set log tcp://[<siderolink-ipv6>]:8092

kernel tftp://${next-server}/talos/vmlinuz-omni \
 ima_template=ima-ng \
 ima_appraise=fix \
 ima_hash=sha512 \
 selinux=1 \
 consoleblank=0 \
 nvme_core.io_timeout=4294967295 \
 initrd=initramfs.xz \
 init_on_alloc=1 \
 slab_nomerge \
 pti=on \
 console=tty0 \
 console=ttyS0 \
 printk.devkmsg=on \
 talos.platform=metal \
 siderolink.api=${api}?jointoken=${token} \
 talos.events.sink=${event} \
 talos.logging.kernel=${log}

initrd tftp://${next-server}/talos/initramfs-omni.xz
boot || goto failed

:shell
shell

:failed
echo Boot failed, press Enter to continue...
read fake
goto start

The api, token, event, and log values come from the Omni console when you generate a join link.

Talos Kernel and Initramfs — Image Factory

The standard Talos release binaries do not include firmware for all hardware. Since Talos 1.6, several older NIC drivers (including Broadcom BNX2 / BCM5709) were removed from the mainline image and made available as extensions via the image factory.

Generate a custom image at factory.talos.dev with the extensions you need (e.g. siderolabs/bnx2), then download the PXE artifacts:

mkdir -p /usr/local/tftp/talos

fetch -o /usr/local/tftp/talos/vmlinuz-omni \
 "https://pxe.factory.talos.dev/image/<IMAGE_ID>/v1.10.1/kernel-amd64"

fetch -o /usr/local/tftp/talos/initramfs-omni.xz \
 "https://pxe.factory.talos.dev/image/<IMAGE_ID>/v1.10.1/initramfs-amd64.xz"

Replace <IMAGE_ID> with the schematic ID from the image factory, and adjust the version tag as needed.

Gotchas

UEFI boot and NIC memory limits — ipxe.efi can be too large to fit in the NIC’s PXE memory buffer on some older hardware. If the UEFI chain fails silently, switch to BIOS/legacy mode and use undionly.kpxe instead.

DHCP options 66/67 conflict — If you have previously set DHCP options 66 (next-server) and 67 (boot file) as raw additional options, remove them. OPNSense’s built-in network boot fields handle this; having both causes conflicts.

BIOS boot order after first boot — Talos writes its own bootloader on first boot. If the BIOS is set to PXE as the primary device, the machine will fall back to the PXE menu on every subsequent reboot. Set disk as the primary boot device once the cluster is up.

Talos Linux in the homelab via Omni

Thu, 14 May 2026 00:00:00 +0000

Getting Talos Linux running in the homelab via PXE boot and Omni — starting with ODEN (SYS-005), an IBM System x3550 M3. The full OPNSense + iPXE configuration lives in the reference note; this covers what actually happened, in order.

Setup

Hardware: ODEN (SYS-005) — IBM x3550 M3, Broadcom BNX2 NICs (BCM5709)
Network: OPNSense router on LAN; ODEN connected via one NIC (start with one — removes variables)
Target: Single-node Talos cluster registered in Omni

Step 1 — OPNSense DHCP and TFTP

Enable network booting on the LAN DHCP server and download the iPXE binaries to the TFTP root. Full field values in the iPXE reference note.

One thing to check first: if you previously set DHCP options 66 and 67 as raw additional options, remove them. OPNSense’s built-in network boot fields do the same job and having both causes conflicts.

Step 2 — iPXE boot script

Write default.ipxe to /usr/local/tftp/. Include a boot menu with at minimum a Talos option and a shell fallback — the shell is genuinely useful when something fails and you need to debug from the boot prompt. Full script in the reference note.

The Talos entry in the menu needs the Omni join token from your Omni console. Generate a join link in Omni; it provides the API endpoint, token, and SideroLink addresses.

Step 3 — Talos kernel and initramfs

The standard Talos release binaries do not include BNX2 firmware. Since around Talos 1.6 those drivers are available as extensions but not in the mainline image. Without them, the node boots, fails to initialise the NIC, and produces can't load firmware bnx2 errors — everything else looks fine until you notice the node never gets an IP and never appears in Omni.

Fix: generate a custom image at factory.talos.dev with the siderolabs/bnx2 extension included, then download the PXE kernel and initramfs from the factory URL. Commands in the reference note.

Step 4 — First boot

Go into BIOS and set the boot device to PXE. On the M3, UEFI boot with ipxe.efi fails silently — the image is too large for the NIC’s PXE memory buffer. Switch to legacy/BIOS mode and use undionly.kpxe instead.

The machine takes a while to POST and boot. This is normal for old enterprise hardware. It is also why demos typically use virtual machines.

Step 5 — Static IP

After the BNX2 fix the node boots Talos successfully but still does not appear in Omni. The DHCP assignment for the node is not being picked up during early boot. Workaround: add a static IP via kernel params in the iPXE script:

ip=192.168.1.171::192.168.1.1:255.255.255.0::eth0:off

Add this to the kernel line in the Talos iPXE entry. The format is ip=<client-ip>::<gateway>:<netmask>::<iface>:off.

Step 6 — Omni registration

With a working NIC and an IP, the node contacts the Omni API using the join token. It appears in the Omni console as an unallocated machine. Create a cluster, assign the machine, and let Omni configure it. The initial cluster bootstrap takes a few minutes.

Step 7 — Fix the BIOS boot order

After the cluster is up, change the BIOS boot order so the disk is first. If PXE remains the primary boot device, every reboot drops the machine back to the iPXE menu instead of booting the installed Talos. Discovered on first reboot. Worth noting it here so you don’t make the same trip to the garage.

Upgrade

Omni makes single-node upgrades straightforward: open the cluster in the Omni console, select a new Talos version, apply. The node reboots once. Single-node means the cluster has downtime during the reboot; that is expected. Nothing else to do.

Result

Single-node Kubernetes cluster running on ODEN, managed via Omni. kubectl and talosctl access via the Omni CLI. Next experiment: Rook + Ceph for persistent storage.

Hardware Provisioning: PXE Booting and Tooling

Tue, 12 May 2026 00:00:00 +0000

When moving beyond manual installs, managing hardware lifecycle through PXE (Preboot Execution Environment) becomes essential. A breakdown of common tools for automating the “power-on to OS ready” process.

Common starting points

Tool	Focus	Complexity	Best for
Cobbler	PXE/repo server	Low–Medium	Stable, static environments needing reliable kickstart or seed installs
Foreman	Full lifecycle mgmt	High	Single pane of glass for provisioning + ongoing config management (Puppet/Ansible)
Digital Rebar	Infrastructure-as-Code	Medium	Modern DevOps teams wanting cloud-like speed on physical gear; evolved from Crowbar
Ironic / Bifrost	BMaaS / scale	High	Bare Metal as a Service at scale; Bifrost runs Ironic standalone without full OpenStack

Broader landscape

Classic PXE / Provisioning

Tool	Type	Strengths	Weaknesses
Cobbler	PXE provisioning server	Simple, mature, easy to understand	Old architecture, static workflows
Foreman	Lifecycle/provisioning platform	Powerful, enterprise-capable, large ecosystem	Heavy footprint, Rails monolith
Uyuni	Systems management	Enterprise lifecycle management (SUSE/Spacewalk lineage)	Less modern provisioning architecture

Dynamic / Policy-Driven

Tool	Type	Strengths	Weaknesses
Razor	Policy-driven provisioning	Dynamic node discovery, elegant lifecycle model	Effectively dormant
Digital Rebar	Workflow provisioning platform	Architecturally modern and flexible	Partially commercialized

Cloud / Hyperscale Bare Metal

Tool	Type	Strengths	Weaknesses
Ironic	OpenStack bare-metal service	Extremely scalable, API-driven	High operational complexity
Bifrost	Standalone Ironic deployment	Easier entry into Ironic ecosystem	Inherits Ironic complexity
MAAS	Bare metal cloud platform	Excellent UX, API-first, machine discovery	Larger footprint, Ubuntu-centric

Kubernetes-Native / Cloud-Native

Tool	Type	Strengths	Weaknesses
Tinkerbell	Cloud-native provisioning	Modern architecture, composable workflows	Microservice complexity
Metal3	Kubernetes operator	Native Kubernetes integration	Requires Kubernetes infrastructure
Omni	Talos cluster orchestration	Very modern UX and lifecycle management	Talos/Kubernetes specific
Matchbox	Minimal PXE/ignition service	Elegant, simple, iPXE-first	Narrow immutable-infra focus

Boot Infrastructure / PXE Utilities

Tool	Type	Strengths	Weaknesses
iPXE	Network boot firmware	Flexible, fast, programmable (HTTP + scripting)	Requires infrastructure around it
netboot.xyz	Dynamic network boot menu	Extremely useful and lightweight	Not a provisioning orchestrator

Architectural Styles

Style	Example Tools	Characteristics
Static config-driven	Cobbler	Profiles + templates + PXE configs
Policy/state-driven	Razor, Digital Rebar	Nodes discovered dynamically, assigned via policies
Cloud resource model	Ironic, MAAS	Bare metal treated as cloud infrastructure
Kubernetes-native	Tinkerbell, Metal3	Bare metal managed via Kubernetes APIs
Immutable OS orchestration	Omni, Matchbox	Minimal provisioning around immutable operating systems

The Gap

There is still no widely adopted FOSS solution that is simultaneously:

lightweight
modern
self-hostable
API-first
iPXE-native
distro-agnostic
easy to operate
single-binary deployable
workflow-capable
not tied to Kubernetes/OpenStack

Most existing systems drift toward enterprise complexity, cloud platform assumptions, Kubernetes dependency, immutable OS specialization, or monolithic lifecycle management.

“A modern lightweight provisioning orchestrator for reproducible bare-metal infrastructure.”