Security on Backend Engineering Strategy Tools

Linux Identity Management — FreeIPA and SSSD

Mon, 01 Jan 2024 00:00:00 +0000

Managing user accounts across many Linux machines by hand — creating the same user on every host, syncing passwords, maintaining sudo rules — breaks down fast. FreeIPA provides centralised identity management: one place to define users, groups, sudo rules, host policies, and SSH keys. SSSD is the daemon that runs on each Linux machine and connects it to FreeIPA (or any LDAP/Kerberos provider), making those central definitions available locally.

FreeIPA

An integrated identity management solution from Red Hat, combining LDAP (389 Directory Server), Kerberos, DNS, a certificate authority, and a web UI into a single deployable stack. Users, groups, sudo rules, HBAC (host-based access control) rules, and SSH public keys are all managed centrally and enforced on enrolled hosts. FreeIPA is the open source upstream of Red Hat Identity Management (IdM).

Install on RHEL/Rocky/AlmaLinux:

dnf install freeipa-server
ipa-server-install --domain=example.com --realm=EXAMPLE.COM

Once the server is running, enroll a client host:

dnf install freeipa-client
ipa-client-install --server=ipa.example.com --domain=example.com

After enrollment, users defined in FreeIPA can log into the host with Kerberos SSO — no separate account needed on the machine.

SSSD

The System Security Services Daemon. SSSD runs on each Linux host and mediates all identity lookups — NSS (name service switch) queries for users and groups, PAM authentication, sudo rule lookups. It caches responses locally so logins still work when the identity server is temporarily unreachable, and it handles the Kerberos ticket lifecycle transparently.

SSSD is not FreeIPA-specific. It supports:

FreeIPA (the natural pairing)
Active Directory (via the ad provider — direct AD integration without Samba)
Generic LDAP and Kerberos

The ipa-client-install command configures SSSD automatically when enrolling a FreeIPA client. For AD integration, the configuration is similar but uses the ad provider:

[domain/example.com]
id_provider = ad
auth_provider = ad
access_provider = ad
ad_domain = example.com
krb5_realm = EXAMPLE.COM

The pairing

FreeIPA and SSSD are complementary halves of the same solution. FreeIPA is the authoritative store — where you create and manage identities. SSSD is the enforcer on each host — it translates FreeIPA’s policies into local authentication decisions, caches them for resilience, and keeps Kerberos tickets current. Neither replaces the other; together they give you centralised identity management with no single point of failure for login availability.

Resources

LUKS — Linux Disk Encryption

Mon, 01 Jan 2024 00:00:00 +0000

LUKS (Linux Unified Key Setup) is the standard for full-disk encryption on Linux. It uses dm-crypt in the kernel to encrypt block devices transparently — the filesystem sits on top of an encrypted layer, and the encryption happens below it. The LUKS header stores the encrypted key material and metadata, supporting up to eight independent key slots (passphrases or keyfiles) that all unlock the same volume.

Setup

# Encrypt a device (destroys existing data)
cryptsetup luksFormat /dev/sdb

# Open the encrypted device, exposing it as a plaintext mapper device
cryptsetup luksOpen /dev/sdb data-encrypted

# Format and use the plaintext device normally
mkfs.ext4 /dev/mapper/data-encrypted
mount /dev/mapper/data-encrypted /mnt/data

# Close when done
umount /mnt/data
cryptsetup luksClose data-encrypted

Key slots

LUKS supports multiple passphrases or keyfiles — useful for adding a recovery key alongside an operational passphrase:

# Add a second key slot (e.g. a recovery keyfile)
cryptsetup luksAddKey /dev/sdb /path/to/recovery.key

# Remove a key slot
cryptsetup luksRemoveKey /dev/sdb

# List key slot usage
cryptsetup luksDump /dev/sdb

Auto-unlock at boot

For encrypted root or data partitions that should unlock automatically on boot, add the device to /etc/crypttab with a keyfile path:

data-encrypted /dev/sdb /etc/keys/data.key luks

Then add the plaintext device to /etc/fstab as normal. On servers, the keyfile is stored on a separate volume or fetched from a secrets manager (Vault, Tang/Clevis for network-bound disk encryption).

Use in Kubernetes

Node-level disk encryption with LUKS protects data at rest on Kubernetes worker nodes — persistent volume data stored on the node’s disks is encrypted before it leaves the kernel. Talos Linux enables LUKS encryption for its state and ephemeral partitions by default.

Resources

Security Scanning & Monitoring

Mon, 01 Jan 2024 00:00:00 +0000

Security tooling broadly splits into three concerns: what vulnerabilities exist in your software before it runs (image scanning), what is actually happening on your systems while they run (endpoint monitoring), and what is moving across your network (intrusion detection). Clair, osquery, and SNORT each cover one of these.

Clair

A static analysis tool for container image vulnerability scanning, from the Quay project (Red Hat). Clair maintains a database of CVEs from multiple sources (NVD, Red Hat, Debian, Alpine, Ubuntu) and matches them against the packages installed in a container image layer by layer. Integrated into a container registry, it scans every image on push and blocks or flags images with known vulnerabilities above a configurable severity threshold. The result is a vulnerability report tied to the image digest — not the running container, but the image itself before it’s ever deployed.

osquery

Facebook’s open source endpoint monitoring tool. osquery exposes the operating system as a relational database — processes, users, network connections, installed packages, kernel modules, scheduled tasks, browser extensions — all queryable with SQL. This makes ad-hoc security investigation fast (a single query answers “which processes are listening on unexpected ports?”) and continuous monitoring straightforward (schedule queries, collect results centrally, alert on anomalies). osquery runs on Linux, macOS, and Windows. Used standalone it is a powerful investigation tool; integrated with a SIEM or fleet management layer (Fleet, Kolide) it becomes a continuous compliance and detection platform.

-- Processes with open network connections
SELECT p.name, p.pid, l.address, l.port
FROM processes p
JOIN listening_ports l ON p.pid = l.pid
WHERE l.port NOT IN (22, 80, 443);

SNORT

A network intrusion detection and prevention system (IDS/IPS). SNORT inspects network traffic in real time against a ruleset — signatures for known attack patterns, protocol anomalies, port scans, exploit attempts. In IDS mode it logs and alerts; in IPS mode it can drop matching packets inline. SNORT rules are expressive and the community ruleset (Snort Community Rules, Emerging Threats) covers a wide range of threats. Placed at a network chokepoint — in front of a server, at the edge of a network segment — it gives visibility into what traffic is actually reaching your systems and can detect lateral movement, exfiltration attempts, and known exploits in transit.

Resources

SSH

Mon, 01 Jan 2024 00:00:00 +0000

SSH (Secure Shell) is the standard protocol for encrypted remote access to Linux and Unix systems. It replaced telnet and rsh by wrapping the session in a cryptographic tunnel — authentication, commands, and data transfer all protected against interception and tampering.

Public key authentication

The default auth mechanism for any serious setup. You generate a key pair: a private key that never leaves your machine, and a public key that goes on the remote host.

ssh-keygen -t ed25519 -C "your@email.com"
ssh-copy-id user@host # appends public key to ~/.ssh/authorized_keys on remote

Prefer ed25519 over the older rsa — smaller keys, faster, stronger. Keep your private key protected with a passphrase; use ssh-agent to avoid typing it repeatedly.

Key deployment at scale

When you have many hosts, distributing public keys manually doesn’t scale. A typical pattern: submit your public key to a central approval system, which then deploys it to the relevant hosts via automation (Ansible, Puppet, etc.).

This keeps a clear audit trail — keys are approved, recorded, and can be revoked centrally without touching individual authorized_keys files.

Certificate authentication

At larger scale, even centralised key distribution gets unwieldy. SSH certificates solve this properly: instead of deploying individual public keys, you run an SSH Certificate Authority (CA). Hosts and users trust the CA, not each other’s individual keys.

Generate a key pair locally
Submit the public key to the SSH CA (backed by your identity provider — FreeIPA, Vault, etc.)
The CA issues a signed certificate with a short TTL
SSH to any host — the host trusts the CA, so the certificate is accepted without any pre-deployed keys

Short-lived certificates (hours, not days) dramatically reduce the blast radius of a compromised credential. No revocation lists to maintain.

SSH config

~/.ssh/config avoids repetitive flags on every connection:

Host bastion
 HostName bastion.example.com
 User deploy
 IdentityFile ~/.ssh/id_ed25519
 ForwardAgent yes

Host internal-*
 ProxyJump bastion
 User deploy

ProxyJump (formerly -J) lets you reach hosts that aren’t directly accessible — you SSH through the bastion transparently.

Port forwarding

SSH can tunnel TCP traffic, useful for reaching services on private networks:

# Local forward: reach remote Postgres via localhost:5432
ssh -L 5432:db.internal:5432 bastion

# Dynamic forward: SOCKS proxy through the bastion
ssh -D 1080 bastion

Hardening basics

Key settings in /etc/ssh/sshd_config:

PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
AllowUsers deploy ansible

Disable password auth entirely once keys are in place. Restrict which users can log in. Run sshd -t to validate config before reloading.

Resources

TLS Certificates — Let's Encrypt, Certbot, cert-manager

Mon, 01 Jan 2024 00:00:00 +0000

TLS certificates prove that a server is who it claims to be and encrypt traffic in transit. Getting and renewing them used to mean manual requests to a CA, waiting days, and calendar reminders to renew before expiry. Let’s Encrypt automated the entire process in 2015 and made it free. Today there is no reason to run a public service without TLS.

Let’s Encrypt

A free, automated, open certificate authority run by the Internet Security Research Group. Let’s Encrypt issues Domain Validation (DV) certificates valid for 90 days, automatically, via the ACME protocol. The short lifetime is intentional — it forces automation and limits the window if a certificate is compromised. Let’s Encrypt is now the largest CA in the world by certificates issued. You never interact with Let’s Encrypt directly; you use an ACME client that speaks the protocol on your behalf.

Certbot

The EFF’s ACME client — the most widely used way to obtain and renew Let’s Encrypt certificates on a Linux server. Certbot handles the ACME challenge (proving you control the domain), fetches the certificate, installs it into your web server config (nginx, Apache), and sets up a cron job or systemd timer for automatic renewal. For a straightforward public-facing server, Certbot is the default choice:

certbot --nginx -d example.com -d www.example.com

Certbot supports HTTP-01 challenges (serve a file over port 80) and DNS-01 challenges (add a TXT record — required for wildcard certificates and useful when port 80 isn’t accessible).

# Wildcard cert via DNS challenge
certbot certonly --manual --preferred-challenges dns -d "*.example.com"

Renewals run automatically. Check the timer is active:

systemctl status certbot.timer

cert-manager

The Kubernetes-native way to manage certificates. cert-manager runs as a controller in the cluster and automates the full certificate lifecycle — requesting, renewing, and storing certificates as Kubernetes Secret resources. It supports Let’s Encrypt via ACME (HTTP-01 and DNS-01 challenges) as well as other issuers (Vault, Venafi, self-signed).

A ClusterIssuer configures the CA once for the whole cluster:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
 name: letsencrypt-prod
spec:
 acme:
 server: https://acme-v02.api.letsencrypt.org/directory
 email: admin@example.com
 privateKeySecretRef:
 name: letsencrypt-prod
 solvers:
 - http01:
 ingress:
 class: alb

A Certificate resource then requests a cert for a specific domain — or an Ingress annotation triggers cert-manager automatically. The certificate is stored as a Secret and mounted into pods or referenced by the Ingress. Renewal happens automatically before expiry.