[{"content":"Clients of api.thalamus.am were occasionally timing out. The gateway access log showed successful requests.\nThe request had arrived. The upstream had answered. The log recorded a 200. There was no obvious pattern by endpoint or time of day, and the failures were rare enough to disappear into otherwise healthy dashboards.\nI spent too long looking for an application error. The useful question turned out to be: which machines did the connection depend on after the application had done its work?\nThe timeouts lined up with node termination Deploy events didn\u0026rsquo;t explain the failures. Traffic spikes didn\u0026rsquo;t either. Node termination times did.\nThe gateway pods weren\u0026rsquo;t on the terminating nodes. Neither were the upstream pods. We used Karpenter consolidation, so node replacement was a normal part of running the cluster. I had been following where the pods lived and had missed a node in the network path.\nThe distinction matters: moving pods off a node doesn\u0026rsquo;t move every connection\u0026rsquo;s kernel state off it. Karpenter\u0026rsquo;s disruption process handles node draining and pod eviction. An application\u0026rsquo;s traffic path can still depend on a machine that no longer hosts one of its pods.\nA node without a gateway pod was still in the connection The gateway was exposed through a Service with externalTrafficPolicy: Cluster. The path below applies to an NLB using instance targets, where traffic reaches a node\u0026rsquo;s NodePort before the gateway pod. NLBs can also use pod IP targets; AWS documents both target types. Don\u0026rsquo;t assume the same path for both.\nWith Cluster, kube-proxy can forward external traffic to a ready endpoint on another node. In the kube-proxy path involved here, source NAT made the receiving node part of the return path too. The Kubernetes source IP walkthrough shows how this happens.\nImagine the connection arriving at node A while the gateway pod runs on node B:\nRequest: client -\u0026gt; NLB -\u0026gt; node A -\u0026gt; gateway pod on node B Response: client \u0026lt;- NLB \u0026lt;- node A \u0026lt;- gateway pod on node B ^ connection/NAT state lives here Node A had no gateway pod to show up in the application\u0026rsquo;s deployment view. It still held state that this connection needed.\nIf node A disappears before the connection drains, that state disappears with it. The gateway on node B may finish handling the request while the response can no longer return through the original path.\nThis is a failure mode to investigate, not a claim that every consolidated node drops traffic. Load balancer deregistration, health checks and termination timing affect what happens. Kubernetes documents connection draining for deleting nodes; a healthy design needs enough time for that process to finish.\nLocal removed that extra node from our path The change was to use externalTrafficPolicy: Local for the gateway Service. In this mode, kube-proxy keeps external Service traffic on node-local endpoints. It doesn\u0026rsquo;t forward a request to a gateway pod on another node.\nWith matching load balancer health checks, nodes without usable local gateway endpoints stop receiving new traffic once the load balancer observes the change. That removed the cross-node forwarding dependency from this setup.\nIt also avoids the source NAT used by that forwarding path. Whether the gateway sees the original client IP still depends on the NLB\u0026rsquo;s target type, protocol and client IP preservation settings. Local doesn\u0026rsquo;t override every other part of the network.\nCheck placement and draining together Changing the policy was only part of the work. A node with no local gateway endpoint can\u0026rsquo;t forward the request elsewhere to make up for it.\nI would review these settings together before making the same change in another cluster:\nSpread gateway replicas across the nodes and zones where traffic must be served. Check that there is enough spare capacity during maintenance. Use a PodDisruptionBudget for eviction-based disruption, and review the Deployment\u0026rsquo;s rollout settings separately. PDBs do not constrain a Deployment\u0026rsquo;s own rolling update. Match the NLB health-check protocol, port and path to the controller and traffic policy. The AWS Load Balancer Controller has specific guidance for Local Services. Allow time for health-check changes, target deregistration and application connection draining before termination. Zone behavior also depends on cross-zone load balancing and DNS health handling. It isn\u0026rsquo;t as simple as every zone permanently receiving an equal share of traffic. AWS explains these details, including fail-open behavior when all targets are unhealthy, in its NLB health-check documentation.\nTest the result from outside the cluster while a gateway rolls out and while a node drains. Keep requests open long enough to exercise the return path. A test from another pod may take a different route and miss the original problem.\nA successful access log isn\u0026rsquo;t a delivery receipt The meaning of a 200 entry depends on the proxy and its log format. It tells you what the proxy recorded at its point in the request lifecycle. It doesn\u0026rsquo;t prove that the client application received and processed the whole response.\nWhen client observations and gateway logs disagree, follow the connection in both directions. In this case, the node that mattered wasn\u0026rsquo;t running a gateway pod. It was holding the connection state on the way back.\n","date":"2026-09-20T00:00:00Z","image":"/posts/the-200-that-nobody-received/editorial-cover.png","permalink":"/posts/the-200-that-nobody-received/","title":"The 200 that nobody received"},{"content":"The third time the compactor volume filled up, I checked what was using the space before making it bigger.\nThe alert was KubePersistentVolumeFillingUp, on the compactor volume of our log store. The first time, we resized the volume. That bought about a month of quiet. It also made the diagnosis feel settled: more logs, not enough disk.\nThen the alert came back. And came back sooner.\nThis time I opened a shell in the pod and compared two numbers:\n$ df -h /data Filesystem Size Used Avail Use% Mounted on /dev/nvme2n1 50G 46G 3.2G 94% /data $ du -sh /data 6.1G /data The filesystem reported roughly 46 GB in use. The files I could reach under /data accounted for about 6 GB. Before another resize, I needed to explain the gap.\nThe two commands count different things df reports usage for the filesystem containing the path. du walks the directory tree and totals the files it can reach. They answer related questions, but they don\u0026rsquo;t measure exactly the same thing.\nA large difference is a clue, not a diagnosis. First check that you\u0026rsquo;re looking at the same filesystem, that du isn\u0026rsquo;t reporting permission errors, and that another mount isn\u0026rsquo;t hiding files. Filesystem metadata and snapshots can also affect the comparison. On GNU systems, du -xsh /data keeps the walk on one filesystem; it still only counts what is reachable under that directory.\nIn our case, the missing space belonged to deleted files that the compactor still had open.\nDeleting a filename doesn\u0026rsquo;t necessarily free its data. If a process still holds the file open, Linux keeps the file available through that open reference. du can\u0026rsquo;t find it through its old name, but its blocks still count in filesystem usage. The unlink(2) manual describes this distinction between removing a name and releasing the file.\nFind the process holding the files Inside this container, the compactor was PID 1. I checked its open descriptors:\n$ ls -l /proc/1/fd | grep \u0026#39; (deleted)$\u0026#39; | wc -l 14312 That was more than fourteen thousand descriptor entries pointing to deleted files. The count alone didn\u0026rsquo;t tell me how many bytes they occupied, or how many distinct files there were. It did give me a process to investigate.\n/proc/1/fd is specific to this container\u0026rsquo;s process layout. In another container, PID 1 might be a wrapper or an init process. Use the actual application\u0026rsquo;s PID, and check other processes if the first one doesn\u0026rsquo;t explain the usage. The /proc/\u0026lt;pid\u0026gt;/fd documentation explains the entries and their access restrictions.\nWhere lsof is installed, this is another useful starting point:\nlsof +L1 +L1 selects open files with no remaining directory links. Inspect the process, device and file information to connect the results to the affected mount. You need permission to inspect the relevant processes, and a container may not expose every process that uses the volume. An empty result doesn\u0026rsquo;t rule out the problem if your view is incomplete.\nThe compactor was removing temporary files while retaining references to them. Each compaction cycle left more space allocated.\nA restart recovered space; the update addressed the leak Restarting the compactor dropped usage from 94% to below 15%. The process exited, its open references went away, and the filesystem could release the deleted files. Updating the compactor addressed the underlying leak in this incident.\nA restart needs the same care here as any other change to a stateful service. Capture the evidence first. Check how the component handles interruption and how it resumes work. If the leak remains, the volume can fill again after the restart.\nThe drop supported what the open-file inspection had shown. It wasn\u0026rsquo;t a universal test for every df/du mismatch. If usage stays high, another process may still hold the files, your inspection may have missed part of the filesystem, or a different cause may be responsible.\nCompare usage before discussing capacity The misleading part was that resizing worked for a while. The graph stopped approaching the limit, the alert cleared, and there was less reason to question the explanation.\nI now want the runbook to answer these questions before treating the next alert as a sizing problem:\nAre df and du looking at the same filesystem and a comparable scope? Did the directory walk finish without access errors? Are processes holding deleted files on that filesystem? Which process owns them? Does the application\u0026rsquo;s version history describe the same symptom? Compare the details before choosing an upgrade. If a controlled restart is needed to recover space, what evidence should be saved first, and what will prevent the buildup from returning? There are real capacity problems too. Retention may be too long, ingestion may have grown, or a workload may need more working space. In this incident, though, I had been sizing the volume around files the compactor should already have released.\nThe next time the alert fires, compare the two numbers before deciding what the disk needs.\n","date":"2026-08-08T00:00:00Z","image":"/posts/df-says-full-du-disagrees/editorial-cover.png","permalink":"/posts/df-says-full-du-disagrees/","title":"df says full, du disagrees"},{"content":"Claude Code writes a useful share of my Terraform now. It drafts Helm values, reviews GitOps configuration and helps me write runbooks. I still decide which changes reach production.\nThe useful output is a diff I can review. Getting that diff faster doesn\u0026rsquo;t remove the need to understand it.\nMy workflow keeps commits, pushes and deployment commands with me. Claude Code\u0026rsquo;s permissions help enforce that split. But a list of blocked commands is only part of the control: a tool that has production credentials may have more than one way to use them.\nStart with Claude Code\u0026rsquo;s permission rules The boundary starts with a practical distinction. The agent can edit files in its working copy. Changing a file there must not, by itself, deploy that change.\nThe deny list from my workflow can be expressed in Claude Code settings like this:\n{ \u0026#34;permissions\u0026#34;: { \u0026#34;deny\u0026#34;: [ \u0026#34;Bash(terraform apply *)\u0026#34;, \u0026#34;Bash(terraform destroy *)\u0026#34;, \u0026#34;Bash(kubectl apply *)\u0026#34;, \u0026#34;Bash(kubectl delete *)\u0026#34;, \u0026#34;Bash(helm install *)\u0026#34;, \u0026#34;Bash(helm upgrade *)\u0026#34;, \u0026#34;Bash(helm uninstall *)\u0026#34;, \u0026#34;Bash(git commit *)\u0026#34;, \u0026#34;Bash(git push *)\u0026#34; ] } } This is a partial example, not a complete isolation policy. Claude Code checks matching deny rules before ask and allow rules. Its documented trailing * form also matches the bare command. Use /permissions to inspect the active rules and where each came from. Claude Code permissions\nThe limit is in what matches. The documentation explicitly says that Bash(git push *) doesn\u0026rsquo;t match git -C . push. Other rules and the active permission mode decide what happens to that form. A command pattern is not a restriction on everything the underlying program can do. Bash rule limits\nThe list also leaves other mutation commands, such as kubectl patch and kubectl edit, to be addressed. A script, another client or a direct API request may reach the same system. Git can change through a hosting API too.\nTo enforce the boundary beyond the runner, restrict the credentials and access available to its process. Use scoped cloud permissions and Kubernetes RBAC, and keep deployment credentials in the controlled deployment environment. If an agent can edit the policy or obtain a more privileged identity, that policy isn\u0026rsquo;t an independent limit on its authority.\nKubernetes\u0026rsquo; RBAC guidance is useful here. A role\u0026rsquo;s name doesn\u0026rsquo;t tell you whether it is harmless; the permissions and the resources it can access do.\nFor shared rules, Claude Code uses .claude/settings.json; personal project overrides use .claude/settings.local.json. Organization-wide requirements belong in administrator-controlled managed settings, whose policy takes precedence. A checked-in settings file alone is not an administrator-enforced boundary. Settings files and precedence\nMake investigation easy, but check what each tool does I want the agent to inspect code, compare configuration and read the evidence it needs without interrupting me for every step.\nCommands such as git diff, helm lint and terraform validate are useful in that workflow. They have different requirements, though. terraform validate checks an initialized configuration for internal consistency. It doesn\u0026rsquo;t validate remote services or prove that an apply will work.\nValidation also runs provider plugins. Review those dependencies and give the process limited access; the command name alone isn\u0026rsquo;t a security boundary.\nterraform plan needs more care than the label \u0026ldquo;read-only\u0026rdquo; suggests. It normally reads remote objects to calculate proposed changes. Providers run as code, and a configuration can invoke an external program through an external data source. Review the configuration and its dependencies before running it with credentials.\nA plan doesn\u0026rsquo;t execute the resource changes it proposes. That doesn\u0026rsquo;t make every program involved in producing it safe to run with unrestricted access.\nThe output needs care too. A saved plan can contain sensitive values in cleartext, even when terminal output hides them. Store it as a restricted artifact, not as another file to commit beside the code.\nFor cluster investigation, scope access to the resources and namespaces needed for the task. Logs can contain credentials or personal data. Allowing all kubectl get commands can also expose resources the task never needed, including Secrets if RBAC permits it.\nThe goal is a useful investigation path with limited access, rather than a broad shell session that happens to avoid one command name.\nFile rules need an enforced boundary My workflow also uses hooks to limit file access and block known secret paths. Claude Code\u0026rsquo;s PreToolUse hooks can inspect a tool call and deny it before execution. Those checks catch ordinary mistakes, and I want them in place.\nA hook\u0026rsquo;s coverage depends on its matcher and code. It doesn\u0026rsquo;t monitor every filesystem operation inside the resulting process. A command-hook timeout falls back to the normal permission flow, so test failure cases as well as successful rejections. Hook timeouts\nDecrypted files remain readable data if the process can reach them. Filenames are a useful hint, not a complete inventory of sensitive content.\nClaude Code\u0026rsquo;s built-in sandbox adds operating-system filesystem and network restrictions to supported shell tools and their child processes. It also incorporates file-deny rules into that boundary. Other tools, including MCP tools, still need their own permission controls. Check the sandbox\u0026rsquo;s exclusions and unsandboxed retry settings; don\u0026rsquo;t assume every process is contained merely because sandboxing is enabled. Sandbox scope and enforcement\nDefault read access is broad, and child processes can inherit credentials through environment variables. Enabling the sandbox alone doesn\u0026rsquo;t remove either source of access.\nFor a stronger boundary, give the process only the workspace and data it needs. Keep deployment credentials outside that environment, restrict network destinations where practical, and protect the permission policy from edits. Hooks then support the sandbox and access policy instead of carrying the entire security claim.\nReview the route from a diff to production A normal session ends with changed files, a summary of the proposed behavior and the checks the agent ran. I read the diff as I would a colleague\u0026rsquo;s work. If something is uncertain, I want that uncertainty named before I approve the change.\nFor Terraform, review the plan associated with the change you intend to apply. If the configuration or relevant state changes afterward, generate and review a new plan. Keep the deployment step on the human-controlled path with the appropriate credentials.\nFor GitOps, the merge matters because it can trigger reconciliation. The boundary therefore needs to cover repository access, merge permissions and CI credentials too. Keeping git push off an allowlist is helpful, but it doesn\u0026rsquo;t replace a protected deployment branch and a review process.\nI keep the final commit and apply steps in my hands in this workflow. Other teams may record an agent\u0026rsquo;s contribution differently. What matters is that a reviewer accepts responsibility for the change and the system records who approved its deployment.\nTry the boundary before relying on it Use a disposable environment to test the restrictions. Verify that an unapproved change can\u0026rsquo;t reach a deployment identity, that a script doesn\u0026rsquo;t bypass the intended command restrictions, and that the agent can\u0026rsquo;t rewrite its own permissions. Check API and connector paths as well as the terminal.\nI still get useful drafts, configuration reviews and better incident notes within those limits. The tradeoff is that some tasks need a narrower input or a human-run check. That\u0026rsquo;s work I can plan for.\nThe agent can propose the change. Before it reaches production, I need to understand it well enough to approve it.\n","date":"2026-07-18T00:00:00Z","image":"/posts/the-apply-is-still-mine/editorial-cover.png","permalink":"/posts/the-apply-is-still-mine/","title":"The apply is still mine"},{"content":"A Kubernetes cluster on your own VMs needs a way to give services an address that other machines can reach. This lab uses kubeadm to build the cluster, Calico for pod networking, and MetalLB for LoadBalancer services.\nThe setup had three Ubuntu VMs: one control plane node and two workers. These notes keep the 2023 versions, including Kubernetes 1.26.4 and Calico 3.25.1. Some steps were missing from the first version, especially the MetalLB network configuration. I explain those gaps below, but the complete setup still needs a fresh test.\nPrepare the same runtime on all three nodes The node preparation runs on every VM. Start with the package update from the original lab:\nsudo apt update sudo apt upgrade The lab uses containerd as the container runtime. Load the kernel modules for this network setup, and arrange to load them again after a reboot:\ncat \u0026lt;\u0026lt;EOF | sudo tee /etc/modules-load.d/containerd.conf overlay br_netfilter EOF sudo modprobe overlay sudo modprobe br_netfilter Enable forwarding and bridge filtering:\ncat \u0026lt;\u0026lt;EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 net.bridge.bridge-nf-call-ip6tables = 1 EOF sudo sysctl --system Next comes the original Docker package repository setup for containerd.io. The notes did not record a containerd version. For a new installation, check the Docker Ubuntu repository instructions and select a runtime version compatible with your Kubernetes release. Installing whatever that repository serves today would not reproduce this lab.\nThe keyring directory and the tools used by these commands must exist first:\nsudo apt-get install -y ca-certificates curl gnupg lsb-release sudo install -m 0755 -d /etc/apt/keyrings curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg echo \u0026#34;deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \\ $(lsb_release -cs) stable\u0026#34; | sudo tee /etc/apt/sources.list.d/docker.list \u0026gt; /dev/null sudo apt-get update sudo apt-get install -y containerd.io On the fresh lab VMs, the next commands generate a default containerd configuration and enable its systemd cgroup driver:\nsudo mkdir -p /etc/containerd sudo containerd config default | sudo tee /etc/containerd/config.toml sudo sed -i \u0026#39;s/SystemdCgroup = false/SystemdCgroup = true/g\u0026#39; /etc/containerd/config.toml sudo systemctl restart containerd sudo systemctl status containerd Generating that file replaces any existing configuration, so this step belongs on the fresh lab VMs. Check that the CRI plugin is enabled, too. The Kubernetes runtime documentation shows different configuration sections for containerd 1.x and 2.x. Read the resulting file rather than assuming the old sed command changed the right setting.\nThis lab disables swap:\nsudo swapoff -a That change lasts until reboot. If the VM enables swap through /etc/fstab, the corresponding swap entry also needs to be disabled for this setup. Other systems may use a different swap service. See the kubeadm prerequisites before preparing new nodes.\nThe original Kubernetes package repository is gone The following commands explain how the 2023 lab installed Kubernetes. They no longer form a working package installation path. The Kubernetes project removed its old Google-hosted repositories in March 2024.\nsudo apt-get update sudo apt-get install -y apt-transport-https curl curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - cat \u0026lt;\u0026lt;EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list deb https://apt.kubernetes.io/ kubernetes-xenial main EOF sudo apt-get update sudo apt-get install -y kubelet=1.26.4-00 kubeadm=1.26.4-00 kubectl=1.26.4-00 sudo apt-mark hold kubelet kubeadm kubectl The hold applies to those three packages, not to every package on the machine. Its purpose was to keep Kubernetes upgrades deliberate. If a package manager is already running, wait for it to finish instead of deleting its lock file.\nFor a new cluster, follow the current kubeadm installation instructions, including the package repository for your chosen Kubernetes minor release.\nInitialize the control plane, then add pod networking The next commands belong on the control plane node. The original pod network was 172.16.0.0/16; check that it does not overlap your VM network or other networks the cluster must reach.\nsudo kubeadm init --pod-network-cidr 172.16.0.0/16 --kubernetes-version 1.26.4 Copy the administrator kubeconfig so your normal user can run kubectl:\nmkdir -p \u0026#34;$HOME/.kube\u0026#34; sudo cp -i /etc/kubernetes/admin.conf \u0026#34;$HOME/.kube/config\u0026#34; sudo chown \u0026#34;$(id -u):$(id -g)\u0026#34; \u0026#34;$HOME/.kube/config\u0026#34; That file grants administrative access to the cluster. Keep it private.\nThe original lab installed Calico from a versioned manifest:\nkubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.25.1/manifests/calico.yaml Before reproducing the lab, inspect that manifest and verify the resulting Calico IP pool against the pod CIDR. The Calico installation guide explains pod CIDR configuration and the supported installation methods for a new cluster.\nCheck the node and network pods:\nkubectl get nodes kubectl get pods -n kube-system A node that stays NotReady needs investigation before you continue. The original notes included a restart of kubelet and containerd, but restarting both services is not a substitute for finding the failure in their logs.\nJoin the workers with the command from this cluster On the control plane, print a fresh join command:\nsudo kubeadm token create --print-join-command Run the resulting sudo kubeadm join ... command on each worker. Use the endpoint, token, and CA hash from your own cluster; the dots here are a placeholder, not a complete command. Treat the join token as a credential.\nBack on the control plane, check the result:\nkubectl get nodes You should now have three nodes in the Ready state. That is the checkpoint for joining the workers. Testing access from outside the cluster comes later.\nKeep MetalLB and its address pool in the same namespace The original lab used the Bitnami MetalLB chart, version 4.4.1. Its install command used namespace metallb, while the address pool below used metallb-system. Those names must agree.\nHere is the historical command with that mismatch corrected:\nhelm install metallb-system bitnami/metallb \\ --namespace metallb-system \\ --create-namespace \\ --version 4.4.1 This assumes the bitnami repository alias is already configured and that the old chart and images remain available. Their availability has not been verified. For a new installation, start with MetalLB\u0026rsquo;s own Helm chart and installation instructions; do not substitute a chart version without reviewing its configuration.\nThe pool tells MetalLB which addresses it may allocate. Save the following historical example as ipaddresspool.yaml:\napiVersion: metallb.io/v1beta1 kind: IPAddressPool metadata: name: first-pool namespace: metallb-system spec: addresses: - 192.168.20.0/24 The /24 is the original example, not a range you should copy into your network. Reserve unused addresses for MetalLB and keep them separate from DHCP leases, node addresses, and the pod network.\nThe original apply step was:\nkubectl apply -f ipaddresspool.yaml An address pool still needs an advertisement This is where the first version stopped too early. An IPAddressPool gives MetalLB addresses to allocate, but those addresses also need to be announced. In Layer 2 mode, associate an L2Advertisement with the pool. A BGP setup needs its own peer and advertisement configuration. The MetalLB configuration guide describes both paths.\nThe old notes also proposed this router entry:\nNetwork/host IP Netmask Gateway 192.168.20.0 255.255.255.0 \u0026lt;K8s NODE IP\u0026gt; Keep that table as a record of the lab, not a general routing recipe. A route through an arbitrary node does not replace an advertisement or establish failover. The intended Layer 2 or BGP topology still needs to be documented and tested.\nA rebuilt lab needs a test LoadBalancer service. Check its assigned address, then try reaching the application from another machine. Also test what happens when the node handling traffic becomes unavailable. Those results would make the missing network behavior visible; they are still pending for this article.\n","date":"2023-04-29T00:00:00Z","image":"/posts/setup-kubernetes-cluster-with-metallb/editorial-cover.png","permalink":"/posts/setup-kubernetes-cluster-with-metallb/","title":"Set up a Kubernetes cluster with MetalLB"},{"content":"A server certificate is only one part of a trust chain. The server presents it, an intermediate CA signs it, and the client needs a trusted root to verify the chain.\nThis 2023 lab uses CFSSL, Cloudflare\u0026rsquo;s PKI toolkit, to create those three certificates. The example domain is example.com. Creating this private CA does not make browsers trust it, and generating the files does not configure a web server.\nExample CA (root) └── Example Intermediate CA └── example.com (server) The original article did not record its CFSSL version. Keep that limit in mind when reproducing it. You\u0026rsquo;ll need cfssl and cfssljson; use the upstream installation instructions for your platform.\nUse a disposable lab directory. These commands write private keys to disk, so keep the directory out of Git and shared storage. A production CA also needs a plan for key protection, renewal and revocation; this lab does not provide one.\nKeep the root, intermediate and server files separate The relative paths below assume you start with these sibling directories:\nmkdir root intermediate server cd root Describe the root before generating its key In root, create ca_csr.json. The subject fields describe the CA; the key block requests an RSA key. These are the names and key settings from the original lab.\n{ \u0026#34;CN\u0026#34;: \u0026#34;Example CA\u0026#34;, \u0026#34;key\u0026#34;: { \u0026#34;algo\u0026#34;: \u0026#34;rsa\u0026#34;, \u0026#34;size\u0026#34;: 2048 }, \u0026#34;names\u0026#34;: [ { \u0026#34;C\u0026#34;: \u0026#34;AM\u0026#34;, \u0026#34;L\u0026#34;: \u0026#34;Yerevan\u0026#34;, \u0026#34;O\u0026#34;: \u0026#34;Thalamus\u0026#34;, \u0026#34;OU\u0026#34;: \u0026#34;IT\u0026#34;, \u0026#34;ST\u0026#34;: \u0026#34;Yerevan\u0026#34; } ] } A signing profile decides what a certificate may do Still in root, create ca-config.json. This file keeps the original profiles for reference. The lab uses intermediate_ca and server; peer and client are unused.\nThe intermediate\u0026rsquo;s is_ca flag allows it to act as a CA. Together, max_path_len: 0 and max_path_len_zero: true prevent it from issuing another subordinate CA. CFSSL documents this pair in its signing configuration reference.\nThe intermediate profile also contains server and client authentication usages. Those broad permissions are part of the historical example, not a suggested production policy. Review each CA\u0026rsquo;s permitted uses before adapting it.\n{ \u0026#34;signing\u0026#34;: { \u0026#34;default\u0026#34;: { \u0026#34;expiry\u0026#34;: \u0026#34;8760h\u0026#34; }, \u0026#34;profiles\u0026#34;: { \u0026#34;intermediate_ca\u0026#34;: { \u0026#34;usages\u0026#34;: [ \u0026#34;signing\u0026#34;, \u0026#34;digital signature\u0026#34;, \u0026#34;key encipherment\u0026#34;, \u0026#34;cert sign\u0026#34;, \u0026#34;crl sign\u0026#34;, \u0026#34;server auth\u0026#34;, \u0026#34;client auth\u0026#34; ], \u0026#34;expiry\u0026#34;: \u0026#34;8760h\u0026#34;, \u0026#34;ca_constraint\u0026#34;: { \u0026#34;is_ca\u0026#34;: true, \u0026#34;max_path_len\u0026#34;: 0, \u0026#34;max_path_len_zero\u0026#34;: true } }, \u0026#34;peer\u0026#34;: { \u0026#34;usages\u0026#34;: [ \u0026#34;signing\u0026#34;, \u0026#34;digital signature\u0026#34;, \u0026#34;key encipherment\u0026#34;, \u0026#34;client auth\u0026#34;, \u0026#34;server auth\u0026#34; ], \u0026#34;expiry\u0026#34;: \u0026#34;8760h\u0026#34; }, \u0026#34;server\u0026#34;: { \u0026#34;usages\u0026#34;: [ \u0026#34;signing\u0026#34;, \u0026#34;digital signature\u0026#34;, \u0026#34;key encipherment\u0026#34;, \u0026#34;server auth\u0026#34; ], \u0026#34;expiry\u0026#34;: \u0026#34;8760h\u0026#34; }, \u0026#34;client\u0026#34;: { \u0026#34;usages\u0026#34;: [ \u0026#34;signing\u0026#34;, \u0026#34;digital signature\u0026#34;, \u0026#34;key encipherment\u0026#34;, \u0026#34;client auth\u0026#34; ], \u0026#34;expiry\u0026#34;: \u0026#34;8760h\u0026#34; } } } } The root signs its own certificate From root, run the original generation command:\ncfssl gencert -initca ca_csr.json | cfssljson -bare ca - CFSSL emits JSON; cfssljson writes ca.pem, ca-key.pem and ca.csr. The certificate can be shared with clients that should trust this CA. The private key must stay private. See the CFSSL output-file reference for the naming rules.\nThe 8760h signing profile above does not set the lifetime of this self-signed root: this command does not read ca-config.json. CFSSL\u0026rsquo;s CA initialization code uses the request\u0026rsquo;s CA settings or its initialization defaults. Inspect the resulting certificate instead of assuming its expiry.\nThe root signs the intermediate\u0026rsquo;s request Move to intermediate and create intermediate.json:\ncd ../intermediate/ { \u0026#34;CN\u0026#34;: \u0026#34;Example Intermediate CA\u0026#34;, \u0026#34;key\u0026#34;: { \u0026#34;algo\u0026#34;: \u0026#34;rsa\u0026#34;, \u0026#34;size\u0026#34;: 2048 }, \u0026#34;names\u0026#34;: [ { \u0026#34;C\u0026#34;: \u0026#34;AM\u0026#34;, \u0026#34;L\u0026#34;: \u0026#34;Yerevan\u0026#34;, \u0026#34;O\u0026#34;: \u0026#34;Thalamus\u0026#34;, \u0026#34;OU\u0026#34;: \u0026#34;IT\u0026#34;, \u0026#34;ST\u0026#34;: \u0026#34;Yerevan\u0026#34; } ], \u0026#34;ca\u0026#34;: { \u0026#34;expiry\u0026#34;: \u0026#34;42720h\u0026#34; } } The two commands below do different jobs. The first creates the intermediate\u0026rsquo;s key, CSR and an initial self-signed certificate. The second signs that CSR with the root and replaces the intermediate certificate with the root-signed one.\ncfssl gencert -initca intermediate.json | cfssljson -bare intermediate_ca cfssl sign -ca ../root/ca.pem \\ -ca-key ../root/ca-key.pem \\ -config ../root/ca-config.json \\ -profile intermediate_ca intermediate_ca.csr | cfssljson -bare intermediate_ca There are two expiry values here. The request\u0026rsquo;s 42720h applies to the initial self-signed certificate. The final signing command selects intermediate_ca, whose expiry is 8760h (365 days). Do not read the request\u0026rsquo;s longer value as the final intermediate\u0026rsquo;s lifetime.\nPut the server\u0026rsquo;s DNS name in the request Move to server and create example.json:\ncd ../server/ { \u0026#34;CN\u0026#34;: \u0026#34;example.com\u0026#34;, \u0026#34;key\u0026#34;: { \u0026#34;algo\u0026#34;: \u0026#34;rsa\u0026#34;, \u0026#34;size\u0026#34;: 2048 }, \u0026#34;names\u0026#34;: [ { \u0026#34;C\u0026#34;: \u0026#34;AM\u0026#34;, \u0026#34;L\u0026#34;: \u0026#34;Yerevan\u0026#34;, \u0026#34;O\u0026#34;: \u0026#34;Thalamus\u0026#34;, \u0026#34;OU\u0026#34;: \u0026#34;IT\u0026#34;, \u0026#34;ST\u0026#34;: \u0026#34;Yerevan\u0026#34; } ], \u0026#34;hosts\u0026#34;: [ \u0026#34;example.com\u0026#34; ] } The hosts entry supplies the DNS name for the Subject Alternative Name extension. Replace it with the name your client will connect to. Do not rely on changing only CN.\nSign the request with the intermediate\u0026rsquo;s certificate and private key:\ncfssl gencert -ca ../intermediate/intermediate_ca.pem \\ -ca-key ../intermediate/intermediate_ca-key.pem \\ -config ../root/ca-config.json \\ -profile=server example.json | cfssljson -bare example This produces the server certificate and key. Keep both CA private keys away from the web server. The server needs its own key and the certificate chain required by its TLS configuration.\nCheck the chain before changing system trust The original article jumped straight to installing the root. You can inspect the chain without doing that. From server, the following diagnostic checks the certificate for the example hostname and TLS server purpose:\nopenssl verify -CAfile ../root/ca.pem \\ -untrusted ../intermediate/intermediate_ca.pem \\ -purpose sslserver -verify_hostname example.com example.pem -CAfile identifies the trusted root for this command. -untrusted supplies the intermediate so OpenSSL can build the chain; it does not make the intermediate a trust anchor. An OK result means this verification passed, not that a web server has been configured or tested. The OpenSSL verification manual explains the options.\nAlso inspect validity dates and extensions with openssl x509 -in example.pem -noout -text, then inspect each CA certificate. Check the DNS name, issuer, CA constraints and expiry dates. These inspection steps have not been run against a recreated version of this lab.\nInstalling a root changes what the machine trusts Only install this root on a machine where you intend to trust certificates issued beneath it. The original Debian/Ubuntu commands were:\nsudo cp ../root/ca.pem /usr/local/share/ca-certificates/example.crt sudo update-ca-certificates --fresh Copy ca.pem, never ca-key.pem. Debian\u0026rsquo;s update-ca-certificates manual requires PEM certificates with a .crt extension in this directory, one certificate per file. The original --fresh option rebuilds the certificate symlinks; it is not required just to add this root. Applications with a separate trust store may need their own configuration.\nTo take this lab further, configure a local TLS server to send its server certificate and intermediate, then check the connection from a client that trusts the root. Check the hostname and the chain the server sends. A successful local certificate check alone cannot tell you whether that connection works.\n","date":"2023-01-12T00:00:00Z","image":"/posts/setting-up-a-certificate-authority-ca-hierarchy-with-cfssl/editorial-cover.png","permalink":"/posts/setting-up-a-certificate-authority-ca-hierarchy-with-cfssl/","title":"Build a certificate authority hierarchy with CFSSL"},{"content":"This lab follows a microservices demo from three Ubuntu machines to an application deployed on Kubernetes. Rancher provides the interface for creating the cluster and registering its nodes.\nThe notes and screenshots come from my 2020 demo. The package repositories and Rancher screens have changed since then. The sequence is still useful to read, but it needs a fresh lab run before it can serve as a current tutorial.\nThe three nodes have different jobs The original cluster assigned these roles:\nNode Roles kserver1 etcd, control plane kserver2 etcd, control plane, worker kserver3 etcd, worker The control plane manages the cluster. etcd stores its state. Workers run the application workloads. A node can have more than one role, as kserver2 does here.\nKubernetes diagram You can follow the recorded demo alongside the commands and screenshots below.\nCreating Microservices Deployments▶ Load YouTube videoLoads content from YouTube when you choose to play. Open on YouTube ↗ Docker came first in this lab These commands came from the Docker installation flow used at the time. They include an old apt-key setup and assume an amd64 Ubuntu machine. For a new host, follow the current Docker Ubuntu instructions instead of copying this repository configuration.\nThe first step removed conflicting packages on the lab machines:\nsudo apt remove docker docker-engine docker.io containerd runc That changes the container software installed on the host. Don\u0026rsquo;t treat it as a routine preparation step for a machine already running containers.\nThe remaining historical installation commands were:\nsudo apt install apt-transport-https ca-certificates curl gnupg-agent software-properties-common curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - sudo apt-key fingerprint 0EBFCD88 sudo add-apt-repository \u0026#34;deb [arch=amd64] https://download.docker.com/linux/ubuntu \\ $(lsb_release -cs) \\ stable\u0026#34; sudo apt update sudo apt install docker-ce docker-ce-cli containerd.io The Docker packages were not pinned, so running this again would not reproduce the same versions. The Kubernetes runtime setup has changed, too. A current installation needs a compatible CRI runtime; Docker Engine needs an adapter to provide that interface. The Kubernetes runtime documentation explains that requirement.\nApply the network setting from the file you just wrote The lab enabled bridge filtering on all three nodes. Load the bridge module before setting its parameter:\nsudo modprobe br_netfilter cat \u0026lt;\u0026lt;EOF | sudo tee /etc/sysctl.d/k8s.conf net.bridge.bridge-nf-call-iptables = 1 EOF sudo sysctl -p /etc/sysctl.d/k8s.conf The filename in the last command matters. The old article used sysctl -p on its own, which reads /etc/sysctl.conf rather than the new file in sysctl.d. See the Ubuntu sysctl manual for that behavior. A rebuild should also check that the required module loads after reboot.\nThe original setup disabled swap on every server:\nsudo swapoff -a sudo vi /etc/fstab swapoff changes the running system. Editing the relevant swap entry in /etc/fstab prevents that entry from enabling it again at boot. Leave unrelated mounts alone, and check how the VM actually manages swap before changing the file.\nThe kubectl package commands are historical The lab installed kubectl version 1.15.7-00 from the old Kubernetes repository:\ncurl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - cat \u0026lt;\u0026lt;EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list deb https://apt.kubernetes.io/ kubernetes-xenial main EOF sudo apt update sudo apt install -y kubectl=1.15.7-00 That repository is no longer available. Kubernetes removed it in March 2024. For a new lab, choose a client compatible with the cluster and use the current kubectl installation instructions.\nRancher manages the cluster from a separate interface The original demo started Rancher as a privileged Docker container:\ndocker run -d --name rancher --privileged --restart=unless-stopped \\ -p 80:80 -p 443:443 rancher/rancher:stable This is the original command, not a pinned release you can use to reproduce the old environment. The stable tag can move, and the exact Rancher build was not recorded. Rancher\u0026rsquo;s single-node Docker installation is intended for testing and development. A production setup needs the supported installation design for its chosen release.\nThe following screenshots keep the original flow. Start by setting the administrator password and the URL that the nodes will use to reach Rancher:\nSet a new password Setup Rancher URL Then create the cluster and give it a name:\nCreate cluster Create cluster Specify cluster name Choose the node roles from the table above. The registration command generated by Rancher belongs to that Rancher instance and cluster; don\u0026rsquo;t reuse a command copied from a screenshot.\nChoose cluster role Once the nodes have registered, obtain the kubeconfig for the cluster:\nCopy kubeconfig file Copy kubeconfig file Keep the kubeconfig private. From the machine where you configured it, check which cluster you are using and whether the nodes are ready:\nkubectl config current-context kubectl get nodes Continue only when the context points to the lab cluster and all three expected nodes are Ready.\nThe TLS Secret must match the ingress configuration The original demo generated a self-signed certificate and stored it in a Kubernetes Secret:\nopenssl req -x509 -nodes -days 1095 -newkey rsa:2048 \\ -keyout thalamus.key -out thalamus.crt kubectl create secret tls thalamus-tls --cert=thalamus.crt --key=thalamus.key This is incomplete as a reusable HTTPS example. The command does not explicitly set a hostname or a Subject Alternative Name. The OpenSSL request documentation shows how to set certificate extensions. A rebuilt demo needs a certificate with the intended DNS name and a browser that trusts its issuer. Keep the private key out of Git.\nThe Secret must live in the same namespace as the Ingress, which must reference thalamus-tls. You also need an ingress controller to handle requests. Check how the hostname and Secret connect in the Kubernetes Ingress documentation.\nInspect the application manifests before applying them The demo used the Thalamus microservices repository. The original SSH clone command requires GitHub access through your SSH key:\ngit clone git@github.com:Thalamus-am/microservices-demo.git cd microservices-demo/release/ The notes did not record a commit. Before repeating the deployment, inspect kubernetes-manifests-ingress.yaml at a specific revision. Check its API versions, images, namespace, ingress host, and TLS Secret reference. The repository\u0026rsquo;s current contents have not been revalidated for this article.\nThe historical deployment step was:\nkubectl apply -f kubernetes-manifests-ingress.yaml For a future lab run, check the workloads and service path after applying the reviewed manifest:\nkubectl get pods kubectl get services kubectl get ingress Use the application\u0026rsquo;s namespace if it differs from the current context. If the pods are ready but the browser cannot reach the application, follow the request from DNS to the ingress controller. Then check whether TLS presents the certificate you expected. A rebuilt version of this lab should show that last part working, including the checks used to get there.\n","date":"2020-05-09T00:00:00Z","image":"/posts/creating-microservices-deployments/editorial-cover.png","permalink":"/posts/creating-microservices-deployments/","title":"Deploy a microservices demo with Rancher"},{"content":"In this Java CI/CD lab, Jenkins runs the build jobs, Docker builds container images, and Nexus stores them. Nginx gives Jenkins and Nexus their own HTTPS addresses. Before connecting the pipeline, each service needs to work on its own.\nI split the original project into two parts: install the components by hand first, then automate the setup with Terraform and Ansible. This page covers the manual installation recorded in 2020. It stops before the Java build job and Kubernetes deployment.\nThe original environment was an Ubuntu 18.04 VM on Google Compute Engine. The commands and screenshots below explain that environment. Several package repositories and runtime requirements have changed, so use the linked installation guides when building a new lab. The full setup has not been rerun against current releases.\nEach service has a separate job CI/CD part 1 plan Component Job in this lab Google Compute Engine Hosts the Ubuntu VM. Jenkins Runs build jobs and calls the other tools. Docker Builds and runs containers. Nexus Repository Stores container images in a hosted Docker repository. Nginx Accepts HTTPS requests and forwards them to the right local service. kubectl Talks to a Kubernetes API server when a later deployment step needs it. Keep the three service addresses separate. jenkins.thalamus.am goes to Jenkins on port 8080. nexus.thalamus.am goes to the Nexus web interface on port 8081. dockerhub.thalamus.am goes to the Docker repository connector on port 8123. Despite its name, that last address is the lab\u0026rsquo;s private registry, not Docker Hub.\nThose names belong to the original lab. Use hostnames you control, DNS records that point to your VM, and certificates that cover those names. You\u0026rsquo;ll also need sudo access and a private test environment. Keep the backend ports out of public firewall rules; clients should reach them through the proxy.\nThe recording follows the original installation CI/CD for Java project — part 1.1▶ Load YouTube videoLoads content from YouTube when you choose to play. Open on YouTube ↗ The recording and screenshots use the interfaces available at the time. The notes below flag places where copying the old commands into a new VM would cause trouble.\nDocker needs its own package repository The original Docker installation used apt-key and the Ubuntu 18.04 repository setup. Docker\u0026rsquo;s Ubuntu installation guide now uses a separate keyring and a Signed-By repository entry. Follow that guide for a supported Ubuntu release.\nFor reference, these were the package steps in the historical lab:\n# Historical Ubuntu 18.04 instructions, not a current installation recipe. sudo apt remove docker docker-engine docker.io containerd runc sudo apt install apt-transport-https ca-certificates curl gnupg-agent software-properties-common curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - sudo apt-key fingerprint 0EBFCD88 sudo add-apt-repository \u0026#34;deb [arch=amd64] https://download.docker.com/linux/ubuntu \\ $(lsb_release -cs) \\ stable\u0026#34; sudo apt update sudo apt install docker-ce docker-ce-cli containerd.io On a fresh lab VM, check Docker before introducing Jenkins:\nsudo systemctl status docker sudo docker run --rm hello-world The test container should print its confirmation and exit. If it can\u0026rsquo;t connect to the Docker daemon, fix the service first. If the image can\u0026rsquo;t be downloaded, check the VM\u0026rsquo;s network and registry access.\nJenkins needs a Java runtime that matches its release The lab installed OpenJDK 8 before Jenkins. That combination is historical: Java 8 does not meet current Jenkins runtime requirements. Choose the Java runtime and Jenkins release together using the Jenkins Linux installation guide. The Java version used to build your application is a separate choice from the runtime that starts Jenkins, although individual build plugins can impose extra requirements.\nThe old package commands were:\n# Historical Jenkins repository and Java version. sudo apt install openjdk-8-jdk wget -q -O - https://pkg.jenkins.io/debian/jenkins.io.key | sudo apt-key add - sudo sh -c \u0026#39;echo deb https://pkg.jenkins.io/debian binary/ \u0026gt; \\ /etc/apt/sources.list.d/jenkins.list\u0026#39; sudo apt update sudo apt install jenkins The original lab also gave the Jenkins service account access to Docker:\nsudo usermod -aG docker jenkins sudo systemctl restart jenkins Restarting the service lets its new process pick up the group membership. This is a lab shortcut with a large consequence: the Docker group grants root-level privileges. A build with access to that socket can control the host. For a shared Jenkins installation, run builds on separate agents and review the controller isolation guidance before granting Docker access.\nCheck that Jenkins starts:\nsudo systemctl status jenkins sudo journalctl -u jenkins -n 50 --no-pager Read the initial unlock password on the VM:\nsudo cat /var/lib/jenkins/secrets/initialAdminPassword Use that password in the setup wizard. It is a temporary bootstrap credential, not a password to reuse in a pipeline.\nJenkins initial unlock password The recording installs the suggested plugins:\nInstall suggested plugins Then it creates an administrator account:\nCreate admin user Once the dashboard opens, the Jenkins installation is ready for the proxy step. There is no build job yet.\nNginx sends each hostname to the right backend The original Nginx configuration files are part of the project repository.\nThe recording created a self-signed certificate with this command:\n# Original lab command: it does not configure client trust or DNS SANs. sudo openssl req -x509 -nodes -days 1095 -newkey rsa:2048 \\ -keyout /etc/ssl/private/thalamus.key \\ -out /etc/ssl/certs/thalamus.crt That command alone is not enough for a usable registry certificate. For a new lab, issue a certificate with subject alternative names for your chosen hostnames and configure the clients to trust its issuer. Docker documents how to trust a registry\u0026rsquo;s CA certificate. For public services, use a certificate from a CA your clients already trust. Keep the private key readable only by the accounts that need it.\nInstall Nginx, then disable the packaged default site if it is still the unused default on this lab VM:\nsudo apt install nginx sudo unlink /etc/nginx/sites-enabled/default Check the symlink before removing it on an existing server. Disabling the site this way leaves the original file in sites-available intact.\nCreate /etc/nginx/sites-available/jenkins with the following lab configuration. Replace the hostnames and certificate paths for your environment.\nserver { listen 80; listen [::]:80; server_name jenkins.thalamus.am; return 301 https://jenkins.thalamus.am$request_uri; } server { listen 443 ssl; server_name jenkins.thalamus.am; ssl_certificate /etc/ssl/certs/thalamus.crt; ssl_certificate_key /etc/ssl/private/thalamus.key; access_log /var/log/nginx/jenkins.access.log; error_log /var/log/nginx/jenkins.error.log; location / { proxy_pass http://127.0.0.1:8080; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; } } Use listen 443 ssl to enable TLS on the listener. The old ssl on; directive is absent here because Nginx removed it in 1.25.1.\nThis small configuration shows the routing used in the recording. A new Jenkins setup should use the maintained Jenkins Nginx example, which also handles WebSocket agents and HTTP CLI requests. Set Jenkins\u0026rsquo;s own URL to the external HTTPS address and restrict direct access to port 8080.\nEnable the site, validate the configuration, and reload only if validation succeeds:\nsudo ln -s /etc/nginx/sites-available/jenkins /etc/nginx/sites-enabled/jenkins sudo nginx -t \u0026amp;\u0026amp; sudo systemctl reload nginx If the symlink already exists, edit the existing file instead of creating another link. A 502 Bad Gateway response usually means Nginx couldn\u0026rsquo;t get a usable response from the backend; check Jenkins on 127.0.0.1:8080 and read both services\u0026rsquo; logs.\nNexus stores its data outside the container The lab uses Sonatype\u0026rsquo;s Nexus Repository container image. Its upstream container documentation explains the /nexus-data directory and startup behavior.\nCreate a named volume for repository data. The original command used the unversioned sonatype/nexus3 image, so it did not record a reproducible Nexus release. Choose and test a specific release before using this example; replace the placeholder below with that version.\nNEXUS_IMAGE=\u0026#39;sonatype/nexus3:REPLACE_WITH_A_TESTED_VERSION\u0026#39; sudo docker volume create nexus sudo docker run -d \\ -p 127.0.0.1:8081:8081 \\ -p 127.0.0.1:8123:8123 \\ --name nexus \\ --restart=always \\ -v nexus:/nexus-data \\ \u0026#34;$NEXUS_IMAGE\u0026#34; Port 8081 serves the web interface. Port 8123 will serve the Docker repository after you create its connector. The loopback bindings keep these mappings behind the local Nginx proxy rather than publishing them on every host interface. Check your Docker version and network rules too: Docker\u0026rsquo;s port publishing guide documents a localhost exposure issue in versions before 28.0.0.\nThe volume holds repository content and configuration when the container is replaced. A volume is not a backup; don\u0026rsquo;t delete it while cleaning up the container.\nWatch the startup log before opening the web interface:\nsudo docker logs --tail 100 -f nexus Wait for Nexus to finish starting. If it exits instead, inspect the log and check the selected release\u0026rsquo;s memory and storage requirements.\nCreate /etc/nginx/sites-available/nexus:\nserver { listen 80; server_name nexus.thalamus.am; return 301 https://nexus.thalamus.am$request_uri; } server { listen 443 ssl; server_name nexus.thalamus.am; ssl_certificate /etc/ssl/certs/thalamus.crt; ssl_certificate_key /etc/ssl/private/thalamus.key; access_log /var/log/nginx/nexus.access.log; error_log /var/log/nginx/nexus.error.log; location / { proxy_pass http://127.0.0.1:8081; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; } } Enable it using the same check-before-reload sequence:\nsudo ln -s /etc/nginx/sites-available/nexus /etc/nginx/sites-enabled/nexus sudo nginx -t \u0026amp;\u0026amp; sudo systemctl reload nginx Read the bootstrap password from inside the container:\nsudo docker exec nexus cat /nexus-data/admin.password This avoids depending on where Docker mounted the volume on the host. Use the bootstrap password only for initial setup. For an instance you have already configured, use its existing credentials.\nLogin to Nexus Sign in as admin and follow the setup wizard:\nNexus setup Replace the bootstrap password with your own administrator password:\nChange Nexus admin password The Docker repository has a separate connector In Nexus, create a hosted Docker repository. A hosted repository accepts the images your pipeline pushes; the repository\u0026rsquo;s name and its connector port are separate settings.\nCreate Docker repository Configure Docker repository Choose a unique repository name and set its HTTP connector to 8123 to match the container port mapping. Nginx handles external HTTPS, so this connector receives plain HTTP on the local backend.\nThe old screenshot includes an optional Docker V1 setting. Check the needs of your clients against Sonatype\u0026rsquo;s Docker registry documentation rather than treating every checkbox in the screenshot as required. That guide also covers registry authentication and connector choices.\nDocker repository connector Create /etc/nginx/sites-available/dockerhub for the registry address:\nserver { listen 80; server_name dockerhub.thalamus.am; return 301 https://dockerhub.thalamus.am$request_uri; } server { listen 443 ssl; server_name dockerhub.thalamus.am; proxy_send_timeout 120; proxy_read_timeout 300; proxy_buffering off; tcp_nodelay on; server_tokens off; client_max_body_size 1G; ssl_certificate /etc/ssl/certs/thalamus.crt; ssl_certificate_key /etc/ssl/private/thalamus.key; keepalive_timeout 60; access_log /var/log/nginx/dockerhub.access.log; error_log /var/log/nginx/dockerhub.error.log; location / { proxy_pass http://127.0.0.1:8123; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; } } The 1G upload limit is a lab setting. Choose a limit that fits the image layers your pipeline needs to push. The snippets here explain the proxy path; they are not a complete TLS or registry security configuration.\nEnable the registry site:\nsudo ln -s /etc/nginx/sites-available/dockerhub /etc/nginx/sites-enabled/dockerhub sudo nginx -t \u0026amp;\u0026amp; sudo systemctl reload nginx Opening the Nexus web interface proves that port 8081 works. It doesn\u0026rsquo;t prove that the Docker connector on 8123 is ready. Before adding a pipeline, test registry authentication and a push/pull using a dedicated account with the repository permissions it needs. Follow the selected Nexus release\u0026rsquo;s authentication steps and configure certificate trust on the Docker client.\nkubectl is a client, not a cluster installer The original article installed kubectl version 1.15.7-00 from the old Google-hosted package repository:\n# Historical reference only: this package repository is no longer available. curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - cat \u0026lt;\u0026lt; EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list deb https://apt.kubernetes.io/ kubernetes-xenial main EOF sudo apt update sudo apt install -y kubectl=1.15.7-00 That repository was removed in March 2024. These commands cannot install the client today. Use the current kubectl Linux installation guide and select a client within one minor version of the cluster\u0026rsquo;s API server.\nAfter installing a compatible client, check which binary you have:\nkubectl version --client This checks the local tool. It does not create a Kubernetes cluster or prove that Jenkins can deploy to one. Deployment also needs a cluster, a kubeconfig, and an identity with suitable permissions. Keep those credentials out of source code.\nTrace one image before automating the setup At this point, the lab has places to run jobs and store images. It still needs the Java build job, repository credentials, and a deployment target before it is a working delivery pipeline.\nBefore you automate it, draw the path of one image push: the registry hostname, the Nginx listener, the Nexus connector, and the data volume. Then pick one failure, such as a 502 response, and identify the first log you would read. That\u0026rsquo;s a small exercise, but it tells you whether you understand how the pieces connect.\nThe next step is to test that path in your own lab, record the versions that worked, and only then automate those choices with Terraform and Ansible.\n","date":"2020-04-12T00:00:00Z","image":"/posts/ci-cd-for-java-project-manual-installation/editorial-cover.png","permalink":"/posts/ci-cd-for-java-project-manual-installation/","title":"CI/CD for a Java project: manual installation"},{"content":"A swipe can switch workspaces by triggering the same shortcut you would press on the keyboard. This note records two setups from 2020: libinput-gestures for a touchpad, and an older Touchégg configuration for a touchscreen.\nThe original Ubuntu/Linux Mint release and tool versions were not recorded. The examples below explain the setup, but they still need testing on a specified desktop and release.\nCheck your desktop\u0026rsquo;s own gestures first GNOME documents built-in touchpad and touchscreen gestures. Try the actions you need before adding another gesture service.\nThe xdotool workspace shortcuts below target X11. They do not control a native GNOME Wayland desktop. Touchégg\u0026rsquo;s upstream FAQ also limits it to X11. A gesture configuration that worked on an older Ubuntu session may therefore be the wrong tool for your current session.\nA touchpad swipe becomes a keyboard shortcut In this setup, libinput reports a gesture, libinput-gestures matches a configuration line, and xdotool sends the keys. If the keyboard shortcut doesn\u0026rsquo;t switch workspaces by itself, mapping a swipe to it won\u0026rsquo;t help.\nTry Ctrl+Alt+Left and Ctrl+Alt+Right in the intended X11 session first. Check your desktop\u0026rsquo;s keyboard settings if they do something else.\nGive the gesture reader access to the device The original setup added the desktop user to the input group:\nsudo gpasswd -a \u0026#34;$USER\u0026#34; input This grants access to input devices, including more than the touchpad. Read the project\u0026rsquo;s permissions warning before using it. Group membership needs a new session; the upstream instructions request a reboot. Save your work before restarting.\nInstall the tools used by the old setup The original package command was:\nsudo apt-get install xdotool wmctrl libinput-tools The clone step also needs Git. These are the historical source-install commands; cloning the default branch today will not recover the version used in 2020. Follow the project\u0026rsquo;s installation notes for a new setup and inspect the installer before running it with sudo.\ncd ~ git clone https://github.com/bulletmark/libinput-gestures .libinput-gestures cd ~/.libinput-gestures sudo ./libinput-gestures-setup install Keep the checkout if you want to inspect or update that installation. The old notes deleted it immediately after installation; that cleanup is unnecessary.\nPut your shortcut mappings in a user configuration The original article edited /etc/libinput-gestures.conf. The project also supports a personal copy at ~/.config/libinput-gestures.conf, which avoids changing defaults for other users:\nmkdir -p ~/.config cp /etc/libinput-gestures.conf ~/.config/libinput-gestures.conf vim ~/.config/libinput-gestures.conf Skip the copy if you already have a personal configuration, and edit that file instead. Keep a backup before replacing any existing mappings.\nIn the copy, comment out conflicting gesture lines and add the original mappings:\ngesture swipe up xdotool key ctrl+alt+Up gesture swipe down xdotool key ctrl+alt+Down gesture swipe left xdotool key ctrl+alt+Left gesture swipe right xdotool key ctrl+alt+Right These shortcuts assume the desktop has workspaces in the requested direction. Two-finger scrolling is a separate action; libinput describes swipe gestures as three or more fingers moving together.\nStart the service in your desktop session, after refreshing group membership:\nlibinput-gestures-setup start Once the mappings behave as intended, enable startup at login:\nlibinput-gestures-setup autostart After editing a running setup, use libinput-gestures-setup restart to reload it. If a swipe does nothing, the project\u0026rsquo;s debugging instructions separate missing gesture events from commands that fail to run.\nThe touchscreen example uses the old Touchégg format This March 2020 configuration predates the Touchégg 2.0 rewrite, released that September. Its DRAG actions and inline action values are historical syntax. Do not paste this XML into a current configuration and expect it to work; use the current configuration reference for the installed version.\nThe old package installation command, with the required administrator privilege made explicit, was:\nsudo apt install touchegg A distribution package today may contain a different version from the one behind these notes. For the historical setup, the user configuration lived at ~/.config/touchegg/touchegg.conf:\nvim ~/.config/touchegg/touchegg.conf In this example, a two-finger tap sends a right click and a two-finger drag scrolls. Three-finger drags send the workspace shortcuts. Its five-finger tap closed a window, so leave it disabled while trying the other mappings.\n\u0026lt;touchégg\u0026gt; \u0026lt;settings\u0026gt; \u0026lt;property name=\u0026#34;composed_gestures_time\u0026#34;\u0026gt;0\u0026lt;/property\u0026gt; \u0026lt;/settings\u0026gt; \u0026lt;application name=\u0026#34;All\u0026#34;\u0026gt; \u0026lt;gesture type=\u0026#34;TAP\u0026#34; fingers=\u0026#34;2\u0026#34; direction=\u0026#34;\u0026#34;\u0026gt; \u0026lt;action type=\u0026#34;MOUSE_CLICK\u0026#34;\u0026gt;BUTTON=3\u0026lt;/action\u0026gt; \u0026lt;/gesture\u0026gt; \u0026lt;gesture type=\u0026#34;TAP\u0026#34; fingers=\u0026#34;5\u0026#34; direction=\u0026#34;\u0026#34;\u0026gt; \u0026lt;action type=\u0026#34;CLOSE_WINDOW\u0026#34;\u0026gt;\u0026lt;/action\u0026gt; \u0026lt;/gesture\u0026gt; \u0026lt;gesture type=\u0026#34;DRAG\u0026#34; fingers=\u0026#34;2\u0026#34; direction=\u0026#34;ALL\u0026#34;\u0026gt; \u0026lt;action type=\u0026#34;SCROLL\u0026#34;\u0026gt;SPEED=7:INVERTED=true\u0026lt;/action\u0026gt; \u0026lt;/gesture\u0026gt; \u0026lt;gesture type=\u0026#34;DRAG\u0026#34; fingers=\u0026#34;3\u0026#34; direction=\u0026#34;UP\u0026#34;\u0026gt; \u0026lt;action type=\u0026#34;SEND_KEYS\u0026#34;\u0026gt;Control+Alt+Up\u0026lt;/action\u0026gt; \u0026lt;/gesture\u0026gt; \u0026lt;gesture type=\u0026#34;DRAG\u0026#34; fingers=\u0026#34;3\u0026#34; direction=\u0026#34;DOWN\u0026#34;\u0026gt; \u0026lt;action type=\u0026#34;SEND_KEYS\u0026#34;\u0026gt;Control+Alt+Down\u0026lt;/action\u0026gt; \u0026lt;/gesture\u0026gt; \u0026lt;gesture type=\u0026#34;DRAG\u0026#34; fingers=\u0026#34;3\u0026#34; direction=\u0026#34;LEFT\u0026#34;\u0026gt; \u0026lt;action type=\u0026#34;SEND_KEYS\u0026#34;\u0026gt;Control+Alt+Left\u0026lt;/action\u0026gt; \u0026lt;/gesture\u0026gt; \u0026lt;gesture type=\u0026#34;DRAG\u0026#34; fingers=\u0026#34;3\u0026#34; direction=\u0026#34;RIGHT\u0026#34;\u0026gt; \u0026lt;action type=\u0026#34;SEND_KEYS\u0026#34;\u0026gt;Control+Alt+Right\u0026lt;/action\u0026gt; \u0026lt;/gesture\u0026gt; \u0026lt;/application\u0026gt; \u0026lt;/touchégg\u0026gt; Start any new setup with one action, such as switching to the next workspace. Confirm the shortcut works, then confirm the gesture triggers it. If the swipe fails, you can then check gesture recognition without also wondering whether the shortcut is wrong.\n","date":"2020-03-27T00:00:00Z","image":"/posts/multi-touch-gestures-configuration/editorial-cover.png","permalink":"/posts/multi-touch-gestures-configuration/","title":"Multi-touch gestures configuration"}]