Adding insecure registries to OneKE

Community Support
1

Hello,
I currently have a OneKE installation with one master and one worker.
I’m trying to add an insecure registry (skip tls verification) to be able to deploy images from a private container registry located in my network.

I tried to follow the RKE2 documentation to add insecure registries without result, adding the following:

configs:
  registry-url:5000:
    tls:
      insecure_skip_verify: true

to /etc/rancher/rke2/registries.yaml in each node and restarting rke2-server in the master node and rke2-agent in the worker node.

Other images from public registries are pulled correctly.
I’m able to pull images in my workstation by adding the registry to “insecure-registries” in /etc/docker/daemon.json

Versions of the related components and OS (frontend, hypervisors, VMs):
OneKE 1.27

Steps to reproduce:
Deploy image from insecure container registry

Current results:
Failed to pull image “<registry-url/image>”: rpc error: code = Unknown desc = failed to pull and unpack image “registry-url/image:latest”: failed to resolve reference “registry-url/image:latest”: failed to do request: Head “https://registry-url/v2/image/manifests/latest”: tls: failed to verify certificate: x509

Expected results:
Correct image pull from insecure registry.

2

Hi @SysAdminHorror !

I think your config snippet is not correct for RKE2. Please refer to Containerd Registry Configuration | RKE2

This is some example that worked for me (you have to setup this on nodes that will be running your pod):

/etc/rancher/rke2/registries.yaml:

mirrors:
  asd.123:
    endpoint:
      - "http://192.168.150.1:5000"

Then I executed:

root@oneke-ip-192-168-200-101:~# kubectl run asd --image=asd.123/alpine -it --rm -- /bin/sh
If you don't see a command prompt, try pressing enter.
/ #

I hope that helps. :hugs:

3

Hi @mopala!

I figured it out quite differently, it turns out that the config is actually different between the master and the worker nodes:

If the master node needs access to the registry create the file /etc/rancher/rke2/registries.yaml:

configs:
  "<registry-url>":
    tls:
      insecure_skip_verify: true

Them systemctl restart rke2-server.service

In each worker node that is going to pull images from the registry copy /var/lib/rancher/rke2/agent/etc/containerd/config.toml as /var/lib/rancher/rke2/agent/etc/containerd/config.toml.tmpl and add the following:

[plugins."io.containerd.grpc.v1.cri".registry.configs."<registry-url>".tls]
  insecure_skip_verify = true

Them systemctl restart rke2-agent.service
Refer to Configuring containerd | RKE2.