Act Runner in k8s fail to connect to docker daemon

I got it working - backtracked to the original image i was using 1 hr ago - gitea/act_runner:0.2.9 runs ok with docker:26.1.4-dind

So I used the config.yaml file you specify in your exampleā€¦ and then use the ubuntu-22.04 label to run my loadbuild scripts. My scripts install docker, try to use buildx and the installation went fineā€¦ but then buildx erroredā€¦

ERROR: failed to initialize builder tender_blackwell (tender_blackwell0): Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

I added the ā€˜optionsā€™ section Iā€™d originally missed to my original configmap and still see same type of errorā€¦

ERROR: failed to initialize builder silly_dewdney (silly_dewdney0): 
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. 
Is the docker daemon running?

The runner pod has the DOCKER_HOST set to tcp addressā€¦

I use an image self-hosted in Gitea from here:
gitea/runner-images: Official Gitea Actions runner images - runner-images - Gitea: Git with a cup of tea, ubuntu-act:22.04

The Gitea action in my example is unchanged,

did you try this ?

I havenā€™t tried the whole example as-isā€¦ I tried the configmap.

I believe Iā€™m also use a self-hosted gitea act-runner image for the runner.
image

I then used gitea runner ubuntu22.04 you have in your labels, and I also tried my home-grown ubuntu 22.04 image. Both ubuntu versions have the same socket error.

I will try the gitea action next.

1 Like

I understand

but this label,

is not representing the stock ubuntu 22.04 image.

and in the action you can overide that, with:

jobs:
  build-image:
    name: build-image
    runs-on: ubuntu-latest

    container:
      # image: ghcr.io/catthehacker/ubuntu:act-latest
      # image: gitea/runner-images:ubuntu-latest
      image: gitea.example.com/mariusrugan/ubuntu-act:22.04

The label just allows you to specify runs-on:

name: Build Service
on:
  push:
    branches: [main]
jobs:
  pre-build:
    runs-on: ubuntu-latest
    outputs:

Yeah, I did change the runs-on to a gitea runner when I change the configmap file to the example you postedā€¦

        - "ubuntu-latest:docker://gitea/runner-images:ubuntu-latest"
        - "ubuntu-22.04:docker://gitea/runner-images:ubuntu-22.04"
        - "ubuntu-20.04:docker://gitea/runner-images:ubuntu-20.04"

I got the same socket error. I will double-check.

i understand.

but i think youā€™re not touching the image at all,

do this in your action:

image: gitea/runner-images:ubuntu-22.04

Arh, you meanā€¦ using a container imageā€¦ no, I am not. :blush:

From your exampleā€¦

    container:
      # image: ghcr.io/catthehacker/ubuntu:act-latest
      # image: gitea/runner-images:ubuntu-latest
      image: gitea.example.com/xyz/ubuntu-act:22.04
      credentials:
        username: ${{ gitea.actor }}
        password: ${{ secrets.GA_TOKEN }}
1 Like

And also the stateful set, switch it to gitea/act_runner:0.2.9 because youā€™re using :latest and is not working.

The reason is the latest Dockerfile is changed, and tini doesnā€™t start
here is the reason: act_runner/Dockerfile at main - act_runner - Gitea: Git with a cup of tea

in the stateful set, the #1 container is looking for docker to start, and with :latest this contraption is not working, will fix it and post a message.

The idea was from here:

The fix is very simple:

replace the current command with:

command: ["sh", "-c", "while ! nc -z localhost 2376 </dev/null; do echo 'waiting for docker daemon...'; sleep 5; done; /sbin/tini -- run.sh"]

so image: gitea/act_runner:nightly runs with

Also you can upgrade Docker DinD image to: docker:27.3.1-dind

1 Like

Thanks for the info.

So, I have this workflowā€¦ and Iā€™ve tried with setup.docker.sh installing docker and not installing docker and it fails as soon as I call docker buildx create with the same message about socket.

jobs:
  build:
    runs-on: ubuntu-22.04    
    container:
      image: gitea/runner-images:ubuntu-latest
      
    steps:
      - uses: actions/checkout@v4

      - name: Setup Toolchain (Docker Buildx)
        run: .gitea/workflows/helpers/scripts/setup.docker.sh

      - name: Build and Push Contest Server Docs
        env:
          REGISTRY_LOADBUILDER_PASSWORD: ${{ secrets.REGISTRY_LOADBUILDER_PASSWORD }}
          SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
          BRANCH_NAME: ${{ github.head_ref || github.ref_name }}
          API_VERSION: "dev"
          RELEASE_NAME: "dev"
        run: |
          cd docs/sphinx/top-level/
          ./build-and-push.sh

It is launched now on a gitea runner imageā€¦

act-runner-5587748d6d-vzdhs(version:v0.2.9) received task 4845 of job 4534, be triggered by event: push
workflow prepared
evaluating expression 'success()'
expression 'success()' evaluated to 'true'
šŸš€  Start image=gitea/runner-images:ubuntu-latest
  šŸ³  docker pull image=gitea/runner-images:ubuntu-latest platform= username= forcePull=false
  šŸ³  docker pull gitea/runner-images:ubuntu-latest
...

What i recommend is always have a sleep in there that pauses the flow and hop in the container and debug it.

so at the run: | level

# while [ 1 -eq 1 ];do sleep 3600;done;

The ā€˜setup.docker.shā€™ runs just this nowā€¦

echo "Creating buildx using $DOCKER_HOST"
docker buildx create 

which givesā€¦

Creating buildx using 
ERROR: failed to initialize builder blissful_lumiere (blissful_lumiere0): Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Yeah, will do - I was doing that earlier with the runner and dind containers.
Thanks for all the suggestions and troubleshootingā€¦ I figure I still must have some misconfiguration and will go over this more carefully.

1 Like

iirc

there is more to buildx to set it up correctly.
use the action, iā€™d say

     # https://github.com/docker/setup-buildx-action
      # https://docs.docker.com/engine/reference/commandline/cli/#environment-variables
      - name: Setup Docker Buildx
        id: buildx
        uses: docker/setup-buildx-action@v3
        with:
          # buildkitd-flags: --debug
          endpoint: builders
        env:
          DOCKER_HOST: "tcp://docker:2376/"
          DOCKER_TLS_CERTDIR: "/certs"
          DOCKER_TLS_VERIFY: 1
          DOCKER_CERT_PATH: "/certs/client"

Also this endpoint setup

1 Like

no problem, glad to help

ā€œThe onionā€ is very complex, with many layers, i hate it too.
Github actions running nodejs this that context, endpoint, docker TCP address ā€¦ Everything is worth writing a book :smiley:

I think youā€™re really close,
let me know if any help is needed here.

EDIT:

if you want to do your own thing, build on top of the image from Gitea, setup everything there, and just run one step and a script. Itā€™s perfectly sane and fine.

:checkered_flag: Job succeeded

Ok, ok, okā€¦ it works now! So, I think my problem was I tried to retrofit my original workflow with probably ~70% of your example and ignored some critical onion layers.

Most notably, the DOCKER_* ENVs and tried to home-bake the docker buildx setup instead of using the ā€œSetup Docker Buildxā€ action. I had got quite far on a non-k8s environment but clearly had assumed a lot of magic.

I think there were ~2-3 onion layers I wasnā€™t even thinking about. I had just assumed a gitea runner image would be sufficient. I am not 100% sure why this container is neededā€¦ I assume some magic with nodejs and tcp:// docker host?

But this seemed to be the major thing I was missing for anyone else who has made it this far in the threadā€¦

    container:
      image: gitea/runner-images:ubuntu-latest

Thanks again @mariusrugan - Iā€™m not sure if this is an onion or how you know so much about onions but I really appreciate your help and maintaining a working example.

Hopefully my ā€œjob succeededā€ wasnā€™t a fluke and I dont have to come back here with more head scratching ;o)

1 Like

@mariusrugan - one more question - not 100% on terminology - but is there a way with gitea act runners using this DinD configuration to use an in-cluster mirror. (k3s for example calls this an embedded mirror).

Iā€™m also interested in a docker layer cache that is shared amongst the runners and not having to rebuild everything from scratch. Iā€™ve noticed, in cluster, with this new shiny DinD setup that every workflow job rebuilds a Dockerfile from scratch as there is no existing layers from previous builds.

Any thoughts on how to configure this or if its even possible to get act runner to use a registry mirror or build cache?