How to implement artifacts actions w/o nodejs?

Since i want neither to have nodejs in all my images nor to install it for each run, i want to create composite or docker actions to upload and download artifacts. Unfortunately, there seem to be no documentation on the api routes used for these purposes. I found that there exist /api/actions_pipeline routes which handle that and some notes on them in gitea source code, but i was unable to create working upload action (didn’t even try download yet).
This is what i currently have:

name: "Artifacts upload"
author: "rps"
description: "Action to upload cicd artifacts"
inputs:
  name:
    description: artifact name
    required: true
    default: 'artifact'
  path:
    description: list of paths to upload as artifacts
    required: true
  retention-days:
    description: >
      Duration after which artifact will expire in days. 0 means using default retention.

      Minimum 1 day.
      Maximum 90 days unless changed from the repository settings page.
    default: 1
  token:
    description: token to authenticate to gitea
    required: true
    default: ${{ secrets.GITHUB_TOKEN }}
runs:
  using: "composite"
  steps:
    - name: upload artifacts
      shell: sh
      run: |
        echo curl -v -D /dev/stdout -o resp0.json --header "Authorization: Bearer ${{ inputs.token }}" \
          -X POST --data '{"Type":"actions_storage","Name":"${{ inputs.name }}"}' \
          "${{ gitea.server_url }}/api/actions_pipeline/_apis/pipelines/workflows/${{ gitea.run_id }}/artifacts?api-version=6.0-preview"
        curl -v -D /dev/stdout -o resp0.json --header "Authorization: Bearer ${{ inputs.token }}" \
          -X POST --data '{"Type":"actions_storage","Name":"${{ inputs.name }}"}' \
          "${{ gitea.server_url }}/api/actions_pipeline/_apis/pipelines/workflows/${{ gitea.run_id }}/artifacts?api-version=6.0-preview"
        cat resp0.json
        ls -l resp0.json
        UPLOAD_URL=$(jq -r '.fileContainerResourceUrl' resp0.json)
        for artifact in ${{ inputs.path }}; do
          content_length=$(ls -l "$artifact" | awk '{print $5}')
          echo curl -v -D /dev/stdout -o resp1.json --header "Authorization: Bearer ${{ inputs.token }}" \
            --header "x-tfs-filelength: ${content_length}" \
            --header "content-length: ${content_length}" \
            --header "x-actions-results-md5: $(md5sum "$artifact" | awk '{print $1}')" \
            --header "content-range: 0-$((content_length-1))/${content_length}" \
            -X PUT --data "@$artifact" \
            "${UPLOAD_URL}?retentionDays=${{ inputs.retention-days }}&itemPath=${{ inputs.name }}%2F$artifact"
          curl -v -D /dev/stdout -o resp1.json --header "Authorization: Bearer ${{ inputs.token }}" \
            --header "x-tfs-filelength: ${content_length}" \
            --header "content-length: ${content_length}" \
            --header "x-actions-results-md5: $(md5sum "$artifact" | awk '{print $1}')" \
            --header "content-range: 0-$((content_length-1))/${content_length}" \
            -X PUT --data "@$artifact" \
            "${UPLOAD_URL}?retentionDays=${{ inputs.retention-days }}&itemPath=${{ inputs.name }}%2F$artifact"
          cat resp1.json
          echo curl -v -D /dev/stdout -o resp2.json --header "Authorization: Bearer ${{ inputs.token }}" \
            -X PATCH \
            "${UPLOAD_URL}?itemPath=${{ inputs.name }}%2F$artifact"
          curl -v -D /dev/stdout -o resp2.json --header "Authorization: Bearer ${{ inputs.token }}" \
            -X PATCH \
            "${UPLOAD_URL}?itemPath=${{ inputs.name }}%2F$artifact"
          cat resp2.json
        done

The first POST request is successful, but the following PUT request fails with curl: (56) OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 0 (status 408 reported by nginx). What am i doing wrong and how to properly implement these actions?
Also, are there any plans to document these api routes?

Update: there is no openssl error if i exclude content-length header from PUT request. However, in this case 500 error is given. In gitea debug log the following lines appear:

2024/06/21 22:21:37 ...actions/artifacts.go:262:uploadArtifact() [D] [artifact] upload chunk, name: testing, path: testfile1, size: 9, retention days: 1
2024/06/21 22:21:38 .../artifacts_chunks.go:51:saveUploadChunkBase() [I] [artifact] check chunk md5, sum: 72VMQKtPF0f8aZkV1PcJAg==, header: 73d643ec3f4beb9020eef0beed440ad0
2024/06/21 22:21:38 ...actions/artifacts.go:278:uploadArtifact() [E] Error save upload chunk: md5 not match

The md5 from header is actual md5 from file name, size is also correct. Which sum does the server compute and how? If i try to decode base64 from this log, i get some bytes sequence which doesn’t look like original file at all.

UPDATE1: cksum -a md5 --base64 actually produces the required hash for header.

Finally, this is a minimal working action for uploading artifacts:

name: "Artifacts upload"
author: "rps"
description: "Action to upload cicd artifacts"
inputs:
  name:
    description: artifact name
    required: true
    default: 'artifact'
  path:
    description: list of paths to upload as artifacts
    required: true
  retention-days:
    description: >
      Duration after which artifact will expire in days. 0 means using default retention.

      Minimum 1 day.
      Maximum 90 days unless changed from the repository settings page.
    default: 1
  token:
    description: token to authenticate to gitea
    required: true
    default: ${{ secrets.GITHUB_TOKEN }}
runs:
  using: "composite"
  steps:
    - name: upload artifacts
      shell: sh
      run: |
        curl --fail -v -D /dev/stdout -o resp0.json --header "Authorization: Bearer ${{ inputs.token }}" \
          -X POST --data '{"Type":"actions_storage","Name":"${{ inputs.name }}"}' \
          "${{ gitea.server_url }}/api/actions_pipeline/_apis/pipelines/workflows/${{ gitea.run_id }}/artifacts?api-version=6.0-preview"
        cat resp0.json
        UPLOAD_URL=$(jq -r '.fileContainerResourceUrl' resp0.json)
        for artifact in ${{ inputs.path }}; do
          content_length=$(ls -l "$artifact" | awk '{print $5}')
          md5=$(cksum -a md5 --base64 --untagged "$artifact" | awk '{print $1}')
          curl --fail -v -D /dev/stdout -o resp1.json --header "Authorization: Bearer ${{ inputs.token }}" \
            --header "x-actions-results-md5: $md5" \
            --header "x-tfs-filelength: ${content_length}" \
            --header "content-range: bytes 0-$((content_length-1))/${content_length}" \
            -X PUT --data-binary "@$artifact" \
            "${UPLOAD_URL}?retentionDays=${{ inputs.retention-days }}&itemPath=${{ inputs.name }}%2F$artifact"
          cat resp1.json
          curl --fail -v -D /dev/stdout -o resp2.json --header "Authorization: Bearer ${{ inputs.token }}" \
            -X PATCH \
            "${{ gitea.server_url }}/api/actions_pipeline/_apis/pipelines/workflows/${{ gitea.run_id }}/artifacts?api-version=6.0-preview&artifactName=${{ inputs.name }}"
          cat resp2.json
          rm resp1.json resp2.json
        done
        rm resp0.json

Artifacts made by this job can be successfully consumed by cmmon download artifacts action (v3).
I’ll try to implement curl-based download action in a while.

1 Like

Here is what i came from for download:

name: "Artifacts download"
author: "rps"
description: "Action to download cicd artifacts"
inputs:
  name:
    description: artifact name
    required: true
  path:
    description: destination path
    required: true
  token:
    description: token to authenticate to gitea
    required: true
    default: ${{ gitea.token }}
runs:
  using: "composite"
  steps:
    - name: download artifacts
      shell: sh
      run: |
        curl --fail -v -D /dev/stdout -o resp0.json --header "Authorization: Bearer ${{ inputs.token }}" \
          "${{ gitea.server_url }}/api/actions_pipeline/_apis/pipelines/workflows/${{ gitea.run_id }}/artifacts?api-version=6.0-preview"
        cat resp0.json
        container_url=$(jq -r '.value.[] | select(.name == "${{ inputs.name }}").fileContainerResourceUrl' resp0.json)
        echo $container_url
        rm resp0.json
        artifact_urlencode=$(echo "${{ inputs.name }}" | jq -Rr '@uri')
        echo $artifact_urlencode
        curl --fail -v -D /dev/stdout -o resp1.json --header "Authorization: Bearer ${{ inputs.token }}" \
          "$container_url?itemPath=$artifact_urlencode"
        cat resp1.json
        for key in $(jq -r '.value[].path' resp1.json); do
          artifact_url=$(jq -r ".value[] | select(.path == \"$key\").contentLocation" resp1.json)
          key_urlencode=$(echo $key | jq -Rr '@uri')
          curl --fail -v -D /dev/stdout -o "${{ inputs.path }}/$key" --create-dirs --header "Authorization: Bearer ${{ inputs.token }}" \
            "$artifact_url?itemPath=$key_urlencode"
        done
        rm resp1.json

There is a difference between nodejs implementation and mine: the former only saves and restores basenames of each artifact file whereas the latter saves and restores paths relative to workdir.