Upgrade now giving failed push error

Upgraded from 1.17.3 to 1.21.4, saw no major changes in the changelog, and was able to restart and it loads up fine, but now the pre-receive hook is declined.

remote: Gitea: Internal Server Error Decoding Failed
To https://<Mygiteaserver>/my-group/Spack.git
 ! [remote rejected] main -> main (pre-receive hook declined)
error: failed to push some refs to 'https://<Mygiteaserver>/my-group/Spack.git'

I have run

/opt/gitea --config /local/gitea/custom/conf/app.ini admin regenerate hooks

and that has not fixed it. Log entry from attempting a push below:

2024/02/16 10:34:59 ...eb/routing/logger.go:102:func1() [I] router: completed GET /my-group/Spack.git/info/refs?service=git-receive-pack for <MYIP>:36536, 401 Unauthorized in 5.9ms @ repo/githttp.go:532(repo.GetInfoRefs)
2024/02/16 10:34:59 ...dules/git/command.go:290:Run() [D] git.Command.RunDir(/local/gitea/data/gitea-repositories/my-group/spack.git): /usr/bin/git -c protocol.version=2 -c credential.helper= -c filter.lfs.required= -c filter.lfs.smudge= -c filter.lfs.clean= receive-pack --stateless-rpc --advertise-refs .
2024/02/16 10:34:59 ...eb/routing/logger.go:102:func1() [I] router: completed GET /my-group/Spack.git/info/refs?service=git-receive-pack for <MYIP>:36536, 200 OK in 45.1ms @ repo/githttp.go:532(repo.GetInfoRefs)
2024/02/16 10:35:00 ...dules/git/command.go:290:Run() [D] git.Command.RunDir(/local/gitea/data/gitea-repositories/my-group/spack.git): /usr/bin/git -c protocol.version=2 -c credential.helper= -c filter.lfs.required= -c filter.lfs.smudge= -c filter.lfs.clean= receive-pack --stateless-rpc /local/gitea/data/gitea-repositories/my-group/spack.git
2024/02/16 10:35:00 ...eb/routing/logger.go:102:func1() [I] router: completed POST /My-Repo/Spack.git/git-receive-pack for <MYIP>:36536, 200 OK in 261.2ms @ repo/githttp.go:500(repo.ServiceReceivePack)

Surely I’m missing something simple?

From /opt/gitea --config /local/gitea/custom/conf/app.ini doctor check --all

[1] Garbage collect LFS
 - [I] Found 0 (0 B) LFS file(s)
OK

[2] Check paths and basic configuration
 - [I] Configuration File Path:    "/local/gitea/custom/conf/app.ini"
 - [I] Repository Root Path:       "/local/gitea/data/gitea-repositories"
 - [I] Data Root Path:             "/local/gitea/data"
 - [I] Custom File Root Path:      "/local/gitea/custom"
 - [I] Work directory:             "/local/gitea"
 - [I] Log Root Path:              "/var/log"
OK

[3] Check if there are orphaned archives in storage
 - [I] Found 0 (0 B) repo archive(s)
OK

[4] Check if there are orphaned attachments in storage
 - [I] Found 3 (1.2 MiB) attachment(s)
OK

[5] Check if there are orphaned avatars in storage
 - [W] Found 5/17 (17 KiB/1.7 MiB) orphaned avatar(s)
 - [I] Found 0 (0 B) repo avatar(s)
OK

[6] Check if there are orphaned lfs files in storage
 - [I] Found 0 (0 B) LFS file(s)
OK

[7] Check if there are orphaned package blobs in storage
 - [I] Found 0 (0 B) package blob(s)
OK

[8] Check if there are orphaned storage files
 - [I] Found 3 (1.2 MiB) attachment(s)
 - [I] Found 0 (0 B) LFS file(s)
 - [W] Found 5/17 (17 KiB/1.7 MiB) orphaned avatar(s)
 - [I] Found 0 (0 B) repo avatar(s)
 - [I] Found 0 (0 B) repo archive(s)
 - [I] Found 0 (0 B) package blob(s)
OK

[9] Check Database Version
 - [I] Expected database version: 280
OK

[10] Check consistency of database
 - [W] Found 2 Orphaned OAuth2Application without existing User
OK

[11] Check if user with wrong type exist
OK

[12] Check if OpenSSH authorized_keys file is up-to-date
 - [E] Unable to open authorized_keys file. ERROR: open /dev/null/.ssh/authorized_keys: not a directory
ERROR

[13] Deleted all content related to orphaned repos
OK

[14] Check if SCRIPT_TYPE is available
 - [I] ScriptType bash is on the current PATH at /usr/bin/bash
OK

[15] Check if hook files are up-to-date and executable
OK

[16] Recalculate Stars number for all user
 - [I] No check available for User Stars numbers (skipped)
OK

[17] Check old archives
 - [I] 0 old archives in repository need to be deleted
OK

[18] Check that all git repositories have receive.advertisePushOptions set to true
 - [I] Checked 59 repositories, 0 need updates.
OK

[19] Check for incorrectly dumped repo_units (See #16961)
 - [I] Found no broken repo_units
OK

[20] Check for incorrect can_create_org_repo for org owner teams
 - [I] Found no team with incorrect can_create_org_repo
OK

[21] Recalculate merge bases
 - [I] All 4 PRs in 59 repos have a correct mergebase
OK

[22] Synchronize repo HEADs
 - [I] All 59 repos have their HEADs in the correct state
OK

[23] Check git-daemon-export-ok files
 - [I] Checked 59 repositories, 2 need updates.
OK

[24] Check commit-graphs
 - [I] Checked 59 repositories, 59 without commit-graphs.
OK

[25] Check if users has an valid email address
 - [I] All users have a valid e-mail.
OK

[26] Check if users have a valid username
 - [I] All users have a valid username.
OK

All done (checks: 26).

The auth-keys file error makes sense. The gitea user doesn’t have a home dir in our config, and shouldn’t actually need auth-keys.

Went digging through old bugs. Removed

LOCAL_ROOT_URL = http://localhost:443/

from config. restarted, tried again, worked? Is this intended?

Where I found the idea: [remote rejected] main -> main (pre-receive hook declined) · Issue #21528 · go-gitea/gitea · GitHub