Hey all,
since upgrading to 1.22 I have been getting errors on my gitea, where it says it cannot load the index.js file, and interestingly i get 2 different errors on chrome and firefox. Chrome says its a http2 erorr, while firefox says its a partial send error? (screenshots attached)
I searched the forum and came across this post: Upgraded to gitea-1.20.1-1 from 1.11, Fedora 38
And yes, my instance is also behind a proxy (HAproxy running on pfsense), but i don’t do any filtering on paths, its a very basic listen on port 443, match the sni, do the ssl termination, and proxy_pass the traffic to the VM via port 3000. It’s been working flawlessly this whole time and for multiple version upgrades.
Any ideas on further investigations to figure out this issue?
I found lunny say HTTP2 isn’t support here: Reverse proxy (nginx) setup question - #2 by lunny
So perhaps this is the issue. But i’m confused what would of changed all of a sudden for http2 to start being used? my HAproxy hasn’t been updated at all, nor has the config changed.
Did browsers suddenly implement HTTP2 as a default?
This is my HAproxy config
# Automaticaly generated, dont edit manually.
# Generated on: 2024-05-29 15:35
global
maxconn 10000
log /var/run/log local0 debug
stats socket /tmp/haproxy.socket level admin expose-fd listeners
uid 80
gid 80
nbthread 1
hard-stop-after 15m
chroot /tmp/haproxy_chroot
daemon
log-send-hostname haproxy-devel
server-state-file /tmp/haproxy_server_state
listen HAProxyLocalStats
bind 127.0.0.1:2200 name localstats
mode http
stats enable
stats refresh 10
stats admin if TRUE
stats show-legends
stats uri /haproxy/haproxy_stats.php?haproxystats=1
timeout client 5000
timeout connect 5000
timeout server 5000
frontend wan-http-to-https
bind [redacted public ip]:8080 name [redacted public ip]:8080
mode http
log global
option log-separate-errors
option http-keep-alive
option forwardfor
acl https ssl_fc
http-request set-header X-Forwarded-Proto http if !https
http-request set-header X-Forwarded-Proto https if https
timeout client 30000
option httpslog
log-format "IP:%ci Req:%HM %HU %U bytes in FE:%f BE:%s Waited:%Tt ms Resp:%ST %B bytes out"
http-request redirect scheme https
frontend shared-wan-https-merged
bind [redacted public ip]:8443 name [redacted public ip]:8443 ssl crt-list /var/etc/haproxy/shared-wan-https.crt_list
mode http
log global
option log-separate-errors
option httplog
option http-keep-alive
option forwardfor
acl https ssl_fc
http-request set-header X-Forwarded-Proto http if !https
http-request set-header X-Forwarded-Proto https if https
timeout client 30000
option httpslog
capture request header Host len 30
log-format "IP:%ci Host: %hr Req: %HM %HU %U bytes in FE:%f BE:%s Waited:%Tt ms Resp:%ST %B bytes out"
acl tld var(txn.txnhost) -m str -i [redacted domain].dev
acl root-path var(txn.txnpath) -m str -i /
acl local-ips src [redacted private ip]/24
acl robots-txt var(txn.txnpath) -m str -i /robots.txt
acl aclcrt_shared-wan-https var(txn.txnhost) -m reg -i ^[redacted domain]\.dev(:([0-9]){1,5})?$
acl aclcrt_shared-wan-https var(txn.txnhost) -m reg -i ^([^\.]*)\.[redacted domain]\.dev(:([0-9]){1,5})?$
acl gitea-dns var(txn.txnhost) -m str -i gitea.[redacted domain].dev
http-request set-var(txn.txnhost) hdr(host)
http-request set-var(txn.txnpath) path
http-request set-log-level silent if local-ips aclcrt_shared-wan-https
http-request redirect code 301 location https://www.[redacted domain].dev if tld aclcrt_shared-wan-https
http-request return status 200 content-type text/plain string "User-agent: *\nDisallow: /admin\nDisallow: /actions\nDisallow: /open-source\nUser-agent: Amazonbot\nUser-agent: SemrushBot\nDisallow: *" if robots-txt aclcrt_shared-wan-https
use_backend gitea_ipvANY if gitea-dns
use_backend no-match_ipvANY if aclcrt_shared-wan-https
backend no-match_ipvANY
mode http
id 129
log global
timeout connect 30000
timeout server 30000
retries 3
load-server-state-from-file global
http-request deny deny_status 404
backend gitea_ipvANY
mode http
id 102
log global
timeout connect 30000
timeout server 30000
retries 3
load-server-state-from-file global
server gitea [redacted private ip]:3000 id 103 check inter 60000
Does anyone have any insights to this problem? This still occurring and I have no idea why.
Please is there anyone available to give some clues as to what this problem may be and how to fix it? This actually stops much of the functionality of my gitea server For example when i try to setup a mirror of a github repo i can’t because the javascript doesn’t load on any page and so it seems to break the requests being sent and the repo never loads. Even just trying to navigate my gitea server, even though technically does load existing repos, gives this pop up every time you go to a new page and much of the assets do not load on the page
@always-self-hosted
So I created the account just to tell you.
I also was facing this issue. Started from this exact version.
For some reason, disabling the mailer fixed it for me. I changed it to false and restarted the container. See the docker logs; maybe you have the same issue or something else.
I could see that it was not able to send emails. Maybe you see the same error?
1 Like
Hi monkeymoon.
Thanks so much for making the effort to share you experience! its very disappointing you are the first to do so Seems gitea community / dev is not very active at all
I do have the mailer enabled, and its configured correctly as i use it to send emails on signup, but i dont seem to have anything in /var/lib/gitea/log
other than access.log
. Was there a specific place you found the mailer log? I don’t use the docker image, im running the binary on a debian machine.
I could see the logs in the container itself, not sure which file it was coming from.
I would suggest posting it in their repo; maybe it will get more sets of eyes.
Also, see the documentation for reverse proxy.
I see some bits have changed for HAproxy between 1.22 and 1.21.11.
Ok after A LOT of investigating by myself (sad ) I think i finally solved my problem.
So for me, it seems like the machine the VM was running on for my gitea was being effected by another VM (nodejs if anyone is curious ). It seems like another VM was causing a huge amount of io wait for the host, and so these unable to load js file errors were because the gitea vm was unable to read from disk. (although does that make sense? because i thought these static files are embedded in the binary and therefore would be loaded into ram at runtime?)
After i shut down the offending vm causing huge io wait, this problem immediately went away. thank god its finally solved. This was pure luck that i solved this by the way, its not like gitea was pointing to down a road of an io problem. There was no errors or information on the machine that hinted at what this issue was. this was purely my other nodejs server having issues and investigating that server i noticed the nodejs eventloop was super slow and i can see it was async disk write calls taking forever and thats what lead me thinking the host machine itself is struggling and the gitea vm may be caught up in this, and it was indeed being effected.