[solved] Migration Error Upgrading to 1.9.4

Hello,

I tried to upgrade from gitea 1.5.0(armv7) to 1.9.4(armv6).
However the 1.9.4 executable always crashes trying to migrate the databases. It tries 10 times then prints an error and because I set the service to restart itself it then starts again.

2019/10/18 21:58:06 routers/init.go:51:initDBEngine() [I] ORM engine initialization attempt #10/10...
2019/10/18 21:58:06 ...-xorm/xorm/engine.go:329:Ping() [I] PING DATABASE mysql
2019/10/18 21:58:06 .../xorm/session_raw.go:143:queryBytes() [I] [SQL] SELECT `TABLE_NAME` from `INFORMATION_SCHEMA`.`TABLES` WHERE `TABLE_SCHEMA`=? and `TABLE_NAME`=? []interface {}{"gitea", "version"}
2019/10/18 21:58:06 ...m.io/core/dialect.go:161:HasRecords() [I] [SQL] SELECT `COLUMN_NAME` FROM `INFORMATION_SCHEMA`.`COLUMNS` WHERE `TABLE_SCHEMA` = ? AND `TABLE_NAME` = ? AND `COLUMN_NAME` = ? [gitea version
id]
2019/10/18 21:58:06 ...m.io/core/dialect.go:161:HasRecords() [I] [SQL] SELECT `COLUMN_NAME` FROM `INFORMATION_SCHEMA`.`COLUMNS` WHERE `TABLE_SCHEMA` = ? AND `TABLE_NAME` = ? AND `COLUMN_NAME` = ? [gitea version
version]
2019/10/18 21:58:06 .../xorm/session_get.go:99:nocacheGet() [I] [SQL] SELECT `id`, `version` FROM `version` WHERE `id`=? LIMIT 1 []interface {}{1}
2019/10/18 21:58:06 ...ations/migrations.go:272:Migrate() [I] Migration[82]: hot fix for wrong release sha1 on release table
2019/10/18 21:58:06 ...xorm/xormCould someone /session.go:845:saveLastSQL() [I] [SQL] BEGIN TRANSACTION
2019/10/18 21:58:06 ...xorm/session_find.go:199:noCacheFind() [I] [SQL] SELECT `id`, `repo_id`, `sha1`, `tag_name` FROM `release` WHERE (is_tag=?) ORDER BY `id` ASC LIMIT 100 []interface {}{false}
2019/10/18 21:58:06 .../xorm/session_get.go:99:nocacheGet() [I] [SQL] SELECT `id`, `owner_id`, `name` FROM `repository` WHERE `id`=? LIMIT 1 []interface {}{2}
2019/10/18 21:58:06 .../xorm/session_get.go:99:nocacheGet() [I] [SQL] SELECT `id`, `name` FROM `user` WHERE `id`=? LIMIT 1 []interface {}{2}
2019/10/18 21:58:06 ...xorm/xorm/session.go:845:saveLastSQL() [I] [SQL] ROLL BACK
2019/10/18 21:58:06 routers/init.go:86:GlobalInit() [F] ORM engine initialization failed: migrate: do migrate: Error 1054: Unknown column 'name' in 'field list'
2019/10/18 21:58:09 ...dules/setting/log.go:269:newLogService() [I] Gitea Log Mode: File(File:info)
2019/10/18 21:58:09 ...les/setting/cache.go:42:newCacheService() [I] Cache Service Enabled
2019/10/18 21:58:09 ...s/setting/session.go:45:newSessionService() [I] Session Service Enabled
2019/10/18 21:58:09 routers/init.go:49:initDBEngine() [I] Beginning ORM engine initialization.
2019/10/18 21:58:09 routers/init.go:51:initDBEngine() [I] ORM engine initialization attempt #1/10...

I tried to use various other versions inbetween, but they did not work either, e.g. 1.64(arm6 and arm7) exited with en segfault /SEGV) without writing to the logfile.
The hardware gitea is running on is a RaspberryPi 3B+.

Does someone know what causes this and could help me upgrading ?

Please try with 1.5.0 --> 1.5.3 --> 1.6.0 --> 1.6.4 --> 1.7.0 --> 1.7.6 --> 1.9.4

I’m not sure about all the intermediate steps; you can experiment a little.

1.5.3 segfaulted immediately.
I then downloaded and tried 1.51 (arm6 and arm7).
Running the arm6 version manually without pointing it to my app.ini seemed to work fine, so I exited with Ctrl+C.
I now first restorted the sql databases to my 1.5.0 backup, then tested 1.5.1-arm7. The arm7 version segfaulted.

And now comes the part that I do not understand:
The arm6 version of 1.5.1 does now also segfault.

Terminal output:

$ gpg --verify gitea-1.5.1-linux-arm-6.asc gitea-1.5.1-linux-arm-6
gpg: Signature made Mo 03 Sep 2018 04:45:50 CEST
gpg:                using RSA key CC64B1DB67ABBEECAB24B6455FC346329753F4B0
gpg: Good signature from "Teabot <teabot@gitea.io>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 7C9E 6815 2594 6888 62D6  2AF6 2D9A E806 EC15 92E2
     Subkey fingerprint: CC64 B1DB 67AB BEEC AB24  B645 5FC3 4632 9753 F4B0
$ shasum -c gitea-1.5.1-linux-arm-6.sha256 
gitea-1.5.1-linux-arm-6: OK
$ chmod a+x gitea-1.5.1-linux-arm-6
$ ./gitea-1.5.1-linux-arm-6 web
2019/10/22 01:07:47 [W] Custom config '/home/git/gitea/1.5.1-arm6/custom/conf/app.ini' not found, ignore this if you're running first time
2019/10/22 01:07:47 [T] AppPath: /home/git/gitea/1.5.1-arm6/gitea-1.5.1-linux-arm-6
2019/10/22 01:07:47 [T] AppWorkPath: /home/git/gitea/1.5.1-arm6
2019/10/22 01:07:47 [T] Custom path: /home/git/gitea/1.5.1-arm6/custom
2019/10/22 01:07:47 [T] Log path: /home/git/gitea/1.5.1-arm6/log
2019/10/22 01:07:47 [I] Gitea v1.5.1 built with: bindata, sqlite
2019/10/22 01:07:47 [I] Log Mode: Console(Info)
2019/10/22 01:07:47 [I] XORM Log Mode: Console(Info)
2019/10/22 01:07:47 [I] Cache Service Enabled
2019/10/22 01:07:47 [I] Session Service Enabled
2019/10/22 01:07:47 [I] SQLite3 Supported
2019/10/22 01:07:47 [I] Run Mode: Development
2019/10/22 01:07:48 [I] Listen: http://0.0.0.0:3000
2019/10/22 01:07:48 Serving [::]:3000 with pid 1147
^C2019/10/22 01:08:13 Exiting pid 1147.
$ cd ../1.5.1-arm7
$ wget https://dl.gitea.io/gitea/1.5.1/gitea-1.5.1-linux-arm-7 https://dl.gitea.io/gitea/1.5.1/gitea-1.5.1-linux-arm-7.asc https://dl.gitea.io/gitea/1.5.1/gitea-1.5.1-linux-arm-7.sha256
$ mariadb -u gitea -p gitea < /home/git-backups/gitea-1.5.0_20191015_sql/gitea.sql 
Enter password: 
$ chmod a+x gitea-1.5.1-linux-arm-7
$ gpg --verify gitea-1.5.1-linux-arm-7.asc gitea-1.5.1-linux-arm-7
gpg: Signature made Mo 03 Sep 2018 04:45:53 CEST
gpg:                using RSA key CC64B1DB67ABBEECAB24B6455FC346329753F4B0
gpg: Good signature from "Teabot <teabot@gitea.io>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 7C9E 6815 2594 6888 62D6  2AF6 2D9A E806 EC15 92E2
     Subkey fingerprint: CC64 B1DB 67AB BEEC AB24  B645 5FC3 4632 9753 F4B0
$ shasum -c gitea-1.5.1-linux-arm-7.sha256 
gitea-1.5.1-linux-arm-7: OK
$ ./gitea-1.5.1-linux-arm-7 web
Segmentation fault
g$ cd ../1.5.1-arm6/
$ ./gitea-1.5.1-linux-arm-6 web
Segmentation fault

Neither rebooting nor deleting the newly create ./custom/conf/app.ini did solve this. It all still segfaults. Checking the shasums I can verify that the binary file is still the same.
The 1.5.0 binary segfaults if another gitea instances failed previously, but works again after reboot.
I must admit that I neither understand this behaviour nor do I know what to do about it.

I just found this issue with the raspian buster kernel:

It seems likely that a kernel update might resolve this issue and this one:
https://forum.gitea.com/t/seg-fault-on-dietpi-buster/1593/4

Until the new kernel is available via apt, running the binary via gdb seems to avoid the issue due to different memory handling. I was able to succesfully upgrade to 1.5.3 with gdb and will see if incremental updates will continue to work up to 1.9.4.

If using the installaton as described here: docs.gitea .io/en-us/install-from-binary :

gdb --args gitea web -c /etc/gitea/app.ini
> set cwd /var/lib/gitea
> set environment HOME /home/git
> set environment GITEA_WORK_DIR /var/lib/gitea
> run

A new kernel version (4.19.93-v7+) is now available for the RaspberryPi on raspian in the default stable repositories and can be installed with apt-get.
The segfaults seem to be resolved. I tested 1.10.3 and 1.10.2 and everything worked fine up until now.
(Previous available kernel version: 4.19.75-v7+)