# Validating and documenting migration files **URL:** \ **Category:** Install/Maintain/Configure\ **Created:** [January 5, 2022, 11:16pm UTC](https://forum.gitea.com/t/validating-and-documenting-migration-files/4358 "2022-01-05T23:16:50Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![dachary](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.gitea.com/dachary/32/4990_2.png) [@dachary](https://forum.gitea.com/u/dachary)\ **Post date:** [January 5, 2022, 11:16pm UTC](https://forum.gitea.com/t/validating-and-documenting-migration-files/4358/1 "2022-01-05T23:16:50Z")
Bonjour, When [restore-repo](https://docs.gitea.io/en-us/command-line/#restore-repo) reads YAML file that were previously created by [dump-repo](https://docs.gitea.io/en-us/command-line/#dump-repo), it would be helpful to sanity check their content. It would also make is easier for third party software (other forges or any other) to create such files for Gitea to ingest. Or use files dumped by Gitea for other purposes. The schema used for validation can also include a documentation and it could grow into a de-facto file format. To illustrate this, [a diff implements](https://lab.forgefriends.org/forgefriends/forgefriends/-/merge_requests/32/diffs) this idea for issues only but it would be trivial to similarly cover other data structures. - The [–validation option is added to restore-repo](https://lab.forgefriends.org/forgefriends/forgefriends/-/merge_requests/32/diffs#0f107e817d97e964345dbd15ebe7c22b0364ce66_46_46) - [JSON schema](http://json-schema.org/) describes [issues](https://lab.forgefriends.org/forgefriends/forgefriends/-/merge_requests/32/diffs#4cad0b7e1df11ce5a0cbd495667a960b24b0b21a_0_1), [labels](https://lab.forgefriends.org/forgefriends/forgefriends/-/merge_requests/32/diffs#45a29603e1a5d7421a130216297a50b879778db6_0_1) and [reactions](https://lab.forgefriends.org/forgefriends/forgefriends/-/merge_requests/32/diffs#064c1c4fd327b669a56bd3705d45e25275446213_0_1) - A [helper loads the YAML dump file](https://lab.forgefriends.org/forgefriends/forgefriends/-/merge_requests/32/diffs#41437f6ec54dd42a7b305843798daf56b055f6c2_0_20) and optionally (if --validation is set) validates its content against the corresponding schema. - The [helper is used in RestoreRepo](https://lab.forgefriends.org/forgefriends/forgefriends/-/merge_requests/32/diffs#62124b6c5d1f1ccfaeb9eff817c6278ba10609af_193_195) and helps keep it DRY - The schemas are [added in a bindata file](https://lab.forgefriends.org/forgefriends/forgefriends/-/merge_requests/32/diffs#736141301dbda89d45dbb4664b6e0e1ab0b7bc08_0_1) to be [used in the binary](https://lab.forgefriends.org/forgefriends/forgefriends/-/merge_requests/32/diffs#f455c43c0d39adb4b71061e648ba2cd4e56a5462_0_10) - Tests [verify validation border cases and error conditions](https://lab.forgefriends.org/forgefriends/forgefriends/-/merge_requests/32/diffs#55f9bc094cb2b005750f5bd46a90ea872b6366e1) How do people feel about going in this direction? ---
**Author:** ![lunny](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.gitea.com/lunny/32/4701_2.png) [@lunny](https://forum.gitea.com/u/lunny)\ **Post date:** [January 6, 2022, 10:17am UTC](https://forum.gitea.com/t/validating-and-documenting-migration-files/4358/2 "2022-01-06T10:17:10Z")
Because I think it’s not very stable of the file format. If we want to share it with others, we need to make a standard at first. ---
**Author:** ![dachary](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.gitea.com/dachary/32/4990_2.png) [@dachary](https://forum.gitea.com/u/dachary)\ **Post date:** [January 6, 2022, 10:29am UTC](https://forum.gitea.com/t/validating-and-documenting-migration-files/4358/3 "2022-01-06T10:29:26Z")
Right: at the moment it is an internal format that is not designed or intended to be shared. Adding validation is a baby step that is both useful immediately and helps create a standard to be shared with others later on. A use case where it can be immediately useful is when someone **dump-repo** a project with gitea 14.2 and a much later tries to **restore-repo** the same project using gitea 18.3. If the format changed, validation will display an error instead of trying to ingest files that will be interpreted incorrectly. > [@lunny](#): > > If we want to share it with others, we need to make a standard at first. The standard internal to Gitea is defined by these files [issues](https://lab.forgefriends.org/forgefriends/forgefriends/-/merge_requests/32/diffs#4cad0b7e1df11ce5a0cbd495667a960b24b0b21a_0_1), [labels](https://lab.forgefriends.org/forgefriends/forgefriends/-/merge_requests/32/diffs#45a29603e1a5d7421a130216297a50b879778db6_0_1), [reactions](https://lab.forgefriends.org/forgefriends/forgefriends/-/merge_requests/32/diffs#064c1c4fd327b669a56bd3705d45e25275446213_0_1) etc. It can then become a standard shared outside of Gitea. Does that make sense? ---
**Author:** ![lunny](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.gitea.com/lunny/32/4701_2.png) [@lunny](https://forum.gitea.com/u/lunny)\ **Post date:** [January 6, 2022, 1:24pm UTC](https://forum.gitea.com/t/validating-and-documenting-migration-files/4358/4 "2022-01-06T13:24:25Z")
Maybe you could send a PR back to Gitea. 🙂 ---
**Author:** ![dachary](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.gitea.com/dachary/32/4990_2.png) [@dachary](https://forum.gitea.com/u/dachary)\ **Post date:** [January 6, 2022, 6:02pm UTC](https://forum.gitea.com/t/validating-and-documenting-migration-files/4358/5 "2022-01-06T18:02:50Z")
For archive (discussion in chat) a related PR: > **[Escape string and blob results from dump more correctly](https://gitea.com/xorm/xorm/pulls/2091)** > > dumpTables currently badly handles BLOB and TEXT data containing control > characters: > > \* MySQL will interpret and unescape string literals e.g.\`\\r\` will become > carriage return. > > \* Postgres will not allow string literals to contain NUL nor will > SQLite... ---
**Author:** ![lunny](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.gitea.com/lunny/32/4701_2.png) [@lunny](https://forum.gitea.com/u/lunny)\ **Post date:** [January 7, 2022, 3:30am UTC](https://forum.gitea.com/t/validating-and-documenting-migration-files/4358/6 "2022-01-07T03:30:16Z")
I have reviewed that and will merge it after CI PASS. ---
**Author:** ![dachary](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.gitea.com/dachary/32/4990_2.png) [@dachary](https://forum.gitea.com/u/dachary)\ **Post date:** [January 9, 2022, 7:26pm UTC](https://forum.gitea.com/t/validating-and-documenting-migration-files/4358/7 "2022-01-09T19:26:15Z")
The [pull request was submitted](https://github.com/go-gitea/gitea/pull/18203) by @realaravinth and passes CI. Thanks a lot for your review 🙂