# Code walkthrough: Gitea mirror service

**URL:** <https://forum.gitea.com/t/code-walkthrough-gitea-mirror-service/4082>\
**Category:** Install/Maintain/Configure\
**Created:** [November 11, 2021, 12:03pm UTC](https://forum.gitea.com/t/code-walkthrough-gitea-mirror-service/4082 "2021-11-11T12:03:25Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![dachary](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.gitea.com/dachary/32/4990_2.png) [@dachary](https://forum.gitea.com/u/dachary)\
**Post date:** [November 11, 2021, 12:03pm UTC](https://forum.gitea.com/t/code-walkthrough-gitea-mirror-service/4082/1 "2021-11-11T12:03:25Z")

</div>

_This is one of the [Code walkthrough](https://forum.gitea.com/search?q=code%20walkthrough%20in%3Atitle%20%40dachary%20order%3Alatest) I wrote while exploring the [Gitea codebase](https://github.com/go-gitea/gitea/blob/de9625e48510c2e6ea65abfe50db6e552b655744/)._

When Gitea starts, it [initializes the mirror service](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/routers/init.go#L149) which [creates the “mirror” queue](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/services/mirror/mirror.go#L109-L116) singleton which [runs](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/services/mirror/mirror.go#L115) in a [RunWithShutdownFns](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/modules/graceful/manager.go#L74) goroutine. The “mirror” [queue](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/modules/queue/queue.go#L58-L62) is created by [CreateUniqueQueue](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/modules/queue/setting.go#L72) via [NewQueue](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/modules/queue/queue.go#L191) and is backed by a persistent storage. It handles retries, timeouts etc.

The [queueHandle](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/services/mirror/mirror.go#L101-L106) function is [called](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/services/mirror/mirror.go#L113) when a [new SyncRequest is pushed to the queue](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/services/mirror/mirror.go#L124-L127). It [calls](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/services/mirror/mirror.go#L37-L46) either [SyncPullMirror](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/services/mirror/mirror_pull.go#L351) or [SyncPushMirror](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/services/mirror/mirror_push.go#L77) to perform the mirroring and [schedules the next run](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/services/mirror/mirror_pull.go#L375) by updating the [LastUpdateUnix](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/models/repo_pushmirror.go#L32) database field (date and time). When the [Update](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/services/mirror/mirror.go#L49) function runs [from cron every 10 minutes](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/modules/cron/tasks_basic.go#L27), it [iterates](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/services/mirror/mirror.go#L93) on the database records and [runs the handler](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/models/repo_pushmirror.go#L110) when it is due.

The [handler](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/services/mirror/mirror.go#L56) does not perform the action, it pushes a [SyncRequest](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/services/mirror/mirror.go#L72-L75) onto the [“mirror” queue](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/services/mirror/mirror.go#L85) and it will be passed on to the [queueHandle](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/services/mirror/mirror.go#L101-L106) function that was registered when the “mirror” queue was initialized. There can only be one `SyncPushMirror` for a given repository in the queue. That’s what the [“Unique” in “UniqueQueue”](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/modules/queue/unique_queue.go#L11-L21) stands for: the redis unique queue [enforces this constraint](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/modules/queue/unique_queue_redis.go#L93) by using [SADD](https://redis.io/commands/sadd) and [failing](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/modules/queue/unique_queue_redis.go#L98) on duplicate entries.

When [the mirror is removed](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/routers/web/repo/setting.go#L270), the [git remote is removed](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/services/mirror/mirror_push.go#L60) and [the database record deleted](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/models/repo_pushmirror.go#L77).

---

<div class="post-metadata">

**Author:** ![dachary](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.gitea.com/dachary/32/4990_2.png) [@dachary](https://forum.gitea.com/u/dachary)\
**Post date:** [November 11, 2021, 5:09pm UTC](https://forum.gitea.com/t/code-walkthrough-gitea-mirror-service/4082/2 "2021-11-11T17:09:35Z")

</div>

> [@dachary](#):
>
> updating the [LastUpdateUnix](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/models/repo_pushmirror.go#L32) database field (date and time). When the [Update](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/services/mirror/mirror.go#L49) function runs [from cron every 10 minutes](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/modules/cron/tasks_basic.go#L27), it [iterates](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/services/mirror/mirror.go#L93) on the database records and [runs the handler](https://github.com/go-gitea/gitea/blob/90eb9fb889e5d3a5845576dbc63e3792f3da33f2/models/repo_pushmirror.go#L110) when it is due.

I’m not sure to understand the benefit of having a `UniqueQueue` here. It looks like it would work exactly the same if there was only the database and a query on the existing mirrors at regular intervals to figure out what needs mirroring. I must be missing something. Ideas?
