diff options
author | Zeger-Jan van de Weg <git@zjvandeweg.nl> | 2020-04-22 13:30:55 +0300 |
---|---|---|
committer | Zeger-Jan van de Weg <git@zjvandeweg.nl> | 2020-04-22 13:30:55 +0300 |
commit | 167fe3654959361c3d0897e47f0667f087220dff (patch) | |
tree | b68d723d040b5d71bf5d03dd4c32a8c22a703c1f | |
parent | bd06a8ede38155e0928c3506734495a3089c2cf4 (diff) | |
parent | 48fafa6556c9c834b09965fa1edc98ba05f36365 (diff) |
Merge branch 'jv-doc-logs' into 'master'
Add developer documentation about logs
Closes #2628
See merge request gitlab-org/gitaly!2080
-rw-r--r-- | doc/README.md | 1 | ||||
-rw-r--r-- | doc/logging.md | 56 |
2 files changed, 57 insertions, 0 deletions
diff --git a/doc/README.md b/doc/README.md index 5ce5435d1..fc79345a2 100644 --- a/doc/README.md +++ b/doc/README.md @@ -34,6 +34,7 @@ For configuration please read [praefects configuration documentation](doc/config - [Tips for reading Git source code](reading_git_source.md) - [gitaly-ssh](../cmd/gitaly-ssh/README.md) - [Git object quarantine during git push](object_quarantine.md) +- [Logging in Gitaly](logging.md) #### Proposals diff --git a/doc/logging.md b/doc/logging.md new file mode 100644 index 000000000..88250be7c --- /dev/null +++ b/doc/logging.md @@ -0,0 +1,56 @@ +# Logging in Gitaly + +Gitaly creates several kinds of log data. + +## Go application logs + +The main Gitaly process uses logrus to writes structured logs to +stdout. These logs use either the text or the json format of logrus, +depending on setting in the Gitaly config file. + +The main Gitaly process writes log messages with global scope and +request scope. Request scoped messages can be recognized and filtered +by their correlation ID. + +The main application logs include an access log for all requests. +Request-scoped errors are printed with the request correlation ID +attached. + +Many Gitaly RPC's spawn Git processes which may write errors or +warnings to stderr. Gitaly will capture these stderr messages and +include them in its main log, tagged with the request correlation ID. + +## Gitaly-ruby application logs + +Gitaly-ruby writes logs to stdout. These logs are not structured. The +main Gitaly process captures the gitaly-ruby process log messages and +converts each line into a structured message that includes information +about the gitaly-ruby process such as the PID. These logs then get +printed as part of the log stream of the main Gitaly process. + +There is no attribution of log messages in gitaly-ruby beyond the +gitaly-ruby process ID. If an RPC implemented in gitaly-ruby runs a +Git command, and if that Git command prints to stderr, it will show up +as untagged data in the log stream for the gitaly-ruby parent process. + +Because of these properties, gitaly-ruby logs are often hard to read, +and it is often not possible to attribute log messages to individual +RPC requests. + +## Log files + +In a few cases, Gitaly spawns process that cannot log to stderr +because stderr gets relayed to the end user, and we would risk leaking +information about the machine Gitaly runs on. One (the only?) example +is Git hooks. Because of this, the Gitaly config file also has a log +directory setting. Hooks that must log to a file will use that +directory. + +Examples are: + +- `gitlab-shell.log` +- `gitaly_hooks.log` + +There is another log file called `githost.log`. This log is generated +by legacy code in gitaly-ruby. The way it is used, it might as well +write to stdout. |