diff options
author | Achilleas Pipinellis <axil@gitlab.com> | 2017-11-20 11:56:49 +0300 |
---|---|---|
committer | Achilleas Pipinellis <axil@gitlab.com> | 2017-11-20 11:56:49 +0300 |
commit | cbee84ca29dd7eb20603ad2231394e3d33595075 (patch) | |
tree | 3d21d97abdcbe12402affa1cbb953cc4c2638da2 /doc | |
parent | 33b4da0a3c0f32f8535e1c5fa94c848cf435d3ab (diff) | |
parent | b001487d94a131fa698d7feb1be8d9168c877e55 (diff) |
Merge branch 'sh-document-strace-unicorn' into 'master'
Document how to troubleshoot internal API calls
See merge request gitlab-org/gitlab-ce!15337
Diffstat (limited to 'doc')
-rw-r--r-- | doc/administration/troubleshooting/debug.md | 28 |
1 files changed, 28 insertions, 0 deletions
diff --git a/doc/administration/troubleshooting/debug.md b/doc/administration/troubleshooting/debug.md index be538ea250a..83a714810c1 100644 --- a/doc/administration/troubleshooting/debug.md +++ b/doc/administration/troubleshooting/debug.md @@ -163,6 +163,34 @@ separate Rails process to debug the issue: 1. In a new window, run `top`. It should show this ruby process using 100% CPU. Write down the PID. 1. Follow step 2 from the previous section on using gdb. +### GitLab: API is not accessible + +This often occurs when gitlab-shell attempts to request authorization via the +internal API (e.g., `http://localhost:8080/api/v4/internal/allowed`), and +something in the check fails. There are many reasons why this may happen: + +1. Timeout connecting to a database (e.g., PostgreSQL or Redis) +1. Error in Git hooks or push rules +1. Error accessing the repository (e.g., stale NFS handles) + +To diagnose this problem, try to reproduce the problem and then see if there +is a unicorn worker that is spinning via `top`. Try to use the `gdb` +techniques above. In addition, using `strace` may help isolate issues: + +```shell +strace -tt -T -f -s 1024 -p <PID of unicorn worker> -o /tmp/unicorn.txt +``` + +If you cannot isolate which Unicorn worker is the issue, try to run `strace` +on all the Unicorn workers to see where the `/internal/allowed` endpoint gets +stuck: + +```shell +ps auwx | grep unicorn | awk '{ print " -p " $2}' | xargs strace -tt -T -f -s 1024 -o /tmp/unicorn.txt +``` + +The output in `/tmp/unicorn.txt` may help diagnose the root cause. + # More information * [Debugging Stuck Ruby Processes](https://blog.newrelic.com/2013/04/29/debugging-stuck-ruby-processes-what-to-do-before-you-kill-9/) |