diff options
Diffstat (limited to 'doc/development')
| -rw-r--r-- | doc/development/architecture.md | 6 | ||||
| -rw-r--r-- | doc/development/ee_features.md | 70 | ||||
| -rw-r--r-- | doc/development/emails.md | 94 | ||||
| -rw-r--r-- | doc/development/gotchas.md | 8 | ||||
| -rw-r--r-- | doc/development/logging.md | 82 | ||||
| -rw-r--r-- | doc/development/profiling.md | 8 | ||||
| -rw-r--r-- | doc/development/routing.md | 34 | ||||
| -rw-r--r-- | doc/development/sql.md | 4 |
8 files changed, 163 insertions, 143 deletions
diff --git a/doc/development/architecture.md b/doc/development/architecture.md index 8319603fea2..b645a72567c 100644 --- a/doc/development/architecture.md +++ b/doc/development/architecture.md @@ -484,9 +484,11 @@ When making a request to an HTTP Endpoint (think `/users/sign_in`) the request w Below we describe the different pathing that HTTP vs. SSH Git requests will take. There is some overlap with the Web Request Cycle but also some differences. ### Web Request (80/443) + TODO ### SSH Request (22) + TODO ## System Layout @@ -505,7 +507,9 @@ To summarize here's the [directory structure of the `git` user home directory](. ### Processes - ps aux | grep '^git' +```sh +ps aux | grep '^git' +``` GitLab has several components to operate. As a system user (i.e. any user that is not the `git` user) it requires a persistent database (MySQL/PostreSQL) and redis database. It also uses Apache httpd or Nginx to proxypass Unicorn. As the `git` user it starts Sidekiq and Unicorn (a simple ruby HTTP server running on port `8080` by default). Under the GitLab user there are normally 4 processes: `unicorn_rails master` (1 process), `unicorn_rails worker` (2 processes), `sidekiq` (1 process). diff --git a/doc/development/ee_features.md b/doc/development/ee_features.md index dc17b59a9a0..7131b717353 100644 --- a/doc/development/ee_features.md +++ b/doc/development/ee_features.md @@ -182,52 +182,52 @@ There are a few gotchas with it: pattern](https://en.wikipedia.org/wiki/Template_method_pattern). For example, given this base: - ```ruby - class Base - def execute - return unless enabled? + ```ruby + class Base + def execute + return unless enabled? - # ... - # ... - end + # ... + # ... end - ``` + end + ``` - Instead of just overriding `Base#execute`, we should update it and extract - the behaviour into another method: + Instead of just overriding `Base#execute`, we should update it and extract + the behaviour into another method: - ```ruby - class Base - def execute - return unless enabled? + ```ruby + class Base + def execute + return unless enabled? - do_something - end + do_something + end - private + private - def do_something - # ... - # ... - end + def do_something + # ... + # ... end - ``` + end + ``` - Then we're free to override that `do_something` without worrying about the - guards: + Then we're free to override that `do_something` without worrying about the + guards: - ```ruby - module EE::Base - extend ::Gitlab::Utils::Override + ```ruby + module EE::Base + extend ::Gitlab::Utils::Override - override :do_something - def do_something - # Follow the above pattern to call super and extend it - end + override :do_something + def do_something + # Follow the above pattern to call super and extend it end - ``` + end + ``` - This would require updating CE first, or make sure this is back ported to CE. + This would require updating CE first, or make sure this is back ported to CE. When prepending, place them in the `ee/` specific sub-directory, and wrap class or module in `module EE` to avoid naming conflicts. @@ -1038,8 +1038,8 @@ to avoid conflicts during CE to EE merge. Until the work completed to merge the ce and ee codebases, which is tracked on [epic &802](https://gitlab.com/groups/gitlab-org/-/epics/802), there exists times in which some changes for EE require specific changes to the CE code base. Examples of backports include the following: -* Features intended or originally built for EE that are later decided to move to CE -* Sometimes some code in CE may impact the EE feature +- Features intended or originally built for EE that are later decided to move to CE +- Sometimes some code in CE may impact the EE feature Here is a workflow to make sure those changes end up backported safely into CE too. diff --git a/doc/development/emails.md b/doc/development/emails.md index 1fd4139c91e..e6af075a282 100644 --- a/doc/development/emails.md +++ b/doc/development/emails.md @@ -26,51 +26,51 @@ See the [Rails guides] for more info. feature and fill in the details for your specific IMAP server and email account: - Configuration for Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com - - ```yaml - incoming_email: - enabled: true - - # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. - # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`). - address: "gitlab-incoming+%{key}@gmail.com" - - # Email account username - # With third party providers, this is usually the full email address. - # With self-hosted email servers, this is usually the user part of the email address. - user: "gitlab-incoming@gmail.com" - # Email account password - password: "[REDACTED]" - - # IMAP server host - host: "imap.gmail.com" - # IMAP server port - port: 993 - # Whether the IMAP server uses SSL - ssl: true - # Whether the IMAP server uses StartTLS - start_tls: false - - # The mailbox where incoming mail will end up. Usually "inbox". - mailbox: "inbox" - # The IDLE command timeout. - idle_timeout: 60 - ``` - - As mentioned, the part after `+` is ignored, and this will end up in the mailbox for `gitlab-incoming@gmail.com`. + Configuration for Gmail / Google Apps, assumes mailbox `gitlab-incoming@gmail.com`: + + ```yaml + incoming_email: + enabled: true + + # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. + # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`). + address: "gitlab-incoming+%{key}@gmail.com" + + # Email account username + # With third party providers, this is usually the full email address. + # With self-hosted email servers, this is usually the user part of the email address. + user: "gitlab-incoming@gmail.com" + # Email account password + password: "[REDACTED]" + + # IMAP server host + host: "imap.gmail.com" + # IMAP server port + port: 993 + # Whether the IMAP server uses SSL + ssl: true + # Whether the IMAP server uses StartTLS + start_tls: false + + # The mailbox where incoming mail will end up. Usually "inbox". + mailbox: "inbox" + # The IDLE command timeout. + idle_timeout: 60 + ``` + + As mentioned, the part after `+` is ignored, and this will end up in the mailbox for `gitlab-incoming@gmail.com`. 1. Run this command in the GitLab root directory to launch `mail_room`: - ```sh - bundle exec mail_room -q -c config/mail_room.yml - ``` + ```sh + bundle exec mail_room -q -c config/mail_room.yml + ``` 1. Verify that everything is configured correctly: - ```sh - bundle exec rake gitlab:incoming_email:check RAILS_ENV=development - ``` + ```sh + bundle exec rake gitlab:incoming_email:check RAILS_ENV=development + ``` 1. Reply by email should now be working. @@ -91,10 +91,10 @@ for the format of the email key: Examples of valid email keys: - - `gitlab-org-gitlab-ce-20-Author_Token12345678-issue` (create a new issue) - - `gitlab-org-gitlab-ce-20-Author_Token12345678-merge-request` (create a new merge request) - - `1234567890abcdef1234567890abcdef-unsubscribe` (unsubscribe from a conversation) - - `1234567890abcdef1234567890abcdef` (reply to a conversation) +- `gitlab-org-gitlab-ce-20-Author_Token12345678-issue` (create a new issue) +- `gitlab-org-gitlab-ce-20-Author_Token12345678-merge-request` (create a new merge request) +- `1234567890abcdef1234567890abcdef-unsubscribe` (unsubscribe from a conversation) +- `1234567890abcdef1234567890abcdef` (reply to a conversation) Please note that the action `-issue-` is used in GitLab Premium as the handler for the Service Desk feature. @@ -103,10 +103,10 @@ Please note that the action `-issue-` is used in GitLab Premium as the handler f Although we continue to support the older legacy format, no new features should use a legacy format. These are the only valid legacy formats for an email handler: - - `path/to/project+namespace` - - `path/to/project+namespace+action` - - `namespace` - - `namespace+action` +- `path/to/project+namespace` +- `path/to/project+namespace+action` +- `namespace` +- `namespace+action` Please note that `path/to/project` is used in GitLab Premium as handler for the Service Desk feature. diff --git a/doc/development/gotchas.md b/doc/development/gotchas.md index 1b9ebb50c29..13dda17bb7d 100644 --- a/doc/development/gotchas.md +++ b/doc/development/gotchas.md @@ -101,10 +101,10 @@ end in a prepended module, which is very likely the case in EE. We could see error like this: - ``` - 1.1) Failure/Error: expect_any_instance_of(ApplicationSetting).to receive_messages(messages) - Using `any_instance` to stub a method (elasticsearch_indexing) that has been defined on a prepended module (EE::ApplicationSetting) is not supported. - ``` + ``` + 1.1) Failure/Error: expect_any_instance_of(ApplicationSetting).to receive_messages(messages) + Using `any_instance` to stub a method (elasticsearch_indexing) that has been defined on a prepended module (EE::ApplicationSetting) is not supported. + ``` ### Alternative: `expect_next_instance_of` diff --git a/doc/development/logging.md b/doc/development/logging.md index d61441813b2..4f63c84fc0e 100644 --- a/doc/development/logging.md +++ b/doc/development/logging.md @@ -30,8 +30,8 @@ Completed 200 OK in 166ms (Views: 117.4ms | ActiveRecord: 27.2ms) These logs suffer from a number of problems: 1. They often lack timestamps or other contextual information (e.g. project ID, user) -2. They may span multiple lines, which make them hard to find via Elasticsearch. -3. They lack a common structure, which make them hard to parse by log +1. They may span multiple lines, which make them hard to find via Elasticsearch. +1. They lack a common structure, which make them hard to parse by log forwarders, such as Logstash or Fluentd. This also makes them hard to search. @@ -67,46 +67,46 @@ importer progresses. Here's what to do: make it easy for people to search pertinent logs in one place. For example, `geo.log` contains all logs pertaining to GitLab Geo. To create a new file: - 1. Choose a filename (e.g. `importer_json.log`). - 1. Create a new subclass of `Gitlab::JsonLogger`: - - ```ruby - module Gitlab - module Import - class Logger < ::Gitlab::JsonLogger - def self.file_name_noext - 'importer' - end + 1. Choose a filename (e.g. `importer_json.log`). + 1. Create a new subclass of `Gitlab::JsonLogger`: + + ```ruby + module Gitlab + module Import + class Logger < ::Gitlab::JsonLogger + def self.file_name_noext + 'importer' end - end - end - ``` + end + end + end + ``` - 1. In your class where you want to log, you might initialize the logger as an instance variable: + 1. In your class where you want to log, you might initialize the logger as an instance variable: - ```ruby - attr_accessor :logger + ```ruby + attr_accessor :logger - def initialize - @logger = Gitlab::Import::Logger.build - end - ``` + def initialize + @logger = Gitlab::Import::Logger.build + end + ``` - Note that it's useful to memoize this because creating a new logger - each time you log will open a file, adding unnecessary overhead. + Note that it's useful to memoize this because creating a new logger + each time you log will open a file, adding unnecessary overhead. 1. Now insert log messages into your code. When adding logs, make sure to include all the context as key-value pairs: - ```ruby - # BAD - logger.info("Unable to create project #{project.id}") - ``` + ```ruby + # BAD + logger.info("Unable to create project #{project.id}") + ``` - ```ruby - # GOOD - logger.info(message: "Unable to create project", project_id: project.id) - ``` + ```ruby + # GOOD + logger.info(message: "Unable to create project", project_id: project.id) + ``` 1. Be sure to create a common base structure of your log messages. For example, all messages might have `current_user_id` and `project_id` to make it easier @@ -116,16 +116,16 @@ importer progresses. Here's what to do: logs properly if you [mix integer and string types](https://www.elastic.co/guide/en/elasticsearch/guide/current/mapping.html#_avoiding_type_gotchas): - ```ruby - # BAD - logger.info(message: "Import error", error: 1) - logger.info(message: "Import error", error: "I/O failure") - ``` + ```ruby + # BAD + logger.info(message: "Import error", error: 1) + logger.info(message: "Import error", error: "I/O failure") + ``` - ```ruby - # GOOD - logger.info(message: "Import error", error_code: 1, error: "I/O failure") - ``` + ```ruby + # GOOD + logger.info(message: "Import error", error_code: 1, error: "I/O failure") + ``` ## Additional steps with new log files diff --git a/doc/development/profiling.md b/doc/development/profiling.md index 795523b82aa..e1d1d2e33fa 100644 --- a/doc/development/profiling.md +++ b/doc/development/profiling.md @@ -95,7 +95,9 @@ Sherlock is a custom profiling tool built into GitLab. Sherlock is _only_ available when running GitLab in development mode _and_ when setting the environment variable `ENABLE_SHERLOCK` to a non empty value. For example: - ENABLE_SHERLOCK=1 bundle exec rails s +```sh +ENABLE_SHERLOCK=1 bundle exec rails s +``` Recorded transactions can be found by navigating to `/sherlock/transactions`. @@ -106,7 +108,9 @@ Bullet adds quite a bit of logging noise it's disabled by default. To enable Bullet, set the environment variable `ENABLE_BULLET` to a non-empty value before starting GitLab. For example: - ENABLE_BULLET=true bundle exec rails s +```sh +ENABLE_BULLET=true bundle exec rails s +``` Bullet will log query problems to both the Rails log as well as the Chrome console. diff --git a/doc/development/routing.md b/doc/development/routing.md index e9c0ad8d4e8..a25eb48b73c 100644 --- a/doc/development/routing.md +++ b/doc/development/routing.md @@ -7,11 +7,15 @@ support subgroups, GitLab project and group routes use the wildcard character to match project and group routes. For example, we might have a path such as: - /gitlab-com/customer-success/north-america/west/customerA +``` +/gitlab-com/customer-success/north-america/west/customerA +``` However, paths can be ambiguous. Consider the following example: - /gitlab-com/edit +``` +/gitlab-com/edit +``` It's ambiguous whether there is a subgroup named `edit` or whether this is a special endpoint to edit the `gitlab-com` group. @@ -25,8 +29,10 @@ number of [reserved names](../user/reserved_names.md). We have a number of global routes. For example: - /-/health - /-/metrics +``` +/-/health +/-/metrics +``` ## Group routes @@ -34,10 +40,12 @@ Every group route must be under the `/-/` scope. Examples: - gitlab-org/-/edit - gitlab-org/-/activity - gitlab-org/-/security/dashboard - gitlab-org/serverless/-/activity +``` +gitlab-org/-/edit +gitlab-org/-/activity +gitlab-org/-/security/dashboard +gitlab-org/serverless/-/activity +``` To achieve that, use the `scope '-'` method. @@ -48,10 +56,12 @@ client or other software requires something different. Examples: - gitlab-org/gitlab-ce/-/activity - gitlab-org/gitlab-ce/-/jobs/123 - gitlab-org/gitlab-ce/-/settings/repository - gitlab-org/serverless/runtimes/-/settings/repository +``` +gitlab-org/gitlab-ce/-/activity +gitlab-org/gitlab-ce/-/jobs/123 +gitlab-org/gitlab-ce/-/settings/repository +gitlab-org/serverless/runtimes/-/settings/repository +``` Currently, only some project routes are placed under the `/-/` scope. However, you can help us migrate more of them! To migrate project routes: diff --git a/doc/development/sql.md b/doc/development/sql.md index edeca7fb298..a256fd46c09 100644 --- a/doc/development/sql.md +++ b/doc/development/sql.md @@ -94,7 +94,9 @@ on the amount of data indexed). To keep naming of these indexes consistent please use the following naming pattern: - index_TABLE_on_COLUMN_trigram +``` +index_TABLE_on_COLUMN_trigram +``` For example, a GIN/trigram index for `issues.title` would be called `index_issues_on_title_trigram`. |