diff options
| author | GitLab Bot <gitlab-bot@gitlab.com> | 2023-12-19 14:01:45 +0300 |
|---|---|---|
| committer | GitLab Bot <gitlab-bot@gitlab.com> | 2023-12-19 14:01:45 +0300 |
| commit | 9297025d0b7ddf095eb618dfaaab2ff8f2018d8b (patch) | |
| tree | 865198c01d1824a9b098127baa3ab980c9cd2c06 /doc/development | |
| parent | 6372471f43ee03c05a7c1f8b0c6ac6b8a7431dbe (diff) | |
Add latest changes from gitlab-org/gitlab@16-7-stable-eev16.7.0-rc42
Diffstat (limited to 'doc/development')
437 files changed, 3101 insertions, 1588 deletions
diff --git a/doc/development/activitypub/actors/group.md b/doc/development/activitypub/actors/group.md index 1ebc9310e05..eec4ead7d78 100644 --- a/doc/development/activitypub/actors/group.md +++ b/doc/development/activitypub/actors/group.md @@ -1,7 +1,7 @@ --- stage: Create group: Source Code -info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments" remove_date: '2024-01-22' redirect_to: 'index.md' --- diff --git a/doc/development/activitypub/actors/index.md b/doc/development/activitypub/actors/index.md index 0a56cc7b093..6d82e79b9a0 100644 --- a/doc/development/activitypub/actors/index.md +++ b/doc/development/activitypub/actors/index.md @@ -1,7 +1,7 @@ --- stage: Create group: Source Code -info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments" --- # Implement an ActivityPub actor **(EXPERIMENT)** diff --git a/doc/development/activitypub/actors/project.md b/doc/development/activitypub/actors/project.md index 37c753b36e6..ed6280ad7f0 100644 --- a/doc/development/activitypub/actors/project.md +++ b/doc/development/activitypub/actors/project.md @@ -1,7 +1,7 @@ --- stage: Create group: Source Code -info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments" remove_date: '2024-01-22' redirect_to: 'index.md' --- diff --git a/doc/development/activitypub/actors/releases.md b/doc/development/activitypub/actors/releases.md index 009b98b6adf..ae3aabaa34a 100644 --- a/doc/development/activitypub/actors/releases.md +++ b/doc/development/activitypub/actors/releases.md @@ -1,7 +1,7 @@ --- stage: Create group: Source Code -info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments" --- # Activities for following releases actor **(EXPERIMENT)** diff --git a/doc/development/activitypub/actors/topic.md b/doc/development/activitypub/actors/topic.md index 2cac837791f..fe2e0bf5eff 100644 --- a/doc/development/activitypub/actors/topic.md +++ b/doc/development/activitypub/actors/topic.md @@ -1,7 +1,7 @@ --- stage: Create group: Source Code -info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments" remove_date: '2024-01-22' redirect_to: 'index.md' --- diff --git a/doc/development/activitypub/actors/user.md b/doc/development/activitypub/actors/user.md index fa4e948f27a..7ddab8d2700 100644 --- a/doc/development/activitypub/actors/user.md +++ b/doc/development/activitypub/actors/user.md @@ -1,7 +1,7 @@ --- stage: Create group: Source Code -info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments" remove_date: '2024-01-22' redirect_to: 'index.md' --- diff --git a/doc/development/activitypub/index.md b/doc/development/activitypub/index.md index ae00290f901..66bd9cc27f5 100644 --- a/doc/development/activitypub/index.md +++ b/doc/development/activitypub/index.md @@ -1,7 +1,7 @@ --- stage: Create group: Source Code -info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments" --- # ActivityPub **(EXPERIMENT)** diff --git a/doc/development/adding_service_component.md b/doc/development/adding_service_component.md index 3ce303d429a..7996a9b1195 100644 --- a/doc/development/adding_service_component.md +++ b/doc/development/adding_service_component.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Adding a new Service Component to GitLab diff --git a/doc/development/advanced_search.md b/doc/development/advanced_search.md index 805459cb4ee..a552b22226d 100644 --- a/doc/development/advanced_search.md +++ b/doc/development/advanced_search.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Global Search -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Advanced search development guidelines @@ -32,7 +32,7 @@ See the [Elasticsearch GDK setup instructions](https://gitlab.com/gitlab-org/git - `gitlab:elastic:test:index_size`: Tells you how much space the current index is using, as well as how many documents are in the index. - `gitlab:elastic:test:index_size_change`: Outputs index size, reindexes, and outputs index size again. Useful when testing improvements to indexing size. -Additionally, if you need large repositories or multiple forks for testing, please consider [following these instructions](rake_tasks.md#extra-project-seed-options) +Additionally, if you need large repositories or multiple forks for testing, consider [following these instructions](rake_tasks.md#extra-project-seed-options) ## How does it work? @@ -40,7 +40,7 @@ The Elasticsearch integration depends on an external indexer. We ship an [indexe After initial indexing is complete, create, update, and delete operations for all models except projects (see [#207494](https://gitlab.com/gitlab-org/gitlab/-/issues/207494)) are tracked in a Redis [`ZSET`](https://redis.io/docs/manual/data-types/#sorted-sets). A regular `sidekiq-cron` `ElasticIndexBulkCronWorker` processes this queue, updating many Elasticsearch documents at a time with the [Bulk Request API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html). -Search queries are generated by the concerns found in [`ee/app/models/concerns/elastic`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/app/models/concerns/elastic). These concerns are also in charge of access control, and have been a historic source of security bugs so please pay close attention to them! +Search queries are generated by the concerns found in [`ee/app/models/concerns/elastic`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/app/models/concerns/elastic). These concerns are also in charge of access control, and have been a historic source of security bugs so pay close attention to them! ### Custom routing @@ -62,13 +62,13 @@ The following analyzers and tokenizers are defined in [`ee/lib/elastic/latest/co Used when indexing blobs' paths. Uses the `path_tokenizer` and the `lowercase` and `asciifolding` filters. -Please see the `path_tokenizer` explanation below for an example. +See the `path_tokenizer` explanation below for an example. #### `sha_analyzer` Used in blobs and commits. Uses the `sha_tokenizer` and the `lowercase` and `asciifolding` filters. -Please see the `sha_tokenizer` explanation later below for an example. +See the `sha_tokenizer` explanation later below for an example. #### `code_analyzer` @@ -76,7 +76,7 @@ Used when indexing a blob's filename and content. Uses the `whitespace` tokenize The `whitespace` tokenizer was selected to have more control over how tokens are split. For example the string `Foo::bar(4)` needs to generate tokens like `Foo` and `bar(4)` to be properly searched. -Please see the `code` filter for an explanation on how tokens are split. +See the `code` filter for an explanation on how tokens are split. NOTE: The [Elasticsearch `code_analyzer` doesn't account for all code cases](../integration/advanced_search/elasticsearch_troubleshooting.md#elasticsearch-code_analyzer-doesnt-account-for-all-code-cases). diff --git a/doc/development/ai_architecture.md b/doc/development/ai_architecture.md index 54ad52f0c39..0717872a12b 100644 --- a/doc/development/ai_architecture.md +++ b/doc/development/ai_architecture.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # AI Architecture @@ -19,7 +19,7 @@ The following diagram from the [architecture blueprint](../architecture/blueprin ## SaaS-based AI abstraction layer -GitLab currently operates a cloud-hosted AI architecture. We will allow access to it for licensed self managed instances using the AI-gateway. Please see [the blueprint](../architecture/blueprints/ai_gateway) for details +GitLab currently operates a cloud-hosted AI architecture. We will allow access to it for licensed self managed instances using the AI-gateway. See [the blueprint](../architecture/blueprints/ai_gateway) for details. There are two primary reasons for this: the best AI models are cloud-based as they often depend on specialized hardware designed for this purpose, and operating self-managed infrastructure capable of AI at-scale and with appropriate performance is a significant undertaking. We are actively [tracking self-managed customers interested in AI](https://gitlab.com/gitlab-org/gitlab/-/issues/409183). diff --git a/doc/development/ai_features/duo_chat.md b/doc/development/ai_features/duo_chat.md index ad044f4a923..d7f88997fca 100644 --- a/doc/development/ai_features/duo_chat.md +++ b/doc/development/ai_features/duo_chat.md @@ -1,7 +1,7 @@ --- stage: AI-powered group: Duo Chat -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # GitLab Duo Chat @@ -14,7 +14,7 @@ Use [this snippet](https://gitlab.com/gitlab-org/gitlab/-/snippets/2554994) for 1. [Enable Anthropic API features](index.md#configure-anthropic-access). 1. [Ensure the embedding database is configured](index.md#set-up-the-embedding-database). 1. Ensure that your current branch is up-to-date with `master`. -1. To access the GitLab Duo Chat interface, in the lower-left corner of any page, select **Help** and **Ask GitLab Duo Chat**. +1. Enable the feature in Rails console: `Feature.enable(:tanuki_bot_breadcrumbs_entry_point)` ## Working with GitLab Duo Chat @@ -85,7 +85,7 @@ gdk start tail -f log/llm.log ``` -## Testing GitLab Duo Chat against real LLMs +## Testing GitLab Duo Chat against real LLMs locally Because success of answers to user questions in GitLab Duo Chat heavily depends on toolchain and prompts of each tool, it's common that even a minor change in a @@ -101,7 +101,7 @@ export ANTHROPIC_API_KEY='<key>' # can use dev value of Gitlab::CurrentSettings export VERTEX_AI_CREDENTIALS='<vertex-ai-credentials>' # can set as dev value of Gitlab::CurrentSettings.vertex_ai_credentials export VERTEX_AI_PROJECT='<vertex-project-name>' # can use dev value of Gitlab::CurrentSettings.vertex_ai_project -REAL_AI_REQUEST=1 bundle exec rspec ee/spec/lib/gitlab/llm/chain/agents/zero_shot/executor_real_requests_spec.rb +REAL_AI_REQUEST=1 bundle exec rspec ee/spec/lib/gitlab/llm/completions/chat_real_requests_spec.rb ``` When you need to update the test questions that require documentation embeddings, @@ -109,21 +109,96 @@ make sure a new fixture is generated and committed together with the change. ## Running the rspecs tagged with `real_ai_request` -The rspecs tagged with the metadata `real_ai_request` can be run in GitLab project's CI by triggering -`rspec-ee unit gitlab-duo-chat`. -The former runs with Vertex APIs enabled. The CI jobs are optional and allowed to fail to account for -the non-deterministic nature of LLM responses. +The following CI jobs for GitLab project run the rspecs tagged with `real_ai_request`: + +- `rspec-ee unit gitlab-duo-chat-zeroshot`: + the job runs `ee/spec/lib/gitlab/llm/completions/chat_real_requests_spec.rb`. + The job is optionally triggered and allowed to fail. + +- `rspec-ee unit gitlab-duo-chat-qa`: + The job runs the QA evaluation tests in + `ee/spec/lib/gitlab/llm/chain/agents/zero_shot/qa_evaluation_spec.rb`. + The job is optionally triggered and allowed to fail. + Read about [GitLab Duo Chat QA Evaluation Test](#gitlab-duo-chat-qa-evaluation-test). + +- `rspec-ee unit gitlab-duo-chat-qa-fast`: + The job runs a single QA evaluation test from `ee/spec/lib/gitlab/llm/chain/agents/zero_shot/qa_evaluation_spec.rb`. + The job is always run and not allowed to fail. Although there's a chance that the QA test still might fail, + it is cheap and fast to run and intended to prevent a regression in the QA test helpers. + +- `rspec-ee unit gitlab-duo pg14`: + This job runs tests to ensure that the GitLab Duo features are functional without running into system errors. + The job is always run and not allowed to fail. + This job does NOT conduct evaluations. The quality of the feature is tested in the other jobs such as QA jobs. ### Management of credentials and API keys for CI jobs All API keys required to run the rspecs should be [masked](../../ci/variables/index.md#mask-a-cicd-variable) The exception is GCP credentials as they contain characters that prevent them from being masked. -Because `rspec-ee unit gitlab-duo-chat` needs to run on MR branches, GCP credentials cannot be added as a protected variable +Because the CI jobs need to run on MR branches, GCP credentials cannot be added as a protected variable and must be added as a regular CI variable. For security, the GCP credentials and the associated project added to GitLab project's CI must not be able to access any production infrastructure and sandboxed. +### GitLab Duo Chat QA Evaluation Test + +Evaluation of a natural language generation (NLG) system such as +GitLab Duo Chat is a rapidly evolving area with many unanswered questions and ambiguities. + +A practical working assumption is LLMs can generate a reasonable answer when given a clear question and a context. +With the assumption, we are exploring using LLMs as evaluators +to determine the correctness of a sample of questions +to track the overall accuracy of GitLab Duo Chat's responses and detect regressions in the feature. + +For the discussions related to the topic, +see [the merge request](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/merge_requests/431) +and [the issue](https://gitlab.com/gitlab-org/gitlab/-/issues/427251). + +The current QA evaluation test consists of the following components. + +#### Epic and issue fixtures + +The fixtures are the replicas of the _public_ issues and epics from projects and groups _owned by_ GitLab. +The internal notes were excluded when they were sampled. The fixtures have been commited into the canonical `gitlab` repository. +See [the snippet](https://gitlab.com/gitlab-org/gitlab/-/snippets/3613745) used to create the fixtures. + +#### RSpec and helpers + +1. [The RSpec file](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/spec/lib/gitlab/llm/chain/agents/zero_shot/qa_evaluation_spec.rb) + and the included helpers invoke the Chat service, an internal interface with the question. + +1. After collecting the Chat service's answer, + the answer is injected into a prompt, also known as an "evaluation prompt", that instructs + a LLM to grade the correctness of the answer based on the question and a context. + The context is simply a JSON serialization of the issue or epic being asked about in each question. + +1. The evaluation prompt is sent to two LLMs, Claude and Vertex. + +1. The evaluation responses of the LLMs are saved as JSON files. + +1. For each question, RSpec will regex-match for `CORRECT` or `INCORRECT`. + +#### Collection and tracking of QA evaluations via CI/CD automation + +The `gitlab` project's CI configurations have been setup to +run the RSpec, +collect the evaluation response as artifacts +and execute [a reporter script](https://gitlab.com/gitlab-org/gitlab/-/blob/master/scripts/duo_chat/reporter.rb) +that automates collection and tracking of evaluations. + +When `rspec-ee unit gitlab-duo-chat-qa` job runs in a pipeline for a merge request, +the reporter script uses the evaluations saved as CI artifacts +to generate a Markdown report and posts it as a note in the merge request. + +When `rspec-ee unit gitlab-duo-chat-qa` is run in a pipeline for a commit on `master` branch, +the reporter script instead +posts the generated report as an issue, +saves the evaluations artfacts as a snippet, +and updates the tracking issue in +[`gitlab-org/ai-powered/ai-framework/qa-evaluation#1`](https://gitlab.com/gitlab-org/ai-powered/ai-framework/qa-evaluation/-/issues/1) +in the project [`gitlab-org/ai-powered/ai-framework/qa-evaluation`](https://gitlab.com/gitlab-org/ai-powered/ai-framework/qa-evaluation). + ## GraphQL Subscription The GraphQL Subscription for Chat behaves slightly different because it's user-centric. A user could have Chat open on multiple browser tabs, or also on their IDE. @@ -134,3 +209,14 @@ We therefore need to broadcast messages to multiple clients to keep them in sync Note that we still broadcast chat messages and currently used tools using the `userId` and `resourceId` as identifier. However, this is deprecated and should no longer be used. We want to remove `resourceId` on the subscription as part of [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/420296). + +## Testing GitLab Duo Chat in production-like environments + +GitLab Duo Chat is enabled in the [Staging](https://staging.gitlab.com) and +[Staging Ref](https://staging-ref.gitlab.com/) GitLab environments. + +Because GitLab Duo Chat is currently only available to members of groups in the +Ultimate tier, Staging Ref may be an easier place to test changes as a GitLab +team member because +[you can make yourself an instance Admin in Staging Ref](https://about.gitlab.com/handbook/engineering/infrastructure/environments/staging-ref/#admin-access) +and, as an Admin, easily create licensed groups for testing. diff --git a/doc/development/ai_features/glossary.md b/doc/development/ai_features/glossary.md new file mode 100644 index 00000000000..be856639b83 --- /dev/null +++ b/doc/development/ai_features/glossary.md @@ -0,0 +1,81 @@ +--- +stage: AI-powered +group: AI Framework +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# GitLab Duo Glossary + +This is a list of terms that may have a general meaning but also may have a +specific meaning at GitLab. If you encounter a piece of technical jargon related +to AI that you think could benefit from being in this list, add it! + +- **AI Gateway**: standalone service used to give access to AI features to + non-SaaS GitLab users. This logic will be moved to Cloud Connector when that + service is ready. Eventually, the AI Gateway will be used to host endpoints that + proxy requests to AI providers, removing the need for the GitLab Rails monolith + to integrate and communicate directly with third-party LLMs. + [Blueprint](../../architecture/blueprints/ai_gateway/index.md). +- **Chat Evaluation**: automated mechanism for determining the helpfulness and + accuracy of GitLab Duo Chat to various user questions. The MVC is an RSpec test + run via GitLab CI that asks a set of questions to Chat and then has a + two different third-party LLMs determine if the generated answer is accurate or not. + [MVC](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/134610). + [Design doc for next iteration](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/136127). +- **Cloud Connector**: Today, Cloud Connector is not a system. It is an umbrella + term for all the projects we engage in that make existing SaaS-only features + available to self-managed and GitLab Dedicated customers. Today, the only + feature available through Cloud Connector is Code Suggestions. + Cloud Connector also refer to a planned GitLab-hosted edge service which would + act as a way for non-SaaS GitLab instances to access SaaS offerings. + [Cloud Connector MVC](../cloud_connector/code_suggestions_for_sm.md). + [Blueprint for future Cloud Connector service](../../architecture/blueprints/cloud_connector/index.md). +- **Consensus Filtering**: method for LLM evaluation where you instruct an LLM + to evaluate the output of another LLM based on the question and context that + resulted in the output. This is the method of evaluation being used for the Chat + Evaluation MVC. + [Issue from Model Validation team](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/prompt-library/-/issues/91#metric-2-consensus-filtering-with-llm-based-evaluation). +- **Context**: relevant information that surrounds a data point, an event, or a + piece of information, which helps to clarify its meaning and implications. + For GitLab Duo Chat, context is the attributes of the Issue or Epic being + referenced in a user question. +- **Golden Questions**: a small subset of the types of questions we think a user + should be able to ask GitLab Duo Chat. Used to generate data for Chat evaluation. + [Questions for Chat Beta](https://gitlab.com/groups/gitlab-org/-/epics/10550#what-the-user-can-ask). +- **Ground Truth**: data that is determined to be the true + output for a given input, representing the reality that the AI model aims to + learn and predict. Ground truth data is usually human-annotated. +- **Model Validation**: group within the AI-powered Stage working on the Prompt + Library and researching AI/ML models to support other use-cases for AI at GitLab. + [Team handbook section](https://about.gitlab.com/handbook/product/categories/features/#ai-poweredai-model-validation-group). +- **Prompt library**: The ["Prompt Library"](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/prompt-library) is a Python library that provides a CLI for testing different prompting techniques with LLMs. It enables data-driven improvements to LLM applications by facilitating hypothesis testing. Key features include the ability to manage and run dataflow pipelines using Apache Beam, and the execution of multiple evaluation experiments in a single pipeline run. + on prompts with various third-party AI Services. + [Code](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/prompt-library). +- **Prompt Registry**: stored, versioned prompts used to interact with third-party + AI Services. [Blueprint](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/135872). +- **Prompt**: instructions sent to an LLM to perform certain tasks. [Prompt guidelines](prompts.md). +- **RAG Pipeline**: (Retrieval-Augmented Generation) is a mechanism used to take + an input (such as a user question) into a system, retrieve any relevant data + for that input, augment the input with additional context, and then + synthesize the information to generate a coherent, contextualy-relevant answer. + This design pattern is helpful in open-domain question answering with LLMs, + which is why we use this design pattern for answering questions to GitLab Duo Chat. +- **Similarity Score**: method to determine the likeness between answers produced by an LLM and the reference ground truth answers. + [Issue from Model Validation team](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/prompt-library/-/issues/91#metric-1-similarity-score-as-comparisons-for-llms). +- **Tool**: logic that performs a specific LLM-related task; each tool has a + description and its own prompt. [How to add a new tool](duo_chat.md#adding-a-new-tool). +- **Word-Level Metrics**: method for LLM evaluation that compares aspects of + text at the granularity of individual words. + [Issue from Model Validation team](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/prompt-library/-/issues/98#metric-3-word-level-metrics). +- **Zero-shot agent**: in the general world of AI, a learning model or system + that can perform tasks without having seen any examples of that task during + training. At GitLab, we use this term to refer specifically to a piece of our + code that serves as a sort of LLM-powered air traffic controller for GitLab Duo Chat. + The GitLab zero-shot agent has a system prompt that explains how an LLM should + interpret user input from GitLab Duo Chat as well as a list of tool descriptions. + Using this information, the agent determines which tool to use to answer a + user's question. The agent may decide that no tools are required and answer the + question directly. If a tool is used, the answer from the tool is fed back to + the zero-shot agent to evaluate if the answer is sufficient or if an additional + tool must be used to answer the question. + [Code](https://gitlab.com/gitlab-org/gitlab/-/blob/6b747cbd7c6a71145a8bfb8201db3c857b5aed6a/ee/lib/gitlab/llm/chain/agents/zero_shot/executor.rb). [Zero-shot agent in action](https://gitlab.com/gitlab-org/gitlab/-/issues/427979). diff --git a/doc/development/ai_features/index.md b/doc/development/ai_features/index.md index df1627f2dc3..e99a49f3bb9 100644 --- a/doc/development/ai_features/index.md +++ b/doc/development/ai_features/index.md @@ -1,7 +1,7 @@ --- stage: AI-powered group: AI Framework -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # AI features based on 3rd-party integrations @@ -37,7 +37,7 @@ Apply the following two feature flags to any AI feature work: - A general flag (`ai_global_switch`) that applies to all AI features. - A flag specific to that feature. The feature flag name [must be different](../feature_flags/index.md#feature-flags-for-licensed-features) than the licensed feature name. -See the [feature flag tracker](https://gitlab.com/gitlab-org/gitlab/-/issues/405161) for the list of all feature flags and how to use them. +See the [feature flag tracker epic](https://gitlab.com/groups/gitlab-org/-/epics/10524) for the list of all feature flags and how to use them. ## Implement a new AI action @@ -50,8 +50,14 @@ All AI features are experimental. ## Test AI features locally -NOTE: -Use [this snippet](https://gitlab.com/gitlab-org/gitlab/-/snippets/2554994) for help automating the following section. +**One-line setup** + +```shell +# Replace the <test-group-name> by the group name you want to enable GitLab Duo features. If the group doesn't exist, it creates a new one. +RAILS_ENV=development bundle exec rake gitlab:duo:setup['<test-group-name>'] +``` + +**Manual way** 1. Enable the required general feature flags: @@ -61,20 +67,18 @@ Use [this snippet](https://gitlab.com/gitlab-org/gitlab/-/snippets/2554994) for 1. Ensure you have followed [the process to obtain an EE license](https://about.gitlab.com/handbook/developer-onboarding/#working-on-gitlab-ee-developer-licenses) for your local instance 1. Simulate the GDK to [simulate SaaS](../ee_features.md#simulate-a-saas-instance) and ensure the group you want to test has an Ultimate license -1. Enable `Experimental features`: +1. Enable `Experiment & Beta features` 1. Go to the group with the Ultimate license 1. **Group Settings** > **General** -> **Permissions and group features** - 1. Enable **Experiment features** + 1. Enable **Experiment & Beta features** 1. Enable the specific feature flag for the feature you want to test +1. You can use Rake task `rake gitlab:duo:enable_feature_flags` to enable all feature flags that are assigned to group AI Framework 1. Set the required access token. To receive an access token: 1. For Vertex, follow the [instructions below](#configure-gcp-vertex-access). - 1. For all other providers, like Anthropic, create an access request where `@m_gill`, `@wayne`, and `@timzallmann` are the tech stack owners. + 1. For Anthropic, create an access request ### Set up the embedding database -NOTE: -Use [this snippet](https://gitlab.com/gitlab-org/gitlab/-/snippets/2554994) for help automating the following section. - For features that use the embedding database, additional setup is needed. 1. Enable [`pgvector`](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/pgvector.md#enable-pgvector-in-the-gdk) in GDK @@ -87,19 +91,13 @@ For features that use the embedding database, additional setup is needed. 1. Run `gdk reconfigure` 1. Run database migrations to create the embedding database -### Setup for GitLab documentation chat (legacy chat) - -To populate the embedding database for GitLab chat: - -1. Open a rails console -1. Run [this script](https://gitlab.com/gitlab-com/gl-infra/production/-/issues/10588#note_1373586079) to populate the embedding database - ### Configure GCP Vertex access -In order to obtain a GCP service key for local development, please follow the steps below: +In order to obtain a GCP service key for local development, follow the steps below: - Create a sandbox GCP project by visiting [this page](https://about.gitlab.com/handbook/infrastructure-standards/#individual-environment) and following the instructions, or by requesting access to our existing group GCP project by using [this template](https://gitlab.com/gitlab-com/it/infra/issue-tracker/-/issues/new?issuable_template=gcp_group_account_iam_update_request). - If you are using an individual GCP project, you may also need to enable the Vertex AI API: + 1. Visit [welcome page](https://console.cloud.google.com/welcome), choose your project (e.g. jdoe-5d23dpe). 1. Go to **APIs & Services > Enabled APIs & services**. 1. Select **+ Enable APIs and Services**. 1. Search for `Vertex AI API`. @@ -141,7 +139,7 @@ we can add a few selected embeddings to the table from a pre-generated fixture. For instance, to test that the question "How can I reset my password" is correctly retrieving the relevant embeddings and answered, we can extract the top N closet embeddings to the question into a fixture and only restore a small number of embeddings quickly. -To faciliate an extraction process, a Rake task been written. +To facilitate an extraction process, a Rake task has been written. You can add or remove the questions needed to be tested in the Rake task and run the task to generate a new fixture. ```shell @@ -277,65 +275,6 @@ I --> J[GraphqlTriggers.ai_completion_response] J --> K[::GitlabSchema.subscriptions.trigger] ``` -## CircuitBreaker - -The CircuitBreaker concern is a reusable module that you can include in any class that needs to run code with circuit breaker protection. The concern provides a `run_with_circuit` method that wraps a code block with circuit breaker functionality, which helps prevent cascading failures and improves system resilience. For more information about the circuit breaker pattern, see: - -- [What is Circuit breaker](https://martinfowler.com/bliki/CircuitBreaker.html). -- [The Hystrix documentation on CircuitBreaker](https://github.com/Netflix/Hystrix/wiki/How-it-Works#circuit-breaker). - -### Use CircuitBreaker - -To use the CircuitBreaker concern, you need to include it in a class. For example: - -```ruby -class MyService - include Gitlab::Llm::Concerns::CircuitBreaker - - def call_external_service - run_with_circuit do - # Code that interacts with external service goes here - - raise InternalServerError - end - end -end -``` - -The `call_external_service` method is an example method that interacts with an external service. -By wrapping the code that interacts with the external service with `run_with_circuit`, the method is executed within the circuit breaker. -The circuit breaker is created and configured by the `circuit` method, which is called automatically when the `CircuitBreaker` module is included. -The method should raise `InternalServerError` error which will be counted towards the error threshold if raised during the execution of the code block. - -The circuit breaker tracks the number of errors and the rate of requests, -and opens the circuit if it reaches the configured error threshold or volume threshold. -If the circuit is open, subsequent requests fail fast without executing the code block, and the circuit breaker periodically allows a small number of requests through to test the service's availability before closing the circuit again. - -### Configuration - -The circuit breaker is configured with two constants which control the number of errors and requests at which the circuit will open: - -- `ERROR_THRESHOLD` -- `VOLUME_THRESHOLD` - -You can adjust these values as needed for the specific service and usage pattern. -The `InternalServerError` is the exception class counted towards the error threshold if raised during the execution of the code block. -This is the exception class that triggers the circuit breaker when raised by the code that interacts with the external service. - -NOTE: -The `CircuitBreaker` module depends on the `Circuitbox` gem to provide the circuit breaker implementation. By default, the service name is inferred from the class name where the concern module is included. Override the `service_name` method if the name needs to be different. - -### Testing - -To test code that uses the `CircuitBreaker` concern, you can use `RSpec` shared examples and pass the `service` and `subject` variables: - -```ruby -it_behaves_like 'has circuit breaker' do - let(:service) { dummy_class.new } - let(:subject) { service.dummy_method } -end -``` - ## How to implement a new action ### Register a new method @@ -523,7 +462,7 @@ module Gitlab end def to_prompt - <<-PROMPT + <<~PROMPT You are an assistant that writes code for the following context: context: #{user_input} diff --git a/doc/development/ai_features/prompts.md b/doc/development/ai_features/prompts.md index f4738055f6b..b3a2462331b 100644 --- a/doc/development/ai_features/prompts.md +++ b/doc/development/ai_features/prompts.md @@ -1,7 +1,7 @@ --- stage: AI-powered group: AI Framework -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Working with AI prompts diff --git a/doc/development/api_graphql_styleguide.md b/doc/development/api_graphql_styleguide.md index 318f9bed6d3..cfe82fe9b81 100644 --- a/doc/development/api_graphql_styleguide.md +++ b/doc/development/api_graphql_styleguide.md @@ -1,13 +1,27 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # GraphQL API style guide This document outlines the style guide for the GitLab [GraphQL API](../api/graphql/index.md). +## Vision + +We want the GraphQL API to be the **primary** means of interacting +programmatically with GitLab. To achieve this, it needs full coverage - anything +possible in the REST API should also be possible in the GraphQL API. + +To help us meet this vision, the frontend should use GraphQL in preference to +the REST API for new features. +The GraphQL API is [versionless](https://graphql.org/learn/best-practices/#versioning). + +There are no plans to deprecate the REST API. To reduce the technical burden of +supporting two APIs in parallel, they should share implementations as much as +possible. + ## How GitLab implements GraphQL <!-- vale gitlab.Spelling = NO --> @@ -44,7 +58,7 @@ For example, the one for [GitLab.com](https://gitlab.com/-/graphql-explorer). The GraphQL framework has some specific gotchas to be aware of, and domain expertise is required to ensure they are satisfied. -If you are asked to review a merge request that modifies any GraphQL files or adds an endpoint, please have a look at +If you are asked to review a merge request that modifies any GraphQL files or adds an endpoint, have a look at [our GraphQL review guide](graphql_guide/reviewing.md). ## Reading GraphQL logs @@ -78,7 +92,7 @@ a connection. ### Max complexity -Complexity is explained [on our client-facing API page](../api/graphql/index.md#max-query-complexity). +Complexity is explained [on our client-facing API page](../api/graphql/index.md#maximum-query-complexity). Fields default to adding `1` to a query's complexity score, but developers can [specify a custom complexity](#field-complexity) when defining a field. @@ -174,8 +188,8 @@ See the [deprecating schema items](#deprecating-schema-items) section for how to ### Breaking change exemptions -Schema items [marked as alpha](#mark-schema-items-as-alpha) are exempt from the deprecation process, -and can be removed or changed at any time without notice. +See the +[GraphQL API breaking change exemptions documentation](../api/graphql/index.md#breaking-change-exemptions). ## Global IDs @@ -190,6 +204,7 @@ See also: - [Exposing Global IDs](#exposing-global-ids). - [Mutation arguments](#object-identifier-arguments). - [Deprecating Global IDs](#deprecate-global-ids). +- [Customer-facing Global ID documentation](../api/graphql/index.md#global-ids). We have a custom scalar type (`Types::GlobalIDType`) which should be used as the type of input and output arguments when the value is a `GlobalID`. The benefits @@ -205,6 +220,35 @@ it is perfectly possible to parameterize this type with a concern or a supertype, if you want to accept a wider range of objects (such as `GlobalIDType[Issuable]` vs `GlobalIDType[Issue]`). +## Optimizations + +By default, GraphQL tends to introduce N+1 problems unless you actively try to minimize them. + +For stability and scalability, you must ensure that our queries do not suffer from N+1 +performance issues. + +The following are a list of tools to help you to optimize your GraphQL code: + +- [Look ahead](#look-ahead) allows you to preload data based on which fields are selected in the query. +- [Batch loading](graphql_guide/batchloader.md) allows you batch database queries together to be executed in one statement. +- [`BatchModelLoader`](graphql_guide/batchloader.md#the-batchmodelloader) is the recommended way to lookup + records by ID to leverage batch loading. +- [`before_connection_authorization`](#before_connection_authorization) allows you to address N+1 problems + specific to [type authorization](#authorization) permission checks. +- [Limit maximum field call count](#limit-maximum-field-call-count) allows you to restrict how many times + a field can return data where optimizations cannot be improved. + +## How to see N+1 problems in development + +N+1 problems can be discovered during development of a feature by: + +- Tailing `development.log` while you execute GraphQL queries that return collections of data. + [Bullet](../development/profiling.md#bullet) may help. +- Observing the [performance bar](../administration/monitoring/performance/performance_bar.md) if + executing queries in the GitLab UI. +- Adding a [request spec](#testing-tips-and-tricks) that asserts there are no (or limited) N+1 + problems with the feature. + ## Types We use a code-first schema, and we declare what type everything is in Ruby. @@ -447,7 +491,7 @@ field :tags, ### Field complexity The GitLab GraphQL API uses a _complexity_ score to limit performing overly complex queries. -Complexity is described in [our client documentation](../api/graphql/index.md#max-query-complexity) on the topic. +Complexity is described in [our client documentation](../api/graphql/index.md#maximum-query-complexity) on the topic. Complexity limits are defined in [`app/graphql/gitlab_schema.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/graphql/gitlab_schema.rb). @@ -806,9 +850,10 @@ The documentation mentions that the old Global ID style is now deprecated. You can mark GraphQL schema items (fields, arguments, enum values, and mutations) as [Alpha](../policy/experiment-beta-support.md#experiment). -An item marked as Alpha is [exempt from the deprecation process](#breaking-change-exemptions) and can be removed -at any time without notice. Mark an item as Alpha when it is -subject to change and not ready for public use. +An item marked as Alpha is +[exempt from the deprecation process](../api/graphql/index.md#breaking-change-exemptions) and can be +removed at any time without notice. Mark an item as Alpha when it is subject to +change and not ready for public use. NOTE: Only mark new items as Alpha. Never mark existing items @@ -833,8 +878,8 @@ mount_mutation Mutations::Ci::JobArtifact::BulkDestroy, alpha: { milestone: '15. Alpha GraphQL items is a custom GitLab feature that leverages GraphQL deprecations. An Alpha item appears as deprecated in the GraphQL schema. Like all deprecated schema items, you can test an -Alpha field in [GraphiQL](../api/graphql/index.md#graphiql). However, be aware that the GraphiQL -autocomplete editor doesn't suggest deprecated fields. +Alpha field in the [interactive GraphQL explorer](../api/graphql/index.md#interactive-graphql-explorer) (GraphiQL). +However, be aware that the GraphiQL autocomplete editor doesn't suggest deprecated fields. The item shows as Alpha in our generated GraphQL documentation and its GraphQL schema description. @@ -1091,7 +1136,7 @@ To find objects to display in a field, we can add resolvers to `app/graphql/resolvers`. Arguments can be defined in the resolver in the same way as in a mutation. -See the [Mutation arguments](#object-identifier-arguments) section. +See the [Arguments](#arguments) section. To limit the amount of queries performed, we can use [BatchLoader](graphql_guide/batchloader.md). @@ -1260,32 +1305,9 @@ field :jobs, resolver: JobsResolver, description: 'All jobs.' field :job, resolver: JobsResolver.single, description: 'A single job.' ``` -### Correct use of `Resolver#ready?` - -Resolvers have two public API methods as part of the framework: `#ready?(**args)` and `#resolve(**args)`. -We can use `#ready?` to perform set-up, validation or early-return without invoking `#resolve`. - -Good reasons to use `#ready?` include: +### Optimizing Resolvers -- validating mutually exclusive arguments (see [validating arguments](#validating-arguments)) -- Returning `Relation.none` if we know before-hand that no results are possible -- Performing setup such as initializing instance variables (although consider lazily initialized methods for this) - -Implementations of [`Resolver#ready?(**args)`](https://graphql-ruby.org/api-doc/1.10.9/GraphQL/Schema/Resolver#ready%3F-instance_method) should -return `(Boolean, early_return_data)` as follows: - -```ruby -def ready?(**args) - [false, 'have this instead'] -end -``` - -For this reason, whenever you call a resolver (mainly in tests - as framework -abstractions Resolvers should not be considered re-usable, finders are to be -preferred), remember to call the `ready?` method and check the boolean flag -before calling `resolve`! An example can be seen in our [`GraphqlHelpers`](https://gitlab.com/gitlab-org/gitlab/-/blob/2d395f32d2efbb713f7bc861f96147a2a67e92f2/spec/support/helpers/graphql_helpers.rb#L20-27). - -### Look-Ahead +#### Look-Ahead The full query is known in advance during execution, which means we can make use of [lookahead](https://graphql-ruby.org/queries/lookahead.html) to optimize our @@ -1368,6 +1390,54 @@ end For an example of real world use, see [`ResolvesMergeRequests`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/graphql/resolvers/concerns/resolves_merge_requests.rb). +#### `before_connection_authorization` + +A `before_connection_authorization` hook can help resolvers eliminate N+1 problems that originate from +[type authorization](graphql_guide/authorization.md#type-authorization) permission checks. + +The `before_connection_authorization` method receives the resolved nodes and the current user. In +the block, use `ActiveRecord::Associations::Preloader` or a `Preloaders::` class to preload data +for the type authorization check. + +Example: + +```ruby +class LabelsResolver < BaseResolver + before_connection_authorization do |labels, current_user| + Preloaders::LabelsPreloader.new(labels, current_user).preload_all + end +end +``` + +#### BatchLoading + +See [GraphQL BatchLoader](graphql_guide/batchloader.md). + +### Correct use of `Resolver#ready?` + +Resolvers have two public API methods as part of the framework: `#ready?(**args)` and `#resolve(**args)`. +We can use `#ready?` to perform set-up, validation, or early-return without invoking `#resolve`. + +Good reasons to use `#ready?` include: + +- Validating mutually exclusive arguments. +- Returning `Relation.none` if we know before-hand that no results are possible. +- Performing setup such as initializing instance variables (although consider lazily initialized methods for this). + +Implementations of [`Resolver#ready?(**args)`](https://graphql-ruby.org/api-doc/1.10.9/GraphQL/Schema/Resolver#ready%3F-instance_method) should +return `(Boolean, early_return_data)` as follows: + +```ruby +def ready?(**args) + [false, 'have this instead'] +end +``` + +For this reason, whenever you call a resolver (mainly in tests because framework +abstractions Resolvers should not be considered re-usable, finders are to be +preferred), remember to call the `ready?` method and check the boolean flag +before calling `resolve`! An example can be seen in our [`GraphqlHelpers`](https://gitlab.com/gitlab-org/gitlab/-/blob/2d395f32d2efbb713f7bc861f96147a2a67e92f2/spec/support/helpers/graphql_helpers.rb#L20-27). + ### Negated arguments Negated filters can filter some resources (for example, find all issues that @@ -1555,85 +1625,6 @@ Examples: If you need advice for mutation naming, canvass the Slack `#graphql` channel for feedback. -### Arguments - -Arguments for a mutation are defined using `argument`. - -Example: - -```ruby -argument :my_arg, GraphQL::Types::String, - required: true, - description: "A description of the argument." -``` - -#### Nullability - -Arguments can be marked as `required: true` which means the value must be present and not `null`. -If a required argument's value can be `null`, use the `required: :nullable` declaration. - -Example: - -```ruby -argument :due_date, - Types::TimeType, - required: :nullable, - description: 'The desired due date for the issue. Due date is removed if null.' -``` - -In the above example, the `due_date` argument must be given, but unlike the GraphQL spec, the value can be `null`. -This allows 'unsetting' the due date in a single mutation rather than creating a new mutation for removing the due date. - -```ruby -{ due_date: null } # => OK -{ due_date: "2025-01-10" } # => OK -{ } # => invalid (not given) -``` - -#### Keywords - -Each GraphQL `argument` defined is passed to the `#resolve` method -of a mutation as keyword arguments. - -Example: - -```ruby -def resolve(my_arg:) - # Perform mutation ... -end -``` - -#### Input Types - -`graphql-ruby` wraps up arguments into an -[input type](https://graphql.org/learn/schema/#input-types). - -For example, the -[`mergeRequestSetDraft` mutation](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/graphql/mutations/merge_requests/set_draft.rb) -defines these arguments (some -[through inheritance](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/graphql/mutations/merge_requests/base.rb)): - -```ruby -argument :project_path, GraphQL::Types::ID, - required: true, - description: "The project the merge request to mutate is in." - -argument :iid, GraphQL::Types::String, - required: true, - description: "The IID of the merge request to mutate." - -argument :draft, - GraphQL::Types::Boolean, - required: false, - description: <<~DESC - Whether or not to set the merge request as a draft. - DESC -``` - -These arguments automatically generate an input type called -`MergeRequestSetDraftInput` with the 3 arguments we specified and the -`clientMutationId`. - ### Object identifier arguments In keeping with the GitLab use of [Global IDs](#global-ids), mutation @@ -1947,36 +1938,110 @@ code so that we have a single source of truth and we do not trigger a subscripti For more information, see [GraphQL pagination](graphql_guide/pagination.md). -## Validating arguments +## Arguments -For validations of single arguments, use the -[`prepare` option](https://github.com/rmosolgo/graphql-ruby/blob/master/guides/fields/arguments.md) -as usual. +[Arguments](https://graphql-ruby.org/fields/arguments.html) for a resolver or mutation are defined using `argument`. -Sometimes a mutation or resolver may accept a number of optional -arguments, but we still want to validate that at least one of the optional -arguments is provided. In this situation, consider using the `#ready?` -method in your mutation or resolver to provide the validation. The -`#ready?` method is called before any work is done in the -`#resolve` method. +Example: + +```ruby +argument :my_arg, GraphQL::Types::String, + required: true, + description: "A description of the argument." +``` + +### Nullability + +Arguments can be marked as `required: true` which means the value must be present and not `null`. +If a required argument's value can be `null`, use the `required: :nullable` declaration. Example: ```ruby -def ready?(**args) - if args.values_at(:body, :position).compact.blank? - raise Gitlab::Graphql::Errors::ArgumentError, - 'body or position arguments are required' - end +argument :due_date, + Types::TimeType, + required: :nullable, + description: 'The desired due date for the issue. Due date is removed if null.' +``` - # Always remember to call `#super` - super +In the above example, the `due_date` argument must be given, but unlike the GraphQL spec, the value can be `null`. +This allows 'unsetting' the due date in a single mutation rather than creating a new mutation for removing the due date. + +```ruby +{ due_date: null } # => OK +{ due_date: "2025-01-10" } # => OK +{ } # => invalid (not given) +``` + +#### Nullability and required: false + +If an argument is marked `required: false` the client is permitted to send `null` as a value. +Often this is undesirable. + +If an argument is optional but `null` is not an allowed value, use validation to ensure that passing `null` returns an error: + +```ruby +argument :name, GraphQL::Types::String, + required: false, + validates: { allow_null: false } +``` + +Alternatively, if you wish to allow `null` when it is not an allowed value, you can replace it with a default value: + +```ruby +argument :name, GraphQL::Types::String, + required: false, + default_value: "No Name Provided", + replace_null_with_default: true +``` + +See [Validation](https://graphql-ruby.org/fields/validation.html), +[Nullability](https://graphql-ruby.org/fields/arguments.html#nullability) and +[Default Values](https://graphql-ruby.org/fields/arguments.html#default-values) for more details. + +### Keywords + +Each GraphQL `argument` defined is passed to the `#resolve` method +of a mutation as keyword arguments. + +Example: + +```ruby +def resolve(my_arg:) + # Perform mutation ... end ``` -In the future this may be able to be done using `OneOf Input Objects` if -[this RFC](https://github.com/graphql/graphql-spec/pull/825) -is merged. +### Input Types + +`graphql-ruby` wraps up arguments into an +[input type](https://graphql.org/learn/schema/#input-types). + +For example, the +[`mergeRequestSetDraft` mutation](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/graphql/mutations/merge_requests/set_draft.rb) +defines these arguments (some +[through inheritance](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/graphql/mutations/merge_requests/base.rb)): + +```ruby +argument :project_path, GraphQL::Types::ID, + required: true, + description: "The project the merge request to mutate is in." + +argument :iid, GraphQL::Types::String, + required: true, + description: "The IID of the merge request to mutate." + +argument :draft, + GraphQL::Types::Boolean, + required: false, + description: <<~DESC + Whether or not to set the merge request as a draft. + DESC +``` + +These arguments automatically generate an input type called +`MergeRequestSetDraftInput` with the 3 arguments we specified and the +`clientMutationId`. ## GitLab custom scalars @@ -2063,7 +2128,7 @@ additional items: Integration tests can also verify the following items, because they invoke the full stack: -- An argument or scalar's [`prepare`](#validating-arguments) applies correctly. +- An argument or scalar's validations apply correctly. - Logic in a resolver or mutation's [`#ready?` method](#correct-use-of-resolverready) applies correctly. - An [argument's `default_value`](https://graphql-ruby.org/fields/arguments.html) applies correctly. - Objects resolve successfully, and there are no N+1 issues. @@ -2214,101 +2279,6 @@ end `spec/requests/api/graphql/ci/pipeline_spec.rb` regardless of the query being used to fetch the pipeline data. -- There can be possible cyclic dependencies in our GraphQL types. - See [Adding field with resolver on a Type causes "Can't determine the return type " error on a different Type](https://github.com/rmosolgo/graphql-ruby/issues/3974#issuecomment-1084444214) - and [Fix unresolved name due to cyclic definition](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/84202/diffs#diff-content-32d14251082fd45412e1fdbf5820e62d157e70d2). - - In particular, this can happen with `connection_type`. Typically we might use the following in a resolver: - - ```ruby - type Types::IssueType.connection_type, null: true - ``` - - However this might cause a cyclic definition, which can result in errors like: - - ```ruby - NameError: uninitialized constant Resolvers::GroupIssuesResolver - - or - - GraphQL::Pagination::Connections::ImplementationMissingError - ``` - - though you might see something different. - - To fix this, we must create a new file that encapsulates the connection type, - and then reference it using double quotes. This gives a delayed resolution, - and the proper connection type. For example: - - [app/graphql/resolvers/base_issues_resolver.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/graphql/resolvers/base_issues_resolver.rb) - originally contained the line - - ```ruby - type Types::IssueType.connection_type, null: true - ``` - - Running the specs locally for this file caused the - `NameError: uninitialized constant Resolvers::GroupIssuesResolver` error. - - The fix was to create a new file, [app/graphql/types/issue_connection.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/graphql/types/issue_connection.rb) with the - line: - - ```ruby - Types::IssueConnection = Types::IssueType.connection_type - ``` - - and in [app/graphql/resolvers/base_issues_resolver.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/graphql/resolvers/base_issues_resolver.rb) - we use the line - - ```ruby - type "Types::IssueConnection", null: true - ``` - - Only use this style if you are having spec failures. We should not typically - use this pattern. This issue should disappear after we've upgraded to `2.x`. - -- There can be instances where a spec fails because the class is not loaded correctly. - It relates to the - [circular dependencies problem](https://github.com/rmosolgo/graphql-ruby/issues/1929) and - [Adding field with resolver on a Type causes "Can't determine the return type " error on a different Type](https://github.com/rmosolgo/graphql-ruby/issues/3974). - - Unfortunately, the errors generated don't really indicate what the problem is. For example, - remove the quotes from the `Rspec.describe` in - [ee/spec/graphql/resolvers/compliance_management/merge_requests/compliance_violation_resolver_spec.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/spec/graphql/resolvers/compliance_management/merge_requests/compliance_violation_resolver_spec.rb). - Then run `rspec ee/spec/graphql/resolvers/compliance_management/merge_requests/compliance_violation_resolver_spec.rb`. - - This generates errors with the expectations. For example: - - ```ruby - 1) Resolvers::ComplianceManagement::MergeRequests::ComplianceViolationResolver#resolve user is authorized filtering the results when given an array of project IDs finds the filtered compliance violations - Failure/Error: expect(subject).to contain_exactly(compliance_violation) - - expected collection contained: [#<MergeRequests::ComplianceViolation id: 4, violating_user_id: 26, merge_request_id: 4, reason: "approved_by_committer", severity_level: "low">] - actual collection contained: [#<MergeRequests::ComplianceViolation id: 4, violating_user_id: 26, merge_request_id: 4, reason: "app...er_id: 27, merge_request_id: 5, reason: "approved_by_merge_request_author", severity_level: "high">] - the extra elements were: [#<MergeRequests::ComplianceViolation id: 5, violating_user_id: 27, merge_request_id: 5, reason: "approved_by_merge_request_author", severity_level: "high">] - # ./ee/spec/graphql/resolvers/compliance_management/merge_requests/compliance_violation_resolver_spec.rb:55:in `block (6 levels) in <top (required)>' - ``` - - However, this is not a case of the wrong result being generated, it's because of the loading order - of the `ComplianceViolationResolver` class. - - The only way we've found to fix this is by quoting the class name in the spec. For example, changing - - ```ruby - RSpec.describe Resolvers::ComplianceManagement::MergeRequests::ComplianceViolationResolver do - ``` - - into: - - ```ruby - RSpec.describe 'Resolvers::ComplianceManagement::MergeRequests::ComplianceViolationResolver' do - ``` - - See [this merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87295#note_946174036) for some discussion. - - Only use this style if you are having spec failures. We should not typically use this pattern. - This issue may disappear after we've upgraded to `2.x`. - - When testing resolvers using `GraphqlHelpers#resolve`, arguments for the resolver can be handled two ways. 1. 95% of the resolver specs use arguments that are Ruby objects, as opposed to when using the GraphQL API diff --git a/doc/development/api_styleguide.md b/doc/development/api_styleguide.md index 45568c700c7..714115d474a 100644 --- a/doc/development/api_styleguide.md +++ b/doc/development/api_styleguide.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # API style guide @@ -10,7 +10,7 @@ This style guide recommends best practices for API development. ## Instance variables -Please do not use instance variables, there is no need for them (we don't need +Don't use instance variables, there is no need for them (we don't need to access them as we do in Rails views), local variables are fine. ## Entities @@ -321,7 +321,7 @@ it's own file in the [`validators`](https://gitlab.com/gitlab-org/gitlab/-/blob/ ## Internal API -The [internal API](internal_api/index.md) is documented for internal use. Please keep it up to date so we know what endpoints +The [internal API](internal_api/index.md) is documented for internal use. Keep it up to date so we know what endpoints different components are making use of. ## Avoiding N+1 problems diff --git a/doc/development/application_limits.md b/doc/development/application_limits.md index 40d157a4411..d84614e0751 100644 --- a/doc/development/application_limits.md +++ b/doc/development/application_limits.md @@ -1,7 +1,7 @@ --- stage: Systems group: Distribution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Application limits development diff --git a/doc/development/application_secrets.md b/doc/development/application_secrets.md index 33bba2b3285..3217f0500f8 100644 --- a/doc/development/application_secrets.md +++ b/doc/development/application_secrets.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Application secrets @@ -46,4 +46,4 @@ GitLab.com environments prior to changing this file. ## Further iteration We may either deprecate or remove this automatic secret generation `01_secret_token.rb` in the future. -Please see [issue 222690](https://gitlab.com/gitlab-org/gitlab/-/issues/222690) for more information. +See [issue 222690](https://gitlab.com/gitlab-org/gitlab/-/issues/222690) for more information. diff --git a/doc/development/application_settings.md b/doc/development/application_settings.md new file mode 100644 index 00000000000..835fd782633 --- /dev/null +++ b/doc/development/application_settings.md @@ -0,0 +1,58 @@ +--- +stage: none +group: unassigned +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. +--- + +# Application settings development + +This document provides a development guide for contributors to add application +settings to GitLab. + +Application settings are stored in the `application_settings` table. Each setting has its own column and there should only be one row. + +## Add a new application setting + +First of all, you have to decide if it is necessary to add an application setting. +Consider our [configuration principles](https://about.gitlab.com/handbook/product/product-principles/#configuration-principles) when adding a new setting. + +To add a new setting, you have to: + +- Add a new column to the `application_settings` table. +- Add the new setting to the [list of visible attributes](https://gitlab.com/gitlab-org/gitlab/-/blob/6f33ad46ffeac454c6c9ce92d6ba328a72f062fd/app/helpers/application_settings_helper.rb#L215). +- Add the new setting to it to [`ApplicationSettingImplementation#defaults`](https://gitlab.com/gitlab-org/gitlab/-/blob/6f33ad46ffeac454c6c9ce92d6ba328a72f062fd/app/models/application_setting_implementation.rb#L36), if the setting has a default value. +- Add a [test for the default value](https://gitlab.com/gitlab-org/gitlab/-/blob/6f33ad46ffeac454c6c9ce92d6ba328a72f062fd/spec/models/application_setting_spec.rb#L20), if the setting has a default value. +- Add a validation for the new field to the [`ApplicationSetting` model](https://gitlab.com/gitlab-org/gitlab/-/blob/6f33ad46ffeac454c6c9ce92d6ba328a72f062fd/app/models/application_setting.rb). +- Add a [model test](https://gitlab.com/gitlab-org/gitlab/-/blob/6f33ad46ffeac454c6c9ce92d6ba328a72f062fd/spec/models/application_setting_spec.rb) for the validation and default value +- Find the [right view file](https://gitlab.com/gitlab-org/gitlab/-/tree/26ad8f4086c03283814bda50ff6e7043902cdbff/app/views/admin/application_settings) or create a new one and add a form field to the new setting. +- Update the [API documentation](https://gitlab.com/gitlab-org/gitlab/-/blob/6f33ad46ffeac454c6c9ce92d6ba328a72f062fd/doc/api/settings.md). Application settings will automatically be available on the REST API. + +### Database migration example + +```ruby +class AddNewSetting < Gitlab::Database::Migration[2.1] + disable_ddl_transaction! + + def up + with_lock_retries do + add_column :application_settings, :new_setting, :text, if_not_exists: true + end + + add_text_limit :application_settings, :new_setting, 255 + end + + def down + with_lock_retries do + remove_column :application_settings, :new_setting, if_exists: true + end + end +end +``` + +### Model validation example + +```ruby +validates :new_setting, + length: { maximum: 255, message: N_('is too long (maximum is %{count} characters)') }, + allow_blank: true +``` diff --git a/doc/development/application_slis/index.md b/doc/development/application_slis/index.md index 30bd6011a67..c5cdc6edc5a 100644 --- a/doc/development/application_slis/index.md +++ b/doc/development/application_slis/index.md @@ -1,7 +1,7 @@ --- stage: Platforms group: Scalability -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # GitLab Application Service Level Indicators (SLIs) @@ -179,7 +179,7 @@ In [this project](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/614) we are extending this so alerts for SLIs with a `feature_category` label in the source metrics can also be routed. -For any question, please don't hesitate to create an issue in +For any question, don't hesitate to create an issue in [the Scalability issue tracker](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues) or come find us in [#g_scalability](https://gitlab.slack.com/archives/CMMF8TKR9) on Slack. diff --git a/doc/development/application_slis/rails_request.md b/doc/development/application_slis/rails_request.md index e5a02c20472..2d18e2a8a15 100644 --- a/doc/development/application_slis/rails_request.md +++ b/doc/development/application_slis/rails_request.md @@ -1,7 +1,7 @@ --- stage: Platforms group: Scalability -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Rails request SLIs (service level indicators) @@ -126,7 +126,7 @@ a case-by-case basis. Take the following into account: view. We cannot scale up the fleet fast enough to accommodate for the incoming slow requests alongside the regular traffic. -When lowering the urgency for an existing endpoint, please involve a +When lowering the urgency for an existing endpoint, involve a [Scalability team member](https://about.gitlab.com/handbook/engineering/infrastructure/team/scalability/#team-members) in the review. We can use request rates and durations available in the logs to come up with a recommendation. You can pick a threshold @@ -172,7 +172,7 @@ information in the logs to check: the target duration we want to set. As decreasing a threshold too much could result in alerts for the -Apdex degradation, please also involve a Scalability team member in +Apdex degradation, also involve a Scalability team member in the merge request. ## How to adjust the urgency @@ -184,10 +184,10 @@ are available: | Urgency | Duration in seconds | Notes | |------------|---------------------|-----------------------------------------------| -| `:high` | 0.25s | | -| `:medium` | 0.5s | | -| `:default` | 1s | The default when nothing is specified. | -| `:low` | 5s | | +| `:high` | [0.25s](https://gitlab.com/gitlab-org/gitlab/-/blob/2f7a38fe48934b78f04233c4d2c81cde88a06da7/lib/gitlab/endpoint_attributes/config.rb#L8) | | +| `:medium` | [0.5s](https://gitlab.com/gitlab-org/gitlab/-/blob/2f7a38fe48934b78f04233c4d2c81cde88a06da7/lib/gitlab/endpoint_attributes/config.rb#L9) | | +| `:default` | [1s](https://gitlab.com/gitlab-org/gitlab/-/blob/2f7a38fe48934b78f04233c4d2c81cde88a06da7/lib/gitlab/endpoint_attributes/config.rb#L10) | The default when nothing is specified. | +| `:low` | [5s](https://gitlab.com/gitlab-org/gitlab/-/blob/2f7a38fe48934b78f04233c4d2c81cde88a06da7/lib/gitlab/endpoint_attributes/config.rb#L11) | | ### Rails controller diff --git a/doc/development/application_slis/sidekiq_execution.md b/doc/development/application_slis/sidekiq_execution.md index dfe7f3864d5..cc787978066 100644 --- a/doc/development/application_slis/sidekiq_execution.md +++ b/doc/development/application_slis/sidekiq_execution.md @@ -1,7 +1,7 @@ --- stage: Platforms group: Scalability -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Sidekiq execution SLIs (service level indicators) diff --git a/doc/development/architecture.md b/doc/development/architecture.md index 094853f5c42..2f6f93fc15b 100644 --- a/doc/development/architecture.md +++ b/doc/development/architecture.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # GitLab architecture overview @@ -12,6 +12,7 @@ There are two software distributions of GitLab: - The open source [Community Edition](https://gitlab.com/gitlab-org/gitlab-foss/) (CE). - The open core [Enterprise Edition](https://gitlab.com/gitlab-org/gitlab/) (EE). + **Note:** The EE repository has been archived. GitLab now operates [under a single codebase](https://about.gitlab.com/blog/2019/08/23/a-single-codebase-for-gitlab-community-and-enterprise-edition/). GitLab is available under [different subscriptions](https://about.gitlab.com/pricing/). @@ -763,7 +764,7 @@ See our [Redis guidelines](redis.md) for more information about how GitLab uses - [Source](../administration/packages/container_registry.md#enable-the-container-registry) - [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/registry.md) - Layer: Core Service (Processor) -- GitLab.com: [GitLab Container Registry](../user/packages/container_registry/build_and_push_images.md#use-gitlab-cicd) +- GitLab.com: [GitLab container registry](../user/packages/container_registry/build_and_push_images.md#use-gitlab-cicd) The registry is what users use to store their own Docker images. The bundled registry uses NGINX as a load balancer and GitLab as an authentication manager. diff --git a/doc/development/audit_event_guide/index.md b/doc/development/audit_event_guide/index.md index aeab188fa76..28af4c3e0bb 100644 --- a/doc/development/audit_event_guide/index.md +++ b/doc/development/audit_event_guide/index.md @@ -1,7 +1,7 @@ --- stage: Govern group: Compliance -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Audit event development guidelines @@ -23,7 +23,7 @@ While any events could trigger an Audit Event, not all events should. In general - Are tracking information for product feature adoption. - Are covered in the direction page's discussion on [what is not planned](https://about.gitlab.com/direction/govern/compliance/audit-events/#what-is-not-planned-right-now). -If you have any questions, please reach out to `@gitlab-org/govern/compliance` to see if an Audit Event, or some other approach, may be best for your event. +If you have any questions, reach out to `@gitlab-org/govern/compliance` to see if an Audit Event, or some other approach, may be best for your event. ## Audit Event Schemas @@ -199,7 +199,7 @@ In addition to recording to the database, we also write these events to > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/367847) in GitLab 15.4. -All new audit events must have a type definition stored in `config/audit_events/types/` that contains a single source of truth for every auditable event in GitLab. +All new audit events must have a type definition stored in `config/audit_events/types/` or `ee/config/audit_events/types/` that contains a single source of truth for every auditable event in GitLab. ### Add a new audit event type @@ -244,7 +244,7 @@ to check if the documentation is up-to-date. ## Event streaming All events where the entity is a `Group` or `Project` are recorded in the audit log, and also streamed to one or more -[event streaming destinations](../../administration/audit_event_streaming.md). When the entity is a: +[event streaming destinations](../../administration/audit_event_streaming/index.md). When the entity is a: - `Group`, events are streamed to the group's root ancestor's event streaming destinations. - `Project`, events are streamed to the project's root ancestor's event streaming destinations. diff --git a/doc/development/auto_devops.md b/doc/development/auto_devops.md index ccbad7f7314..d5a011e7e55 100644 --- a/doc/development/auto_devops.md +++ b/doc/development/auto_devops.md @@ -1,7 +1,7 @@ --- stage: Deploy group: Environments -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Auto DevOps development guidelines diff --git a/doc/development/avoiding_required_stops.md b/doc/development/avoiding_required_stops.md index d523a16a882..74db216a1f0 100644 --- a/doc/development/avoiding_required_stops.md +++ b/doc/development/avoiding_required_stops.md @@ -1,7 +1,7 @@ --- stage: Systems group: Distribution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Avoiding required stops @@ -32,7 +32,7 @@ Wherever possible, a required stop should be avoided. If it can't be avoided, the required stop should be aligned to a _scheduled_ required stop. In cases where we are considering retroactively declaring an unplanned required stop, -please contact the [Distribution team product manager](https://about.gitlab.com/handbook/product/categories/#distributionbuild-group) to advise on next steps. If there +contact the [Distribution team product manager](https://about.gitlab.com/handbook/product/categories/#distributionbuild-group) to advise on next steps. If there is uncertainty about whether we should declare a required stop, the Distribution product manager may escalate to GitLab product leadership (VP or Chief Product Officer) to make a final determination. This may happen, for example, if a change might require a stop for diff --git a/doc/development/backend/create_source_code_be/gitaly_touch_points.md b/doc/development/backend/create_source_code_be/gitaly_touch_points.md index 98607c7f6c7..73e5708ca16 100644 --- a/doc/development/backend/create_source_code_be/gitaly_touch_points.md +++ b/doc/development/backend/create_source_code_be/gitaly_touch_points.md @@ -1,7 +1,7 @@ --- stage: Create group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Source Code - Gitaly Touch Points diff --git a/doc/development/backend/create_source_code_be/index.md b/doc/development/backend/create_source_code_be/index.md index d894a14b006..3ae3c989966 100644 --- a/doc/development/backend/create_source_code_be/index.md +++ b/doc/development/backend/create_source_code_be/index.md @@ -1,25 +1,79 @@ --- stage: Create group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- -# Create: Source Code backend +# Source Code Management -The Create: Source Code backend (BE) team focuses on the GitLab suite of Source Code Management -(SCM) tools. It is responsible for all backend aspects of the product categories +The Source Code Management team is responsible for all backend aspects of the product categories that fall under the [Source Code group](https://about.gitlab.com/handbook/product/categories/#source-code-group) of the [Create stage](https://about.gitlab.com/handbook/product/categories/#create-stage) of the [DevOps lifecycle](https://about.gitlab.com/handbook/product/categories/#devops-stages). -We interface with the Gitaly and Code Review teams, and work closely with the -[Create: Source Code frontend team](https://about.gitlab.com/handbook/engineering/development/dev/create/create-source-code-fe/). The features -we work with are listed on the +The Source Code Management team interfaces with the Gitaly and Code Review teams and works across three codebases: Workhorse, GitLab Shell and GitLab Rails. + +## Source Code Features Reference + +Features owned by the Source Code Management group are listed on the [Features by Group Page](https://about.gitlab.com/handbook/product/categories/features/#createsource-code-group). -The team works across three codebases: Workhorse, GitLab Shell and GitLab Rails. +### Code Owners + +Source Code Management shares ownership of Code Owners with the Code Review group. + +- [Feature homepage](../../../user/project/codeowners/index.md) +- [Developer Reference](../../code_owners/index.md) + +### Approval Rules + +- [Approval Rules](../../merge_request_concepts/approval_rules.md) + +### Push Rules + +- [Push Rules development guidelines](../../push_rules/index.md) + +### Protected Branches + +Details about Protected Branches models can be found in the [Code Owners](../../code_owners/index.md#related-models) technical reference page. + +### Repositories + +- [Project Repository Storage Moves](../../repository_storage_moves/index.md) + +### Project Templates + +- [Custom group-level project templates development guidelines](../../project_templates/index.md) + +### Git LFS + +- [Git LFS Development guidelines](../../lfs.md) + +## Technical Stack -## Workhorse +## GitLab Rails + +### Gitaly touch points + +[Gitaly](../../../administration/gitaly/index.md) provides high-level RPC access to Git repositories. +It is present in every GitLab installation and coordinates Git repository storage and retrieval. +Gitaly implements a client-server architecture with Gitaly as the server and Gitaly clients, also +known as _Gitaly consumers_, including: + +- GitLab Rails +- GitLab Shell +- GitLab Workhorse + +Gitaly Rails provides API endpoints that are counterparts of Gitaly RPCs. For more information, read [Gitaly touch points](gitaly_touch_points.md). + +### Annotated Rails Source Code + +The `:source_code_management` annotation indicates which code belongs to the Source Code Management +group in the Rails codebase. The annotated objects are presented on +[this page](https://gitlab-com.gitlab.io/gl-infra/platform/stage-groups-index/source-code.html) along +with the [Error Budgets dashboards](https://dashboards.gitlab.net/d/stage-groups-source_code/stage-groups3a-source-code3a-group-dashboard?orgId=1). + +## GitLab Workhorse [GitLab Workhorse](../../workhorse/index.md) is a smart reverse proxy for GitLab. It handles "large" HTTP requests such as file downloads, file uploads, `git push`, `git pull` and `git` archive downloads. @@ -34,18 +88,3 @@ For more information, refer to the [GitLab Shell documentation](../../gitlab_she To learn about the reasoning behind our creation of `gitlab-sshd`, read the blog post [Why we implemented our own SSHD solution](https://about.gitlab.com/blog/2022/08/17/why-we-have-implemented-our-own-sshd-solution-on-gitlab-sass/). - -## GitLab Rails - -### Gitaly touch points - -Gitaly is a Go RPC service which handles all the `git` calls made by GitLab. -GitLab is not exposed directly, and all traffic comes through Create: Source Code. -For more information, read [Gitaly touch points](gitaly_touch_points.md). - -### Source Code REST API Endpoints - -Create: Source Code has over 100 REST endpoints, being a mixture of Grape API endpoints and Rails controller endpoints. -For a detailed list, refer to [Source Code REST Endpoints](rest_endpoints.md). - -An alternative list of the [Source Code endpoints and other owned objects](https://gitlab-com.gitlab.io/gl-infra/platform/stage-groups-index/source-code.html) is available. diff --git a/doc/development/backend/create_source_code_be/rest_endpoints.md b/doc/development/backend/create_source_code_be/rest_endpoints.md index 2130d76d014..ccbb11288d5 100644 --- a/doc/development/backend/create_source_code_be/rest_endpoints.md +++ b/doc/development/backend/create_source_code_be/rest_endpoints.md @@ -1,112 +1,11 @@ --- -stage: Create -group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: 'index.md' +remove_date: '2024-03-13' --- -# Source Code REST endpoints +This document was moved to [another location](index.md). -The Create :: Source Code team maintains these endpoints: - -| Endpoint | Threshold | Source | -| -----------------------------------------------------------------------------------|---------------------------------------|--------------------------------------------------------------------------------------| -| `DELETE /api/:version/projects/:id/protected_branches/:name` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/protected_branches.rb) | -| `GET /api/:version/internal/authorized_keys` | `:high` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/internal/base.rb) | | | -| `GET /api/:version/internal/lfs` | `:high` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/internal/lfs.rb)| -| `GET /api/:version/projects/:id/approval_rules` | `:low` | | -| `GET /api/:version/projects/:id/approval_settings` | default | | -| `GET /api/:version/projects/:id/approvals` | default | | -| `GET /api/:version/projects/:id/forks` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/projects.rb) | -| `GET /api/:version/projects/:id/groups` | default | [source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/projects.rb) | -| `GET /api/:version/projects/:id/languages` | `:medium` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/projects.rb) | -| `GET /api/:version/projects/:id/merge_request_approval_setting` | `:medium` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/api/merge_request_approval_settings.rb) | -| `GET /api/:version/projects/:id/merge_requests/:merge_request_iid/approval_rules` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/api/merge_request_approval_rules.rb) | -| `GET /api/:version/projects/:id/merge_requests/:merge_request_iid/approval_settings` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/api/project_approval_settings.rb) | -| `GET /api/:version/projects/:id/merge_requests/:merge_request_iid/approval_state` | `:low` | [source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/merge_request_approvals.rb) | -| `GET /api/:version/projects/:id/merge_requests/:merge_request_iid/approvals` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/merge_request_approvals.rb) | -| `GET /api/:version/projects/:id/protected_branches` | default |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/protected_branches.rb) | -| `GET /api/:version/projects/:id/protected_branches/:name` | default |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/protected_branches.rb) | -| `GET /api/:version/projects/:id/protected_tags` | default | | -| `GET /api/:version/projects/:id/protected_tags/:name` | default | | -| `GET /api/:version/projects/:id/push_rule` | default | | -| `GET /api/:version/projects/:id/remote_mirrors` | default | | -| `GET /api/:version/projects/:id/repository/archive` | default | | -| `GET /api/:version/projects/:id/repository/blobs/:sha` | default | | -| `GET /api/:version/projects/:id/repository/blobs/:sha/raw` | default | | -| `GET /api/:version/projects/:id/repository/branches` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/branches.rb) | -| `GET /api/:version/projects/:id/repository/branches/:branch` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/branches.rb) | -| `GET /api/:version/projects/:id/repository/commits` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/commits.rb)| -| `GET /api/:version/projects/:id/repository/commits/:sha` | default | [source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/commits.rb) | -| `GET /api/:version/projects/:id/repository/commits/:sha/comments` | default | [source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/commits.rb) | -| `GET /api/:version/projects/:id/repository/commits/:sha/diff` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/commits.rb) | -| `GET /api/:version/projects/:id/repository/commits/:sha/merge_requests` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/commits.rb)| -| `GET /api/:version/projects/:id/repository/commits/:sha/refs` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/commits.rb) | -| `GET /api/:version/projects/:id/repository/compare` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/repositories.rb) | -| `GET /api/:version/projects/:id/repository/contributors` | default | | -| `GET /api/:version/projects/:id/repository/files/:file_path` | default | | -| `GET /api/:version/projects/:id/repository/files/:file_path/raw` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/files.rb) | -| `GET /api/:version/projects/:id/repository/tags` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/tags.rb) | -| `GET /api/:version/projects/:id/repository/tree` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/repositories.rb) | -| `GET /api/:version/projects/:id/statistics` | default | | -| `GraphqlController#execute` | default | | -| `HEAD /api/:version/projects/:id/repository/files/:file_path` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/files.rb) | -| `HEAD /api/:version/projects/:id/repository/files/:file_path/raw` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/files.rb) | -| `POST /api/:version/internal/allowed` | default | [source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/internal/base.rb) | -| `POST /api/:version/internal/lfs_authenticate` | `:high` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/internal/base.rb) | -| `POST /api/:version/internal/post_receive` | default | [source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/internal/base.rb) | -| `POST /api/:version/internal/pre_receive` | `:high` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/internal/base.rb) | -| `POST /api/:version/projects/:id/approvals` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/api/project_approvals.rb) | -| `POST /api/:version/projects/:id/merge_requests/:merge_request_iid/approvals` | `:low` | [source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/merge_request_approvals.rb) | -| `POST /api/:version/projects/:id/merge_requests/:merge_request_iid/approve` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/merge_request_approvals.rb) | -| `POST /api/:version/projects/:id/merge_requests/:merge_request_iid/unapprove` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/merge_request_approvals.rb)| -| `POST /api/:version/projects/:id/protected_branches` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/protected_branches.rb)| -| `POST /api/:version/projects/:id/repository/commits` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/commits.rb)| -| `POST /api/:version/projects/:id/repository/files/:file_path` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/files.rb) | -| `PUT /api/:version/projects/:id/push_rule` | default | | -| `PUT /api/:version/projects/:id/repository/files/:file_path` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/files.rb) | -| `Projects::BlameController#show` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/blame_controller.rb) | -| `Projects::BlobController#create` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/blob_controller.rb) | -| `Projects::BlobController#diff` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/blob_controller.rb) | -| `Projects::BlobController#edit` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/blob_controller.rb) | -| `Projects::BlobController#show` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/blob_controller.rb) | -| `Projects::BlobController#update` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/blob_controller.rb) | -| `Projects::BranchesController#create` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/branches_controller.rb) | -| `Projects::BranchesController#destroy` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/branches_controller.rb) | -| `Projects::BranchesController#diverging_commit_counts` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/branches_controller.rb) | -| `Projects::BranchesController#index` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/branches_controller.rb) | -| `Projects::BranchesController#new` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/branches_controller.rb) | -| `Projects::CommitController#branches` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/commit_controller.rb) | -| `Projects::CommitController#merge_requests` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/commit_controller.rb) | -| `Projects::CommitController#pipelines` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/commit_controller.rb) | -| `Projects::CommitController#show` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/commit_controller.rb) | -| `Projects::CommitsController#show` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/commits_controller.rb)| -| `Projects::CommitsController#signatures` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/commits_controller.rb) | -| `Projects::CompareController#create` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/commits_controller.rb) | -| `Projects::CompareController#index` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/compare_controller.rb) | -| `Projects::CompareController#show` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/compare_controller.rb) | -| `Projects::CompareController#signatures` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/compare_controller.rb) | -| `Projects::FindFileController#list` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/find_file_controller.rb) | -| `Projects::FindFileController#show` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/find_file_controller.rb) | -| `Projects::ForksController#index` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/forks_controller.rb) | -| `Projects::GraphsController#show` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/graphs_controller.rb) | -| `Projects::NetworkController#show` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/network_controller.rb) | -| `Projects::PathLocksController#index` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/controllers/projects/path_locks_controller.rb) | -| `Projects::RawController#show` | default | | -| `Projects::RefsController#logs_tree` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/refs_controller.rb) | -| `Projects::RefsController#switch` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/refs_controller.rb) | -| `Projects::RepositoriesController#archive` | default | | -| `Projects::Settings::RepositoryController#show` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/settings/repository_controller.rb) | -| `Projects::TagsController#index` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/tags_controller.rb) | -| `Projects::TagsController#new` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/tags_controller.rb) | -| `Projects::TagsController#show` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/tags_controller.rb) | -| `Projects::TemplatesController#names` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/templates_controller.rb) | -| `Projects::TreeController#show` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/tree_controller.rb) | -| `ProjectsController#refs` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects_controller.rb) | -| `Repositories::GitHttpController#git_receive_pack` | default | | -| `Repositories::GitHttpController#git_upload_pack` | default | | -| `Repositories::GitHttpController#info_refs` | default | | -| `Repositories::LfsApiController#batch` | `:medium` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/repositories/lfs_api_controller.rb) | -| `Repositories::LfsLocksApiController#verify` | default | | -| `Repositories::LfsStorageController#download` | `:medium` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/repositories/lfs_storage_controller.rb) | -| `Repositories::LfsStorageController#upload_authorize` | `:medium` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/repositories/lfs_storage_controller.rb) | -| `Repositories::LfsStorageController#upload_finalize` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/repositories/lfs_storage_controller.rb) | +<!-- This redirect file can be deleted after <2024-03-13>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/development/backend/ruby_style_guide.md b/doc/development/backend/ruby_style_guide.md index 384d8122ccf..73c4d8719a0 100644 --- a/doc/development/backend/ruby_style_guide.md +++ b/doc/development/backend/ruby_style_guide.md @@ -1,8 +1,7 @@ --- -type: reference, dev stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Ruby style guide diff --git a/doc/development/bitbucket_cloud_importer.md b/doc/development/bitbucket_cloud_importer.md index 8a59743a243..98620ca39eb 100644 --- a/doc/development/bitbucket_cloud_importer.md +++ b/doc/development/bitbucket_cloud_importer.md @@ -1,16 +1,11 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Bitbucket Cloud importer developer documentation -The Bitbucket Cloud importer can be configured with the `bitbucket_parallel_importer` feature flag. When the feature flag is: - -- Enabled, the importer uses Sidekiq to schedule work asynchronously. -- Disabled, the importer does all the work in a single thread. - ## Prerequisites You must be authenticated with Bitbucket: diff --git a/doc/development/bitbucket_server_importer.md b/doc/development/bitbucket_server_importer.md new file mode 100644 index 00000000000..3a1f5a4febd --- /dev/null +++ b/doc/development/bitbucket_server_importer.md @@ -0,0 +1,84 @@ +--- +stage: none +group: unassigned +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. +--- + +# Bitbucket Server importer developer documentation + +## Prerequisites + +To test imports, you need a Bitbucket Server instance running locally. For information on running a local instance, see +[these instructions](https://gitlab.com/gitlab-org/manage/import-and-integrate/team/-/blob/main/integrations/bitbucket_server.md). + +## Code structure + +The importer's codebase is broken up into the following directories: + +- `lib/gitlab/bitbucket_server_import`: this directory contains most of the code such as + the classes used for importing resources. +- `app/workers/gitlab/bitbucket_server_import`: this directory contains the Sidekiq + workers. + +## How imports advance + +When a Bitbucket Server project is imported, work is divided into separate stages, with +each stage consisting of a set of Sidekiq jobs that are executed. + +Between every stage, a job called `Gitlab::BitbucketServerImport::AdvanceStageWorker` +is scheduled that periodically checks if all work of the current stage is completed. If +all the work is complete, the job advances the import process to the next stage. + +## Stages + +### 1. Stage::ImportRepositoryWorker + +This worker imports the repository and schedules the next stage when +done. + +### 2. Stage::ImportPullRequestsWorker + +This worker imports all pull requests. For every pull request, a job for the +`Gitlab::BitbucketImport::ImportPullRequestWorker` worker is scheduled. + +Bitbucket Server keeps tracks of references for open pull requests in +`refs/heads/pull-requests`, but closed and merged requests get moved +into hidden internal refs under `stash-refs/pull-requests`. + +As a result, they are not fetched by default. To prevent merge requests from not having +commits and therefore having empty diffs, we fetch affected source and target +commits from the server before importing the pull request. +We save the fetched commits as refs so that Git doesn't remove them, which can happen +if they are unused. +Source commits are saved as `#{commit}:refs/merge-requests/#{pull_request.iid}/head` +and target commits are saved as `#{commit}:refs/keep-around/#{commit}`. + +When creating a pull request, we need to match Bitbucket users with GitLab users for +the author and reviewers. Whenever a matching user is found, the GitLab user ID is cached +for 24 hours so that it doesn't have to be searched for again. + +### 3. Stage::ImportNotesWorker + +This worker imports notes (comments) for all merge requests. + +For every merge request, a job for the `Gitlab::BitbucketServerImport::ImportPullRequestNotesWorker` +worker is scheduled which imports all standalone comments, inline comments, merge events, and +approved events for the merge request. + +### 4. Stage::ImportLfsObjectsWorker + +Imports LFS objects from the source project by scheduling a +`Gitlab::BitbucketServerImport::ImportLfsObjectsWorker` job for every LFS object. + +### 5. Stage::FinishImportWorker + +This worker completes the import process by performing some housekeeping +such as marking the import as completed. + +## Pull request mentions + +Pull request descriptions and notes can contain @mentions to users. If a user with the +same email does not exist on GitLab, this can lead to incorrect users being tagged. + +To get around this, we build a cache containing all users who have access to the Bitbucket +project and then convert mentions in pull request descriptions and notes. diff --git a/doc/development/build_test_package.md b/doc/development/build_test_package.md index bc8cebed3fc..336baab51c6 100644 --- a/doc/development/build_test_package.md +++ b/doc/development/build_test_package.md @@ -1,7 +1,7 @@ --- stage: Systems group: Distribution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Building a package for testing diff --git a/doc/development/bulk_import.md b/doc/development/bulk_import.md index 502bee97c9c..6c0bed8e204 100644 --- a/doc/development/bulk_import.md +++ b/doc/development/bulk_import.md @@ -1,7 +1,7 @@ --- stage: Manage group: Import and Integrate -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Group migration by direct transfer diff --git a/doc/development/cached_queries.md b/doc/development/cached_queries.md index 4696f05873c..f2f4593bfc1 100644 --- a/doc/development/cached_queries.md +++ b/doc/development/cached_queries.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Cloud Connector -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Cached queries guidelines diff --git a/doc/development/caching.md b/doc/development/caching.md index c3385c8499d..9a02c795bbe 100644 --- a/doc/development/caching.md +++ b/doc/development/caching.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Caching guidelines diff --git a/doc/development/cascading_settings.md b/doc/development/cascading_settings.md index 74695f03451..11387d6cfb6 100644 --- a/doc/development/cascading_settings.md +++ b/doc/development/cascading_settings.md @@ -1,7 +1,7 @@ --- stage: Govern group: Authentication -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Cascading Settings diff --git a/doc/development/cells/index.md b/doc/development/cells/index.md index 1ab88e0d8c6..cd222e7b209 100644 --- a/doc/development/cells/index.md +++ b/doc/development/cells/index.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Tenant Scale -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # GitLab Cells Development Guidelines diff --git a/doc/development/changelog.md b/doc/development/changelog.md index 486808f25e7..f2ba1aa13a9 100644 --- a/doc/development/changelog.md +++ b/doc/development/changelog.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Changelog entries diff --git a/doc/development/chaos_endpoints.md b/doc/development/chaos_endpoints.md index 196ec14bffa..0a11e76e8d9 100644 --- a/doc/development/chaos_endpoints.md +++ b/doc/development/chaos_endpoints.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Generating chaos in a test GitLab instance diff --git a/doc/development/chatops_on_gitlabcom.md b/doc/development/chatops_on_gitlabcom.md index 002470f7325..e283c1a0987 100644 --- a/doc/development/chatops_on_gitlabcom.md +++ b/doc/development/chatops_on_gitlabcom.md @@ -1,7 +1,7 @@ --- stage: Deploy group: Environments -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # ChatOps on GitLab.com diff --git a/doc/development/cicd/cicd_reference_documentation_guide.md b/doc/development/cicd/cicd_reference_documentation_guide.md index e358b24c60f..ccd952f586c 100644 --- a/doc/development/cicd/cicd_reference_documentation_guide.md +++ b/doc/development/cicd/cicd_reference_documentation_guide.md @@ -1,7 +1,7 @@ --- stage: Verify group: Pipeline Authoring -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Documenting the `.gitlab-ci.yml` keywords diff --git a/doc/development/cicd/cicd_tables.md b/doc/development/cicd/cicd_tables.md index 8cfb0faca00..465597ca455 100644 --- a/doc/development/cicd/cicd_tables.md +++ b/doc/development/cicd/cicd_tables.md @@ -1,7 +1,7 @@ --- stage: Verify group: Pipeline Execution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Add new tables to the CI database diff --git a/doc/development/cicd/index.md b/doc/development/cicd/index.md index 41ae4fe14b4..18781f9315a 100644 --- a/doc/development/cicd/index.md +++ b/doc/development/cicd/index.md @@ -1,15 +1,14 @@ --- stage: Verify group: Pipeline Execution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: index, concepts, howto +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # CI/CD development guidelines Development guides that are specific to CI/CD are listed here: -- If you are creating new CI/CD templates, please read [the development guide for GitLab CI/CD templates](templates.md). +- If you are creating new CI/CD templates, read [the development guide for GitLab CI/CD templates](templates.md). - If you are adding a new keyword or changing the CI schema, check the [CI schema guide](schema.md) See the [CI/CD YAML reference documentation guide](cicd_reference_documentation_guide.md) diff --git a/doc/development/cicd/pipeline_wizard.md b/doc/development/cicd/pipeline_wizard.md index f322ec2e2bf..0c0c0f3cc45 100644 --- a/doc/development/cicd/pipeline_wizard.md +++ b/doc/development/cicd/pipeline_wizard.md @@ -1,7 +1,7 @@ --- stage: none group: Incubation Engineering -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Pipeline Wizard @@ -133,8 +133,7 @@ is planned to add the ability to create a MR from here. should be committed to - `default-branch` (required): The branch that will be pre-selected during the commit step. This can be changed by the user. -- `default-filename` (optional, default: `.gitlab-ci.yml`): The Filename - to be used for the file. This can be overridden in the template file. +- `default-filename` (optional, default: `.gitlab-ci.yml`): The file name to be used for the file. This can be overridden in the template file. ### Events diff --git a/doc/development/cicd/schema.md b/doc/development/cicd/schema.md index 26e63fb53d8..e9a0b93b5f3 100644 --- a/doc/development/cicd/schema.md +++ b/doc/development/cicd/schema.md @@ -1,8 +1,7 @@ --- stage: Verify group: Pipeline Authoring -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: index, howto +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Contribute to the CI/CD Schema diff --git a/doc/development/cicd/templates.md b/doc/development/cicd/templates.md index f977a70ac05..a2b490b9106 100644 --- a/doc/development/cicd/templates.md +++ b/doc/development/cicd/templates.md @@ -1,12 +1,22 @@ --- stage: Verify group: Pipeline Authoring -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: index, concepts, howto +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Development guide for GitLab CI/CD templates +NOTE: +With the introduction of the [CI/CD Catalog](../../ci/components/index.md#cicd-catalog), +GitLab is no longer accepting contributions of new CI/CD templates to the codebase. Instead, +we encourage team members to create [CI/CD components](../../ci/components/index.md) +for the catalog. This transition enhances the modularity and maintainability of our +shared CI/CD resources, and avoids the complexities of contributing new CI/CD templates. +If you need to update an existing template, you must also update the matching CI/CD component. +If no component exists that matches the CI/CD template yet, consider creating the matching component. +This ensures that template and component functionality remain in sync, aligning with +our new development practices. + This document explains how to develop [GitLab CI/CD templates](../../ci/examples/index.md#cicd-templates). ## Requirements for CI/CD templates @@ -285,7 +295,7 @@ the user's `.gitlab-ci.yml` immediately causes a lint error because there are no such jobs named `performance` in the included template anymore. Therefore, users have to fix their `.gitlab-ci.yml` that could annoy their workflow. -Please read [versioning](#versioning) section for introducing breaking change safely. +Read [versioning](#versioning) section for introducing breaking change safely. ## Versioning @@ -378,7 +388,7 @@ Each CI/CD template must be tested to make sure that it's safe to be published. ### Manual QA It's always good practice to test the template in a minimal demo project. -To do so, please follow the following steps: +To do so, follow the following steps: 1. Create a public sample project on <https://gitlab.com>. 1. Add a `.gitlab-ci.yml` to the project with the proposed template. @@ -482,6 +492,6 @@ If you're unsure if it's secure or not, you must ask security experts for cross- After your CI/CD template MR is created and labeled with `ci::templates`, DangerBot suggests one reviewer and one maintainer that can review your code. When your merge -request is ready for review, please [mention](../../user/discussions/index.md#mentions) +request is ready for review, [mention](../../user/discussions/index.md#mentions) the reviewer and ask them to review your CI/CD template changes. See details in the merge request that added [a DangerBot task for CI/CD template MRs](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/44688). diff --git a/doc/development/cloud_connector/code_suggestions_for_sm.md b/doc/development/cloud_connector/code_suggestions_for_sm.md index bd8a39bc0d6..6393eca815c 100644 --- a/doc/development/cloud_connector/code_suggestions_for_sm.md +++ b/doc/development/cloud_connector/code_suggestions_for_sm.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Cloud Connector -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Cloud Connector MVC: Code Suggestions for Self-Managed/GitLab Dedicated diff --git a/doc/development/code_comments.md b/doc/development/code_comments.md index 306f371f306..51643f3e7ed 100644 --- a/doc/development/code_comments.md +++ b/doc/development/code_comments.md @@ -1,13 +1,13 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Code comments Whenever you add comment to the code that is expected to be addressed at any time -in future, please create a technical debt issue for it. Then put a link to it +in future, create a technical debt issue for it. Then put a link to it to the code comment you've created. This allows other developers to quickly check if a comment is still relevant and what needs to be done to address it. diff --git a/doc/development/code_intelligence/index.md b/doc/development/code_intelligence/index.md index 2034f57d995..945916d8fcd 100644 --- a/doc/development/code_intelligence/index.md +++ b/doc/development/code_intelligence/index.md @@ -1,7 +1,7 @@ --- stage: Create group: Code Review -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Code intelligence development guidelines diff --git a/doc/development/code_owners/index.md b/doc/development/code_owners/index.md index d962a36b497..45c632d5adc 100644 --- a/doc/development/code_owners/index.md +++ b/doc/development/code_owners/index.md @@ -1,7 +1,7 @@ --- stage: Create group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Code Owners development guidelines diff --git a/doc/development/code_review.md b/doc/development/code_review.md index c2f2a7643ae..84d2537d058 100644 --- a/doc/development/code_review.md +++ b/doc/development/code_review.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Code Review Guidelines @@ -179,7 +179,7 @@ by a reviewer before passing it to a maintainer as described in the | `~UX` user-facing changes <sup>3</sup> | [Product Designer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_reviewers_UX). Refer to the [design and user interface guidelines](contributing/design.md) for details. | | Adding a new JavaScript library <sup>1</sup> | - [Frontend foundations member](https://about.gitlab.com/direction/manage/foundations/) if the library significantly increases the [bundle size](https://gitlab.com/gitlab-org/frontend/playground/webpack-memory-metrics/-/blob/master/doc/report.md).<br/>- A [legal department member](https://about.gitlab.com/handbook/legal/) if the license used by the new library hasn't been approved for use in GitLab.<br/><br/>More information about license compatibility can be found in our [GitLab Licensing and Compatibility documentation](licensing.md). | | A new dependency or a file system change | - [Distribution team member](https://about.gitlab.com/company/team/). See how to work with the [Distribution team](https://about.gitlab.com/handbook/engineering/development/enablement/systems/distribution/#how-to-work-with-distribution) for more details.<br/>- For RubyGems, request an [AppSec review](gemfile.md#request-an-appsec-review). | -| `~documentation` or `~UI text` changes | [Technical writer](https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments) based on assignments in the appropriate [DevOps stage group](https://about.gitlab.com/handbook/product/categories/#devops-stages). | +| `~documentation` or `~UI text` changes | [Technical writer](https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments) based on assignments in the appropriate [DevOps stage group](https://about.gitlab.com/handbook/product/categories/#devops-stages). | | Changes to development guidelines | Follow the [review process](development_processes.md#development-guidelines-review) and get the approvals accordingly. | | End-to-end **and** non-end-to-end changes <sup>4</sup> | [Software Engineer in Test](https://about.gitlab.com/handbook/engineering/quality/#individual-contributors). | | Only End-to-end changes <sup>4</sup> **or** if the MR author is a [Software Engineer in Test](https://about.gitlab.com/handbook/engineering/quality/#individual-contributors) | [Quality maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_qa). | @@ -187,7 +187,8 @@ by a reviewer before passing it to a maintainer as described in the | Analytics Instrumentation (telemetry or analytics) changes | [Analytics Instrumentation engineer](https://gitlab.com/gitlab-org/analytics-section/analytics-instrumentation/engineers). | | An addition of, or changes to a [Feature spec](testing_guide/testing_levels.md#frontend-feature-tests) | [Quality maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_qa) or [Quality reviewer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_reviewers_qa). | | A new service to GitLab (Puma, Sidekiq, Gitaly are examples) | [Product manager](https://about.gitlab.com/company/team/). See the [process for adding a service component to GitLab](adding_service_component.md) for details. | -| Changes related to authentication or authorization | [Manage:Authentication and Authorization team member](https://about.gitlab.com/company/team/). Check the [code review section on the group page](https://about.gitlab.com/handbook/engineering/development/dev/manage/authentication-and-authorization/#additional-considerations) for more details. Patterns for files known to require review from the team are listed in the in the `Authentication and Authorization` section of the [`CODEOWNERS`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/CODEOWNERS) file, and the team will be listed in the approvers section of all merge requests that modify these files. | +| Changes related to authentication | [Manage:Authentication](https://about.gitlab.com/company/team/). Check the [code review section on the group page](https://about.gitlab.com/handbook/engineering/development/dev/manage/authentication/#additional-considerations) for more details. Patterns for files known to require review from the team are listed in the in the `Authentication` section of the [`CODEOWNERS`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/CODEOWNERS) file, and the team will be listed in the approvers section of all merge requests that modify these files. | +| Changes related to custom roles or policies | [Manage:Authorization Engineer](https://gitlab.com/gitlab-org/govern/authorization/approvers). | 1. Specs other than JavaScript specs are considered `~backend` code. Haml markup is considered `~frontend` code. However, Ruby code in Haml templates is considered `~backend` code. When in doubt, request both a frontend and backend review. 1. We encourage you to seek guidance from a database maintainer if your merge @@ -199,8 +200,39 @@ by a reviewer before passing it to a maintainer as described in the Designers do not require a Product Designer to approve feature changes, unless the changes are community contributions. 1. End-to-end changes include all files in the `qa` directory. +#### CODEOWNERS approval + +Some merge requests require mandatory approval by specific groups. +See `.gitlab/CODEOWNERS` for definitions. + +Mandatory sections in `.gitlab/CODEOWNERS` should only be limited to cases where +it is necessary due to: + +- compliance +- availability +- security + +When adding a mandatory section, you should track the impact on the new mandatory section +on merge request rates. +See the [Verify issue](https://gitlab.com/gitlab-org/gitlab/-/issues/411559) for a good example. + +All other cases should not use mandatory sections as we favor +[responsibility over ridigity](https://handbook.gitlab.com/handbook/values/#freedom-and-responsibility-over-rigidity). + +Additionally, the current structure of the monolith means that merge requests +are likely to touch seemingly unrelated parts. +Multiple mandatory approvals means that such merge requests require the author +to seek approvals, which is not efficient. + +Efforts to improve this are in: + +- <https://gitlab.com/groups/gitlab-org/-/epics/11624> +- <https://gitlab.com/gitlab-org/gitlab/-/issues/377326> + #### Acceptance checklist +<!-- When editing, remember to announce the change to Engineering Division --> + This checklist encourages the authors, reviewers, and maintainers of merge requests (MRs) to confirm changes were analyzed for high-impact risks to quality, performance, reliability, security, observability, and maintainability. Using checklists improves quality in software engineering. This checklist is a straightforward tool to support and bolster the skills of contributors to the GitLab codebase. @@ -250,7 +282,7 @@ See the [test engineering process](https://about.gitlab.com/handbook/engineering 1. You have reviewed the documentation regarding [internal application security reviews](https://about.gitlab.com/handbook/security/#internal-application-security-reviews) for **when** and **how** to request a security review and requested a security review if this is warranted for this change. 1. If there are security scan results that are blocking the MR (due to the [scan result policies](https://gitlab.com/gitlab-com/gl-security/security-policies)): - For true positive findings, they should be corrected before the merge request is merged. This will remove the AppSec approval required by the scan result policy. - - For false positive findings, something that should be discussed for risk acceptance, or anything questionable, please ping `@gitlab-com/gl-security/appsec`. + - For false positive findings, something that should be discussed for risk acceptance, or anything questionable, ping `@gitlab-com/gl-security/appsec`. ##### Deployment @@ -465,7 +497,7 @@ Here is a summary of the changes, also reflected in this section above. ### Having your merge request reviewed -Please keep in mind that code review is a process that can take multiple +Keep in mind that code review is a process that can take multiple iterations, and reviewers may spot things later that they may not have seen the first time. @@ -619,9 +651,8 @@ WARNING: [very specific cases](https://about.gitlab.com/handbook/engineering/workflow/#criteria-for-merging-during-broken-master). For other cases, follow these [handbook instructions](https://about.gitlab.com/handbook/engineering/workflow/#merging-during-broken-master). - If the latest pipeline was created before the merge request was approved, start a new pipeline to ensure that full RSpec suite has been run. You may skip this step only if the merge request does not contain any backend change. - - If the **latest [merged results pipeline](../ci/pipelines/merged_results_pipelines.md)** was **created less than 6 hours ago**, and **finished less than 2 hours ago**, you - may merge without starting a new pipeline as the merge request is close - enough to `main`. + - If the **latest [merged results pipeline](../ci/pipelines/merged_results_pipelines.md)** was **created less than 4 hours ago**, you + may merge without starting a new pipeline as the merge request is close enough to the target branch. - When you set the MR to auto-merge, you should take over subsequent revisions for anything that would be spotted after that. - For merge requests that have had [Squash and merge](../user/project/merge_requests/squash_and_merge.md) set, diff --git a/doc/development/code_suggestions/index.md b/doc/development/code_suggestions/index.md index 743d06c2b4c..bdf3bcdd520 100644 --- a/doc/development/code_suggestions/index.md +++ b/doc/development/code_suggestions/index.md @@ -1,7 +1,7 @@ --- stage: Create group: Code Creation -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Code Suggestions development guidelines @@ -42,7 +42,7 @@ This should enable everyone to see locally any change in an IDE being sent to th When testing interactions with the Model Gateway, you might want to integrate your local GDK with the deployed staging Model Gateway. To do this: -1. You need a [cloud staging license](../../user/project/repository/code_suggestions/self_managed.md#update-gitlab) that has the Code Suggestions add-on, because add-ons are enabled on staging. Drop a note in the `#s_fulfillment` internal Slack channel to request an add-on to your license. See this [handbook page](https://about.gitlab.com/handbook/developer-onboarding/#working-on-gitlab-ee-developer-licenses) for how to request a license for local development. +1. You need a [cloud staging license](../../user/project/repository/code_suggestions/self_managed.md#upgrade-gitlab) that has the Code Suggestions add-on, because add-ons are enabled on staging. Drop a note in the `#s_fulfillment` internal Slack channel to request an add-on to your license. See this [handbook page](https://about.gitlab.com/handbook/developer-onboarding/#working-on-gitlab-ee-developer-licenses) for how to request a license for local development. 1. Set environment variables to point customers-dot to staging, and the Model Gateway to staging: ```shell diff --git a/doc/development/contributing/design.md b/doc/development/contributing/design.md index c4ec0f66b62..0e922550856 100644 --- a/doc/development/contributing/design.md +++ b/doc/development/contributing/design.md @@ -1,8 +1,7 @@ --- -type: reference, dev stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Design and user interface changes @@ -48,7 +47,7 @@ Check these aspects both when _designing_ and _reviewing_ UI changes. - Use clear and consistent [terminology](https://design.gitlab.com/content/terminology/). - Check grammar and spelling. - Consider help content and follow its [guidelines](https://design.gitlab.com/usability/contextual-help). -- Request review from the [appropriate Technical Writer](https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments), +- Request review from the [appropriate Technical Writer](https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments), indicating any specific files or lines they should review, and how to preview or understand the location/context of the text from the user's perspective. diff --git a/doc/development/contributing/first_contribution.md b/doc/development/contributing/first_contribution.md index 834f34328bc..c6d097574e8 100644 --- a/doc/development/contributing/first_contribution.md +++ b/doc/development/contributing/first_contribution.md @@ -1,20 +1,24 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Tutorial: Make a GitLab contribution Anyone can contribute to the development of GitLab. -Maybe you want to add functionality that you feel is missing. Or maybe -you noticed some UI text that you want to improve. +Maybe you want to add functionality that you feel is missing. +Or maybe you noticed some UI text that you want to improve. This tutorial will walk you through the process of updating UI text and related files by using the GitLab Development Kit and the GitLab community fork. -You can follow this example exactly to familiarize yourself with the process, -or you can choose other UI text to update. +You can follow this example to familiarize yourself with the process. + +NOTE: +We recommend joining our [Discord server](https://discord.gg/gitlab), where GitLab team +members and the wider community are ready and waiting to answer your questions +and ensure [everyone can contribute](https://handbook.gitlab.com/handbook/company/mission/). ## Steps @@ -40,9 +44,11 @@ On your local machine: On GitLab.com: - Create an account. Ensure you can successfully sign in. -- Go to the [`gitlab-community/community-members` group](https://gitlab.com/gitlab-community/community-members) - and select **Request access**. This action will give you access to the GitLab - community fork, where you'll author your changes. +- Click [here](https://gitlab.com/groups/gitlab-community/community-members/-/group_members/request_access) + to request access to the [community forks](https://gitlab.com/gitlab-community). + - The access request will be manually verified and should take no more than a few hours. + - You can get started without access, and only need it prior to pushing your changes to GitLab.com. + - NOTE: If you see an error, you are either: not signed in, or already have access. ## Step 1: Configure the GitLab Development Kit @@ -55,13 +61,40 @@ You can run it on your local machine, or use GitPod to run a remote version.  -If you've never used the GDK before, and you think you might contribute -more than once, you should install it. +We recommend using Gitpod for your first contribution, it will run the GDK regardless +of your local hardware and software (e.g. operating system). + If you already have a working GDK, you should [update it to use the community fork](#an-existing-gdk-installation). +### Using Gitpod + +If you want to contribute without the overhead of setting up a local development environment, +you can use [Gitpod](../../integration/gitpod.md). +Gitpod runs a virtual instance of the GDK. + +Set aside about 15 minutes to launch the GDK in Gitpod. + +1. Launch the GDK in [Gitpod](https://gitpod.io/#https://gitlab.com/gitlab-community/gitlab/-/tree/master/). +1. Create an account if this is your first time using Gitpod. + After registering, you may need to relaunch [GitLab in Gitpod](https://gitpod.io/#https://gitlab.com/gitlab-community/gitlab/-/tree/master/). +1. Select **Continue with GitLab**. +1. If this is your first time using Gitpod with GitLab, select **Authorize** when prompted to + **Authorize Gitpod.io to use your account?**. +1. Leave the default repository URL: `gitlab.com/gitlab-community/gitlab/-/tree/master/`. +1. Select your preferred **Editor**. +1. Leave the default **Class**: `Standard`. +1. Wait for Gitpod to launch. + +You can begin exploring the codebase and making your changes once your editor of choice has launched. + +You will need to wait a little longer for GitLab to be available to preview your changes. + ### A new GDK installation +NOTE: +Skip this step if you're using Gitpod. + Set aside about two hours to install the GDK. If all goes smoothly, it should take about an hour to install. @@ -156,12 +189,6 @@ To confirm it was successful: If you get errors, run `gdk doctor` to troubleshoot. For more advanced troubleshooting, see [the troubleshooting docs](https://gitlab.com/gitlab-org/gitlab-development-kit/-/tree/main/doc/troubleshooting). -### Gitpod - -If you want to contribute without the overhead of setting up a local development environment, -you can use [Gitpod](../../integration/gitpod.md) instead. -Gitpod runs a virtual instance of the GDK. - ## Step 2: Change the code [View an interactive demo of this step](https://gitlab.navattic.com/uu5a0dc5). @@ -177,6 +204,9 @@ I want to change this text: Other settings on the page start with the word `Customize` and skip the `This setting allows you to` part. I'll update this phrase to match the others. +NOTE: +As this text has already been [changed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/116472) when developing this tutorial, you can instead search for `Customize the appearance of the syntax` to find the files that were changed. + 1. Search the `gitlab-development-kit/gitlab` directory for the string `This setting allows you to customize`. The results show one `.haml` file, two `.md` files, one `.pot` file, and diff --git a/doc/development/contributing/index.md b/doc/development/contributing/index.md index abec161bd6e..2c8b5b2af20 100644 --- a/doc/development/contributing/index.md +++ b/doc/development/contributing/index.md @@ -1,8 +1,7 @@ --- -type: reference, dev stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Contribute to GitLab development @@ -143,7 +142,7 @@ Lastly, keep the following in mind when submitting merge requests: ## Contributing to Premium/Ultimate features with an Enterprise Edition license If you would like to work on GitLab features that are within a paid tier, also known as the code that lives in the [EE folder](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee), it requires a GitLab Enterprise Edition license. -Please request an Enterprise Edition Developers License according to the [documented process](https://about.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows.html#contributing-to-the-gitlab-enterprise-edition-ee). +Request an Enterprise Edition Developers License according to the [documented process](https://about.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows#contributing-to-the-gitlab-enterprise-edition-ee). ## Get help diff --git a/doc/development/contributing/issue_workflow.md b/doc/development/contributing/issue_workflow.md index 0b1c7303fc0..b59a77c1f52 100644 --- a/doc/development/contributing/issue_workflow.md +++ b/doc/development/contributing/issue_workflow.md @@ -1,8 +1,7 @@ --- -type: reference, dev stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Issues workflow @@ -27,7 +26,7 @@ Do **not** create publicly viewable issues for suspected security vulnerabilitie ### Feature proposals -To create a feature proposal, open an issue in the issue tracker using the +To create a feature proposal, open an issue in the issue tracker using the [**Feature Proposal - detailed** issue template](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Feature%20proposal%20-%20detailed). In order to help track feature proposals, we use the @@ -35,7 +34,7 @@ In order to help track feature proposals, we use the Users that are not members of the project cannot add labels via the UI. Instead, use [reactive label commands](https://about.gitlab.com/handbook/engineering/quality/triage-operations/#reactive-label-and-unlabel-commands). -Please keep feature proposals as small and simple as possible, complex ones +Keep feature proposals as small and simple as possible, complex ones might be edited to make them small and simple. For changes to the user interface (UI), follow our [design and UI guidelines](design.md), @@ -78,7 +77,7 @@ You are very welcome to help the GitLab team triage issues. The most important thing is making sure valid issues receive feedback from the development team. Therefore the priority is mentioning developers that can help -on those issues. Please select someone with relevant experience from the +on those issues. Select someone with relevant experience from the [GitLab team](https://about.gitlab.com/company/team/). If there is nobody mentioned with that expertise, look in the commit history for the affected files to find someone. @@ -121,7 +120,7 @@ with a reference to an issue describing the regression, and then to update that note with a reference to the merge request that fixes it as it becomes available. If you're a contributor who doesn't have the required permissions to update -other users' notes, please post a new note with a reference to both the issue +other users' notes, post a new note with a reference to both the issue and the merge request. The release manager will diff --git a/doc/development/contributing/merge_request_workflow.md b/doc/development/contributing/merge_request_workflow.md index e739799557e..8ede40e741e 100644 --- a/doc/development/contributing/merge_request_workflow.md +++ b/doc/development/contributing/merge_request_workflow.md @@ -1,8 +1,7 @@ --- -type: reference, dev stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Merge requests workflow @@ -15,7 +14,7 @@ label, but you are free to contribute to any issue you want. ## Working from issues -If you find an issue, please submit a merge request with a fix or improvement, +If you find an issue, submit a merge request with a fix or improvement, if you can, and include tests. If you want to add a new feature that is not labeled, it is best to first create @@ -71,13 +70,13 @@ For a walkthrough of the contribution process, see [Tutorial: Make a GitLab cont - If you would like quick feedback on your merge request feel free to mention someone from the [core team](https://about.gitlab.com/community/core-team/) or one of the [merge request coaches](https://about.gitlab.com/company/team/). When having your code reviewed - and when reviewing merge requests, please keep the [code review guidelines](../code_review.md) + and when reviewing merge requests, keep the [code review guidelines](../code_review.md) in mind. And if your code also makes changes to the database, or does expensive queries, check the [database review guidelines](../database_review.md). ### Keep it simple -*Live by smaller iterations.* Please keep the amount of changes in a single MR **as small as possible**. +*Live by smaller iterations.* Keep the amount of changes in a single MR **as small as possible**. If you want to contribute a large feature, think very carefully about what the [minimum viable change](https://about.gitlab.com/handbook/product/#the-minimally-viable-change) is. Can you split the functionality into two smaller MRs? Can you submit only the @@ -157,7 +156,7 @@ Example commit message template that can be used on your machine that embodies t ## Contribution acceptance criteria -To make sure that your merge request can be approved, please ensure that it meets +To make sure that your merge request can be approved, ensure that it meets the contribution acceptance criteria below: 1. The change is as small as possible. @@ -196,7 +195,7 @@ the contribution acceptance criteria below: ## Definition of done -If you contribute to GitLab, please know that changes involve more than just +If you contribute to GitLab, know that changes involve more than just code. We use the following [definition of done](https://www.agilealliance.org/glossary/definition-of-done/). To reach the definition of done, the merge request must create no regressions and meet all these criteria: @@ -264,7 +263,7 @@ requirements. 1. For tests that use Capybara, read [how to write reliable, asynchronous integration tests](https://thoughtbot.com/blog/write-reliable-asynchronous-integration-tests-with-capybara). 1. [Black-box tests/end-to-end tests](../testing_guide/testing_levels.md#black-box-tests-at-the-system-level-aka-end-to-end-tests) - added if required. Please contact [the quality team](https://about.gitlab.com/handbook/engineering/quality/#teams) + added if required. Contact [the quality team](https://about.gitlab.com/handbook/engineering/quality/#teams) with any questions. 1. The change is tested in a review app where possible and if appropriate. 1. Code affected by a feature flag is covered by [automated tests with the feature flag enabled and disabled](../feature_flags/index.md#feature-flags-in-tests), or both @@ -276,7 +275,7 @@ requirements. 1. Use available components from the GitLab Design System, [Pajamas](https://design.gitlab.com/). 1. The MR must include *Before* and *After* screenshots if UI changes are made. -1. If the MR changes CSS classes, please include the list of affected pages, which +1. If the MR changes CSS classes, include the list of affected pages, which can be found by running `grep css-class ./app -R`. ### Description of changes @@ -295,7 +294,7 @@ requirements. ### Approval -1. The [MR acceptance checklist](../code_review.md#acceptance-checklist) has been checked as confirmed in the MR. +1. The MR was evaluated against the [MR acceptance checklist](../code_review.md#acceptance-checklist). 1. Create an issue in the [infrastructure issue tracker](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues) to inform the Infrastructure department when your contribution is changing default settings or introduces a new setting, if relevant. 1. An agreed-upon [rollout plan](https://about.gitlab.com/handbook/engineering/development/processes/rollout-plans/). 1. Reviewed by relevant reviewers, and all concerns are addressed for Availability, Regressions, and Security. Documentation reviews should take place as soon as possible, but they should not block a merge request. @@ -324,7 +323,7 @@ Contributions do not require approval from the [Product team](https://about.gitl ## Dependencies -If you add a dependency in GitLab (such as an operating system package) please +If you add a dependency in GitLab (such as an operating system package), consider updating the following, and note the applicability of each in your merge request: diff --git a/doc/development/contributing/style_guides.md b/doc/development/contributing/style_guides.md index 80ac0b872d6..e560920cca1 100644 --- a/doc/development/contributing/style_guides.md +++ b/doc/development/contributing/style_guides.md @@ -1,8 +1,7 @@ --- -type: reference, dev stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Development style guides diff --git a/doc/development/contributing/verify/index.md b/doc/development/contributing/verify/index.md index 8d1bd3c76f0..dbc48121dff 100644 --- a/doc/development/contributing/verify/index.md +++ b/doc/development/contributing/verify/index.md @@ -1,5 +1,4 @@ --- -type: reference, dev stage: Verify group: Pipeline Execution --- @@ -243,7 +242,7 @@ scenario relating to a software being built by one of our [early customers](http > could generate a new particle that would then cause the universe to implode. That would be quite an undesirable outcome of a small bug in GitLab CI/CD status -processing. Please take extra care when you are working on CI/CD statuses, +processing. Take extra care when you are working on CI/CD statuses, we don't want to implode our Universe! This is an extreme and unlikely scenario, but presenting data that is not accurate diff --git a/doc/development/dangerbot.md b/doc/development/dangerbot.md index 476d370e7ee..c25c4751ee8 100644 --- a/doc/development/dangerbot.md +++ b/doc/development/dangerbot.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Danger bot @@ -34,6 +34,8 @@ from the start of the merge request. - It's not obvious Danger updates the old comment, thus you need to pay attention to it if it is updated or not. +- When Danger tokens are rotated, it creates confusion/clutter (as old comments + can't be updated). ## Run Danger locally @@ -179,15 +181,19 @@ at GitLab so far: ## Limitations -Danger is run but its output is not added to a merge request comment if working -on a fork. This happens because the secret variable from the canonical project -is not shared to forks. +If working on a personal fork, Danger is run but it's output is not added to a +merge request comment and labels are not applied. +This happens because the secret variable from the canonical project is not shared +to forks. -### Configuring Danger for forks +The best and recommended approach is to work from the [community forks](https://gitlab.com/gitlab-community/meta), +where Danger is already configured. + +### Configuring Danger for personal forks Contributors can configure Danger for their forks with the following steps: -1. Create a [personal API token](https://gitlab.com/-/profile/personal_access_tokens?name=GitLab+Dangerbot&scopes=api) +1. Create a [personal API token](https://gitlab.com/-/user_settings/personal_access_tokens?name=GitLab+Dangerbot&scopes=api) that has the `api` scope set (don't forget to copy it to the clipboard). 1. In your fork, add a [project CI/CD variable](../ci/variables/index.md#for-a-project) called `DANGER_GITLAB_API_TOKEN` with the token copied in the previous step. diff --git a/doc/development/data_science/index.md b/doc/development/data_science/index.md index 832d3999645..632acb8602f 100644 --- a/doc/development/data_science/index.md +++ b/doc/development/data_science/index.md @@ -1,7 +1,9 @@ --- stage: Data Science group: ModelOps -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- +# Data Science + - [Model Registry](model_registry/index.md) diff --git a/doc/development/data_science/model_registry/index.md b/doc/development/data_science/model_registry/index.md index 6ebf6430727..c8f37ca06eb 100644 --- a/doc/development/data_science/model_registry/index.md +++ b/doc/development/data_science/model_registry/index.md @@ -1,7 +1,7 @@ --- stage: Data Science group: ModelOps -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Model Registry diff --git a/doc/development/database/add_foreign_key_to_existing_column.md b/doc/development/database/add_foreign_key_to_existing_column.md index 823fb49a9ab..4a0e3a47633 100644 --- a/doc/development/database/add_foreign_key_to_existing_column.md +++ b/doc/development/database/add_foreign_key_to_existing_column.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Add a foreign key constraint to an existing column diff --git a/doc/development/database/adding_database_indexes.md b/doc/development/database/adding_database_indexes.md index d41800d2df7..05425fdbb21 100644 --- a/doc/development/database/adding_database_indexes.md +++ b/doc/development/database/adding_database_indexes.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Adding Database Indexes diff --git a/doc/development/database/avoiding_downtime_in_migrations.md b/doc/development/database/avoiding_downtime_in_migrations.md index 3b4b45935b9..bf75b7c2ab2 100644 --- a/doc/development/database/avoiding_downtime_in_migrations.md +++ b/doc/development/database/avoiding_downtime_in_migrations.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Avoiding downtime in migrations diff --git a/doc/development/database/background_migrations.md b/doc/development/database/background_migrations.md deleted file mode 100644 index be121cc0bfe..00000000000 --- a/doc/development/database/background_migrations.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'index.md' -remove_date: '2023-10-11' ---- - -This document was moved to [another location](index.md). - -<!-- This redirect file can be deleted after <2023-10-11>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/development/database/batched_background_migrations.md b/doc/development/database/batched_background_migrations.md index a6d827df820..ec1a71eb4f4 100644 --- a/doc/development/database/batched_background_migrations.md +++ b/doc/development/database/batched_background_migrations.md @@ -1,8 +1,7 @@ --- -type: reference, dev stage: Data Stores group: Database -info: "See the Technical Writers assigned to Development Guidelines: https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-development-guidelines" +info: "See the Technical Writers assigned to Development Guidelines: https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-development-guidelines" --- # Batched background migrations @@ -258,6 +257,41 @@ Make sure the newly-created data is either migrated, or saved in both the old and new version upon creation. Removals in turn can be handled by defining foreign keys with cascading deletes. +### Finalize a batched background migration + +Finalizing a batched background migration is done by calling +`ensure_batched_background_migration_is_finished`. + +It is important to finalize all batched background migrations when it is safe +to do so. Leaving around old batched background migration is a form of +technical debt that needs to be maintained in tests and in application +behavior. It is important to note that you cannot depend on any batched +background migration being completed until after it is finalized. + +We recommend that batched background migrations are finalized after all of the +following conditions are met: + +- The batched background migration is completed on GitLab.com +- The batched background migration was added in or before the last [required stop](required_stops.md) + +The `ensure_batched_background_migration_is_finished` call must exactly match +the migration that was used to enqueue it. Pay careful attention to: + +- The job arguments: Needs to exactly match or it will not find the queued migration +- The `gitlab_schema`: Needs to exactly match or it will not find the queued + migration. Even if the `gitlab_schema` of the table has changed from + `gitlab_main` to `gitlab_main_cell` in the meantime you must finalize it + with `gitlab_main` if that's what was used when queueing the batched + background migration. + +When finalizing a batched background migration you also need to update the +`finalized_by` in the corresponding `db/docs/batched_background_migrations` +file. The value should be the timestamp/version of the migration you added to +finalize it. + +See the below [Examples](#examples) for specific details on what the actual +migration code should be. + ### Use job arguments `BatchedMigrationJob` provides the `job_arguments` helper method for job classes to define the job arguments they need. @@ -654,6 +688,65 @@ index 4ae3622479f0800c0553959e132143ec9051898e..d556ec7f55adae9d46a56665ce02de78 Create a Draft merge request with your changes and trigger the manual `db:gitlabcom-database-testing` job. +### Establish dependencies + +In some instances, migrations depended on the completion of previously enqueued BBMs. If the BBMs are +still running, the dependent migration fails. For example: introducing an unique index on a large table can depend on +the previously enqueued BBM to handle any duplicate records. + +The following process has been configured to make dependencies more evident while writing a migration. + +- Version of the migration that queued the BBM is stored in _batched_background_migrations_ table and in BBM dictionary file. +- `DEPENDENT_BATCHED_BACKGROUND_MIGRATIONS` constant is added (commented by default) in each migration file. + To establish the dependency, add `queued_migration_version` of the dependent BBMs. If not, remove + the commented line. +- `Migration::UnfinishedDependencies` cop complains if the dependent BBMs are not yet finished. It determines + whether they got finished by looking up the `finalized_by` key in the + [BBM dictionary](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/generators/batched_background_migration/templates/batched_background_migration_dictionary.template). + +Example: + +```ruby +# db/post_migrate/20231113120650_queue_backfill_routes_namespace_id.rb +class QueueBackfillRoutesNamespaceId < Gitlab::Database::Migration[2.1] + MIGRATION = 'BackfillRouteNamespaceId' + + restrict_gitlab_migration gitlab_schema: :gitlab_main + ... + ... + + def up + queue_batched_background_migration( + MIGRATION, + ... + ) + end +end +``` + +```ruby +# This depends on the finalization of QueueBackfillRoutesNamespaceId BBM +class AddNotNullToRoutesNamespaceId < Gitlab::Database::Migration[2.1] + DEPENDENT_BATCHED_BACKGROUND_MIGRATIONS = ["20231113120650"] + + def up + add_not_null_constraint :routes, :namespace_id + end + + def down + remove_not_null_constraint :routes, :namespace_id + end +end +``` + +#### Notes + +- `BackgroundMigration::DictionaryFile` cop ensures the presence of `finalize_after` and `introduced_by_url` keys in the + BBM dictionary. + - `finalize_after`: Captures the (approximate) date after which the BBM is expected to be finalized. + - `introduced_by_url`: After the `finalize_after` date, an issue is created using the labels and author from `introduced_by_url`. + - As of writing (2023-08-11), issue [#424886](https://gitlab.com/gitlab-org/gitlab/-/issues/424886) is still open. + ## Managing NOTE: @@ -1003,6 +1096,19 @@ background migration. end ``` + ```yaml + # db/docs/batched_background_migrations/backfill_route_namespace_id.yml + --- + migration_job_name: BackfillRouteNamespaceId + description: Copies source_id values from routes to namespace_id + feature_category: source_code_management + introduced_by_url: "https://mr_url" + milestone: 16.6 + queued_migration_version: 20231113120650 + finalize_after: "2023-11-15" + finalized_by: # version of the migration that ensured this bbm + ``` + NOTE: When queuing a batched background migration, you need to restrict the schema to the database where you make the actual changes. @@ -1015,8 +1121,8 @@ background migration. - Continues using the data as before. - Ensures that both existing and new data are migrated. -1. Add a new post-deployment migration - that checks that the batched background migration is completed. For example: +1. Add a new post-deployment migration that checks that the batched background migration is complete. Also update + `finalized_by` attribute in BBM dictionary with the version of this migration. ```ruby class FinalizeBackfillRouteNamespaceId < Gitlab::Database::Migration[2.1] @@ -1041,6 +1147,19 @@ background migration. end ``` + ```yaml + # db/docs/batched_background_migrations/backfill_route_namespace_id.yml + --- + migration_job_name: BackfillRouteNamespaceId + description: Copies source_id values from routes to namespace_id + feature_category: source_code_management + introduced_by_url: "https://mr_url" + milestone: 16.6 + queued_migration_version: 20231113120650 + finalize_after: "2023-11-15" + finalized_by: 20231115120912 + ``` + NOTE: If the batched background migration is not finished, the system will execute the batched background migration inline. If you don't want diff --git a/doc/development/database/ci_mirrored_tables.md b/doc/development/database/ci_mirrored_tables.md index 1e37739bdc4..31c16926c51 100644 --- a/doc/development/database/ci_mirrored_tables.md +++ b/doc/development/database/ci_mirrored_tables.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # CI mirrored tables diff --git a/doc/development/database/clickhouse/clickhouse_within_gitlab.md b/doc/development/database/clickhouse/clickhouse_within_gitlab.md index 2f7a3c4dfe0..a459f89b185 100644 --- a/doc/development/database/clickhouse/clickhouse_within_gitlab.md +++ b/doc/development/database/clickhouse/clickhouse_within_gitlab.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # ClickHouse within GitLab @@ -205,6 +205,26 @@ end NOTE: It's important to test and verify efficient batching of database records from PostgreSQL. Consider using the techniques described in the [Iterating tables in batches](../iterating_tables_in_batches.md). +## Implementing Sidekiq workers + +Sidekiq workers leveraging ClickHouse databases should include the `ClickHouseWorker` module. +This ensures that the worker is paused while database migrations are running, +and that migrations do not run while the worker is active. + +```ruby +# events_sync_worker.rb +# frozen_string_literal: true + +module ClickHouse + class EventsSyncWorker + include ApplicationWorker + include ClickHouseWorker + + ... + end +end +``` + ## Testing ClickHouse is enabled on CI/CD but to avoid significantly affecting the pipeline runtime we've decided to run the ClickHouse server for test cases tagged with `:click_house` only. diff --git a/doc/development/database/clickhouse/gitlab_activity_data.md b/doc/development/database/clickhouse/gitlab_activity_data.md index 7c30703a016..af327ba64a8 100644 --- a/doc/development/database/clickhouse/gitlab_activity_data.md +++ b/doc/development/database/clickhouse/gitlab_activity_data.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Store GitLab activity data in ClickHouse diff --git a/doc/development/database/clickhouse/index.md b/doc/development/database/clickhouse/index.md index 52e9aecce4a..f877ce4053f 100644 --- a/doc/development/database/clickhouse/index.md +++ b/doc/development/database/clickhouse/index.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Introduction to ClickHouse use and table design diff --git a/doc/development/database/clickhouse/merge_request_analytics.md b/doc/development/database/clickhouse/merge_request_analytics.md index 34da71d6c4c..db441ca9051 100644 --- a/doc/development/database/clickhouse/merge_request_analytics.md +++ b/doc/development/database/clickhouse/merge_request_analytics.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Merge request analytics with ClickHouse diff --git a/doc/development/database/clickhouse/optimization.md b/doc/development/database/clickhouse/optimization.md index 166bbf7d2b1..91431d150b4 100644 --- a/doc/development/database/clickhouse/optimization.md +++ b/doc/development/database/clickhouse/optimization.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Optimizing query execution diff --git a/doc/development/database/clickhouse/tiered_storage.md b/doc/development/database/clickhouse/tiered_storage.md index d9026f47e28..afa1747b6f8 100644 --- a/doc/development/database/clickhouse/tiered_storage.md +++ b/doc/development/database/clickhouse/tiered_storage.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Tiered Storages in ClickHouse diff --git a/doc/development/database/client_side_connection_pool.md b/doc/development/database/client_side_connection_pool.md index 3c9f8e76d89..04e2d1fc9f2 100644 --- a/doc/development/database/client_side_connection_pool.md +++ b/doc/development/database/client_side_connection_pool.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Client-side connection-pool diff --git a/doc/development/database/constraint_naming_convention.md b/doc/development/database/constraint_naming_convention.md index 4ac1cd2a71d..3b4f2e92b98 100644 --- a/doc/development/database/constraint_naming_convention.md +++ b/doc/development/database/constraint_naming_convention.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Constraints naming conventions diff --git a/doc/development/database/creating_enums.md b/doc/development/database/creating_enums.md index 908656dae84..bc83b1a022b 100644 --- a/doc/development/database/creating_enums.md +++ b/doc/development/database/creating_enums.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Creating enums diff --git a/doc/development/database/database_debugging.md b/doc/development/database/database_debugging.md index 9cc85610e98..271f14e91bc 100644 --- a/doc/development/database/database_debugging.md +++ b/doc/development/database/database_debugging.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Troubleshooting and debugging the database diff --git a/doc/development/database/database_dictionary.md b/doc/development/database/database_dictionary.md index 70691d8746c..bce1d5a43da 100644 --- a/doc/development/database/database_dictionary.md +++ b/doc/development/database/database_dictionary.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Database Dictionary @@ -41,7 +41,7 @@ gitlab_schema: gitlab_main | `feature_categories` | Array(String) | yes | List of feature categories using this table. | | `description` | String | no | Text description of the information stored in the table, and its purpose. | | `introduced_by_url` | URL | no | URL to the merge request or commit which introduced this table. | -| `milestone` | String | no | The milestone that introduced this table. | +| `milestone` | String | yes | The milestone that introduced this table. | | `gitlab_schema` | String | yes | GitLab schema name. | ### Process diff --git a/doc/development/database/database_lab.md b/doc/development/database/database_lab.md index 7cdf034844d..3458dd9b570 100644 --- a/doc/development/database/database_lab.md +++ b/doc/development/database/database_lab.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Database Lab and Postgres.ai diff --git a/doc/development/database/database_migration_pipeline.md b/doc/development/database/database_migration_pipeline.md index 280254d90b0..069665a83c5 100644 --- a/doc/development/database/database_migration_pipeline.md +++ b/doc/development/database/database_migration_pipeline.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Database migration pipeline diff --git a/doc/development/database/database_query_comments.md b/doc/development/database/database_query_comments.md index 946947db502..c6ec29bcf2e 100644 --- a/doc/development/database/database_query_comments.md +++ b/doc/development/database/database_query_comments.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Database query comments with Marginalia diff --git a/doc/development/database/database_reviewer_guidelines.md b/doc/development/database/database_reviewer_guidelines.md index c75503c503b..07785557813 100644 --- a/doc/development/database/database_reviewer_guidelines.md +++ b/doc/development/database/database_reviewer_guidelines.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Database Reviewer Guidelines @@ -84,7 +84,7 @@ topics and use cases. The most frequently required during database reviewing are Database maintainership uses the same process as other projects for identifying maintainers. [Follow the general process documented here](https://about.gitlab.com/handbook/engineering/workflow/code-review/#how-to-become-a-project-maintainer). -For database specific requirements, please see [`Project maintainer process for gitlab-database`](https://about.gitlab.com/handbook/engineering/workflow/code-review/#project-maintainer-process-for-gitlab-database) +For database specific requirements, see [`Project maintainer process for gitlab-database`](https://about.gitlab.com/handbook/engineering/workflow/code-review/#project-maintainer-process-for-gitlab-database) ## What to do if you feel overwhelmed diff --git a/doc/development/database/db_dump.md b/doc/development/database/db_dump.md index b09c601537c..f22636ab042 100644 --- a/doc/development/database/db_dump.md +++ b/doc/development/database/db_dump.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Importing a database dump into a staging environment diff --git a/doc/development/database/dbcheck-migrations-job.md b/doc/development/database/dbcheck-migrations-job.md index 4ebfb40cd03..a2e50096e45 100644 --- a/doc/development/database/dbcheck-migrations-job.md +++ b/doc/development/database/dbcheck-migrations-job.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # db:check-migrations job diff --git a/doc/development/database/deleting_migrations.md b/doc/development/database/deleting_migrations.md index bd03e6c11ae..829477bc755 100644 --- a/doc/development/database/deleting_migrations.md +++ b/doc/development/database/deleting_migrations.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Delete existing migrations diff --git a/doc/development/database/efficient_in_operator_queries.md b/doc/development/database/efficient_in_operator_queries.md index 87976df5711..8e334e5bb5c 100644 --- a/doc/development/database/efficient_in_operator_queries.md +++ b/doc/development/database/efficient_in_operator_queries.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Efficient `IN` operator queries diff --git a/doc/development/database/filtering_by_label.md b/doc/development/database/filtering_by_label.md index a30322176ba..a7e7d7d5fd6 100644 --- a/doc/development/database/filtering_by_label.md +++ b/doc/development/database/filtering_by_label.md @@ -1,7 +1,7 @@ --- stage: Plan group: Project Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Filtering by label diff --git a/doc/development/database/foreign_keys.md b/doc/development/database/foreign_keys.md index 4a457622a49..a341b9d9b72 100644 --- a/doc/development/database/foreign_keys.md +++ b/doc/development/database/foreign_keys.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Foreign keys and associations diff --git a/doc/development/database/hash_indexes.md b/doc/development/database/hash_indexes.md index aba23dbea70..8198459bd39 100644 --- a/doc/development/database/hash_indexes.md +++ b/doc/development/database/hash_indexes.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Hash Indexes diff --git a/doc/development/database/index.md b/doc/development/database/index.md index 70681994229..c5969176d72 100644 --- a/doc/development/database/index.md +++ b/doc/development/database/index.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Database development guidelines diff --git a/doc/development/database/insert_into_tables_in_batches.md b/doc/development/database/insert_into_tables_in_batches.md index 547ca25b264..df843c5d596 100644 --- a/doc/development/database/insert_into_tables_in_batches.md +++ b/doc/development/database/insert_into_tables_in_batches.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. description: "Sometimes it is necessary to store large amounts of records at once, which can be inefficient when iterating collections and performing individual `save`s. With the arrival of `insert_all` in Rails 6, which operates at the row level (that is, using `Hash`es), GitLab has added a set diff --git a/doc/development/database/iterating_tables_in_batches.md b/doc/development/database/iterating_tables_in_batches.md index 44a8c72ea2c..802b7622592 100644 --- a/doc/development/database/iterating_tables_in_batches.md +++ b/doc/development/database/iterating_tables_in_batches.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Iterating tables in batches diff --git a/doc/development/database/keyset_pagination.md b/doc/development/database/keyset_pagination.md index ff8038ee24c..9bb44d2ed71 100644 --- a/doc/development/database/keyset_pagination.md +++ b/doc/development/database/keyset_pagination.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Keyset pagination @@ -87,6 +87,55 @@ Because keyset pagination does not support page numbers, we are restricted to go - Last page - First page +#### Usage in REST API with `paginate_with_strategies` + +For the REST API, the `paginate_with_strategies` helper can be used on a relation in order to use either keyset pagination or offset pagination. + +```ruby + desc 'Get the things related to a project' do + detail 'This feature was introduced in GitLab 16.1' + success code: 200, model: ::API::Entities::Thing + failure [ + { code: 401, message: 'Unauthorized' }, + { code: 403, message: 'Forbidden' }, + { code: 404, message: 'Not Found' } + ] + end + params do + use :pagination + requires :project_id, type: Integer, desc: 'The ID of the project' + optional :cursor, type: String, desc: 'Cursor for obtaining the next set of records' + optional :order_by, type: String, values: %w[id name], default: 'id', + desc: 'Attribute to sort by' + optional :sort, type: String, values: %w[asc desc], default: 'desc', desc: 'Order of sorting' + end + route_setting :authentication + get ':project_id/things' do + project = Project.find_by_id(params[:project_id]) + + not_found! if project.blank? + + things = project.things + + present paginate_with_strategies(things), with: ::API::Entities::Thing + end +``` + +In order for keyset pagination to be used, the following conditions must be met: + +1. `params[:keyset]` must return `'keyset'` +1. `params[:order_by]` and `params[:sort]` must both appear in the object returned by the + `supported_keyset_orderings` class method on the model. In the following example, `Thing` + supports keyset pagination when ordering by ID in either ascending or descending order. + + ```ruby + class Thing < ApplicationRecord + def self.supported_keyset_orderings + { id: [:asc, :desc] } + end + end + ``` + #### Usage in Rails with HAML views Consider the following controller action, where we list the projects ordered by name: diff --git a/doc/development/database/layout_and_access_patterns.md b/doc/development/database/layout_and_access_patterns.md index 06ac24c284c..498edc7155b 100644 --- a/doc/development/database/layout_and_access_patterns.md +++ b/doc/development/database/layout_and_access_patterns.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Best practices for data layout and access patterns diff --git a/doc/development/database/load_balancing.md b/doc/development/database/load_balancing.md index f6e145ac7f5..9ec6ef6fce7 100644 --- a/doc/development/database/load_balancing.md +++ b/doc/development/database/load_balancing.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Database load balancing diff --git a/doc/development/database/loose_foreign_keys.md b/doc/development/database/loose_foreign_keys.md index 08d618a26ae..3003ee970ce 100644 --- a/doc/development/database/loose_foreign_keys.md +++ b/doc/development/database/loose_foreign_keys.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Loose foreign keys @@ -119,9 +119,9 @@ To match foreign key (FK), write one or many filters to match against FROM/TO/CO - scripts/decomposition/generate-loose-foreign-key dast_site_profiles_pipelines ``` -The command accepts a list of filters to match from, to, or column for the purpose of the foreign key generation. -For example, run this to swap all foreign keys for `ci_job_token_project_scope_links` for the -decomposed database: +The command accepts a list of regular expressions to match from, to, or column +for the purpose of the foreign key generation. For example, run this to swap +all foreign keys for `ci_job_token_project_scope_links` for the decomposed database: ```shell scripts/decomposition/generate-loose-foreign-key -c ci_job_token_project_scope_links @@ -133,6 +133,15 @@ To swap only the `source_project_id` of `ci_job_token_project_scope_links` for t scripts/decomposition/generate-loose-foreign-key -c ci_job_token_project_scope_links source_project_id ``` +To match the exact name of a table or columns, you can make use of the regular expressions +position anchors `^` and `$`. For example, this command matches only the +foreign keys on the `events` table only, but not on the table +`incident_management_timeline_events`. + +```shell +scripts/decomposition/generate-loose-foreign-key -n ^events$ +``` + To swap all the foreign keys (all having `_id` appended), but not create a new branch (only commit the changes) and not create RSpec tests, run: diff --git a/doc/development/database/maintenance_operations.md b/doc/development/database/maintenance_operations.md index bfe18068aab..2701c228b74 100644 --- a/doc/development/database/maintenance_operations.md +++ b/doc/development/database/maintenance_operations.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Maintenance operations diff --git a/doc/development/database/migrations_for_multiple_databases.md b/doc/development/database/migrations_for_multiple_databases.md index ae9348a2090..b7f3b37ada0 100644 --- a/doc/development/database/migrations_for_multiple_databases.md +++ b/doc/development/database/migrations_for_multiple_databases.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Migrations for Multiple databases diff --git a/doc/development/database/multiple_databases.md b/doc/development/database/multiple_databases.md index a045d8ad144..e453a7f27ea 100644 --- a/doc/development/database/multiple_databases.md +++ b/doc/development/database/multiple_databases.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Tenant Scale -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Multiple Databases @@ -64,6 +64,136 @@ After a schema has been assigned, the merge request pipeline might fail due to o - [Cross-database transactions](#fixing-cross-database-transactions) - [Cross-database foreign keys](#foreign-keys-that-cross-databases) +### Defining a sharding key for all cell-local tables + +All tables with the following `gitlab_schema` are considered "cell-local": + +- `gitlab_main_cell` +- `gitlab_ci` + +All newly created cell-local tables are required to have a `sharding_key` +defined in the corresponding `db/docs/` file for that table. + +The purpose of the sharding key is documented in the +[Organization isolation blueprint](../../architecture/blueprints/organization/isolation.md), +but in short this column is used to provide a standard way of determining which +Organization owns a particular row in the database. The column will be used in +the future to enforce constraints on data not cross Organization boundaries. It +will also be used in the future to provide a uniform way to migrate data +between Cells. + +The actual name of the foreign key can be anything but it must reference a row +in `projects` or `groups`. The following are examples of valid sharding keys: + +- The table entries belong to a project only: + + ```yaml + sharding_key: + project_id: projects + ``` + +- The table entries belong to a project and the foreign key is `target_project_id`: + + ```yaml + sharding_key: + target_project_id: projects + ``` + +- The table entries belong to a namespace/group only: + + ```yaml + sharding_key: + namespace_id: namespaces + ``` + +- The table entries belong to a namespace/group only and the foreign key is `group_id`: + + ```yaml + sharding_key: + group_id: namespaces + ``` + +- The table entries belong to a namespace or a project: + + ```yaml + sharding_key: + project_id: projects + namespace_id: namespaces + ``` + +#### The sharding key must be immutable + +The choice of a `sharding_key` should always be immutable. Therefore, if your feature +requires a user experience which allows data to be moved between projects or +groups/namespaces, then you may need to redesign the move feature to create new rows. An +example of this can be seen in the +[move an issue feature](../../user/project/issues/managing_issues.md#move-an-issue). +This feature does not actually change the `project_id` column for an existing +`issues` row but instead creates a new `issues` row and creates a link in the +database from the original `issues` row. If there is a particularly challenging +existing feature that needs to allow moving data you will need to reach out to +the Tenant Scale team early on to discuss options for how to manage the +sharding key. + +#### Using the same sharding key for projects and namespaces + +Developers may also choose to use `namespace_id` only for tables that can +belong to a project where the feature used by the table is being developed +following the +[Consolidating Groups and Projects blueprint](../../architecture/blueprints/consolidating_groups_and_projects/index.md). +In that case the `namespace_id` would need to be the ID of the +`ProjectNamespace` and not the group that the namespace belongs to. + +#### Defining a `desired_sharding_key` for automatically backfilling a `sharding_key` + +We need to backfill a `sharding_key` to hundreds of tables that do not have one. +This process will involve creating a merge request like +<https://gitlab.com/gitlab-org/gitlab/-/merge_requests/136800> to add the new +column, backfill the data from a related table in the database, and then create +subsequent merge requests to add indexes, foreign keys and not-null +constraints. + +In order to minimize the amount of repetitive effort for developers we've +introduced a concise declarative way to describe how to backfill the +`sharding_key` for this specific table. This content will later be used in +automation to create all the necessary merge requests. + +An example of the `desired_sharding_key` was added in +<https://gitlab.com/gitlab-org/gitlab/-/merge_requests/139336> and it looks like: + +```yaml +--- # db/docs/security_findings.yml +table_name: security_findings +classes: +- Security::Finding + +... + +desired_sharding_key: + project_id: + references: projects + backfill_via: + parent: + foreign_key: scanner_id + table: vulnerability_scanners + sharding_key: project_id + belongs_to: scanner +``` + +To understand best how this YAML data will be used you can map it onto +the merge request we created manually in +<https://gitlab.com/gitlab-org/gitlab/-/merge_requests/136800>. The idea +will be to automatically create this. The content of the YAML specifies +the parent table and its `sharding_key` to backfill from in the batched +background migration. It also specifies a `belongs_to` relation which +will be added to the model to automatically populate the `sharding_key` in +the `before_save`. + +There are likely edge cases where this `desired_sharding_key` structure is not +suitable for backfilling a `sharding_key`. In such cases the team owning the +table will need to create the necessary merge requests to add the +`sharding_key` manually. + ### The impact of `gitlab_schema` The usage of `gitlab_schema` has a significant impact on the application. diff --git a/doc/development/database/namespaces_storage_statistics.md b/doc/development/database/namespaces_storage_statistics.md index c653cfb145d..07754839a1f 100644 --- a/doc/development/database/namespaces_storage_statistics.md +++ b/doc/development/database/namespaces_storage_statistics.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Database case study: Namespaces storage statistics diff --git a/doc/development/database/new_database_migration_version.md b/doc/development/database/new_database_migration_version.md index b97ecd83f37..f253f99c8a6 100644 --- a/doc/development/database/new_database_migration_version.md +++ b/doc/development/database/new_database_migration_version.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Introducing a new database migration version diff --git a/doc/development/database/not_null_constraints.md b/doc/development/database/not_null_constraints.md index 05b1081fc4d..5615edfcc2a 100644 --- a/doc/development/database/not_null_constraints.md +++ b/doc/development/database/not_null_constraints.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # `NOT NULL` constraints @@ -204,11 +204,117 @@ If you have to clean up a nullable column for a [high-traffic table](../migratio it needs an additional [batched background migration cleaning up](batched_background_migrations.md#cleaning-up-a-batched-background-migration) in the release after adding the data migration. -In that rare case you need 3 releases end-to-end: - -1. Release `N.M` - Add the `NOT NULL` constraint and the background-migration to fix the existing records. -1. Release `N.M+1` - Cleanup the background migration. -1. Release `N.M+2` - Validate the `NOT NULL` constraint. +In this case the number of releases depends on the amount of time needed to migrate existing records. The cleanup is +scheduled after the background migration has completed, which could be several releases after the constraint was added. + +1. Release `N.M`: + - Add the `NOT NULL` constraint without validating it: + + ```ruby + # db/post_migrate/ + class AddMergeRequestDiffsProjectIdNotNullConstraint < Gitlab::Database::Migration[2.2] + disable_ddl_transaction! + milestone '16.7' + + def up + add_not_null_constraint :merge_request_diffs, :project_id, validate: false + end + + def down + remove_not_null_constraint :merge_request_diffs, :project_id + end + end + ``` + + - Add the background-migration to fix the existing records: + + ```ruby + # db/post_migrate/ + class QueueBackfillMergeRequestDiffsProjectId < Gitlab::Database::Migration[2.2] + milestone '16.7' + restrict_gitlab_migration gitlab_schema: :gitlab_main + + MIGRATION = 'BackfillMergeRequestDiffsProjectId' + DELAY_INTERVAL = 2.minutes + + def up + queue_batched_background_migration( + MIGRATION, + :merge_request_diffs, + :id, + job_interval: DELAY_INTERVAL + ) + end + + def down + delete_batched_background_migration(MIGRATION, :merge_request_diffs, :id, []) + end + end + ``` + +1. Release `N.M+X`, where `X` is the number of releases the migration was running: + - Cleanup the background migration: + + ```ruby + # db/post_migrate/ + class FinalizeMergeRequestDiffsProjectIdBackfill < Gitlab::Database::Migration[2.2] + disable_ddl_transaction! + milestone '16.10' + restrict_gitlab_migration gitlab_schema: :gitlab_main + + MIGRATION = 'BackfillMergeRequestDiffsProjectId' + + def up + ensure_batched_background_migration_is_finished( + job_class_name: MIGRATION, + table_name: :merge_request_diffs, + column_name: :id, + job_arguments: [], + finalize: true + ) + end + + def down + # no-op + end + end + ``` + + - **Optional.** For very large tables, schedule asynchronous validation of the `NOT NULL` constraint: + + ```ruby + # db/post_migrate/ + class PrepareMergeRequestDiffsProjectIdNotNullValidation < Gitlab::Database::Migration[2.2] + milestone '16.10' + + CONSTRAINT_NAME = 'check_11c5f029ad' + + def up + prepare_async_check_constraint_validation :merge_request_diffs, name: CONSTRAINT_NAME + end + + def down + unprepare_async_check_constraint_validation :merge_request_diffs, name: CONSTRAINT_NAME + end + end + ``` + +1. Validate the `NOT NULL` constraint (if the constraint was validated asynchronously, wait for this validation to finish): + + ```ruby + # db/post_migrate/ + class ValidateMergeRequestDiffsProjectIdNullConstraint < Gitlab::Database::Migration[2.2] + milestone '16.10' + + def up + validate_not_null_constraint :merge_request_diffs, :project_id + end + + def down + # no-op + end + end + ``` For these cases, consult the database team early in the update cycle. The `NOT NULL` constraint may not be required or other options could exist that do not affect really large diff --git a/doc/development/database/ordering_table_columns.md b/doc/development/database/ordering_table_columns.md index 286313fb3ce..a97e1f60a2d 100644 --- a/doc/development/database/ordering_table_columns.md +++ b/doc/development/database/ordering_table_columns.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Ordering Table Columns in PostgreSQL diff --git a/doc/development/database/pagination_guidelines.md b/doc/development/database/pagination_guidelines.md index 8b07dcada05..77fecccb38e 100644 --- a/doc/development/database/pagination_guidelines.md +++ b/doc/development/database/pagination_guidelines.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Pagination guidelines diff --git a/doc/development/database/pagination_performance_guidelines.md b/doc/development/database/pagination_performance_guidelines.md index b06839979da..f3a08c0240f 100644 --- a/doc/development/database/pagination_performance_guidelines.md +++ b/doc/development/database/pagination_performance_guidelines.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Pagination performance guidelines diff --git a/doc/development/database/poc_tree_iterator.md b/doc/development/database/poc_tree_iterator.md index 453f77f0cde..118d2d605be 100644 --- a/doc/development/database/poc_tree_iterator.md +++ b/doc/development/database/poc_tree_iterator.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Batch iteration in a tree hierarchy (proof of concept) diff --git a/doc/development/database/polymorphic_associations.md b/doc/development/database/polymorphic_associations.md index bb7eb46b448..54ef1c27229 100644 --- a/doc/development/database/polymorphic_associations.md +++ b/doc/development/database/polymorphic_associations.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Polymorphic Associations diff --git a/doc/development/database/post_deployment_migrations.md b/doc/development/database/post_deployment_migrations.md index 8ee9a4ae099..772c34b76d1 100644 --- a/doc/development/database/post_deployment_migrations.md +++ b/doc/development/database/post_deployment_migrations.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Post Deployment Migrations diff --git a/doc/development/database/query_count_limits.md b/doc/development/database/query_count_limits.md index b5fd4395b36..4575669c7af 100644 --- a/doc/development/database/query_count_limits.md +++ b/doc/development/database/query_count_limits.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Query Count Limits diff --git a/doc/development/database/query_performance.md b/doc/development/database/query_performance.md index 1bf9ded3f15..a5fbc2dec17 100644 --- a/doc/development/database/query_performance.md +++ b/doc/development/database/query_performance.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Query performance guidelines diff --git a/doc/development/database/query_recorder.md b/doc/development/database/query_recorder.md index 39d54954268..81fc94ea055 100644 --- a/doc/development/database/query_recorder.md +++ b/doc/development/database/query_recorder.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # QueryRecorder @@ -17,8 +17,8 @@ As a rule, merge requests [should not increase query counts](../merge_request_co This style of test works by counting the number of SQL queries executed by ActiveRecord. First a control count is taken, then you add new records to the database and rerun the count. If the number of queries has significantly increased then an `N+1` queries problem exists. ```ruby -it "avoids N+1 database queries" do - control = ActiveRecord::QueryRecorder.new { visit_some_page } +it "avoids N+1 database queries", :use_sql_query_cache do + control = ActiveRecord::QueryRecorder.new(skip_cached: false) { visit_some_page } create_list(:issue, 5) expect { visit_some_page }.to issue_same_number_of_queries_as(control) end diff --git a/doc/development/database/rename_database_tables.md b/doc/development/database/rename_database_tables.md index 641bba1f131..ad641797496 100644 --- a/doc/development/database/rename_database_tables.md +++ b/doc/development/database/rename_database_tables.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Rename table without downtime diff --git a/doc/development/database/required_stops.md b/doc/development/database/required_stops.md index 361a0495545..d476a2e70eb 100644 --- a/doc/development/database/required_stops.md +++ b/doc/development/database/required_stops.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Adding required stops diff --git a/doc/development/database/serializing_data.md b/doc/development/database/serializing_data.md index b25c8d49b64..64b7e19f528 100644 --- a/doc/development/database/serializing_data.md +++ b/doc/development/database/serializing_data.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Serializing Data diff --git a/doc/development/database/setting_multiple_values.md b/doc/development/database/setting_multiple_values.md index 48be7ff9f10..169a699d5a8 100644 --- a/doc/development/database/setting_multiple_values.md +++ b/doc/development/database/setting_multiple_values.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Update multiple database objects diff --git a/doc/development/database/sha1_as_binary.md b/doc/development/database/sha1_as_binary.md index 5ad9d106086..bd955c61441 100644 --- a/doc/development/database/sha1_as_binary.md +++ b/doc/development/database/sha1_as_binary.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Storing SHA1 Hashes As Binary diff --git a/doc/development/database/single_table_inheritance.md b/doc/development/database/single_table_inheritance.md index 40b608bd110..ecc42a3b5d1 100644 --- a/doc/development/database/single_table_inheritance.md +++ b/doc/development/database/single_table_inheritance.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Single Table Inheritance @@ -53,8 +53,13 @@ class Animal < ActiveRecord::Base def self.inheritance_column = 'species' end -class Dog < Animal; end -class Cat < Animal; end +class Dog < Animal + self.allow_legacy_sti_class = true +end + +class Cat < Animal + self.allow_legacy_sti_class = true +end ``` If your table already has a `*_type`, new classes for the different types can be added as needed. diff --git a/doc/development/database/strings_and_the_text_data_type.md b/doc/development/database/strings_and_the_text_data_type.md index adb715e219d..70d59603e0c 100644 --- a/doc/development/database/strings_and_the_text_data_type.md +++ b/doc/development/database/strings_and_the_text_data_type.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Strings and the Text data type diff --git a/doc/development/database/swapping_tables.md b/doc/development/database/swapping_tables.md index f659d1408d3..1c634c6a83e 100644 --- a/doc/development/database/swapping_tables.md +++ b/doc/development/database/swapping_tables.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Swapping Tables diff --git a/doc/development/database/table_partitioning.md b/doc/development/database/table_partitioning.md index 8477dd180a6..4d3101b40fb 100644 --- a/doc/development/database/table_partitioning.md +++ b/doc/development/database/table_partitioning.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Database table partitioning diff --git a/doc/development/database/transaction_guidelines.md b/doc/development/database/transaction_guidelines.md index 1648cf58937..741a40c660f 100644 --- a/doc/development/database/transaction_guidelines.md +++ b/doc/development/database/transaction_guidelines.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Transaction guidelines diff --git a/doc/development/database/understanding_explain_plans.md b/doc/development/database/understanding_explain_plans.md index 3e8978e1046..22c52c04745 100644 --- a/doc/development/database/understanding_explain_plans.md +++ b/doc/development/database/understanding_explain_plans.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Understanding EXPLAIN plans @@ -571,8 +571,8 @@ what if we slightly change the purpose of it? What if instead of retrieving all projects with `visibility_level` 0 or 20, we retrieve those that a user interacted with somehow? -Fortunately, GitLab has an answer for this, and it's a table called -`user_interacted_projects`. This table has the following schema: +Prior to GitLab 16.7, GitLab used a table named `user_interacted_projects` to track user interactions with projects. +This table had the following schema: ```sql Table "public.user_interacted_projects" diff --git a/doc/development/database/verifying_database_capabilities.md b/doc/development/database/verifying_database_capabilities.md index c1dd29082ba..f39ce0ae30c 100644 --- a/doc/development/database/verifying_database_capabilities.md +++ b/doc/development/database/verifying_database_capabilities.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Verifying Database Capabilities diff --git a/doc/development/database_review.md b/doc/development/database_review.md index ba0423a1a0d..ec4c4f9f2c5 100644 --- a/doc/development/database_review.md +++ b/doc/development/database_review.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Database Review Guidelines @@ -182,6 +182,7 @@ Include in the MR description: - To produce a query plan with enough data, you can use the IDs of: - The `gitlab-org` namespace (`namespace_id = 9970`), for queries involving a group. - The `gitlab-org/gitlab-foss` (`project_id = 13083`) or the `gitlab-org/gitlab` (`project_id = 278964`) projects, for queries involving a project. + - For queries involving membership of projects, `project_namespace_id` of these projects may be required to create a query plan. These are `15846663` (for `gitlab-org/gitlab`) and `15846626` (for `gitlab-org/gitlab-foss`) - The `gitlab-qa` user (`user_id = 1614863`), for queries involving a user. - Optionally, you can also use your own `user_id`, or the `user_id` of a user with a long history within the project or group being used to generate the query plan. - That means that no query plan should return 0 records or less records than the provided limit (if a limit is included). If a query is used in batching, a proper example batch with adequate included results should be identified and provided. diff --git a/doc/development/dependencies.md b/doc/development/dependencies.md index 3b935ceba22..0acd9472075 100644 --- a/doc/development/dependencies.md +++ b/doc/development/dependencies.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Dependencies diff --git a/doc/development/deprecation_guidelines/index.md b/doc/development/deprecation_guidelines/index.md index d586f25ffbf..7d1551cfc6c 100644 --- a/doc/development/deprecation_guidelines/index.md +++ b/doc/development/deprecation_guidelines/index.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Deprecating GitLab features diff --git a/doc/development/developing_with_solargraph.md b/doc/development/developing_with_solargraph.md index 7b9912a865e..61e4d3b124a 100644 --- a/doc/development/developing_with_solargraph.md +++ b/doc/development/developing_with_solargraph.md @@ -1,7 +1,7 @@ --- stage: Create group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Using Solargraph diff --git a/doc/development/development_seed_files.md b/doc/development/development_seed_files.md index 2bf3688fd48..fbbc1cc7efb 100644 --- a/doc/development/development_seed_files.md +++ b/doc/development/development_seed_files.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Development seed files @@ -18,6 +18,7 @@ data for features. |-------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------| | DevOps Adoption | `FILTER=devops_adoption bundle exec rake db:seed_fu` | [31_devops_adoption.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/db/fixtures/development/31_devops_adoption.rb) | | Value Streams Dashboard | `FILTER=cycle_analytics SEED_VSA=1 bundle exec rake db:seed_fu` | [17_cycle_analytics.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/fixtures/development/17_cycle_analytics.rb) | +| Value Streams Dashboard overview counts | `FILTER=vsd_overview_counts SEED_VSD_COUNTS=1 bundle exec rake db:seed_fu` | [93_vsd_overview_counts.rb](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/db/fixtures/development/93_vsd_overview_counts.rb) | | Value Stream Analytics | `FILTER=customizable_cycle_analytics SEED_CUSTOMIZABLE_CYCLE_ANALYTICS=1 bundle exec rake db:seed_fu` | [30_customizable_cycle_analytics](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/db/fixtures/development/30_customizable_cycle_analytics.rb) | | CI/CD analytics | `FILTER=ci_cd_analytics SEED_CI_CD_ANALYTICS=1 bundle exec rake db:seed_fu` | [38_ci_cd_analytics](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/fixtures/development/38_ci_cd_analytics.rb?ref_type=heads) | | Contributions Analytics<br><br>Productivity Analytics<br><br>Code review Analytics<br><br>Merge Request Analytics | `FILTER=productivity_analytics SEED_PRODUCTIVITY_ANALYTICS=1 bundle exec rake db:seed_fu` | [90_productivity_analytics](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/db/fixtures/development/90_productivity_analytics.rb) | diff --git a/doc/development/distributed_tracing.md b/doc/development/distributed_tracing.md index 56c114ba8de..86732c3a8ac 100644 --- a/doc/development/distributed_tracing.md +++ b/doc/development/distributed_tracing.md @@ -1,7 +1,7 @@ --- -stage: Monitor +stage: Service Management group: Respond -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Distributed tracing development guidelines diff --git a/doc/development/distribution/index.md b/doc/development/distribution/index.md index 168e3854eca..2c164b6745d 100644 --- a/doc/development/distribution/index.md +++ b/doc/development/distribution/index.md @@ -1,7 +1,7 @@ --- stage: Systems group: Distribution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. description: "GitLab's development guidelines for Distribution" --- diff --git a/doc/development/documentation/contribute.md b/doc/development/documentation/contribute.md deleted file mode 100644 index 2b166266ddb..00000000000 --- a/doc/development/documentation/contribute.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'index.md' -remove_date: '2023-10-27' ---- - -This document was moved to [another location](index.md). - -<!-- This redirect file can be deleted after <2023-10-27>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/development/documentation/drawers.md b/doc/development/documentation/drawers.md index 7ede1a0b652..1dfe50a8c00 100644 --- a/doc/development/documentation/drawers.md +++ b/doc/development/documentation/drawers.md @@ -1,7 +1,7 @@ --- stage: none group: Documentation Guidelines -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. --- # Create content for drawers diff --git a/doc/development/documentation/experiment_beta.md b/doc/development/documentation/experiment_beta.md index fab78082cb5..85f6dc38621 100644 --- a/doc/development/documentation/experiment_beta.md +++ b/doc/development/documentation/experiment_beta.md @@ -1,5 +1,5 @@ --- -info: For assistance with this Style Guide page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects stage: none group: unassigned --- diff --git a/doc/development/documentation/feature_flags.md b/doc/development/documentation/feature_flags.md index a08052bf0e4..617691e8628 100644 --- a/doc/development/documentation/feature_flags.md +++ b/doc/development/documentation/feature_flags.md @@ -1,5 +1,5 @@ --- -info: For assistance with this Style Guide page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. stage: none group: unassigned description: "GitLab development - how to document features deployed behind feature flags" diff --git a/doc/development/documentation/graphql_styleguide.md b/doc/development/documentation/graphql_styleguide.md index 3c96dd06b25..264ba19226f 100644 --- a/doc/development/documentation/graphql_styleguide.md +++ b/doc/development/documentation/graphql_styleguide.md @@ -1,8 +1,7 @@ --- -type: reference, dev stage: none group: unassigned -info: "See the Technical Writers assigned to Development Guidelines: https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-development-guidelines" +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. description: "Writing styles, markup, formatting, and other standards for GraphQL API's GitLab Documentation." --- diff --git a/doc/development/documentation/help.md b/doc/development/documentation/help.md index a921429bf49..59ffe93f904 100644 --- a/doc/development/documentation/help.md +++ b/doc/development/documentation/help.md @@ -1,7 +1,7 @@ --- stage: none group: Documentation Guidelines -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. --- # GitLab /help diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md index c06a396b378..698bc59a951 100644 --- a/doc/development/documentation/index.md +++ b/doc/development/documentation/index.md @@ -1,7 +1,7 @@ --- stage: none group: Documentation Guidelines -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. --- # Contribute to the GitLab documentation @@ -76,7 +76,7 @@ Ask for help from the Technical Writing team if you: To identify someone who can help you: 1. Locate the Technical Writer for the relevant - [DevOps stage group](https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments). + [DevOps stage group](https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments). 1. Either: - If urgent help is required, directly assign the Technical Writer in the issue or in the merge request. - If non-urgent help is required, ping the Technical Writer in the issue or merge request. diff --git a/doc/development/documentation/metadata.md b/doc/development/documentation/metadata.md new file mode 100644 index 00000000000..4e80ea154b3 --- /dev/null +++ b/doc/development/documentation/metadata.md @@ -0,0 +1,92 @@ +--- +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. +stage: none +group: unassigned +--- + +# Metadata + +Each documentation Markdown page contains YAML front matter. +All values in the metadata are treated as strings and are used for the +docs website only. + +## Stage and group metadata + +Each page should have metadata related to the stage and group it +belongs to, as well as an information block. For example: + +```yaml +--- +stage: Example Stage +group: Example Group +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- +``` + +To populate the metadata, include this information: + +- `stage`: The [Stage](https://about.gitlab.com/handbook/product/categories/#devops-stages) + that the majority of the page's content belongs to. +- `group`: The [Group](https://about.gitlab.com/company/team/structure/#product-groups) + that the majority of the page's content belongs to. +- `info`: How to find the Technical Writer associated with the page's stage and + group. + +## Additional metadata + +The following metadata is optional and is not actively maintained. + +- `description`: A short description of what the page is about. See the Google [Best practices for creating quality meta descriptions](https://developers.google.com/search/docs/appearance/snippet#meta-descriptions) for writing tips. This content can be used in search result snippets and is shown in social media previews. +- `feedback`: Set to `false` to not include the "Help & Feedback" footer. +- `noindex`: Set to `false` to prevent the page from being indexed by search engines. +- `redirect_to`: Used to control redirects. For more information, see [Redirects in GitLab documentation](redirects.md). +- `searchbar`: Set to `false` to not include the search bar in the page header. +- `toc`: Set to `false` to not include the "On this page" navigation. + +## Batch updates for TW metadata + +The [`CODEOWNERS`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/CODEOWNERS) +file contains a list of files and the associated technical writers. + +When a merge request contains documentation, the information in the `CODEOWNERS` file determines: + +- The list of users in the **Approvers** section. +- The technical writer that the GitLab Bot pings for community contributions. + +You can use a Rake task to update the `CODEOWNERS` file. + +### Update the `CODEOWNERS` file + +When groups or [TW assignments](https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments) +change, you must update the `CODEOWNERS` file: + +1. Update the [stage and group metadata](#stage-and-group-metadata) for any affected doc pages, if necessary. If there are many changes, you can do this step in a separate MR. +1. Update the [`codeowners.rake`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/tasks/gitlab/tw/codeowners.rake) file with the changes. +1. Go to the root of the `gitlab` repository. +1. Run the Rake task with this command: `bundle exec rake tw:codeowners` +1. Review the changes in the `CODEOWNERS` file. +1. Add and commit all your changes and push your branch up to `origin`. +1. Create a merge request and assign it to a technical writing manager for review. + +When you update the `codeowners.rake` file: + +- To specify multiple writers for a single group, use a space between writer names. Files are assigned to both writers. + + ```ruby + CodeOwnerRule.new('Group Name', '@writer1 @writer2'), + ``` + + - To assign different writers within a group to docs in different directories, use the `path` parameter to specify a directory: + + ```ruby + CodeOwnerRule.new('Group Name', ->(path) { path.start_with?('/doc/user') ? '@writer1' : '@writer2' }), + ``` + + In this example, `writer1` is a code owner for files related to this group that are in `/doc/user`. + For everything else, `writer2` is made code owner. For an example, see [MR 127903](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/127903). + +- For a group that does not have an assigned writer, include the group name in the file and comment out the line: + + ```ruby + # CodeOwnerRule.new('Group Name', ''), + ``` diff --git a/doc/development/documentation/redirects.md b/doc/development/documentation/redirects.md index f080625ea9c..056ef9d3d62 100644 --- a/doc/development/documentation/redirects.md +++ b/doc/development/documentation/redirects.md @@ -1,7 +1,7 @@ --- stage: none group: Documentation Guidelines -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. description: Learn how to contribute to GitLab Documentation. --- diff --git a/doc/development/documentation/restful_api_styleguide.md b/doc/development/documentation/restful_api_styleguide.md index a53330a7e63..66bb662e1c7 100644 --- a/doc/development/documentation/restful_api_styleguide.md +++ b/doc/development/documentation/restful_api_styleguide.md @@ -1,5 +1,5 @@ --- -info: For assistance with this Style Guide page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. stage: none group: unassigned description: 'Writing styles, markup, formatting, and other standards for the GitLab RESTful APIs.' @@ -24,7 +24,7 @@ In the Markdown doc for a resource (AKA endpoint): - Every method must have the REST API request. For example: ```plaintext - GET /projects/:id/repository/branches + GET /api/v4/projects/:id/repository/branches ``` - Every method must have a detailed [description of the attributes](#method-description). @@ -57,7 +57,7 @@ One or two sentence description of what endpoint does. Description of the method. ```plaintext -METHOD /endpoint +METHOD /api/v4/endpoint ``` Supported attributes: diff --git a/doc/development/documentation/review_apps.md b/doc/development/documentation/review_apps.md index 483145d1f44..2402a15e61b 100644 --- a/doc/development/documentation/review_apps.md +++ b/doc/development/documentation/review_apps.md @@ -1,7 +1,7 @@ --- stage: none group: Documentation Guidelines -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. description: Learn how documentation review apps work. --- @@ -44,7 +44,7 @@ the GitLab team to run the job. If you want to know the in-depth details, here's what's really happening: 1. You manually run the `review-docs-deploy` job in a merge request. -1. The job runs the [`scripts/trigger-build.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/scripts/trigger-build.rb) +1. The job downloads and runs the [`scripts/trigger-build.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/scripts/trigger-build.rb) script with the `docs deploy` flag, which triggers the "Triggered from `gitlab-org/gitlab` 'review-docs-deploy' job" pipeline trigger in the `gitlab-org/gitlab-docs` project for the `$DOCS_BRANCH` (defaults to `main`). 1. The preview URL is shown both at the job output and in the merge request @@ -64,6 +64,69 @@ The following GitLab features are used among others: - [Artifacts](../../ci/yaml/index.md#artifacts) - [Merge request pipelines](../../ci/pipelines/merge_request_pipelines.md) +## How to add a new documentation review app + +In case a documentation review app is missing from one of the documentation +projects, you can use the following CI/CD template to add a manually triggered review app: + +```yaml +# Set up documentation review apps +# https://docs.gitlab.com/ee/development/documentation/review_apps.html +.review-docs: + image: ruby:3.1-alpine + needs: [] + before_script: + - gem install gitlab --no-doc + # We need to download the script rather than clone the repo since the + # review-docs-cleanup job will not be able to run when the branch gets + # deleted (when merging the MR). + - apk add --update openssl + - wget https://gitlab.com/gitlab-org/gitlab/-/raw/master/scripts/trigger-build.rb + - chmod 755 trigger-build.rb + variables: + GIT_STRATEGY: none + DOCS_REVIEW_APPS_DOMAIN: docs.gitlab-review.app + # By default, deploy the Review App using the `main` branch of the `gitlab-org/gitlab-docs` project + DOCS_BRANCH: main + when: manual + allow_failure: true + +# Trigger a docs build in gitlab-docs +# Useful to preview the docs changes live +# https://docs.gitlab.com/ee/development/documentation/index.html#previewing-the-changes-live +review-docs-deploy: + extends: + - .review-docs + environment: + name: review-docs/mr-${CI_MERGE_REQUEST_IID} + # DOCS_REVIEW_APPS_DOMAIN and DOCS_GITLAB_REPO_SUFFIX are CI variables + # Discussion: https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/14236/diffs#note_40140693 + auto_stop_in: 2 weeks + url: https://${DOCS_BRANCH}-${DOCS_GITLAB_REPO_SUFFIX}-${CI_MERGE_REQUEST_IID}.${DOCS_REVIEW_APPS_DOMAIN}/${DOCS_GITLAB_REPO_SUFFIX} + on_stop: review-docs-cleanup + script: + - ./trigger-build.rb docs deploy + +# Cleanup remote environment of gitlab-docs +review-docs-cleanup: + extends: + - .review-docs + environment: + name: review-docs/mr-${CI_MERGE_REQUEST_IID} + action: stop + script: + - ./trigger-build.rb docs cleanup +``` + +You may need to add some rules when those jobs run, it depends on the project. +You can find the current implementations: + +- [GitLab](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/docs.gitlab-ci.yml) +- [Omnibus GitLab](https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/ee8699658c8a7d4c635ad503ef0b825ac592dc4b/gitlab-ci-config/gitlab-com.yml#L367-391) +- [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner/-/blob/main/.gitlab/ci/docs.gitlab-ci.yml) +- [GitLab Charts](https://gitlab.com/gitlab-org/charts/gitlab/-/blob/aae7ee8d23a60d6025eec7d1a864ce244f21cd85/.gitlab-ci.yml#L629-679) +- [GitLab Operator](https://gitlab.com/gitlab-org/cloud-native/gitlab-operator/-/blob/5fa29607cf9286b510148a8f5fef7595dca34186/.gitlab-ci.yml#L180-228) + ## Troubleshooting review apps ### `NoSuchKey The specified key does not exist` diff --git a/doc/development/documentation/site_architecture/automation.md b/doc/development/documentation/site_architecture/automation.md index 5b2b02ad97e..75e69e848e8 100644 --- a/doc/development/documentation/site_architecture/automation.md +++ b/doc/development/documentation/site_architecture/automation.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. --- # Automated pages @@ -57,7 +57,7 @@ If you want to automate a page on the docs site: 1. Review [issue 823](https://gitlab.com/gitlab-org/gitlab-docs/-/issues/823) and consider adding feedback there. 1. If that issue does not describe what you need, contact - [the DRI for the docs site backend](https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects). + [the DRI for the docs site backend](https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects). Because automation adds extra complexity and a support burden, we review it on a case-by-case basis. diff --git a/doc/development/documentation/site_architecture/deployment_process.md b/doc/development/documentation/site_architecture/deployment_process.md index 767fdf907d6..7f232ad0c0d 100644 --- a/doc/development/documentation/site_architecture/deployment_process.md +++ b/doc/development/documentation/site_architecture/deployment_process.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. --- # Documentation deployments diff --git a/doc/development/documentation/site_architecture/folder_structure.md b/doc/development/documentation/site_architecture/folder_structure.md index d4a3c856e0a..739c432ee27 100644 --- a/doc/development/documentation/site_architecture/folder_structure.md +++ b/doc/development/documentation/site_architecture/folder_structure.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. --- # Folder structure for documentation diff --git a/doc/development/documentation/site_architecture/global_nav.md b/doc/development/documentation/site_architecture/global_nav.md index 3e0534220a8..0dfe538cfbd 100644 --- a/doc/development/documentation/site_architecture/global_nav.md +++ b/doc/development/documentation/site_architecture/global_nav.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. description: "Learn how GitLab docs' global navigation works and how to add new items." --- diff --git a/doc/development/documentation/site_architecture/index.md b/doc/development/documentation/site_architecture/index.md index 822c7992222..d2de745a8b7 100644 --- a/doc/development/documentation/site_architecture/index.md +++ b/doc/development/documentation/site_architecture/index.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. --- # Documentation site architecture @@ -61,7 +61,7 @@ Then you can use one of these approaches: [to the global navigation](global_nav.md#add-a-navigation-entry), but keep the rest of the documentation in the external repository. The landing page is indexed and searchable on <https://docs.gitlab.com>, but the rest of the documentation is not. - For example, the [GitLab Workflow extension for VS Code](../../../user/project/repository/vscode.md). + For example, the [GitLab Workflow extension for VS Code](../../../editor_extensions/visual_studio_code/index.md). We do not encourage the use of [pages with lists of links](../topic_types/index.md#pages-and-topics-to-avoid), so only use this option if the recommended options are not feasible. @@ -77,3 +77,19 @@ code review. For docs changes in merge requests, whenever a change to files unde is made, Danger Bot leaves a comment with further instructions about the documentation process. This is configured in the `Dangerfile` in the GitLab repository under [/danger/documentation/](https://gitlab.com/gitlab-org/gitlab/-/tree/master/danger/documentation). + +## Request a documentation survey banner + +To reach to a wider audience, you can request +[a survey banner](https://gitlab.com/gitlab-org/gitlab-docs/-/blob/main/doc/maintenance.md#survey-banner). + +Only one banner can exist at any given time. Priority is given based on who +asked for the banner first. + +To request a survey banner: + +1. [Open an issue](https://gitlab.com/gitlab-org/gitlab-docs/-/issues/new?issue[title]=Survey%20banner%20request&issuable_template=Survey%20banner%20request) + in the `gitlab-docs` project and use the "Survey banner request" template. +1. Fill in the details in the issue description. +1. Create the issue and someone from the Technical Writing team will handle your request. +1. When you no longer need the banner, ping the person assigned to the issue and ask them to remove it. diff --git a/doc/development/documentation/styleguide/index.md b/doc/development/documentation/styleguide/index.md index 6158d60a0ba..26660d2eba1 100644 --- a/doc/development/documentation/styleguide/index.md +++ b/doc/development/documentation/styleguide/index.md @@ -1,5 +1,5 @@ --- -info: For assistance with this Style Guide page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. stage: none group: unassigned description: 'Writing styles, markup, formatting, and other standards for GitLab Documentation.' @@ -147,105 +147,61 @@ Use backticks for: - Commands, parameters, and filenames. - Values. For example: "In the **Name** text box, type `test`." -## Metadata - -Each documentation Markdown page contains YAML front matter. -All values in the metadata are treated as strings and are used for the -docs website only. - -### Stage and group metadata - -Each page should have metadata related to the stage and group it -belongs to, as well as an information block. For example: - -```yaml ---- -stage: Example Stage -group: Example Group -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments ---- -``` - -To populate the metadata, include this information: - -- `stage`: The [Stage](https://about.gitlab.com/handbook/product/categories/#devops-stages) - that the majority of the page's content belongs to. -- `group`: The [Group](https://about.gitlab.com/company/team/structure/#product-groups) - that the majority of the page's content belongs to. -- `info`: How to find the Technical Writer associated with the page's stage and - group. +## Language -### Additional metadata +GitLab documentation should be clear and easy to understand. -The following metadata is optional and is not actively maintained. +- Avoid unnecessary words. +- Be clear, concise, and stick to the goal of the topic. +- Write in US English with US grammar. (Tested in [`British.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/British.yml).) -- `description`: A short description of what the page is about. See the Google [Best practices for creating quality meta descriptions](https://developers.google.com/search/docs/appearance/snippet#meta-descriptions) for writing tips. This content can be used in search result snippets and is shown in social media previews. -- `feedback`: Set to `false` to not include the "Help & Feedback" footer. -- `noindex`: Set to `false` to prevent the page from being indexed by search engines. -- `redirect_to`: Used to control redirects. For more information, see [Redirects in GitLab documentation](../redirects.md). -- `searchbar`: Set to `false` to not include the search bar in the page header. -- `toc`: Set to `false` to not include the "On this page" navigation. +### Active voice -### Deprecated metadata +In most cases, text is easier to understand and to translate if you use active voice instead of passive. -The `type` metadata parameter is deprecated but still exists in documentation -pages. You can remove the `type` metadata parameter and its values. +For example, use: -### Batch updates for TW metadata +- The developer writes code for the application. -The [`CODEOWNERS`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/CODEOWNERS) -file contains a list of files and the associated technical writers. +Instead of: -When a merge request contains documentation, the information in the `CODEOWNERS` file determines: +- Application code is written by the developer. -- The list of users in the **Approvers** section. -- The technical writer that the GitLab Bot pings for community contributions. +Sometimes, using `GitLab` as the subject can be awkward. For example, `GitLab exports the report`. +In this case, you can use passive voice instead. For example, `The report is exported`. -You can use a Rake task to update the `CODEOWNERS` file. +### Customer perspective -#### Update the `CODEOWNERS` file +Focus on the functionality and benefits that GitLab brings to customer, +rather than what GitLab has created. -When groups or [TW assignments](https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments) -change, you must update the `CODEOWNERS` file: +For example, use: -1. Update the [stage and group metadata](#stage-and-group-metadata) for any affected doc pages, if necessary. If there are many changes, you can do this step in a separate MR. -1. Update the [`codeowners.rake`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/tasks/gitlab/tw/codeowners.rake) file with the changes. -1. Go to the root of the `gitlab` repository. -1. Run the Rake task with this command: `bundle exec rake tw:codeowners` -1. Review the changes in the `CODEOWNERS` file. -1. Add and commit all your changes and push your branch up to `origin`. -1. Create a merge request and assign it to a technical writing manager for review. +- Use merge requests to compare code in the source and target branches. -When you update the `codeowners.rake` file: +Instead of: -- To specify multiple writers for a single group, use a space between writer names. Files are assigned to both writers. +- GitLab allows you to compare code. +- GitLab created the ability to let you compare code. +- Merge requests let you compare code. - ```ruby - CodeOwnerRule.new('Group Name', '@writer1 @writer2'), - ``` +Words that indicate you are not writing from a customer perspective are +[allow and enable](word_list.md#allow-enable). Try instead to use +[you](word_list.md#you-your-yours) and to speak directly to the user. - - To assign different writers within a group to docs in different directories, use the `path` parameter to specify a directory: +### Building trust - ```ruby - CodeOwnerRule.new('Group Name', ->(path) { path.start_with?('/doc/user') ? '@writer1' : '@writer2' }), - ``` +Product documentation should be focused on providing clear, concise information, +without the addition of sales or marketing text. - In this example, `writer1` is a code owner for files related to this group that are in `/doc/user`. - For everything else, `writer2` is made code owner. For an example, see [MR 127903](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/127903). +- Do not use words like [easily](word_list.md#easily) or [simply](word_list.md#simply-simple). +- Do not use marketing phrases like "This feature will save you time and money." -- For a group that does not have an assigned writer, include the group name in the file and comment out the line: +Instead, focus on facts and achievable goals. Be specific. For example: - ```ruby - # CodeOwnerRule.new('Group Name', ''), - ``` - -## Language - -GitLab documentation should be clear and easy to understand. - -- Avoid unnecessary words. -- Be clear, concise, and stick to the goal of the topic. -- Write in US English with US grammar. (Tested in [`British.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/British.yml).) +- The build time can decrease when you use this feature. +- You can use this feature to save time when you create a project. The API creates the file and you + do not need to manually intervene. ### Capitalization @@ -262,8 +218,6 @@ Use sentence case for topic titles. For example: When referring to specific user interface text, like a button label or menu item, use the same capitalization that's displayed in the user interface. -Standards for this content are listed in the [Pajamas Design System Content section](https://design.gitlab.com/content/punctuation/) -and typically match what's mentioned in this Documentation Style Guide. If you think the user interface text contains style mistakes, create an issue or an MR to propose a change to the user interface text. @@ -350,10 +304,10 @@ Some contractions, however, should be avoided: | Do not use a contraction | Example | Use instead | |-------------------------------|--------------------------------------------------|------------------------------------------------------------------| -| With a proper noun and a verb | The **Container Registry's** a powerful feature. | The **Container Registry** is a powerful feature. | +| With a proper noun and a verb | **Terraform's** a helpful tool. | **Terraform** is a helpful tool. | | To emphasize a negative | **Don't** install X with Y. | **Do not** install X with Y. | | In reference documentation | **Don't** set a limit. | **Do not** set a limit. | -| In error messages | Requests to localhost **aren't** allowed. | Requests to localhost **are not** allowed. | +| In error messages | Requests to localhost **aren't** allowed. | Requests to localhost **are not** allowed. | <!-- vale gitlab.Possessive = YES --> @@ -559,9 +513,13 @@ about styling cURL commands. ## Lists +Use lists to present information in a format that is easier to scan. + +- Make all items in the list parallel. + For example, do not start some bullets with nouns and others with verbs. - Do not use a period if the phrase is not a full sentence. - Use a period after every sentence. Do not use semicolons or commas. -- Majority rules. All items should have the same punctuation. +- Give all items the same punctuation. - Start list items with a capital letter. - Separate the introductory phrase from explanatory text with a colon (`:`). For example: @@ -754,8 +712,20 @@ Instead, follow the [API topic template](../restful_api_styleguide.md#api-topic- ### Footnotes -To indicate a footnote, use the HTML tag `<sup>` with a number. -Put the tag at the end of the sentence or term. For example: +Use footnotes below tables when it's not suitable to include the content in the table +itself. For example, use footnotes when you need to: + +- Provide the same reference information on several table cells. +- Include content that would disrupt the table's layout. + +#### Footnote format + +For each footnote, use the HTML superscript tag `<sup>`. +Put the tag at the end of the sentence or term. + +When you add a footnote, do not re-sort the existing tags in the table. + +For example: ```markdown | App name | Description | @@ -1010,7 +980,9 @@ Guidance for each individual UI element is in [the word list](word_list.md). ### How to write navigation task steps -To be consistent, use these templates when you write navigation steps in a task topic. +To be consistent, use these examples to write navigation steps in a task topic. +Although alternative steps might exist, including items pinned by default, +use these steps instead. To open project settings: @@ -1051,10 +1023,12 @@ To create a group: To open the Admin Area: ```markdown -1. On the left sidebar, select **Search or go to**. -1. Select **Admin Area**. +1. On the left sidebar, at the bottom, select **Admin Area**. +1. Select **Settings > CI/CD**. ``` +You do not need to repeat `On the left sidebar` in your second step. + To open the **Your work** menu item: ```markdown diff --git a/doc/development/documentation/styleguide/word_list.md b/doc/development/documentation/styleguide/word_list.md index 1888d72f991..9d0d59e27c5 100644 --- a/doc/development/documentation/styleguide/word_list.md +++ b/doc/development/documentation/styleguide/word_list.md @@ -1,7 +1,7 @@ --- stage: none group: Documentation Guidelines -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. description: 'Writing styles, markup, formatting, and other standards for GitLab Documentation.' --- @@ -50,7 +50,7 @@ Don't use backticks. Spell out **two-factor authentication** in sentence case for the first use and in topic titles, and **2FA** thereafter. If the first word in a sentence, do not capitalize `factor` or `authentication`. For example: -- Two-factor authentication (2FA) helps secure your account. Set up 2FA when you first log in. +- Two-factor authentication (2FA) helps secure your account. Set up 2FA when you first sign in. ## above @@ -77,28 +77,15 @@ When you create a user, you choose an access level: **Regular**, **Auditor**, or Capitalize these words when you refer to the UI. Otherwise use lowercase. -## active voice - -Use active voice instead of passive. - -Use: - -- The contributor writes the documentation. - -Instead of: - -- The documentation is written by contributors. - -NOTE: -If you can add the phrase "by zombies" to the phrase, -the construction is passive. For example, `The button is selected by zombies` -is passive. `Zombies select the button` is active. - ## Admin Area Use title case for **Admin Area**. This area of the UI says **Admin Area** at the top of the page and on the menu. +## Admin Mode + +Use title case for **Admin Mode**. The UI uses title case. + ## administrator Use **administrator access** instead of **admin** when talking about a user's access level. @@ -235,8 +222,8 @@ Try to avoid **below** when referring to an example or table in a documentation Use uppercase for **Beta**. For example: **The XYZ feature is in Beta.** or **This Beta release is ready to test.** -You might also want to link to [this section](../../../policy/experiment-beta-support.md#beta) -in the handbook when writing about Beta features. +You might also want to link to [this topic](../../../policy/experiment-beta-support.md#beta) +when writing about Beta features. ## blacklist @@ -307,9 +294,18 @@ Use **check out** as a verb. For the Git command, use `checkout`. - Use `git checkout` to check out a branch locally. - Check out the files you want to edit. +## CI, CD + +When talking about GitLab features, use **CI/CD**. Do not use **CI** or **CD** alone. + ## CI/CD -CI/CD is always uppercase. No need to spell it out on first use. +**CI/CD** is always uppercase. No need to spell it out on first use. + +You can omit **CI/CD** when the context is clear, especially after the first use. For example: + +- Test your code in a **CI/CD pipeline**. Configure the **pipeline** to run for merge requests. +- Store the value in a **CI/CD variable**. Set the **variable** to masked. ## CI/CD minutes @@ -384,6 +380,18 @@ Use **compute minutes** instead of these (or similar) terms: For more information, see [epic 2150](https://gitlab.com/groups/gitlab-com/-/epics/2150). +## configuration + +When you update a collection of settings, call it a **configuration**. + +## configure + +Use **configure** after a feature or product has been [set up](#setup-set-up). +For example: + +1. Set up your installation. +1. Configure your installation. + ## confirmation dialog Use **confirmation dialog** to describe the dialog that asks you to confirm an action. For example: @@ -394,7 +402,7 @@ Do not use **confirmation box** or **confirmation dialog box**. See also [**dial ## container registry -When documenting the GitLab container registry features and functionality, use lower case. +When documenting the GitLab container registry features and functionality, use lowercase. Use: @@ -505,6 +513,13 @@ To be more upbeat and precise, do not use **downgrade**. Focus instead on the ac - For changing to earlier GitLab versions, use [**roll back**](#roll-back). - For changing to lower GitLab tiers, use **change the subscription tier**. +## download + +Use **download** to describe saving data to a user's device. For details, see +[the Microsoft style guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/d/download). + +Do not confuse download with [export](#export). + ## dropdown list Use **dropdown list** to refer to the UI element. Do not use **dropdown** without **list** after it. @@ -611,14 +626,41 @@ Use **expand** instead of **open** when you are talking about expanding or colla Use uppercase for **Experiment**. For example: **The XYZ feature is an Experiment.** or **This Experiment is ready to test.** -You might also want to link to [this section](../../../policy/experiment-beta-support.md#experiment) -in the handbook when writing about Experiment features. +You might also want to link to [this topic](../../../policy/experiment-beta-support.md#experiment) +when writing about Experiment features. + +## export + +Use **export** to indicate translating raw data, +which is not represented by a file in GitLab, into a standard file format. + +You can differentiate **export** from **download** because: + +- Often, you can use export options to change the output. +- Exported data is not necessarily downloaded to a user's device. + +For example: + +- Export the contents of your report to CSV format. + +Do not confuse with [download](#download). ## FAQ We want users to find information quickly, and they rarely search for the term **FAQ**. Information in FAQs belongs with other similar information, under an easily searchable topic title. +## feature + +You should rarely need to use the word **feature**. Instead, explain what GitLab does. +For example, use: + +- Use merge requests to incorporate changes into the target branch. + +Instead of: + +- Use the merge request feature to incorporate changes into the target branch. + ## field Use **text box** instead of **field** or **box**. @@ -641,6 +683,12 @@ of the fields at once. For example: Learn more about [documenting multiple fields at once](index.md#documenting-multiple-fields-at-once). +## file name + +Use two words for **file name**. When using file name as a variable, use `<file_name>`. + +([Vale](../testing.md#vale) rule: [`SubstitutionWarning.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/SubstitutionWarning.yml)) + ## filter When you are viewing a list of items, like issues or merge requests, you filter the list by @@ -655,14 +703,19 @@ Do not use **foo** in product documentation. You can use it in our API and contr ## fork A **fork** is a project that was created from a **upstream project** by using the -[forking process](../../../topics/git/terminology.md#fork). +forking process. The **upstream project** (also known as the **source project**) and the **fork** have a **fork relationship** and are **linked**. -If the [**fork relationship** is removed](../../../user/project/repository/forking_workflow.md#unlink-a-fork), the +If the **fork relationship** is removed, the **fork** is **unlinked** from the **upstream project**. +## full screen + +Use two words for **full screen**. +([Vale](../testing.md#vale) rule: [`SubstitutionWarning.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/SubstitutionWarning.yml)) + ## future tense When possible, use present tense instead of future tense. For example, use **after you execute this command, GitLab displays the result** instead of **after you execute this command, GitLab will display the result**. ([Vale](../testing.md#vale) rule: [`FutureTense.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/FutureTense.yml)) @@ -698,7 +751,8 @@ Do not use **Dedicated** by itself. Always use **GitLab Dedicated**. Do not use **Duo** by itself. Always use **GitLab Duo**. -On first use on a page, use **GitLab Duo `<featurename>`**. For example: +On first use on a page, use **GitLab Duo `<featurename>`**. As of Dec, 2023, +the following are the names of GitLab Duo features: - GitLab Duo Chat - GitLab Duo Code Suggestions @@ -709,6 +763,7 @@ On first use on a page, use **GitLab Duo `<featurename>`**. For example: - GitLab Duo Code review summary - GitLab Duo Code explanation - GitLab Duo Vulnerability summary +- GitLab Duo Vulnerability resolution - GitLab Duo Test generation - GitLab Duo Git suggestions - GitLab Duo Root cause analysis @@ -966,7 +1021,15 @@ Do not use **limitations**. Use **known issues** instead. ## log in, log on -Do not use **log in** or **log on**. Use [sign in](#sign-in-sign-in) instead. If the user interface has **Log in**, you can use it. +Do not use: + +- **log in**. +- **log on**. +- **login** + +Use [sign in](#sign-in-sign-in) instead. + +However, if the user interface has **Log in**, you should match the UI. ## logged-in user, logged in user @@ -985,6 +1048,14 @@ Instead of: - In GitLab 14.1 and lower. - In GitLab 14.1 and older. +## machine learning + +Use lowercase for **machine learning**. + +When machine learning is used as an adjective, like **a machine learning model**, +do not hyphenate. While a hyphen might be more grammatically correct, we risk +becoming inconsistent if we try to be more precise. + ## Maintainer When writing about the Maintainer role: @@ -1252,9 +1323,14 @@ Do not use bold. Do not use **Owner permissions**. A user who is assigned the Owner role has a set of associated permissions. An Owner is the highest role a user can have. -## Package Registry +## package registry -Use title case for the GitLab Package Registry. +When documenting the GitLab package registry features and functionality, use lowercase. + +Use: + +- The GitLab package registry supports A, B, and C. +- You can publish a package to your project's package registry. ## page @@ -1289,10 +1365,16 @@ see the [Microsoft Style Guide](https://learn.microsoft.com/en-us/style-guide/a- Use **Premium**, in uppercase, for the subscription tier. When you refer to **Premium** in the context of other subscription tiers, follow [the subscription tier](#subscription-tier) guidance. +## preferences + +Use **preferences** to describe user-specific, system-level settings like theme and layout. + ## prerequisites Use **prerequisites** when documenting the tasks that must be completed or the conditions that must be met before a user can complete a task. Do not use **requirements**. +**Prerequisites** must always be plural, even if the list includes only one item. + For more information, see [the task topic type](../topic_types/task.md). For tutorial page types, use [**before you begin**](#before-you-begin) instead. @@ -1367,6 +1449,17 @@ Do not use **Reporter permissions**. A user who is assigned the Reporter role ha Use title case for **Repository Mirroring**. +## resolution, resolve + +Use **resolution** when the troubleshooting solution fixes the issue permanently. +A resolution usually involves file and code changes to correct the problem. +For example: + +- To resolve this issue, update the `.gitlab-ci.yml` file. +- One resolution is to update the `.gitlab-ci.yml` file. + +See also [workaround](#workaround). + ## requirements When documenting the tasks that must be completed or the conditions that must be met before a user can complete the steps: @@ -1376,6 +1469,10 @@ When documenting the tasks that must be completed or the conditions that must be Do not use **requirements**. +## reset + +Use **reset** to describe the action associated with resetting an item to a new state. + ## respectively Avoid **respectively** and be more precise instead. @@ -1389,6 +1486,10 @@ Instead of: - Select **Create user** or **Save changes** if you created a new user or edited an existing one respectively. +## restore + +See the [Microsoft Style Guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/r/restore) for guidance on **restore**. + ## review app Use lowercase for **review app**. @@ -1508,18 +1609,37 @@ Use **setup** as a noun, and **set up** as a verb. For example: - Your remote office setup is amazing. - To set up your remote office correctly, consider the ergonomics of your work area. +Do not confuse **set up** with [**configure**](#configure). +**Set up** implies that it's the first time you've done something. For example: + +1. Set up your installation. +1. Configure your installation. + +## settings + +A **setting** changes the default behavior of the product. A **setting** consists of a key/value pair, +typically represented by a label with one or more options. + ## sign in, sign-in -Use **sign in** or **sign in to** as a verb to describe the action of signing in. +To describe the action of signing in, use: -Do not use **sign on** or **sign into**, or **log on**, **log in**, or **log into**. +- **sign in**. +- **sign in to** as a verb. For example: Use your password to sign in to GitLab. -If the user interface has different words, use those. +You can also use: + +- **sign-in** as a noun or adjective. For example: **sign-in page** or + **sign-in restrictions**. +- **single sign-on**. + +Do not use: -You can use **sign-in** as a noun or adjective. For example, **sign-in page** or -**sign-in restrictions**. +- **sign on**. +- **sign into**. +- [**log on**, **log in**, or **log into**](#log-in-log-on). -You can use **single sign-on**. +If the user interface has different words, you can use those. ## sign up @@ -1709,6 +1829,10 @@ See also [**enter**](#enter). Use **Ultimate**, in uppercase, for the subscription tier. When you refer to **Ultimate** in the context of other subscription tiers, follow [the subscription tier](#subscription-tier) guidance. +## undo + +See the [Microsoft Style Guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/u/undo) for guidance on **undo**. + ## units of measurement Use a space between the number and the unit of measurement. For example, **128 GB**. @@ -1762,6 +1886,19 @@ Do not use **useful**. If the user doesn't find the process to be useful, we los You create a **user account**. The user account has an [access level](#access-level). When you add a **user account** to a group or project, the user account becomes a **member**. +## using + +Avoid **using** in most cases. It hides the subject and makes the phrase more difficult to translate. +Use **by using**, **that use**, or re-write the sentence. + +For example: + +- Instead of: The files using storage... +- Use: The files that use storage... + +- Instead of: Change directories using the command line. +- Use: Change directories by using the command line. Or even better: To change directories, use the command line. + ## utilize Do not use **utilize**. Use **use** instead. It's more succinct and easier for non-native English speakers to understand. @@ -1777,6 +1914,13 @@ Thereafter, use **Value stream forecasting** by itself. Do not use Latin abbreviations. Use **with**, **through**, or **by using** instead. ([Vale](../testing.md#vale) rule: [`LatinTerms.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/LatinTerms.yml)) +## Vulnerability resolution + +Use sentence case for **Vulnerability resolution**. + +On first mention on a page, use **GitLab Duo Vulnerability resolution**. +Thereafter, use **Vulnerability resolution** by itself. + ## Vulnerability summary Use sentence case for **Vulnerability summary**. @@ -1798,6 +1942,16 @@ Instead of: ([Vale](../testing.md#vale) rule: [`SubstitutionSuggestions.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/SubstitutionSuggestions.yml)) +## workaround + +Use **workaround** when the troubleshooting solution is a temporary fix. +A workaround is usually an immediate fix and might have ongoing issues. +For example: + +- The workaround is to temporarily pin your template to the deprecated version. + +See also [resolution](#resolution-resolve). + ## while Use **while** to refer only to something occurring in time. For example, diff --git a/doc/development/documentation/testing.md b/doc/development/documentation/testing.md index 414c2bede7b..62df2395fbb 100644 --- a/doc/development/documentation/testing.md +++ b/doc/development/documentation/testing.md @@ -1,7 +1,7 @@ --- stage: none group: Documentation Guidelines -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. description: Learn how to contribute to GitLab Documentation. --- @@ -629,8 +629,7 @@ You can disable a specific Vale linting rule or all Vale linting rules for any p document: - To disable a specific rule, add a `<!-- vale gitlab.rulename = NO -->` tag before the text, and a - `<!-- vale gitlab.rulename = YES -->` tag after the text, replacing `rulename` with the filename - of a test in the + `<!-- vale gitlab.rulename = YES -->` tag after the text, replacing `rulename` with the file name of a test in the [GitLab styles](https://gitlab.com/gitlab-org/gitlab/-/tree/master/doc/.linting/vale/styles/gitlab) directory. - To disable all Vale linting rules, add a `<!-- vale off -->` tag before the text, and a diff --git a/doc/development/documentation/topic_types/concept.md b/doc/development/documentation/topic_types/concept.md index 50220b9bebe..fe9a6f8af71 100644 --- a/doc/development/documentation/topic_types/concept.md +++ b/doc/development/documentation/topic_types/concept.md @@ -1,7 +1,7 @@ --- stage: none group: Documentation Guidelines -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. --- # Concept topic type @@ -19,6 +19,8 @@ Don't tell them **how** to do this thing. Tell them **what it is**. If you start describing another concept, start a new concept and link to it. +## Format + Concepts should be in this format: ```markdown diff --git a/doc/development/documentation/topic_types/get_started.md b/doc/development/documentation/topic_types/get_started.md new file mode 100644 index 00000000000..b43f9a1b71c --- /dev/null +++ b/doc/development/documentation/topic_types/get_started.md @@ -0,0 +1,92 @@ +--- +stage: none +group: Style Guide +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. +--- + +# Get started page type + +A **Get started** page is meant to introduce high-level concepts for a broad feature area. +While a specific feature might be defined in the feature documentation, +a **Get started** page is meant to give an introduction to a set of concepts. +When you group the concepts together, you help the user see how they fit together. + +A **Get started** page should familiarize the reader with terms and then quickly +point them to actions they can take to get started. Hopefully the actions are +task-based, but the next step can also be to learn more. + +## When to use a Get started page + +A **Get started** page should be used only for a larger concept, +like CI/CD or security. In general, we describe features in concept topics. +However, if you find you want to explain how multiple concepts fit together, +then a **Get started** page might be what you need. + +To determine if a **Get started** page makes sense, make a list +of the common terms you expect to include. If you have more than four or five, +then this page type might make sense. + +A **Get started** page is different from a tutorial. It's conceptual, while +a tutorial helps the user achieve a task. A **Get started** page should point +to tutorials, however, because tutorials are a great way for a user to get started. + +## Format + +Get started pages should be in this format: + +```markdown +# Get started with abc + +Abc is a thing you use to do xyz. You might use it when you need to blah, +and it can be helpful for etc. + +## Common terms + +If you're new to abc, start by reviewing some of the most commonly used terms. + +### First term + +This thing is this. Describe what it is, not how to do it. + +**Get started:** + +- [Create your first abc](LINK). +- [Learn more about abc](LINK). + +### Second term + +This thing is this. Describe what it is, not how to do it. + +**Get started:** + +- [Create your first abc](LINK). +- [Learn more about abc](LINK). + +## Videos + +- [Video 1](LINK). +- [Video 2](LINK). + +## Related topics + +- [Link 1](LINK). +- [Link 2](LINK). +``` + +- Follow [the video guidance](../styleguide/index.md#link-to-video) + for the links in the Video topic. +- Do not use links inline with content (as part of sentences). + Use them where links are specified only. +- The terms described on this page can exist elsewhere in the docs. + However, the term descriptions on this page should be relatively brief. + +## Get started page titles + +For the title, use `Get started with topic_name`. + +For the left nav, use `Getting started`. + +## Example + +For an example of the Get started page type, +see [Get started with GitLab CI/CD](../../../ci/index.md). diff --git a/doc/development/documentation/topic_types/glossary.md b/doc/development/documentation/topic_types/glossary.md index f6137953cb0..ee72ad76855 100644 --- a/doc/development/documentation/topic_types/glossary.md +++ b/doc/development/documentation/topic_types/glossary.md @@ -1,7 +1,7 @@ --- stage: none group: Documentation Guidelines -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. --- # Glossary topic type diff --git a/doc/development/documentation/topic_types/index.md b/doc/development/documentation/topic_types/index.md index 10e7e3d2014..b4f28ad46d3 100644 --- a/doc/development/documentation/topic_types/index.md +++ b/doc/development/documentation/topic_types/index.md @@ -1,7 +1,7 @@ --- stage: none group: Documentation Guidelines -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. --- # Documentation topic types (CTRT) @@ -24,6 +24,7 @@ The acronym refers to the first letter of each topic type. In addition to the four primary topic types, you can use the following: - Page type: [Tutorials](tutorial.md) +- Page type: [Get started](get_started.md) - Topic type: [Related topics](#related-topics) - Page or topic type: [Glossaries](glossary.md) @@ -36,7 +37,6 @@ You should avoid: - Topics that have one or two sentences only. In these cases: - Incorporate the information in another topic. - If the sentence links to another page, use a [Related topics](#related-topics) link instead. -- Get started topics. To document a procedure for a single feature, use a [task](task.md). For a set of steps, use a [tutorial](tutorial.md). ## Topic title guidelines diff --git a/doc/development/documentation/topic_types/reference.md b/doc/development/documentation/topic_types/reference.md index ef2ca2f791d..f9db71aa9f8 100644 --- a/doc/development/documentation/topic_types/reference.md +++ b/doc/development/documentation/topic_types/reference.md @@ -1,7 +1,7 @@ --- stage: none group: Documentation Guidelines -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. --- # Reference topic type @@ -9,6 +9,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w Reference information should be in an easily-scannable format, like a table or list. It's similar to a dictionary or encyclopedia entry. +## Format + +Reference topics should be in this format: + ```markdown # Title (a noun, like "Pipeline settings" or "Administrator options") diff --git a/doc/development/documentation/topic_types/task.md b/doc/development/documentation/topic_types/task.md index a6e4c01505d..2130a28c56c 100644 --- a/doc/development/documentation/topic_types/task.md +++ b/doc/development/documentation/topic_types/task.md @@ -1,13 +1,15 @@ --- stage: none group: Documentation Guidelines -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. --- # Task topic type A task gives instructions for how to complete a procedure. +## Format + Tasks should be in this format: ```markdown @@ -153,6 +155,8 @@ As a best practice, if the task requires the user to have a role other than Gues put the minimum role in the prerequisites. See [the Word list](../styleguide/word_list.md) for how to write the phrase for each role. +`Prerequisites` must always be plural, even if the list includes only one item. + ## Related topics - [How to write task steps](../styleguide/index.md#navigation) diff --git a/doc/development/documentation/topic_types/troubleshooting.md b/doc/development/documentation/topic_types/troubleshooting.md index a2fe6f8ca2d..aee5bd1377c 100644 --- a/doc/development/documentation/topic_types/troubleshooting.md +++ b/doc/development/documentation/topic_types/troubleshooting.md @@ -1,15 +1,17 @@ --- stage: none group: Documentation Guidelines -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. --- # Troubleshooting topic type Troubleshooting topics should be the final topics on a page. -If a page has more than five troubleshooting topics, put the content on a separate page that has troubleshooting information exclusively. Name the page `Troubleshooting <feature>` -and in the left nav, use the word `Troubleshooting` only. +If a page has five or more troubleshooting topics, put those topics on a separate page. + +- Name the page `Troubleshooting <feature>`. +- In the left nav, use the word `Troubleshooting` only. ## What type of troubleshooting information to include @@ -25,7 +27,7 @@ If you think you have an exception to this rule, contact the Technical Writing t GitLab Support maintains their own [troubleshooting content](../../../administration/troubleshooting/index.md). -## Troubleshooting topic types +## Format Troubleshooting can be one of three types: introductory, task, or reference. @@ -47,7 +49,7 @@ For example, "Run debug tools" or "Verify syntax." ### Troubleshooting reference -This topic includes the error message. For example: +This topic includes the error message. To be consistent, use **workaround** for temporary solutions and **resolution** and **resolve** for permanent solutions. For example: ```markdown ### The error message or a description of it @@ -59,9 +61,11 @@ This issue occurs when... The workaround is... ``` -If multiple causes or workarounds exist, consider putting them into a table format. +If multiple causes or solutions exist, consider putting them into a table format. If you use the exact error message, surround it in backticks so it's styled as code. +For more guidance on solution types, see [workaround](../../documentation/styleguide/word_list.md#workaround) and [resolution, resolve](../../documentation/styleguide/word_list.md#resolution-resolve). + ## Troubleshooting topic titles For the title of a **Troubleshooting reference** topic: diff --git a/doc/development/documentation/topic_types/tutorial.md b/doc/development/documentation/topic_types/tutorial.md index 50976149cf8..c5eeceac4cb 100644 --- a/doc/development/documentation/topic_types/tutorial.md +++ b/doc/development/documentation/topic_types/tutorial.md @@ -1,7 +1,7 @@ --- stage: none group: Documentation Guidelines -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. --- # Tutorial page type diff --git a/doc/development/documentation/versions.md b/doc/development/documentation/versions.md index bd83ed7eff2..22fda69094b 100644 --- a/doc/development/documentation/versions.md +++ b/doc/development/documentation/versions.md @@ -1,5 +1,5 @@ --- -info: For assistance with this Style Guide page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. stage: none group: unassigned description: 'Writing styles, markup, formatting, and other standards for GitLab Documentation.' @@ -69,6 +69,16 @@ If a feature is moved to another subscription tier, use `moved`: > - [Moved](<link-to-issue>) from GitLab Premium to GitLab Free in 12.0. ``` +#### Changing the feature status + +If the feature status changes, use `changed`: + +```markdown +> - [Introduced](<link-to-issue>) as an [Experiment](../../policy/experiment-beta-support.md) in GitLab 15.7. +> - [Changed](<link-to-issue>) to Beta in GitLab 16.0. +> - [Changed](<link-to-issue>) to Generally Available in GitLab 16.3. +``` + #### Features introduced behind feature flags When features are introduced behind feature flags, you must add details about the feature flag to the documentation. @@ -155,7 +165,7 @@ To remove a page: --- stage: Data Stores group: Global Search - info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments + info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments remove_date: '2022-08-02' redirect_to: '../newpath/to/file/index.md' --- diff --git a/doc/development/documentation/workflow.md b/doc/development/documentation/workflow.md index 5c99f5c48df..965559bd217 100644 --- a/doc/development/documentation/workflow.md +++ b/doc/development/documentation/workflow.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: For assistance with this Style Guide page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. --- # Documentation workflow @@ -95,7 +95,7 @@ For these reasons, do not add AI-generated content to the documentation. ## Related topics - [Reviews and levels of edit](https://about.gitlab.com/handbook/product/ux/technical-writing/#reviews) -- [Technical writing assignments](https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments) +- [Technical writing assignments](https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments) - The [Style Guide](styleguide/index.md) - The [Word list](styleguide/word_list.md) - The [Markdown Guide](https://about.gitlab.com/handbook/markdown-guide/) diff --git a/doc/development/ee_features.md b/doc/development/ee_features.md index d05249f3d3f..08045675295 100644 --- a/doc/development/ee_features.md +++ b/doc/development/ee_features.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Guidelines for implementing Enterprise Edition features @@ -133,16 +133,14 @@ version of the product: 1. Enable **Allow use of licensed EE features** to make licensed EE features available to projects only if the project namespace's plan includes the feature. - 1. On the left sidebar, select **Search or go to**. - 1. Select **Admin Area**. + 1. On the left sidebar, at the bottom, select **Admin Area**. 1. On the left sidebar, select **Settings > General**. 1. Expand **Account and limit**. 1. Select the **Allow use of licensed EE features** checkbox. 1. Select **Save changes**. 1. Ensure the group you want to test the EE feature for is actually using an EE plan: - 1. On the left sidebar, select **Search or go to**. - 1. Select **Admin Area**. + 1. On the left sidebar, at the bottom, select **Admin Area**. 1. On the left sidebar, select **Overview > Groups**. 1. Identify the group you want to modify, and select **Edit**. 1. Scroll to **Permissions and group features**. For **Plan**, select `Ultimate`. diff --git a/doc/development/emails.md b/doc/development/emails.md index bf32b71f2b7..c69eade7cb6 100644 --- a/doc/development/emails.md +++ b/doc/development/emails.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Working with email in development diff --git a/doc/development/event_store.md b/doc/development/event_store.md index 918da8fb738..0ffcf8704ff 100644 --- a/doc/development/event_store.md +++ b/doc/development/event_store.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # GitLab EventStore @@ -184,7 +184,7 @@ Changes to the schema require multiple rollouts. While the new version is being - Existing subscribers can consume events using the old version. - Events get persisted in the Sidekiq queue as job arguments, so we could have 2 versions of the schema during deployments. -As changing the schema ultimately impacts the Sidekiq arguments, please refer to our +As changing the schema ultimately impacts the Sidekiq arguments, refer to our [Sidekiq style guide](sidekiq/compatibility_across_updates.md#changing-the-arguments-for-a-worker) with regards to multiple rollouts. #### Add properties @@ -277,6 +277,10 @@ declaring the subscription in `ee/lib/ee/gitlab/event_store.rb`. Subscriptions are stored in memory when the Rails app is loaded and they are immediately frozen. It's not possible to modify subscriptions at runtime. +NOTE: +Before creating a subscription, make sure that the worker is already deployed to GitLab.com as the +newly introduced workers are [not compatible with canary deployments](sidekiq/compatibility_across_updates.md#adding-new-workers). + ### Conditional dispatch of events A subscription can specify a condition when to accept an event: @@ -315,6 +319,26 @@ on the subscriber Sidekiq worker, instead of `perform_async`. This technique is useful when publishing many events and leverage the Sidekiq deduplication. +### Publishing group of events + +In some scenarios we publish multiple events of same type in a single business transaction. +This puts additional load to Sidekiq by invoking a job for each event. In such cases, we can +publish a group of events by calling `Gitlab::EventStore.publish_group`. This method accepts an +array of events of similar type. By default the subscriber worker receives a group of max 10 events, +but this can be configured by defining `group_size` parameter while creating the subscription. +The number of published events are dispatched to the subscriber in batches based on the +configured `group_size`. If the number of groups exceeds 100, we schedule each group with a delay +of 10 seconds, to reduce the load on Sidekiq. + +```ruby +store.subscribe ::Security::RefreshProjectPoliciesWorker, + to: ::ProjectAuthorizations::AuthorizationsChangedEvent, + delay: 1.minute, + group_size: 25 +``` + +The `handle_event` method in the subscriber worker is called for each of the events in the group. + ## Testing ### Testing the publisher diff --git a/doc/development/experiment_guide/experiment_code_reviews.md b/doc/development/experiment_guide/experiment_code_reviews.md index 7242acfbdcf..53016f0f7c6 100644 --- a/doc/development/experiment_guide/experiment_code_reviews.md +++ b/doc/development/experiment_guide/experiment_code_reviews.md @@ -1,7 +1,7 @@ --- stage: Growth group: Acquisition -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Experiment code reviews diff --git a/doc/development/experiment_guide/experiment_rollout.md b/doc/development/experiment_guide/experiment_rollout.md index b2e1b1168d3..d9e8b747a50 100644 --- a/doc/development/experiment_guide/experiment_rollout.md +++ b/doc/development/experiment_guide/experiment_rollout.md @@ -1,7 +1,7 @@ --- stage: Growth group: Acquisition -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Experiment rollouts and feature flags diff --git a/doc/development/experiment_guide/implementing_experiments.md b/doc/development/experiment_guide/implementing_experiments.md index 15b8f8fc192..d38bca41fc3 100644 --- a/doc/development/experiment_guide/implementing_experiments.md +++ b/doc/development/experiment_guide/implementing_experiments.md @@ -1,7 +1,7 @@ --- stage: Growth group: Acquisition -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Implementing an A/B/n experiment diff --git a/doc/development/experiment_guide/index.md b/doc/development/experiment_guide/index.md index 5e68921510e..714d5700de4 100644 --- a/doc/development/experiment_guide/index.md +++ b/doc/development/experiment_guide/index.md @@ -1,7 +1,7 @@ --- stage: Growth group: Acquisition -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Experiment Guide diff --git a/doc/development/experiment_guide/testing_experiments.md b/doc/development/experiment_guide/testing_experiments.md index 2e88029a27f..6e4fa9f528e 100644 --- a/doc/development/experiment_guide/testing_experiments.md +++ b/doc/development/experiment_guide/testing_experiments.md @@ -1,7 +1,7 @@ --- stage: Growth group: Acquisition -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Testing experiments diff --git a/doc/development/export_csv.md b/doc/development/export_csv.md index ce0a6e026ff..23f08802893 100644 --- a/doc/development/export_csv.md +++ b/doc/development/export_csv.md @@ -1,7 +1,7 @@ --- stage: Manage group: Import and Integrate -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review.ment-guidelines-review.ment-guidelines-review.ment-guidelines-review. --- # Export to CSV diff --git a/doc/development/fe_guide/accessibility/automated_testing.md b/doc/development/fe_guide/accessibility/automated_testing.md index 2c0d598dc58..b7f50452802 100644 --- a/doc/development/fe_guide/accessibility/automated_testing.md +++ b/doc/development/fe_guide/accessibility/automated_testing.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Automated accessibility testing diff --git a/doc/development/fe_guide/accessibility/best_practices.md b/doc/development/fe_guide/accessibility/best_practices.md index 37c28f99116..d9186278643 100644 --- a/doc/development/fe_guide/accessibility/best_practices.md +++ b/doc/development/fe_guide/accessibility/best_practices.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Accessibility best practices diff --git a/doc/development/fe_guide/accessibility/index.md b/doc/development/fe_guide/accessibility/index.md index 5274fa644e1..fef97aadaa5 100644 --- a/doc/development/fe_guide/accessibility/index.md +++ b/doc/development/fe_guide/accessibility/index.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Accessibility diff --git a/doc/development/fe_guide/architecture.md b/doc/development/fe_guide/architecture.md index 810d9af2de7..971c527ef68 100644 --- a/doc/development/fe_guide/architecture.md +++ b/doc/development/fe_guide/architecture.md @@ -1,16 +1,65 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Architecture -When building new features, consider reaching out to relevant stakeholders as early as possible in the process. +At GitLab, there are no dedicated "software architects". Everyone is encouraged to make their own decisions and document them appropriately. To know how or where to document these decisions, read on. -Architectural decisions should be accessible to everyone. Document -them in the relevant Merge Request discussions or by updating our documentation -when appropriate by adding an entry to this section. +## Documenting decisions + +When building new features, consider the scope and scale of what you are about to build. Depending on the answer, there are several tools or processes that could support your endeavor. We aim to keep the process of building features as efficient as possible. As a general rule, use the simplest process possible unless you need the additional support and structure of more time consuming solutions. + +### Merge requests + +When a change impacts is limited within a group or has a single contributor, the smallest possible documentation of architecture decisions is a commit and by extension a merge request (MR). MRs or commits can still be referenced even after they are merged, so it is vital to leave a good description, comments and commit messages to explain certain decisions in case it needs to be referenced later. Even a MR that is intended to be reviewed within a group should contain all relevant decision-making explicitly. + +### Architectural Issue + +When a unit of work starts to get big enough that it might impact an entire group's direction, it may be a good idea to create an architecture issue to discuss the technical direction. This process is informal and has no official requirements. Create an issue within the GitLab project where you can propose a plan for the work to be done and invite collaborators to refine the proposal with you. + +This structure allows the group to think through a proposed change, gather feedback and iterate. It also allows them to use the issue as a source of truth rather than a comments thread on the feature issue or the MRs themselves. Consider adding some kind of visual support (like a schema) to facilitate the discussion. For example, you can reference this [architectural issue of the CI/CD Catalog](https://gitlab.com/gitlab-org/gitlab/-/issues/393225). + +### Design Documents + +When the work ahead may affect more than a single group, stage or potentially an entirement department (for example, all of the Frontend team) then it is likely that there is need for a [Design Document](https://about.gitlab.com/handbook/engineering/architecture/workflow/). + +This is well documented in the handbook, but to touch on it shortly, it is **the best way** to propose large changes and gather the required feedback and support to move forward. These documents are version controlled, keep evolving with time and are a great way to share a complex understanding across the entire organization. They also require a coach, which is a great way to involve someone with a lot of experience with larger changes. This process is shared across all engineering departments and is owned by the CTO. + +To see all Design Documents, you can check the [Architecture at GitLab page](../../architecture/index.md) + +### Frontend RFCs (deprecated) + +In the past, we had a [Frontend RFC project](https://gitlab.com/gitlab-org/frontend/rfcs) which goal was to propose larger changes and get opinions from the entire department. This project is no longer used for a couple of reasons: + +1. Issues created in this project had a very low participation rate (less than 20%) +1. Controversial issues would stall with no clear way to resolve them +1. Issues that were completed often did not need a RFC in the first place (small issues) +1. Changes were often proposed "naively" without clear time and resource allocation + +In most instances where we would have created a RFC, a Design Document can be used instead as it will have it's own RFC attached to it. This makes the conversation centered around the technical design and RFCs are just a way to further the completion of the design. + +### Entry in the Frontend documentation + +Adding an architecture section to the docs is a way to tell frontend engineers how to use or build upon an existing architecture. Use it to help "onboard" engineers to a part of the application that may not be self-evident. Try to avoid documenting your group's architecture here if it has no impact on other teams. + +### Which to choose? + +As a general rule, the wider the scope of your change, the more likely it is that you and your team would benefit from a Design Document. Also consider whether your change is a true two-way door decision: changes that can easily be reverted require less thinking ahead than those that cannot. + +Work that can be achieved within a single milestone probably only needs Merge requests. Work that may take several milestone to complete, but where you are the only DRI is probably also easier done through MRs. + +When multiple DRIs are involved, ask yourself if the work ahead is clear for all of you. If the work you do is complex and affects each others, consider gathering technical feedback from your team before you start working on an Architectural issue. Write a clear proposal, involve all stakeholders early and keep yourselves accountable to the decisions made on the issue. + +Very small changes may have a very broad impact. For example, a change to any ESLint rule will impact all of engineering, but might not require a Design Document. Consider sending your proposal through Slack to gauge interest ("Should we enable X rule?") and then simply create a MR. Finally, share widely to the appropriate channels to gather feedback. + +For recommending certain code patterns in our documentation, you can write the MR that apply your proposed change, share it broadly with the department and if no strong objections are raised, merge your change. This is more efficient than RFCs because of the bias for action, while also gathering all the feedback necessary for everyone to feel included. + +If you'd like to propose a major change to the technological stack (Vue to React, JavaScript to TypeScript, etc.), start by reaching out on Slack to gauge interest. Always ask yourself whether or not the problems that you see can be fixed from our current tech stack, as we should always try to fix our problems with the tools we already have. Other departments, such as Backend and QA, do not have a clear process to propose technological changes either. That is because these changes would require huge investements from the company and probably cannot be decided without involving high-ranking executives from engineering. + +Instead, consider starting a Design Document that explains the problem and try to solve it with our current tools. Invite contribution from the department and research this thoroughly as there can only be two outcomes. Either the problem **can** be solved with our current tools or it cannot. If it can, this is a huge with for our teams since we've fixed and issue without the need to completly change our stack, and if it cannot, then the Design Document can be the start of the larger conversation around the technological change. ## Widget Architecture diff --git a/doc/development/fe_guide/axios.md b/doc/development/fe_guide/axios.md index 876855b807c..57d8bb39321 100644 --- a/doc/development/fe_guide/axios.md +++ b/doc/development/fe_guide/axios.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Axios diff --git a/doc/development/fe_guide/content_editor.md b/doc/development/fe_guide/content_editor.md index fdeabc99ea4..7a76b39edbc 100644 --- a/doc/development/fe_guide/content_editor.md +++ b/doc/development/fe_guide/content_editor.md @@ -1,7 +1,7 @@ --- stage: Plan group: Knowledge -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Rich text editor development guidelines diff --git a/doc/development/fe_guide/customizable_dashboards.md b/doc/development/fe_guide/customizable_dashboards.md index be0794b95d0..93b5b20b047 100644 --- a/doc/development/fe_guide/customizable_dashboards.md +++ b/doc/development/fe_guide/customizable_dashboards.md @@ -1,7 +1,7 @@ --- -stage: Analyze +stage: Monitor group: Product Analytics -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Customizable dashboards @@ -206,7 +206,8 @@ export default { ## Dashboard designer -> Introduced in GitLab 16.1 [with a flag](../../administration/feature_flags.md) named `combined_analytics_dashboards_editor`. Disabled by default. +> - Introduced in GitLab 16.1 [with a flag](../../administration/feature_flags.md) named `combined_analytics_dashboards_editor`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/411407) in GitLab 16.6. Feature flag `combined_analytics_dashboards_editor` removed. The dashboard designer provides a graphical interface for users to modify the panels and add new ones on user-defined dashboards. Is is not available on diff --git a/doc/development/fe_guide/dark_mode.md b/doc/development/fe_guide/dark_mode.md index 185bd60dd9a..3af65edf317 100644 --- a/doc/development/fe_guide/dark_mode.md +++ b/doc/development/fe_guide/dark_mode.md @@ -1,8 +1,7 @@ --- -type: reference, dev stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- This page is about developing dark mode for GitLab. For more information on how to enable dark mode, see [Change the syntax highlighting theme](../../user/profile/preferences.md#change-the-syntax-highlighting-theme). diff --git a/doc/development/fe_guide/dependencies.md b/doc/development/fe_guide/dependencies.md index b9bacada499..0ba38c04d7b 100644 --- a/doc/development/fe_guide/dependencies.md +++ b/doc/development/fe_guide/dependencies.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Frontend dependencies diff --git a/doc/development/fe_guide/design_patterns.md b/doc/development/fe_guide/design_patterns.md index 44238ff5dc5..b8be003776f 100644 --- a/doc/development/fe_guide/design_patterns.md +++ b/doc/development/fe_guide/design_patterns.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Design Patterns diff --git a/doc/development/fe_guide/design_tokens.md b/doc/development/fe_guide/design_tokens.md index b47c2661e19..1556602e298 100644 --- a/doc/development/fe_guide/design_tokens.md +++ b/doc/development/fe_guide/design_tokens.md @@ -1,8 +1,7 @@ --- -type: reference, dev stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Design tokens diff --git a/doc/development/fe_guide/emojis.md b/doc/development/fe_guide/emojis.md index c93e1bb34c5..f1e4c55f985 100644 --- a/doc/development/fe_guide/emojis.md +++ b/doc/development/fe_guide/emojis.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Emojis diff --git a/doc/development/fe_guide/frontend_faq.md b/doc/development/fe_guide/frontend_faq.md index ab75cc27b6a..b8e98b47cac 100644 --- a/doc/development/fe_guide/frontend_faq.md +++ b/doc/development/fe_guide/frontend_faq.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Frontend FAQ diff --git a/doc/development/fe_guide/frontend_goals.md b/doc/development/fe_guide/frontend_goals.md index 4f39e82c72e..d2ae9ceff28 100644 --- a/doc/development/fe_guide/frontend_goals.md +++ b/doc/development/fe_guide/frontend_goals.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Frontend Goals diff --git a/doc/development/fe_guide/getting_started.md b/doc/development/fe_guide/getting_started.md index 14e704d567e..5687d912de3 100644 --- a/doc/development/fe_guide/getting_started.md +++ b/doc/development/fe_guide/getting_started.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Getting started diff --git a/doc/development/fe_guide/graphql.md b/doc/development/fe_guide/graphql.md index 5807c9c5621..22b977519be 100644 --- a/doc/development/fe_guide/graphql.md +++ b/doc/development/fe_guide/graphql.md @@ -1,8 +1,7 @@ --- -type: reference, dev stage: none group: unassigned -info: "See the Technical Writers assigned to Development Guidelines: https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-development-guidelines" +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # GraphQL @@ -105,7 +104,7 @@ Default client accepts two parameters: `resolvers` and `config`. ### Multiple client queries for the same object -If you are making multiple queries to the same Apollo client object you might encounter the following error: `Cache data may be lost when replacing the someProperty field of a Query object. To address this problem, either ensure all objects of SomeEntityhave an id or a custom merge function`. We are already checking `id` presence for every GraphQL type that has an `id`, so this shouldn't be the case (unless you see this warning when running unit tests; in this case please ensure your mocked responses contain an `id` whenever it's requested). +If you are making multiple queries to the same Apollo client object you might encounter the following error: `Cache data may be lost when replacing the someProperty field of a Query object. To address this problem, either ensure all objects of SomeEntityhave an id or a custom merge function`. We are already checking `id` presence for every GraphQL type that has an `id`, so this shouldn't be the case (unless you see this warning when running unit tests; in this case ensure your mocked responses contain an `id` whenever it's requested). When `SomeEntity` type doesn't have an `id` property in the GraphQL schema, to fix this warning we need to define a custom merge function. @@ -974,6 +973,31 @@ const data = store.readQuery({ Read more about the `@connection` directive in [Apollo's documentation](https://www.apollographql.com/docs/react/caching/advanced-topics/#the-connection-directive). +### Batching similar queries + +By default, the Apollo client sends one HTTP request from the browser per query. You can choose to +batch several queries in a single outgoing request and lower the number of requests by defining a +[batchKey](https://www.apollographql.com/docs/react/api/link/apollo-link-batch-http/#batchkey). + +This can be helpful when a query is called multiple times from the same component but you +want to update the UI once. In this example we use the component name as the key: + +```javascript +export default { + name: 'MyComponent' + apollo: { + user: { + query: QUERY_IMPORT, + context: { + batchKey: 'MyComponent', + }, + } + }, +}; +``` + +The batch key can be the name of the component. + #### Polling and Performance While the Apollo client has support for simple polling, for performance reasons, our [ETag-based caching](../polling.md) is preferred to hitting the database each time. diff --git a/doc/development/fe_guide/guides.md b/doc/development/fe_guide/guides.md index dc2fffcf10a..ce53d3df0fc 100644 --- a/doc/development/fe_guide/guides.md +++ b/doc/development/fe_guide/guides.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Guides diff --git a/doc/development/fe_guide/haml.md b/doc/development/fe_guide/haml.md index 1dc8cf63de9..88f4a785a70 100644 --- a/doc/development/fe_guide/haml.md +++ b/doc/development/fe_guide/haml.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # HAML diff --git a/doc/development/fe_guide/icons.md b/doc/development/fe_guide/icons.md index df296f13f48..d1882644c02 100644 --- a/doc/development/fe_guide/icons.md +++ b/doc/development/fe_guide/icons.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Icons and SVG Illustrations diff --git a/doc/development/fe_guide/index.md b/doc/development/fe_guide/index.md index 6bea22bd6bf..40ff704edfa 100644 --- a/doc/development/fe_guide/index.md +++ b/doc/development/fe_guide/index.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Frontend Development Guidelines @@ -11,7 +11,7 @@ across the GitLab frontend team. ## Overview -GitLab is built on top of [Ruby on Rails](https://rubyonrails.org). It uses [Haml](https://haml.info/) and a JavaScript-based frontend with [Vue.js](https://vuejs.org). If you are not sure when to use Vue on top of Haml-page, please read [this explanation](vue.md#when-to-add-vue-application). +GitLab is built on top of [Ruby on Rails](https://rubyonrails.org). It uses [Haml](https://haml.info/) and a JavaScript-based frontend with [Vue.js](https://vuejs.org). If you are not sure when to use Vue on top of Haml-page, read [this explanation](vue.md#when-to-add-vue-application). <!-- vale gitlab.Spelling = NO --> @@ -19,19 +19,19 @@ Be wary of [the limitations that come with using Hamlit](https://github.com/k0ku <!-- vale gitlab.Spelling = YES --> -When it comes to CSS, we use a utils-based CSS approach. GitLab has its own CSS utils which are packaged inside the `gitlab-ui` project and can be seen [in the repository](https://gitlab.com/gitlab-org/gitlab-ui/-/tree/main/src/scss/utility-mixins) or on [UNPKG](https://unpkg.com/browse/@gitlab/ui@latest/src/scss/utility-mixins/). Please favor using these before adding or using any SCSS classes. +When it comes to CSS, we use a utils-based CSS approach. GitLab has its own CSS utils which are packaged inside the `gitlab-ui` project and can be seen [in the repository](https://gitlab.com/gitlab-org/gitlab-ui/-/tree/main/src/scss/utility-mixins) or on [UNPKG](https://unpkg.com/browse/@gitlab/ui@latest/src/scss/utility-mixins/). Favor using these before adding or using any SCSS classes. We also use [SCSS](https://sass-lang.com) and plain JavaScript with modern ECMAScript standards supported through [Babel](https://babeljs.io/) and ES module support through [webpack](https://webpack.js.org/). -When making API calls, we use [GraphQL](graphql.md) as [the first choice](../../api/graphql/index.md#vision). There are still instances where GitLab REST API is used such as when creating new simple HAML pages or in legacy part of the codebase, but we should always default to GraphQL when possible. +When making API calls, we use [GraphQL](graphql.md) as [the first choice](../api_graphql_styleguide.md#vision). There are still instances where GitLab REST API is used such as when creating new simple HAML pages or in legacy part of the codebase, but we should always default to GraphQL when possible. We use [Apollo](https://www.apollographql.com/) as our global state manager and [GraphQL client](graphql.md). [VueX](vuex.md) is still in use across the codebase, but it is no longer the recommended global state manager. You should **not** [use VueX and Apollo together](graphql.md#using-with-vuex), and should [avoid adding new VueX stores](migrating_from_vuex.md) whenever possible. -For copy strings and translations, we have frontend utilities available. Please see the JavaScript section of [Preparing a page for translation](../i18n/externalization.md#javascript-files) for more information. +For copy strings and translations, we have frontend utilities available. See the JavaScript section of [Preparing a page for translation](../i18n/externalization.md#javascript-files) for more information. Working with our frontend assets requires Node (v12.22.1 or greater) and Yarn (v1.10.0 or greater). You can find information on how to install these on our diff --git a/doc/development/fe_guide/keyboard_shortcuts.md b/doc/development/fe_guide/keyboard_shortcuts.md index aeeee72e6be..35ca240a2ad 100644 --- a/doc/development/fe_guide/keyboard_shortcuts.md +++ b/doc/development/fe_guide/keyboard_shortcuts.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Implementing keyboard shortcuts diff --git a/doc/development/fe_guide/logging.md b/doc/development/fe_guide/logging.md index 750bf95e8b2..debe9cad27d 100644 --- a/doc/development/fe_guide/logging.md +++ b/doc/development/fe_guide/logging.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Client-side logging for frontend development diff --git a/doc/development/fe_guide/merge_request_widget_extensions.md b/doc/development/fe_guide/merge_request_widget_extensions.md index 99206d99590..5319048dfa5 100644 --- a/doc/development/fe_guide/merge_request_widget_extensions.md +++ b/doc/development/fe_guide/merge_request_widget_extensions.md @@ -1,7 +1,7 @@ --- stage: Create group: Code Review -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Merge request widget extensions diff --git a/doc/development/fe_guide/migrating_from_vuex.md b/doc/development/fe_guide/migrating_from_vuex.md index 8dca744e192..45a1ea0b7ee 100644 --- a/doc/development/fe_guide/migrating_from_vuex.md +++ b/doc/development/fe_guide/migrating_from_vuex.md @@ -1,14 +1,14 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Migrating from Vuex ## Why? -We have defined [GraphQL as our primary API](../../api/graphql/index.md#vision) for all user-facing features, +We have defined the [GraphQL API](../../api/graphql/index.md) as [the primary API](../api_graphql_styleguide.md#vision) for all user-facing features, so we can safely assume that whenever GraphQL is present, so will the Apollo Client. We [do not want to use Vuex with Apollo](graphql.md#using-with-vuex), so the VueX stores count will naturally decline over time as we move from the REST API to GraphQL. diff --git a/doc/development/fe_guide/onboarding_course/index.md b/doc/development/fe_guide/onboarding_course/index.md index 0b0ffc69f1b..d8d4cdad8a6 100644 --- a/doc/development/fe_guide/onboarding_course/index.md +++ b/doc/development/fe_guide/onboarding_course/index.md @@ -1,7 +1,7 @@ --- stage: Manage group: Foundations -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Frontend onboarding course @@ -46,7 +46,7 @@ A fortnightly 1-on-1 mentoring sessions are also available to each participant. There are 10 places available on the course. The date will be set after the course material has been prepared. -Please complete the [Frontend Onboarding Course Application Form](https://forms.gle/39Rs4w4ZxQuByhE4A) to apply. +Complete the [Frontend Onboarding Course Application Form](https://forms.gle/39Rs4w4ZxQuByhE4A) to apply. You may also participate in the course informally at your own pace, without the benefit of the synchronous office hours or mentoring session. GitLab team members are happy to support you regardless. diff --git a/doc/development/fe_guide/onboarding_course/lesson_1.md b/doc/development/fe_guide/onboarding_course/lesson_1.md index e82d350f854..ec26da8c4eb 100644 --- a/doc/development/fe_guide/onboarding_course/lesson_1.md +++ b/doc/development/fe_guide/onboarding_course/lesson_1.md @@ -1,7 +1,7 @@ --- stage: manage group: foundations -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Lesson 1 diff --git a/doc/development/fe_guide/performance.md b/doc/development/fe_guide/performance.md index 5e8581663b6..aedb391ed25 100644 --- a/doc/development/fe_guide/performance.md +++ b/doc/development/fe_guide/performance.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Performance diff --git a/doc/development/fe_guide/registry_architecture.md b/doc/development/fe_guide/registry_architecture.md index cf267e80619..bc46c2643de 100644 --- a/doc/development/fe_guide/registry_architecture.md +++ b/doc/development/fe_guide/registry_architecture.md @@ -1,7 +1,7 @@ --- stage: Package group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Registry architecture @@ -12,8 +12,8 @@ already exists to unify the user and developer experiences. Existing registries: -- Package Registry -- Container Registry +- Package registry +- Container registry - Terraform Module Registry - Dependency Proxy @@ -82,7 +82,7 @@ main pieces of the desired UI and UX of a registry page. The most important comp When adding a new registry: - Leverage the shared components that already exist. It's good to look at how the components are - structured and used in the more mature registries (for example, the Package Registry). + structured and used in the more mature registries (for example, the package registry). - If it's in line with the backend requirements, we suggest using GraphQL for the API. This helps in dealing with the innate performance issue of registries. - If possible, we recommend using [Vue Router](https://v3.router.vuejs.org/) diff --git a/doc/development/fe_guide/security.md b/doc/development/fe_guide/security.md index 4e06c22b383..e4b748f7e7c 100644 --- a/doc/development/fe_guide/security.md +++ b/doc/development/fe_guide/security.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Security diff --git a/doc/development/fe_guide/sentry.md b/doc/development/fe_guide/sentry.md index 95a170b7976..fa85d51afb0 100644 --- a/doc/development/fe_guide/sentry.md +++ b/doc/development/fe_guide/sentry.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Sentry monitoring in the frontend development of GitLab diff --git a/doc/development/fe_guide/source_editor.md b/doc/development/fe_guide/source_editor.md index 943ac2969f3..e65ede53a13 100644 --- a/doc/development/fe_guide/source_editor.md +++ b/doc/development/fe_guide/source_editor.md @@ -1,7 +1,7 @@ --- stage: Create group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Source Editor diff --git a/doc/development/fe_guide/storybook.md b/doc/development/fe_guide/storybook.md index cbda9d5efa2..05d4397933f 100644 --- a/doc/development/fe_guide/storybook.md +++ b/doc/development/fe_guide/storybook.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Storybook diff --git a/doc/development/fe_guide/style/html.md b/doc/development/fe_guide/style/html.md index 9d8809f19c7..c9168a68614 100644 --- a/doc/development/fe_guide/style/html.md +++ b/doc/development/fe_guide/style/html.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # HTML style guide diff --git a/doc/development/fe_guide/style/index.md b/doc/development/fe_guide/style/index.md index 552b769d6f6..31b5db82153 100644 --- a/doc/development/fe_guide/style/index.md +++ b/doc/development/fe_guide/style/index.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Frontend style guides diff --git a/doc/development/fe_guide/style/javascript.md b/doc/development/fe_guide/style/javascript.md index cccaefe8a19..01caf65be5f 100644 --- a/doc/development/fe_guide/style/javascript.md +++ b/doc/development/fe_guide/style/javascript.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # JavaScript style guide diff --git a/doc/development/fe_guide/style/scss.md b/doc/development/fe_guide/style/scss.md index 400b178d9a4..a78fcf67665 100644 --- a/doc/development/fe_guide/style/scss.md +++ b/doc/development/fe_guide/style/scss.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # SCSS style guide diff --git a/doc/development/fe_guide/style/typescript.md b/doc/development/fe_guide/style/typescript.md index 529459097b4..567b80489fb 100644 --- a/doc/development/fe_guide/style/typescript.md +++ b/doc/development/fe_guide/style/typescript.md @@ -1,8 +1,7 @@ --- -type: reference, dev stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # TypeScript diff --git a/doc/development/fe_guide/style/vue.md b/doc/development/fe_guide/style/vue.md index a3ab1c1af30..83d725d453b 100644 --- a/doc/development/fe_guide/style/vue.md +++ b/doc/development/fe_guide/style/vue.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Vue.js style guide diff --git a/doc/development/fe_guide/tech_stack.md b/doc/development/fe_guide/tech_stack.md index 9c0d50ea7bd..87a5e42f75e 100644 --- a/doc/development/fe_guide/tech_stack.md +++ b/doc/development/fe_guide/tech_stack.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Tech Stack diff --git a/doc/development/fe_guide/tips_and_tricks.md b/doc/development/fe_guide/tips_and_tricks.md index dcacdb8387b..ae19ecd3fe5 100644 --- a/doc/development/fe_guide/tips_and_tricks.md +++ b/doc/development/fe_guide/tips_and_tricks.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Tips and tricks diff --git a/doc/development/fe_guide/tooling.md b/doc/development/fe_guide/tooling.md index c1084ab4453..536cf5c4a2a 100644 --- a/doc/development/fe_guide/tooling.md +++ b/doc/development/fe_guide/tooling.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Tooling diff --git a/doc/development/fe_guide/troubleshooting.md b/doc/development/fe_guide/troubleshooting.md index 3e4c5007b64..743eaf7e494 100644 --- a/doc/development/fe_guide/troubleshooting.md +++ b/doc/development/fe_guide/troubleshooting.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Troubleshooting frontend development issues diff --git a/doc/development/fe_guide/type_hinting.md b/doc/development/fe_guide/type_hinting.md index 026bf855e27..0252b5343d4 100644 --- a/doc/development/fe_guide/type_hinting.md +++ b/doc/development/fe_guide/type_hinting.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Type hinting overview diff --git a/doc/development/fe_guide/view_component.md b/doc/development/fe_guide/view_component.md index cfd78597501..a616a3d7c48 100644 --- a/doc/development/fe_guide/view_component.md +++ b/doc/development/fe_guide/view_component.md @@ -1,7 +1,7 @@ --- stage: Manage group: Foundations -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # ViewComponent diff --git a/doc/development/fe_guide/vue.md b/doc/development/fe_guide/vue.md index cfc3ac466d5..69967a5a2be 100644 --- a/doc/development/fe_guide/vue.md +++ b/doc/development/fe_guide/vue.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Vue @@ -39,9 +39,9 @@ In the past, we added interactivity to the page piece-by-piece, adding multiple - multiple applications lead to unpredictable user experience, increased page complexity, harder debugging process; - the way apps communicate with each other affects Web Vitals numbers. -Because of these reasons, we want to be cautious about adding new Vue applications to the pages where another Vue application is already present (this does not include old or new navigation). Before adding a new app, please make sure that it is absolutely impossible to extend an existing application to achieve a desired functionality. When in doubt, please feel free to ask for the architectural advise on `#frontend` or `#frontend-maintainers` Slack channel. +Because of these reasons, we want to be cautious about adding new Vue applications to the pages where another Vue application is already present (this does not include old or new navigation). Before adding a new app, make sure that it is absolutely impossible to extend an existing application to achieve a desired functionality. When in doubt, feel free to ask for the architectural advise on `#frontend` or `#frontend-maintainers` Slack channel. -If you still need to add a new application, please make sure it shares local state with existing applications (preferably via Apollo Client, or Vuex if we use REST API) +If you still need to add a new application, make sure it shares local state with existing applications (preferably via Apollo Client, or Vuex if we use REST API) ## Vue architecture @@ -860,7 +860,7 @@ component under test, with the `computed` property, for example). Remember to us We should test for events emitted in response to an action in our component. This testing verifies the correct events are being fired with the correct arguments. -For any DOM events we should use [`trigger`](https://v1.test-utils.vuejs.org/api/wrapper/#trigger) +For any native DOM events we should use [`trigger`](https://v1.test-utils.vuejs.org/api/wrapper/#trigger) to fire out event. ```javascript @@ -892,6 +892,20 @@ it('should fire the itemClicked event', () => { We should verify an event has been fired by asserting against the result of the [`emitted()`](https://v1.test-utils.vuejs.org/api/wrapper/#emitted) method. +It is a good practice to prefer to use `vm.$emit` over `trigger` when emitting events from child components. + +Using `trigger` on the component means we treat it as a white box: we assume that the root element of child component has a native `click` event. Also, some tests fail in Vue3 mode when using `trigger` on child components. + + ```javascript + const findButton = () => wrapper.findComponent(GlButton); + + // bad + findButton().trigger('click'); + + // good + findButton().vm.$emit('click'); + ``` + ## Vue.js Expert Role You should only apply to be a Vue.js expert when your own merge requests and your reviews show: diff --git a/doc/development/fe_guide/vue3_migration.md b/doc/development/fe_guide/vue3_migration.md index b9e819a95bd..6eb1d4c2413 100644 --- a/doc/development/fe_guide/vue3_migration.md +++ b/doc/development/fe_guide/vue3_migration.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Migration to Vue 3 diff --git a/doc/development/fe_guide/vuex.md b/doc/development/fe_guide/vuex.md index 06d7070b855..aff3588a503 100644 --- a/doc/development/fe_guide/vuex.md +++ b/doc/development/fe_guide/vuex.md @@ -1,12 +1,12 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Vuex -[Vuex](https://vuex.vuejs.org) should no longer be considered a preferred path to store management and is currently in its legacy phase. This means it is acceptable to add upon existing `Vuex` stores, but we strongly recommend reducing store sizes over time and eventually [migrating away from VueX entirely](migrating_from_vuex.md). Before adding any new `Vuex` store to an application, first ensure that the `Vue` application you plan to add it into **does not use** `Apollo`. `Vuex` and `Apollo` should not be combined unless absolutely necessary. Please consider reading through [our GraphQL documentation](../fe_guide/graphql.md) for more guidelines on how you can build `Apollo` based applications. +[Vuex](https://vuex.vuejs.org) should no longer be considered a preferred path to store management and is currently in its legacy phase. This means it is acceptable to add upon existing `Vuex` stores, but we strongly recommend reducing store sizes over time and eventually [migrating away from VueX entirely](migrating_from_vuex.md). Before adding any new `Vuex` store to an application, first ensure that the `Vue` application you plan to add it into **does not use** `Apollo`. `Vuex` and `Apollo` should not be combined unless absolutely necessary. Consider reading through [our GraphQL documentation](../fe_guide/graphql.md) for more guidelines on how you can build `Apollo` based applications. The information included in this page is explained in more detail in the official [Vuex documentation](https://vuex.vuejs.org). diff --git a/doc/development/fe_guide/widgets.md b/doc/development/fe_guide/widgets.md index 6cd8e6c091c..de4ac7620c2 100644 --- a/doc/development/fe_guide/widgets.md +++ b/doc/development/fe_guide/widgets.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Widgets diff --git a/doc/development/feature_categorization/index.md b/doc/development/feature_categorization/index.md index 2f4a7380bff..dfd327270f2 100644 --- a/doc/development/feature_categorization/index.md +++ b/doc/development/feature_categorization/index.md @@ -1,7 +1,7 @@ --- stage: Enablement group: Infrastructure -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Feature Categorization diff --git a/doc/development/feature_development.md b/doc/development/feature_development.md index 4dbde42b0ff..a5a6439b420 100644 --- a/doc/development/feature_development.md +++ b/doc/development/feature_development.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: "See the Technical Writers assigned to Development Guidelines: https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-development-guidelines" +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Feature development @@ -96,6 +96,7 @@ Consult these topics for information on contributing to specific GitLab features - [Value Stream Analytics development guide](value_stream_analytics.md) - [Application limits](application_limits.md) - [AI features](ai_features/index.md) +- [Application settings](application_settings.md) ### Import and Export diff --git a/doc/development/feature_flags/controls.md b/doc/development/feature_flags/controls.md index 6e0f0e8dbcf..5006b21965e 100644 --- a/doc/development/feature_flags/controls.md +++ b/doc/development/feature_flags/controls.md @@ -1,8 +1,7 @@ --- -type: reference, dev stage: none group: unassigned -info: "See the Technical Writers assigned to Development Guidelines: https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-development-guidelines" +info: "See the Technical Writers assigned to Development Guidelines: https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-development-guidelines" --- # Use ChatOps to enable and disable feature flags @@ -43,7 +42,7 @@ The GitLab feature library (using [Feature flags process](https://about.gitlab.com/handbook/product-development-flow/feature-flag-lifecycle/) guide) supports rolling out changes to a percentage of time to users. This in turn can be controlled using [GitLab ChatOps](../../ci/chatops/index.md). -For an up to date list of feature flag commands please see +For an up to date list of feature flag commands see [the source code](https://gitlab.com/gitlab-com/chatops/blob/master/lib/chatops/commands/feature.rb). Note that all the examples in that file must be preceded by `/chatops run`. @@ -105,7 +104,7 @@ Guidelines: - Consider notifying `#support_gitlab-com` beforehand. So in case if the feature has any side effects on user experience, they can mitigate and disable the feature flag to reduce some impact. - If the feature meets the requirements for creating a [Change Management](https://about.gitlab.com/handbook/engineering/infrastructure/change-management/#feature-flags-and-the-change-management-process) issue, create a Change Management issue per [criticality guidelines](https://about.gitlab.com/handbook/engineering/infrastructure/change-management/#change-request-workflows). - For simple, low-risk, easily reverted features, proceed and [enable the feature in `#production`](#process). -- For support requests to toggle feature flags for specific groups or projects, please follow the process outlined in the [support workflows](https://about.gitlab.com/handbook/support/workflows/saas_feature_flags.html). +- For support requests to toggle feature flags for specific groups or projects, follow the process outlined in the [support workflows](https://about.gitlab.com/handbook/support/workflows/saas_feature_flags.html). #### Guideline for which percentages to choose during the rollout @@ -204,7 +203,7 @@ Before enabling a feature flag, verify that you are not violating any [Productio The following `/chatops` commands should be performed in the Slack `#production` channel. -When you begin to enable the feature, please link to the relevant +When you begin to enable the feature, link to the relevant feature flag rollout issue within a Slack thread of the first `/chatops` command you make so people can understand the change if they need to. @@ -510,5 +509,5 @@ record still exists in the database that the flag was deployed too. The record can be deleted once the MR is deployed to all the environments: ```shell -/chatops run feature delete <feature-flag-name> --dev --ops --pre --staging --staging-ref --production +/chatops run feature delete <feature-flag-name> --dev --pre --staging --staging-ref --production ``` diff --git a/doc/development/feature_flags/index.md b/doc/development/feature_flags/index.md index c1a5963e97f..17dd8432f7b 100644 --- a/doc/development/feature_flags/index.md +++ b/doc/development/feature_flags/index.md @@ -1,8 +1,7 @@ --- -type: reference, dev stage: none group: unassigned -info: "See the Technical Writers assigned to Development Guidelines: https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-development-guidelines" +info: "See the Technical Writers assigned to Development Guidelines: https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-development-guidelines" --- # Feature flags in the development of GitLab @@ -20,7 +19,7 @@ All newly-introduced feature flags should be [used with an actor](controls.md#pe This document is the subject of continued work as part of an epic to [improve internal usage of feature flags](https://gitlab.com/groups/gitlab-org/-/epics/3551). Raise any suggestions as new issues and attach them to the epic. -For an [overview of the feature flag lifecycle](https://about.gitlab.com/handbook/product-development-flow/feature-flag-lifecycle/#feature-flag-lifecycle), or if you need help deciding [if you should use a feature flag](https://about.gitlab.com/handbook/product-development-flow/feature-flag-lifecycle/#when-to-use-feature-flags) or not, please see the [feature flag lifecycle](https://about.gitlab.com/handbook/product-development-flow/feature-flag-lifecycle/) handbook page. +For an [overview of the feature flag lifecycle](https://about.gitlab.com/handbook/product-development-flow/feature-flag-lifecycle/#feature-flag-lifecycle), or if you need help deciding [if you should use a feature flag](https://about.gitlab.com/handbook/product-development-flow/feature-flag-lifecycle/#when-to-use-feature-flags) or not, see the [feature flag lifecycle](https://about.gitlab.com/handbook/product-development-flow/feature-flag-lifecycle/) handbook page. ## When to use feature flags @@ -605,7 +604,7 @@ with the untested code path should be manually tested before deployment to produ When using the testing environment, all feature flags are enabled by default. Flags can be disabled by default in the [`spec/spec_helper.rb` file](https://gitlab.com/gitlab-org/gitlab/-/blob/b61fba42eea2cf5bb1ca64e80c067a07ed5d1921/spec/spec_helper.rb#L274). -Please add a comment inline to explain why the flag needs to be disabled. You can also attach the issue URL for reference if possible. +Add a comment inline to explain why the flag needs to be disabled. You can also attach the issue URL for reference if possible. WARNING: This does not apply to end-to-end (QA) tests, which [do not enable feature flags by default](#end-to-end-qa-tests). There is a different [process for using feature flags in end-to-end tests](../testing_guide/end_to_end/feature_flags.md). diff --git a/doc/development/features_inside_dot_gitlab.md b/doc/development/features_inside_dot_gitlab.md index f8de93a2243..1217c2bf596 100644 --- a/doc/development/features_inside_dot_gitlab.md +++ b/doc/development/features_inside_dot_gitlab.md @@ -1,13 +1,13 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Features inside the `.gitlab/` directory We have implemented standard features that depend on configuration files in the `.gitlab/` directory. You can find `.gitlab/` in various GitLab repositories. -When implementing new features, please refer to these existing features to avoid conflicts: +When implementing new features, refer to these existing features to avoid conflicts: - [Issue Templates](../user/project/description_templates.md#create-an-issue-template): `.gitlab/issue_templates/`. - [Merge request Templates](../user/project/description_templates.md#create-a-merge-request-template): `.gitlab/merge_request_templates/`. diff --git a/doc/development/file_storage.md b/doc/development/file_storage.md index 39833a441ee..550b56af520 100644 --- a/doc/development/file_storage.md +++ b/doc/development/file_storage.md @@ -1,14 +1,14 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # File Storage in GitLab We use the [CarrierWave](https://github.com/carrierwaveuploader/carrierwave) gem to handle file upload, store and retrieval. -File uploads should be accelerated by workhorse, for details please refer to [uploads development documentation](uploads/index.md). +File uploads should be accelerated by workhorse, for details refer to [uploads development documentation](uploads/index.md). There are many places where file uploading is used, according to contexts: diff --git a/doc/development/fips_compliance.md b/doc/development/fips_compliance.md index 60677abf292..3563f7fc2d5 100644 --- a/doc/development/fips_compliance.md +++ b/doc/development/fips_compliance.md @@ -1,7 +1,7 @@ --- stage: Create group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # FIPS compliance @@ -40,7 +40,7 @@ when FIPS mode is enabled. | Ubuntu 20.04 Libgcrypt Cryptographic Module | [3902](https://csrc.nist.gov/projects/cryptographic-module-validation-program/certificate/3902) | EC2 instances | `gpg`, `sshd` | | Amazon Linux 2 Kernel Crypto API Cryptographic Module | [3709](https://csrc.nist.gov/projects/cryptographic-module-validation-program/certificate/3709) | EKS nodes | Linux kernel | | Amazon Linux 2 OpenSSL Cryptographic Module | [3553](https://csrc.nist.gov/projects/cryptographic-module-validation-program/certificate/3553) | EKS nodes | NGINX | -| RedHat Enterprise Linux 8 OpenSSL Cryptographic Module | [4271](https://csrc.nist.gov/projects/cryptographic-module-validation-program/certificate/4271) | EKS nodes | UBI containers: Workhorse, Pages, Container Registry, Rails (Puma/Sidekiq), Security Analyzers | +| RedHat Enterprise Linux 8 OpenSSL Cryptographic Module | [4271](https://csrc.nist.gov/projects/cryptographic-module-validation-program/certificate/4271) | EKS nodes | UBI containers: Workhorse, Pages, container registry, Rails (Puma/Sidekiq), Security Analyzers | | RedHat Enterprise Linux 8 Libgcrypt Cryptographic Module | [3784](https://csrc.nist.gov/projects/cryptographic-module-validation-program/certificate/3784) | EKS nodes | UBI containers: GitLab Shell, `gpg` | ### Supported Operating Systems diff --git a/doc/development/gemfile.md b/doc/development/gemfile.md index ed38f6481e7..4801a8c2557 100644 --- a/doc/development/gemfile.md +++ b/doc/development/gemfile.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Gemfile development guidelines diff --git a/doc/development/gems.md b/doc/development/gems.md index 54d6e6dc30d..476ed8f916b 100644 --- a/doc/development/gems.md +++ b/doc/development/gems.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Gems development guidelines diff --git a/doc/development/geo.md b/doc/development/geo.md index ceaa2dbb056..d8b000c090a 100644 --- a/doc/development/geo.md +++ b/doc/development/geo.md @@ -1,7 +1,7 @@ --- stage: Systems group: Geo -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Geo (development) @@ -628,7 +628,7 @@ As another example, [Dependency Proxy](../administration/packages/dependency_pro If a new feature introduces a new kind of data which is not a Git repository, or a blob, or a combination of the two, then contact the Geo team to discuss how to handle it. -As an example, Container Registry data does not easily fit into the above categories. It is backed by a registry service which owns the data, and GitLab interacts with the registry service's API. So a one off approach is required for Geo support of Container Registry. Still, we are able to reuse much of the glue code of [the Geo self-service framework](geo/framework.md#repository-replicator-strategy). +As an example, container registry data does not easily fit into the above categories. It is backed by a registry service which owns the data, and GitLab interacts with the registry service's API. So a one off approach is required for Geo support of container registry. Still, we are able to reuse much of the glue code of [the Geo self-service framework](geo/framework.md#repository-replicator-strategy). ## History of communication channel diff --git a/doc/development/geo/framework.md b/doc/development/geo/framework.md index 60529db5ce6..4ec3309dc56 100644 --- a/doc/development/geo/framework.md +++ b/doc/development/geo/framework.md @@ -1,7 +1,7 @@ --- stage: Systems group: Geo -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Geo self-service framework diff --git a/doc/development/geo/proxying.md b/doc/development/geo/proxying.md index 45c60bc370e..49777771b2e 100644 --- a/doc/development/geo/proxying.md +++ b/doc/development/geo/proxying.md @@ -1,7 +1,7 @@ --- stage: Systems group: Geo -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Geo proxying diff --git a/doc/development/git_object_deduplication.md b/doc/development/git_object_deduplication.md index 65d2338cd65..f3e35f9cdf8 100644 --- a/doc/development/git_object_deduplication.md +++ b/doc/development/git_object_deduplication.md @@ -1,7 +1,7 @@ --- stage: Systems group: Gitaly -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # How Git object deduplication works in GitLab diff --git a/doc/development/gitaly.md b/doc/development/gitaly.md index ed7fb6325d6..a391daf8962 100644 --- a/doc/development/gitaly.md +++ b/doc/development/gitaly.md @@ -1,7 +1,7 @@ --- stage: Systems group: Gitaly -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Gitaly development guidelines @@ -73,12 +73,12 @@ when Gitaly is called more than 30 times in a single Rails request or Sidekiq ex As a temporary measure, export `GITALY_DISABLE_REQUEST_LIMITS=1` to suppress the error. This disables the n+1 detection in your development environment. -Please raise an issue in the GitLab CE or EE repositories to report the issue. Include the labels ~Gitaly -~performance ~"technical debt". Please ensure that the issue contains the full stack trace and error message of the +Raise an issue in the GitLab CE or EE repositories to report the issue. Include the labels ~Gitaly +~performance ~"technical debt". Ensure that the issue contains the full stack trace and error message of the `TooManyInvocationsError`. Also include any known failing tests if possible. Isolate the source of the n+1 problem. This is usually a loop that results in Gitaly being called for each -element in an array. If you are unable to isolate the problem, please contact a member +element in an array. If you are unable to isolate the problem, contact a member of the [Gitaly Team](https://gitlab.com/groups/gl-gitaly/group_members) for assistance. After the source has been found, wrap it in an `allow_n_plus_1_calls` block, as follows: diff --git a/doc/development/github_importer.md b/doc/development/github_importer.md index 9ce95cf7da1..56a4ea898df 100644 --- a/doc/development/github_importer.md +++ b/doc/development/github_importer.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # GitHub importer developer documentation @@ -209,7 +209,8 @@ When you schedule a job, `AdvanceStageWorker` is given a project ID, a list of Redis keys, and the name of the next stage. The Redis keys (produced by `Gitlab::JobWaiter`) are used to check if the running stage has been completed or not. If the stage has not yet been -completed `AdvanceStageWorker` reschedules itself. After a stage finishes +completed `AdvanceStageWorker` reschedules itself. After a stage finishes, +or if more jobs have been finished after the last invocation. `AdvanceStageworker` refreshes the import JID (more on this below) and schedule the worker of the next stage. @@ -221,18 +222,19 @@ also reduces pressure on the system as a whole. ## Refreshing import job IDs GitLab includes a worker called `Gitlab::Import::StuckProjectImportJobsWorker` -that periodically runs and marks project imports as failed if they have been -running for more than 24 hours. For GitHub projects, this poses a bit of a -problem: importing large projects could take several hours depending on how +that periodically runs and marks project imports as failed if they have not been +refreshed for more than 24 hours. For GitHub projects, this poses a bit of a +problem: importing large projects could take several days depending on how often we hit the GitHub rate limit (more on this below), but we don't want `Gitlab::Import::StuckProjectImportJobsWorker` to mark our import as failed because of this. To prevent this from happening we periodically refresh the expiration time of -the import process. This works by storing the JID of the import job in the +the import. This works by storing the JID of the import job in the database, then refreshing this JID TTL at various stages throughout the import -process. This is done by calling `ProjectImportState#refresh_jid_expiration`. By -refreshing this TTL we can ensure our import does not get marked as failed so -long we're still performing work. +process. This is done either by calling `ProjectImportState#refresh_jid_expiration`, +or by using the RefreshImportJidWorker and passing in the current worker's jid. +By refreshing this TTL we can ensure our import does not get marked as failed so +long as we're still performing work. ## GitHub rate limit @@ -297,6 +299,54 @@ The code for this resides in: - `lib/gitlab/github_import/user_finder.rb` - `lib/gitlab/github_import/caching.rb` +## Increasing Sidekiq interrupts + +When a Sidekiq process shut downs, it waits for a period of time for running +jobs to finish before it then interrupts them. An interrupt terminates +the job and requeues it again. Our +[vendored `sidekiq-reliable-fetcher` gem](https://gitlab.com/gitlab-org/gitlab/-/blob/master/vendor/gems/sidekiq-reliable-fetch/README.md) +puts a limit of `3` interrupts before a job is no longer requeued and is +permanently terminated. Jobs that have been interrupted log a +`json.interrupted_count` in Kibana. + +This limit offers protection from jobs that can never be completed in +the time between Sidekiq restarts. + +For large imports, our GitHub [stage](#stages) workers (namespaced in +`Stage::`) take many hours to finish. By default, the import is at risk +of failing because of `sidekiq-reliable-fetcher` permanently stopping these +workers before they can complete. + +Stage workers that pick up from where they left off when restarted can +increase the interrupt limit of `sidekiq-reliable-fetcher` to `20` by +calling `.resumes_work_when_interrupted!`: + +```ruby +module Gitlab + module GithubImport + module Stage + class MyWorker + resumes_work_when_interrupted! + + # ... + end + end + end +end +``` + +Stage workers that do not fully resume their work when restarted should +not call this method. For example, a worker that skips already imported +objects, but starts its loop from the beginning each time. + +Examples of stage workers that do resume work fully are ones that +execute services that: + +- [Continue paging](https://gitlab.com/gitlab-org/gitlab/-/blob/487521cc/lib/gitlab/github_import/parallel_scheduling.rb#L114-117) + an endpoint from where it left off. +- [Continue their loop](https://gitlab.com/gitlab-org/gitlab/-/blob/487521cc26c1e2bdba4fc67c14478d2b2a5f2bfa/lib/gitlab/github_import/importer/attachments/issues_importer.rb#L27) + from where it left off. + ## Mapping labels and milestones To reduce pressure on the database we do not query it when setting labels and diff --git a/doc/development/gitlab_flavored_markdown/index.md b/doc/development/gitlab_flavored_markdown/index.md index cde83bff32e..fe743787fb8 100644 --- a/doc/development/gitlab_flavored_markdown/index.md +++ b/doc/development/gitlab_flavored_markdown/index.md @@ -1,7 +1,7 @@ --- stage: Create group: IDE -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- <!-- vale gitlab.GitLabFlavoredMarkdown = NO --> diff --git a/doc/development/gitlab_flavored_markdown/specification_guide/index.md b/doc/development/gitlab_flavored_markdown/specification_guide/index.md index a0eb04d7cad..f8097d48739 100644 --- a/doc/development/gitlab_flavored_markdown/specification_guide/index.md +++ b/doc/development/gitlab_flavored_markdown/specification_guide/index.md @@ -1,7 +1,7 @@ --- stage: Create group: IDE -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- <!-- vale gitlab.GitLabFlavoredMarkdown = NO --> diff --git a/doc/development/gitlab_shell/features.md b/doc/development/gitlab_shell/features.md index f7931c4b94d..c34e430e520 100644 --- a/doc/development/gitlab_shell/features.md +++ b/doc/development/gitlab_shell/features.md @@ -1,7 +1,7 @@ --- stage: Create group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # GitLab Shell feature list diff --git a/doc/development/gitpod_internals.md b/doc/development/gitpod_internals.md index a4b340916dd..33e18dc2133 100644 --- a/doc/development/gitpod_internals.md +++ b/doc/development/gitpod_internals.md @@ -1,7 +1,7 @@ --- stage: none group: Engineering Productivity -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Gitpod internal configuration @@ -19,7 +19,7 @@ The current settings are: A webhook that starts with `https://gitpod.io/` is created to enable prebuilds (see [Enabling Prebuilds](https://www.gitpod.io/docs/configure/authentication/gitlab#enabling-prebuilds) for more details). The webhook is maintained by an [Engineering Productivity team](https://about.gitlab.com/handbook/engineering/quality/engineering-productivity/). -You can find this webhook in [Webhook Settings in `gitlab-org/gitlab`](https://gitlab.com/gitlab-org/gitlab/-/hooks). If you cannot access this setting, please chat to the [Engineering Productivity team](https://about.gitlab.com/handbook/engineering/quality/engineering-productivity/). +You can find this webhook in [Webhook Settings in `gitlab-org/gitlab`](https://gitlab.com/gitlab-org/gitlab/-/hooks). If you cannot access this setting, chat to the [Engineering Productivity team](https://about.gitlab.com/handbook/engineering/quality/engineering-productivity/). ### Troubleshooting a failed webhook diff --git a/doc/development/go_guide/dependencies.md b/doc/development/go_guide/dependencies.md index f2b1ae9e7b8..aaf38d98625 100644 --- a/doc/development/go_guide/dependencies.md +++ b/doc/development/go_guide/dependencies.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Dependency Management in Go diff --git a/doc/development/go_guide/go_upgrade.md b/doc/development/go_guide/go_upgrade.md index 6bfa4ced229..5386221ea7e 100644 --- a/doc/development/go_guide/go_upgrade.md +++ b/doc/development/go_guide/go_upgrade.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Managing Go versions @@ -153,7 +153,7 @@ if you need help finding the correct person or labels: | Gitaly | [Issue Tracker](https://gitlab.com/gitlab-org/gitaly/-/issues) | | GitLab CLI (`glab`). | [Issue Tracker](https://gitlab.com/gitlab-org/cli/-/issues) | GitLab Compose Kit | [Issuer Tracker](https://gitlab.com/gitlab-org/gitlab-compose-kit/-/issues) | -| GitLab Container Registry | [Issue Tracker](https://gitlab.com/gitlab-org/container-registry) | +| GitLab container registry | [Issue Tracker](https://gitlab.com/gitlab-org/container-registry) | | GitLab Elasticsearch Indexer | [Issue Tracker](https://gitlab.com/gitlab-org/gitlab-elasticsearch-indexer/-/issues) | | GitLab agent server for Kubernetes (KAS) | [Issue Tracker](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/issues) | | GitLab Pages | [Issue Tracker](https://gitlab.com/gitlab-org/gitlab-pages/-/issues) | diff --git a/doc/development/go_guide/index.md b/doc/development/go_guide/index.md index c6d7b231b72..a9f4b22c778 100644 --- a/doc/development/go_guide/index.md +++ b/doc/development/go_guide/index.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Go standards and style guidelines @@ -26,7 +26,7 @@ can still have specifics. They are described in their respective The Go upgrade documentation [provides an overview](go_upgrade.md#overview) of how GitLab manages and ships Go binary support. -If a GitLab component requires a newer version of Go, please +If a GitLab component requires a newer version of Go, follow the [upgrade process](go_upgrade.md#updating-go-version) to ensure no customer, team, or component is adversely impacted. Sometimes, individual projects must also [manage builds with multiple versions of Go](go_upgrade.md#supporting-multiple-go-versions). diff --git a/doc/development/gotchas.md b/doc/development/gotchas.md index 59362dc33c0..0f872bfff01 100644 --- a/doc/development/gotchas.md +++ b/doc/development/gotchas.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Gotchas @@ -76,7 +76,7 @@ When run, this spec doesn't do what we might expect: This is because FactoryBot sequences are not reset for each example. -Please remember that sequence-generated values exist only to avoid having to +Remember that sequence-generated values exist only to avoid having to explicitly set attributes that have a uniqueness constraint when using a factory. ### Solution diff --git a/doc/development/graphql_guide/authorization.md b/doc/development/graphql_guide/authorization.md index 7b69ce36071..d40115bdb03 100644 --- a/doc/development/graphql_guide/authorization.md +++ b/doc/development/graphql_guide/authorization.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # GraphQL Authorization diff --git a/doc/development/graphql_guide/batchloader.md b/doc/development/graphql_guide/batchloader.md index 5f06f1faf1e..a4295cb3214 100644 --- a/doc/development/graphql_guide/batchloader.md +++ b/doc/development/graphql_guide/batchloader.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # GraphQL BatchLoader diff --git a/doc/development/graphql_guide/graphql_pro.md b/doc/development/graphql_guide/graphql_pro.md index 7c5770a4410..f6af4c88a47 100644 --- a/doc/development/graphql_guide/graphql_pro.md +++ b/doc/development/graphql_guide/graphql_pro.md @@ -1,7 +1,7 @@ --- stage: Manage group: Import and Integrate -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # GraphQL Pro diff --git a/doc/development/graphql_guide/index.md b/doc/development/graphql_guide/index.md index ef30d489832..5de90f8e858 100644 --- a/doc/development/graphql_guide/index.md +++ b/doc/development/graphql_guide/index.md @@ -1,7 +1,7 @@ --- stage: Manage group: Import and Integrate -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # GraphQL development guidelines diff --git a/doc/development/graphql_guide/monitoring.md b/doc/development/graphql_guide/monitoring.md index f92963dbdee..6e5c8e730eb 100644 --- a/doc/development/graphql_guide/monitoring.md +++ b/doc/development/graphql_guide/monitoring.md @@ -1,7 +1,7 @@ --- stage: Manage group: Import and Integrate -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Reading GraphQL logs diff --git a/doc/development/graphql_guide/pagination.md b/doc/development/graphql_guide/pagination.md index a858d9be681..117a9c691df 100644 --- a/doc/development/graphql_guide/pagination.md +++ b/doc/development/graphql_guide/pagination.md @@ -1,7 +1,7 @@ --- stage: Manage group: Import and Integrate -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # GraphQL pagination diff --git a/doc/development/graphql_guide/reviewing.md b/doc/development/graphql_guide/reviewing.md index 9c32179a89d..9216ea2884d 100644 --- a/doc/development/graphql_guide/reviewing.md +++ b/doc/development/graphql_guide/reviewing.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # GraphQL API merge request checklist @@ -54,7 +54,7 @@ GraphQL is a framework with many moving parts. It's important that the framework - Do not manually invoke framework bits. For example, do not instantiate resolvers during execution and instead let the framework do that. - You can subclass resolvers, as in `MyResolver.single` (see [deriving resolvers](../api_graphql_styleguide.md#deriving-resolvers)). - Use the `ready?` method for more complex argument logic (see [correct use of resolver#ready](../api_graphql_styleguide.md#correct-use-of-resolverready)). -- Use the `prepare` method for more complex argument validation (see [validating arguments](../api_graphql_styleguide.md#validating-arguments)). +- Use the `prepare` method for more complex argument validation (see [Preprocessing](https://graphql-ruby.org/fields/arguments.html#preprocessing)). For details, see [resolver guide](../api_graphql_styleguide.md#writing-resolvers). @@ -68,7 +68,8 @@ For details, see [authorization guide](authorization.md). Ensure: -- You avoid N+1s with [BatchLoader](batchloader.md) or [Lookahead](../api_graphql_styleguide.md#look-ahead) when appropriate. +- You have [checked for N+1s](../api_graphql_styleguide.md#how-to-see-n1-problems-in-development) and + used [optimizations](../api_graphql_styleguide.md#optimizations) to remove N+1s whenever possible. - You use [laziness](../api_graphql_styleguide.md#laziness) appropriately. ### Use appropriate types diff --git a/doc/development/i18n/externalization.md b/doc/development/i18n/externalization.md index 1ce35b254f1..47936dc6f70 100644 --- a/doc/development/i18n/externalization.md +++ b/doc/development/i18n/externalization.md @@ -1,7 +1,7 @@ --- stage: Manage group: Import and Integrate -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Internationalization for GitLab @@ -27,23 +27,27 @@ After you have the GitLab project ready, you can start working on the translatio The following tools are used: +- Custom written tools to aid day-to-day development work with translations: + + - `tooling/bin/gettext_extractor locale/gitlab.pot`: scan all source files for [new content to translate](#updating-the-po-files-with-the-new-content) + - `rake gettext:compile`: reads the contents of the PO files and generates JS files which + contain all the available translations for the Frontend. + - `rake gettext:lint`: [validate PO files](#validating-po-files) + - [`gettext_i18n_rails`](https://github.com/grosser/gettext_i18n_rails): - this gem allows us to translate content from models, views, and controllers. It also gives us - access to the following Rake tasks: + this gem allows us to translate content from models, views, and controllers. + It uses [`fast_gettext`](https://github.com/grosser/fast_gettext) under the hood. + It also provides access to the following Rake tasks, which are rarely needed in day-to-day: + + - `rake gettext:add_language[language]`: [adding a new language](#adding-a-new-language) - `rake gettext:find`: parses almost all the files from the Rails application looking for content marked for translation. It then updates the PO files with this content. - `rake gettext:pack`: processes the PO files and generates the binary MO files that the application uses. -- [`gettext_i18n_rails_js`](https://github.com/webhippie/gettext_i18n_rails_js): - this gem makes the translations available in JavaScript. It provides the following Rake task: - - - `rake gettext:compile`: reads the contents of the PO files and generates JS files which - contain all the available translations. - -- PO editor: there are multiple applications that can help us work with PO files. A good option is - [Poedit](https://poedit.net/download), +- PO editor: there are multiple applications that can help us work with PO files. + A good option is [Poedit](https://poedit.net/download), which is available for macOS, GNU/Linux, and Windows. ## Preparing a page for translation @@ -258,6 +262,11 @@ expect(wrapper.text()).toBe('There was an error: Please refresh and hope for the For more details you can see how we [keep translations dynamic](#keep-translations-dynamic). +## Making changes to translated strings + +If you change the source strings in GitLab, you must [update the `pot` file](#updating-the-po-files-with-the-new-content) before pushing your changes. +If the `pot` file is out of date, pre-push checks and a pipeline job for `gettext` fail. + ## Working with special content ### Interpolation @@ -839,7 +848,7 @@ When in doubt, try to follow the best practices described in this [Mozilla Devel ### Always pass string literals to the translation helpers -The `bin/rake gettext:regenerate` script parses the codebase and extracts all the strings from the +The `tooling/bin/gettext_extractor locale/gitlab.pot` script parses the codebase and extracts all the strings from the [translation helpers](#preparing-a-page-for-translation) ready to be translated. The script cannot resolve the strings if they are passed as variables or function calls. Therefore, @@ -865,7 +874,7 @@ Now that the new content is marked for translation, run this command to update t `locale/gitlab.pot` files: ```shell -bin/rake gettext:regenerate +tooling/bin/gettext_extractor locale/gitlab.pot ``` This command updates the `locale/gitlab.pot` file with the newly externalized strings and removes diff --git a/doc/development/i18n/index.md b/doc/development/i18n/index.md index b7e385a774a..9e39d5554ab 100644 --- a/doc/development/i18n/index.md +++ b/doc/development/i18n/index.md @@ -1,7 +1,7 @@ --- stage: Manage group: Import and Integrate -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Translate GitLab to your language @@ -29,6 +29,10 @@ strings. See [Externalization for GitLab](externalization.md). +### Editing externalized strings + +If you edit externalized strings in GitLab, you must [update the `pot` file](externalization.md#updating-the-po-files-with-the-new-content) before pushing your changes. + ## Translate strings The translation process is managed at [https://crowdin.com/project/gitlab-ee](https://crowdin.com/project/gitlab-ee) diff --git a/doc/development/i18n/merging_translations.md b/doc/development/i18n/merging_translations.md index 4f36cbe125a..d33b8e701e9 100644 --- a/doc/development/i18n/merging_translations.md +++ b/doc/development/i18n/merging_translations.md @@ -1,7 +1,7 @@ --- stage: Manage group: Import and Integrate -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Merging translations from Crowdin diff --git a/doc/development/i18n/proofreader.md b/doc/development/i18n/proofreader.md index f24ebacab18..35e423b28e9 100644 --- a/doc/development/i18n/proofreader.md +++ b/doc/development/i18n/proofreader.md @@ -1,7 +1,7 @@ --- stage: Manage group: Import and Integrate -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Proofread Translations @@ -33,7 +33,7 @@ are very appreciative of the work done by translators and proofreaders! - Huang Tao - [GitLab](https://gitlab.com/htve), [Crowdin](https://crowdin.com/profile/htve) - Victor Wu - [GitLab](https://gitlab.com/_victorwu_), [Crowdin](https://crowdin.com/profile/victorwu) - Xiaogang Wen - [GitLab](https://gitlab.com/xiaogang_cn), [Crowdin](https://crowdin.com/profile/xiaogang_gitlab) - - Qi Zhao - [GitLab](https://gitlab.com/zhaoqi01), [Crowdin](https://crowdin.com/profile/zhaoqi01) + - Zhiyuan Lu - [GitLab](https://gitlab.com/luzhiyuan.deer), [Crowdin](https://crowdin.com/profile/luzhiyuan.deer) - Chinese Traditional 繁體中文 - Weizhe Ding - [GitLab](https://gitlab.com/d.weizhe), [Crowdin](https://crowdin.com/profile/d.weizhe) - Yi-Jyun Pan - [GitLab](https://gitlab.com/pan93412), [Crowdin](https://crowdin.com/profile/pan93412) diff --git a/doc/development/i18n/translation.md b/doc/development/i18n/translation.md index cf6ee16f157..7149d431c30 100644 --- a/doc/development/i18n/translation.md +++ b/doc/development/i18n/translation.md @@ -1,7 +1,7 @@ --- stage: Manage group: Import and Integrate -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Translating GitLab @@ -96,7 +96,7 @@ For example, in German, the word _user_ can be translated into _Benutzer_ (male) ### Updating the glossary -To propose additions to the glossary, please +To propose additions to the glossary, [open an issue](https://gitlab.com/gitlab-org/gitlab/-/issues?scope=all&utf8=✓&state=all&label_name[]=Category%3AInternationalization). ## French translation guidelines diff --git a/doc/development/identity_verification.md b/doc/development/identity_verification.md index 7d1bf8586be..8058ce8b282 100644 --- a/doc/development/identity_verification.md +++ b/doc/development/identity_verification.md @@ -1,7 +1,7 @@ --- stage: Govern group: Anti-Abuse -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Identity verification development diff --git a/doc/development/image_scaling.md b/doc/development/image_scaling.md index 48b780b50bf..05792bfd5a3 100644 --- a/doc/development/image_scaling.md +++ b/doc/development/image_scaling.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Tenant Scale -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Image scaling guide diff --git a/doc/development/img/snowplow_flow.png b/doc/development/img/snowplow_flow.png Binary files differdeleted file mode 100644 index aae597edc13..00000000000 --- a/doc/development/img/snowplow_flow.png +++ /dev/null diff --git a/doc/development/import_export.md b/doc/development/import_export.md index 11bddf190fa..909c31c1b62 100644 --- a/doc/development/import_export.md +++ b/doc/development/import_export.md @@ -1,7 +1,7 @@ --- stage: Manage group: Import and Integrate -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Import/Export development documentation @@ -10,6 +10,8 @@ General development guidelines and tips for the [Import/Export feature](../user/ <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> This document is originally based on the [Import/Export 201 presentation available on YouTube](https://www.youtube.com/watch?v=V3i1OfExotE). +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For more context you can watch this [Deep dive on Import / Export Development on YouTube](https://www.youtube.com/watch?v=IYD39JMGK78). + ## Security The Import/Export feature is constantly updated (adding new things to export). However, @@ -286,13 +288,13 @@ Fixtures used in Import/Export specs live in `spec/fixtures/lib/gitlab/import_ex There are two versions of each of these fixtures: - A human readable single JSON file with all objects, called either `project.json` or `group.json`. -- A folder named `tree`, containing a tree of files in `ndjson` format. **Please do not edit files under this folder manually unless strictly necessary.** +- A folder named `tree`, containing a tree of files in `ndjson` format. **Do not edit files under this folder manually unless strictly necessary.** The tools to generate the NDJSON tree from the human-readable JSON files live in the [`gitlab-org/memory-team/team-tools`](https://gitlab.com/gitlab-org/memory-team/team-tools/-/blob/master/import-export/) project. ### Project -**Please use `legacy-project-json-to-ndjson.sh` to generate the NDJSON tree.** +**Use `legacy-project-json-to-ndjson.sh` to generate the NDJSON tree.** The NDJSON tree looks like: @@ -326,7 +328,7 @@ tree ### Group -**Please use `legacy-group-json-to-ndjson.rb` to generate the NDJSON tree.** +**Use `legacy-group-json-to-ndjson.rb` to generate the NDJSON tree.** The NDJSON tree looks like this: @@ -353,4 +355,4 @@ tree ``` WARNING: -When updating these fixtures, please ensure you update both `json` files and `tree` folder, as the tests apply to both. +When updating these fixtures, ensure you update both `json` files and `tree` folder, as the tests apply to both. diff --git a/doc/development/import_project.md b/doc/development/import_project.md index 0be17ea5873..f592bfc5d35 100644 --- a/doc/development/import_project.md +++ b/doc/development/import_project.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Test import project diff --git a/doc/development/index.md b/doc/development/index.md index abc19645ecb..b3a3729ed10 100644 --- a/doc/development/index.md +++ b/doc/development/index.md @@ -1,8 +1,7 @@ --- -type: index, dev stage: none group: unassigned -info: "See the Technical Writers assigned to Development Guidelines: https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-development-guidelines" +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. description: "Development Guidelines: learn how to contribute to GitLab." --- diff --git a/doc/development/integrations/index.md b/doc/development/integrations/index.md index 382dba58ae3..23a22d2c121 100644 --- a/doc/development/integrations/index.md +++ b/doc/development/integrations/index.md @@ -1,7 +1,7 @@ --- stage: Manage group: Import and Integrate -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. description: "GitLab's development guidelines for Integrations" --- @@ -63,13 +63,37 @@ You should always access the fields through their `getters` and not interact wit You **must not** write to the `properties` hash, you **must** use the generated setter method instead. Direct writes to this hash are not persisted. -You should also define validations for all your properties. To see how these fields are exposed in the frontend form for the integration, see [Customize the frontend form](#customize-the-frontend-form). Other approaches include using `Integration.prop_accessor` or `Integration.data_field`, which you might see in earlier versions of integrations. You should not use these approaches for new integrations. +### Define validations + +You should define Rails validations for all of your fields. + +Validations should only apply when the integration is enabled, by testing the `#activated?` method. + +Any field with the [`required:` property](#customize-the-frontend-form) should have a +corresponding validation for `presence`, as the `required:` field property is only for the frontend. + +For example: + +```ruby +module Integrations + class FooBar < Integration + with_options if: :activated? do + validates :key, presence: true, format: { with: KEY_REGEX } + validates :bar, inclusion: [true, false] + end + + field :key, required: true + field :bar, type: :checkbox + end +end +``` + ### Define trigger events Integrations are triggered by calling their `#execute` method in response to events in GitLab, @@ -194,7 +218,7 @@ This method should return an array of hashes for each field, where the keys can |:---------------|:--------|:---------|:-----------------------------|:-- | `type:` | symbol | true | `:text` | The type of the form field. Can be `:text`, `:textarea`, `:password`, `:checkbox`, or `:select`. | `name:` | string | true | | The property name for the form field. -| `required:` | boolean | false | `false` | Specify if the form field is required or optional. +| `required:` | boolean | false | `false` | Specify if the form field is required or optional. Note [backend validations](#define-validations) for presence are still needed. | `title:` | string | false | Capitalized value of `name:` | The label for the form field. | `placeholder:` | string | false | | A placeholder for the form field. | `help:` | string | false | | A help text that displays below the form field. @@ -353,6 +377,5 @@ You can refer to these issues for examples of adding new integrations: - [Datadog](https://gitlab.com/gitlab-org/gitlab/-/issues/270123): Metrics collector, similar to the Prometheus integration. - [EWM/RTC](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/36662): External issue tracker. -- [Shimo](https://gitlab.com/gitlab-org/gitlab/-/issues/343386): External wiki, similar to the Confluence and External Wiki integrations. - [Webex Teams](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31543): Chat notifications. - [ZenTao](https://gitlab.com/gitlab-org/gitlab/-/issues/338178): External issue tracker with custom issue views, similar to the Jira integration. diff --git a/doc/development/integrations/jenkins.md b/doc/development/integrations/jenkins.md index f52098394dc..2a9903a0665 100644 --- a/doc/development/integrations/jenkins.md +++ b/doc/development/integrations/jenkins.md @@ -1,7 +1,7 @@ --- stage: Manage group: Import and Integrate -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # How to run Jenkins in development environment (on macOS) @@ -24,9 +24,8 @@ brew services start jenkins GitLab does not allow requests to localhost or the local network by default. When running Jenkins on your local machine, you need to enable local access. 1. Log into your GitLab instance as an administrator. -1. On the left sidebar, select **Search or go to**. -1. Select **Admin Area**. -1. On the left sidebar, select **Settings > Network**. +1. On the left sidebar, at the bottom, select **Admin Area**. +1. Select **Settings > Network**. 1. Expand **Outbound requests**, and select the following checkboxes: - **Allow requests to the local network from webhooks and integrations** diff --git a/doc/development/integrations/jira_connect.md b/doc/development/integrations/jira_connect.md index c7fbb73ca36..fd5ff66243a 100644 --- a/doc/development/integrations/jira_connect.md +++ b/doc/development/integrations/jira_connect.md @@ -1,7 +1,7 @@ --- stage: Manage group: Import and Integrate -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Set up a Jira development environment @@ -73,7 +73,7 @@ To avoid external dependencies like Gitpod and a Jira Cloud instance, use the [J ``` 1. Restart GDK. -1. Go to `http://127.0.0.1:3000/-/profile/personal_access_tokens`. +1. Go to `http://127.0.0.1:3000/-/user_settings/personal_access_tokens`. 1. Create a new token with the `api` scope and copy the token. 1. Go to `http://localhost:9292`. 1. Paste the token and select **Install GitLab.com Jira Cloud app**. diff --git a/doc/development/integrations/secure.md b/doc/development/integrations/secure.md index 8c6e3145000..3fb89605bdd 100644 --- a/doc/development/integrations/secure.md +++ b/doc/development/integrations/secure.md @@ -1,7 +1,7 @@ --- stage: Secure group: Static Analysis -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Security scanner integration @@ -239,7 +239,7 @@ Success also includes the case when vulnerabilities are found. When a CI job fails, security report results are not ingested by GitLab, even if the job [allows failure](../../ci/yaml/index.md#allow_failure). However, the report artifacts are still uploaded to GitLab and available -for [download in the pipeline security tab](../../user/application_security/vulnerability_report/pipeline.md#download-security-scan-outputs). +for [download in the pipeline security tab](../../user/application_security/vulnerability_report/pipeline.md#downloading-security-scan-results). ### Logging diff --git a/doc/development/integrations/secure_partner_integration.md b/doc/development/integrations/secure_partner_integration.md index ab94cad3bdc..53c333a6f13 100644 --- a/doc/development/integrations/secure_partner_integration.md +++ b/doc/development/integrations/secure_partner_integration.md @@ -1,7 +1,7 @@ --- stage: Secure group: Static Analysis -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Secure Partner Integration - Onboarding Process @@ -104,7 +104,7 @@ and complete an integration with the Secure stage. - If you specified `remediations` in your artifact, it is proposed through our [remediation](../../user/application_security/vulnerabilities/index.md#resolve-a-vulnerability) interface. 1. Demo the integration to GitLab: - - After you have tested and are ready to demo your integration please + - After you have tested and are ready to demo your integration, [reach out](https://about.gitlab.com/partners/technology-partners/integrate/) to us. If you skip this step you won't be able to do supported marketing. 1. Begin doing supported marketing of your GitLab integration. @@ -119,4 +119,4 @@ that may be helpful as part of this process. This covers various topics related tool. If you have any issues while working through your integration or the steps -above, please create an issue to discuss with us further. +above, create an issue to discuss with us further. diff --git a/doc/development/interacting_components.md b/doc/development/interacting_components.md index 7fb711795c1..b9da531f4f5 100644 --- a/doc/development/interacting_components.md +++ b/doc/development/interacting_components.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Developing against interacting components or features diff --git a/doc/development/internal_analytics/index.md b/doc/development/internal_analytics/index.md index b0e47233777..b5403f56600 100644 --- a/doc/development/internal_analytics/index.md +++ b/doc/development/internal_analytics/index.md @@ -1,7 +1,7 @@ --- -stage: Analyze +stage: Monitor group: Analytics Instrumentation -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Internal analytics @@ -97,6 +97,7 @@ FROM common_mart.mart_behavior_structured_event WHERE event_action = 'feature_used' AND event_category = 'InternalEventTracking' AND behavior_date > '2023-08-01' --restricted minimum date for performance +AND app_id='gitlab' -- use gitlab for production events and gitlab-staging for events from staging GROUP BY 1 ORDER BY 1 desc ``` diff --git a/doc/development/internal_analytics/internal_event_instrumentation/event_definition_guide.md b/doc/development/internal_analytics/internal_event_instrumentation/event_definition_guide.md index 807e27d546e..2c53185a5db 100644 --- a/doc/development/internal_analytics/internal_event_instrumentation/event_definition_guide.md +++ b/doc/development/internal_analytics/internal_event_instrumentation/event_definition_guide.md @@ -1,7 +1,7 @@ --- -stage: Analyze +stage: Monitor group: Analytics Instrumentation -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Event definition guide @@ -49,7 +49,7 @@ identifiers: - user - namespace product_section: dev -product_stage: analyze +product_stage: monitor product_group: group::product analytics milestone: "16.4" introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/128029 diff --git a/doc/development/internal_analytics/internal_event_instrumentation/index.md b/doc/development/internal_analytics/internal_event_instrumentation/index.md index 35f9f31351e..a1683d4df85 100644 --- a/doc/development/internal_analytics/internal_event_instrumentation/index.md +++ b/doc/development/internal_analytics/internal_event_instrumentation/index.md @@ -1,7 +1,7 @@ --- -stage: Analyze +stage: Monitor group: Analytics Instrumentation -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Internal Event Tracking diff --git a/doc/development/internal_analytics/internal_event_instrumentation/introduction.md b/doc/development/internal_analytics/internal_event_instrumentation/introduction.md index e776691fdf0..1becf2da6ae 100644 --- a/doc/development/internal_analytics/internal_event_instrumentation/introduction.md +++ b/doc/development/internal_analytics/internal_event_instrumentation/introduction.md @@ -1,7 +1,7 @@ --- -stage: Analyze +stage: Monitor group: Analytics Instrumentation -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Internal event tracking diff --git a/doc/development/internal_analytics/internal_event_instrumentation/local_setup_and_debugging.md b/doc/development/internal_analytics/internal_event_instrumentation/local_setup_and_debugging.md index d9f45a2d93e..56e83184060 100644 --- a/doc/development/internal_analytics/internal_event_instrumentation/local_setup_and_debugging.md +++ b/doc/development/internal_analytics/internal_event_instrumentation/local_setup_and_debugging.md @@ -1,7 +1,7 @@ --- -stage: Analyze +stage: Monitor group: Analytics Instrumentation -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Local setup and debugging @@ -54,9 +54,8 @@ On GitLab.com events are sent to a collector configured by GitLab. By default, s You can configure your self-managed GitLab instance to use a custom Snowplow collector. -1. On the left sidebar, select **Search or go to**. -1. Select **Admin Area**. -1. On the left sidebar, select **Settings > General**. +1. On the left sidebar, at the bottom, select **Admin Area**. +1. Select **Settings > General**. 1. Expand **Snowplow**. 1. Select **Enable Snowplow tracking** and enter your Snowplow configuration information. For example: diff --git a/doc/development/internal_analytics/internal_event_instrumentation/migration.md b/doc/development/internal_analytics/internal_event_instrumentation/migration.md index 2a3a3560292..2ef439e21e9 100644 --- a/doc/development/internal_analytics/internal_event_instrumentation/migration.md +++ b/doc/development/internal_analytics/internal_event_instrumentation/migration.md @@ -1,7 +1,7 @@ --- -stage: Analyze +stage: Monitor group: Analytics Instrumentation -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Migrating existing tracking to internal event tracking @@ -23,7 +23,7 @@ The event triggered by Internal Events has some special properties compared to p 1. The `label`, `property` and `value` attributes are not used within Internal Events and are always empty. 1. The `category` is automatically set to `InternalEventTracking` -Please make sure that you are okay with this change before you migrate and dashboards are changed accordingly. +Make sure that you are okay with this change before you migrate and dashboards are changed accordingly. ### Backend @@ -42,19 +42,14 @@ Gitlab::InternalEvents.track_event('ci_templates_unique', namespace: namespace, In addition, you have to create definitions for the metrics that you would like to track. -To generate metric definitions, you can use the generator like this: +To generate metric definitions, you can use the generator: ```shell -bin/rails g gitlab:analytics:internal_events \ - --time_frames=7d 28d\ - --group=project_management \ - --stage=plan \ - --section=dev \ - --event=ci_templates_unique \ - --unique=user.id \ - --mr=https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121544 +ruby scripts/internal_events/cli.rb ``` +The generator walks you through the required inputs step-by-step. + ### Frontend If you are using the `Tracking` mixin in the Vue component, you can replace it with the `InternalEvents` mixin. diff --git a/doc/development/internal_analytics/internal_event_instrumentation/quick_start.md b/doc/development/internal_analytics/internal_event_instrumentation/quick_start.md index 15ad4266d1b..6f48f83e7ca 100644 --- a/doc/development/internal_analytics/internal_event_instrumentation/quick_start.md +++ b/doc/development/internal_analytics/internal_event_instrumentation/quick_start.md @@ -1,7 +1,7 @@ --- -stage: Analyze +stage: Monitor group: Analytics Instrumentation -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Quick start for Internal Event Tracking @@ -17,36 +17,17 @@ In order to instrument your code with Internal Events Tracking you need to do th ## Defining event and metrics -<div class="video-fallback"> - See the video about <a href="https://www.youtube.com/watch?v=QICKWznLyy0">adding events and metrics using the generator</a> -</div> -<figure class="video_container"> - <iframe src="https://www.youtube-nocookie.com/embed/QICKWznLyy0" frameborder="0" allowfullscreen="true"> </iframe> -</figure> - -To create an event and metric definitions you can use the `internal_events` generator. - -This example creates an event definition for an event called `project_created` and two metric definitions, which are aggregated every 7 and 28 days. +To create event and/or metric definitions, use the `internal_events` generator from the `gitlab` directory: ```shell -bundle exec rails generate gitlab:analytics:internal_events \ ---time_frames=7d 28d \ ---group=project_management \ ---stage=plan \ ---section=dev \ ---event=project_created \ ---unique=user.id \ ---mr=https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121544 +ruby scripts/internal_events/cli.rb ``` -Where: - -- `time_frames`: Valid options are `7d` and `28d` if you provide a `unique` value and `all` for metrics without `unique`. We are working to make `7d` and `28d` work for metrics with `all` time frame in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/411264). -- `unique`: Valid options are `user.id`, `project.id`, and `namespace.id`, as they are logged as part of the standard context. We [are actively working](https://gitlab.com/gitlab-org/gitlab/-/issues/411255) on a way to define uniqueness on arbitrary properties sent with the event, such as `merge_request.id`. +This CLI will help you create the correct defintion files based on your specific use-case, then provide code examples for instrumentation and testing. ## Trigger events -Triggering an event and thereby updating a metric is slightly different on backend and frontend. Please refer to the relevant section below. +Triggering an event and thereby updating a metric is slightly different on backend and frontend. Refer to the relevant section below. ### Backend tracking @@ -54,17 +35,27 @@ To trigger an event, call the `Gitlab::InternalEvents.track_event` method with t ```ruby Gitlab::InternalEvents.track_event( - "i_code_review_user_apply_suggestion", - user: user, - namespace: namespace, - project: project - ) + "i_code_review_user_apply_suggestion", + user: user, + namespace: namespace, + project: project +) ``` This method automatically increments all RedisHLL metrics relating to the event `i_code_review_user_apply_suggestion`, and sends a corresponding Snowplow event with all named arguments and standard context (SaaS only). +If you have defined a metric with a `unique` property such as `unique: project.id` it is required that you provide the `project` argument. + +It is encouraged to fill out as many of `user`, `namespace` and `project` as possible as it increases the data quality and make it easier to define metrics in the future. + +If a `project` but no `namespace` is provided, the `project.namespace` is used as the `namespace` for the event. + ### Frontend tracking +Any frontend tracking call automatically passes the values `user.id`, `namespace.id`, and `project.id` from the current context of the page. + +If you need to pass any further properties, such as `extra`, `context`, `label`, `property`, and `value`, you can use the [deprecated snowplow implementation](https://docs.gitlab.com/16.4/ee/development/internal_analytics/snowplow/implementation.html). In this case, let us know about your specific use-case in our [feedback issue for Internal Events](https://gitlab.com/gitlab-org/analytics-section/analytics-instrumentation/internal/-/issues/690). + #### Vue components In Vue components, tracking can be done with [Vue mixin](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/tracking/internal_events.js#L29). @@ -148,27 +139,3 @@ Sometimes we want to send internal events when the component is rendered or load = render Pajamas::ButtonComponent.new(button_options: { data: { event_tracking_load: 'true', event_tracking: 'i_devops' } }) do = _("New project") ``` - -### Props - -Apart from `eventName`, the `trackEvent` method also supports `extra` and `context` props. - -`extra`: Use this property to append supplementary information to GitLab standard context. -`context`: Use this property to attach an additional context, if needed. - -The following example shows how to use the `extra` and `context` props with the `trackEvent` method: - -```javascript -this.trackEvent('i_code_review_user_apply_suggestion', { - extra: { - projectId : 123, - }, - context: { - schema: 'iglu:com.gitlab/design_management_context/jsonschema/1-0-0', - data: { - 'design-version-number': '1.0.0', - 'design-is-current-version': '1.0.1', - }, - }, -}); -``` diff --git a/doc/development/internal_analytics/internal_events/index.md b/doc/development/internal_analytics/internal_events/index.md deleted file mode 100644 index d987317a2b0..00000000000 --- a/doc/development/internal_analytics/internal_events/index.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: '../internal_event_tracking/quick_start.md' -remove_date: '2023-10-27' ---- - -This document was moved to [another location](../internal_event_tracking/quick_start.md). - -<!-- This redirect file can be deleted after <2023-10-27>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/development/internal_analytics/metrics/index.md b/doc/development/internal_analytics/metrics/index.md index 45089ec8164..becd81eca51 100644 --- a/doc/development/internal_analytics/metrics/index.md +++ b/doc/development/internal_analytics/metrics/index.md @@ -1,7 +1,7 @@ --- -stage: Analyze +stage: Monitor group: Analytics Instrumentation -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Metrics diff --git a/doc/development/internal_analytics/metrics/metrics_dictionary.md b/doc/development/internal_analytics/metrics/metrics_dictionary.md index 6a3291eaba5..463d5be9100 100644 --- a/doc/development/internal_analytics/metrics/metrics_dictionary.md +++ b/doc/development/internal_analytics/metrics/metrics_dictionary.md @@ -1,7 +1,7 @@ --- -stage: Analyze +stage: Monitor group: Analytics Instrumentation -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Metrics Dictionary Guide @@ -29,28 +29,28 @@ Only metrics with a metric definition YAML and whose status is not `removed` are Each metric is defined in a separate YAML file consisting of a number of fields: -| Field | Required | Additional information | -|---------------------|----------|----------------------------------------------------------------| -| `key_path` | yes | JSON key path for the metric, location in Service Ping payload. | -| `description` | yes | | -| `product_section` | yes | The [section](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/data/sections.yml). | -| `product_stage` | yes | The [stage](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) for the metric. | -| `product_group` | yes | The [group](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) that owns the metric. | -| `value_type` | yes | `string`; one of [`string`, `number`, `boolean`, `object`](https://json-schema.org/understanding-json-schema/reference/type.html). | -| `status` | yes | `string`; [status](#metric-statuses) of the metric, may be set to `active`, `removed`, `broken`. | -| `time_frame` | yes | `string`; may be set to a value like `7d`, `28d`, `all`, `none`. | -| `data_source` | yes | `string`; may be set to a value like `database`, `redis`, `redis_hll`, `prometheus`, `system`, `license`, `internal_events`. | -| `data_category` | yes | `string`; [categories](#data-category) of the metric, may be set to `operational`, `optional`, `subscription`, `standard`. The default value is `optional`.| -| `instrumentation_class` | yes | `string`; [the class that implements the metric](../service_ping/metrics_instrumentation.md). | -| `distribution` | yes | `array`; may be set to one of `ce, ee` or `ee`. The [distribution](https://about.gitlab.com/handbook/marketing/brand-and-product-marketing/product-and-solution-marketing/tiers/#definitions) where the tracked feature is available. | -| `performance_indicator_type` | no | `array`; may be set to one of [`gmau`, `smau`, `paid_gmau`, `umau` or `customer_health_score`](https://about.gitlab.com/handbook/business-technology/data-team/data-catalog/xmau-analysis/). | +| Field | Required | Additional information | +|---------------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `key_path` | yes | JSON key path for the metric, location in Service Ping payload. | +| `description` | yes | | +| `product_section` | yes | The [section](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/data/sections.yml). | +| `product_stage` | yes | The [stage](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) for the metric. | +| `product_group` | yes | The [group](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) that owns the metric. | +| `value_type` | yes | `string`; one of [`string`, `number`, `boolean`, `object`](https://json-schema.org/understanding-json-schema/reference/type.html). | +| `status` | yes | `string`; [status](#metric-statuses) of the metric, may be set to `active`, `removed`, `broken`. | +| `time_frame` | yes | `string`; may be set to a value like `7d`, `28d`, `all`, `none`. | +| `data_source` | yes | `string`; may be set to a value like `database`, `redis`, `redis_hll`, `prometheus`, `system`, `license`, `internal_events`. | +| `data_category` | yes | `string`; [categories](#data-category) of the metric, may be set to `operational`, `optional`, `subscription`, `standard`. The default value is `optional`. | +| `instrumentation_class` | no | `string`; used for metrics with `data_source` other than `internal_events`. See [the class that implements the metric](../service_ping/metrics_instrumentation.md). | +| `distribution` | yes | `array`; may be set to one of `ce, ee` or `ee`. The [distribution](https://about.gitlab.com/handbook/marketing/brand-and-product-marketing/product-and-solution-marketing/tiers/#definitions) where the tracked feature is available. | +| `performance_indicator_type` | no | `array`; may be set to one of [`gmau`, `smau`, `paid_gmau`, `umau` or `customer_health_score`](https://about.gitlab.com/handbook/business-technology/data-team/data-catalog/xmau-analysis/). | | `tier` | yes | `array`; may contain one or a combination of `free`, `premium` or `ultimate`. The [tier](https://about.gitlab.com/handbook/marketing/brand-and-product-marketing/product-and-solution-marketing/tiers/#definitions) where the tracked feature is available. This should be verbose and contain all tiers where a metric is available. | -| `milestone` | yes | The milestone when the metric is introduced and when it's available to self-managed instances with the official GitLab release. | -| `milestone_removed` | no | The milestone when the metric is removed. Required for removed metrics. | -| `introduced_by_url` | no | The URL to the merge request that introduced the metric to be available for self-managed instances. | -| `removed_by_url` | no | The URL to the merge request that removed the metric. Required for removed metrics. | -| `repair_issue_url` | no | The URL of the issue that was created to repair a metric with a `broken` status. | -| `options` | no | `object`: options information needed to calculate the metric value. | +| `milestone` | yes | The milestone when the metric is introduced and when it's available to self-managed instances with the official GitLab release. | +| `milestone_removed` | no | The milestone when the metric is removed. Required for removed metrics. | +| `introduced_by_url` | no | The URL to the merge request that introduced the metric to be available for self-managed instances. | +| `removed_by_url` | no | The URL to the merge request that removed the metric. Required for removed metrics. | +| `repair_issue_url` | no | The URL of the issue that was created to repair a metric with a `broken` status. | +| `options` | no | `object`: options information needed to calculate the metric value. | ### Metric `key_path` diff --git a/doc/development/internal_analytics/metrics/metrics_instrumentation.md b/doc/development/internal_analytics/metrics/metrics_instrumentation.md index df0b3ff9a6a..fa3d4069955 100644 --- a/doc/development/internal_analytics/metrics/metrics_instrumentation.md +++ b/doc/development/internal_analytics/metrics/metrics_instrumentation.md @@ -1,7 +1,7 @@ --- -stage: Analyze +stage: Monitor group: Analytics Instrumentation -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Metrics instrumentation guide diff --git a/doc/development/internal_analytics/metrics/metrics_lifecycle.md b/doc/development/internal_analytics/metrics/metrics_lifecycle.md index 2361b8d2ca6..681992b4379 100644 --- a/doc/development/internal_analytics/metrics/metrics_lifecycle.md +++ b/doc/development/internal_analytics/metrics/metrics_lifecycle.md @@ -1,7 +1,7 @@ --- -stage: Analyze +stage: Monitor group: Analytics Instrumentation -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Metric lifecycle @@ -24,7 +24,7 @@ As a result, if you need to change one of the following parts of a metric, you n - **calculation logic**: This means any changes that can produce a different value than the previous implementation - **YAML attributes**: The following attributes are directly used for analysis or calculation: `key_path`, `time_frame`, `value_type`, `data_source`. -If you change the `performance_indicator_type` attribute of a metric or think your case needs an exception from the outlined rules then please notify the Customer Success Ops team (`@csops-team`), Analytics Engineers (`@gitlab-data/analytics-engineers`), and Product Analysts (`@gitlab-data/product-analysts`) teams by `@` mentioning those groups in a comment on the merge request or issue. +If you change the `performance_indicator_type` attribute of a metric or think your case needs an exception from the outlined rules then notify the Customer Success Ops team (`@csops-team`), Analytics Engineers (`@gitlab-data/analytics-engineers`), and Product Analysts (`@gitlab-data/product-analysts`) teams by `@` mentioning those groups in a comment on the merge request or issue. You can change any other attributes without impact to the calculation or analysis. See [this video tutorial](https://youtu.be/bYf3c01KCls) for help updating metric attributes. @@ -91,7 +91,7 @@ To remove a metric: therefore continue to report the removed metric. The Analytics Instrumentation team requires a record of all removed metrics to identify and filter them. - For example please take a look at this [merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60149/diffs#b01f429a54843feb22265100c0e4fec1b7da1240_10_10). + For example, take a look at this [merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60149/diffs#b01f429a54843feb22265100c0e4fec1b7da1240_10_10). 1. After you verify the metric can be safely removed, remove the metric's instrumentation from @@ -99,7 +99,7 @@ To remove a metric: or [`ee/lib/ee/gitlab/usage_data.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/ee/gitlab/usage_data.rb). - For example please take a look at this [merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60149/diffs#6335dc533bd21df26db9de90a02dd66278c2390d_167_167). + For example, take a look at this [merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60149/diffs#6335dc533bd21df26db9de90a02dd66278c2390d_167_167). 1. Remove any other records related to the metric: - The feature flag YAML file at [`config/feature_flags/*/*.yaml`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/config/feature_flags). diff --git a/doc/development/internal_analytics/review_guidelines.md b/doc/development/internal_analytics/review_guidelines.md index bb7491fd20d..eb59b834cbc 100644 --- a/doc/development/internal_analytics/review_guidelines.md +++ b/doc/development/internal_analytics/review_guidelines.md @@ -1,7 +1,7 @@ --- -stage: Analyze +stage: Monitor group: Analytics Instrumentation -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Internal Analytics review guidelines diff --git a/doc/development/internal_analytics/service_ping/index.md b/doc/development/internal_analytics/service_ping/index.md index f010884272b..eb0e384b10d 100644 --- a/doc/development/internal_analytics/service_ping/index.md +++ b/doc/development/internal_analytics/service_ping/index.md @@ -1,7 +1,7 @@ --- -stage: Analyze +stage: Monitor group: Analytics Instrumentation -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Service Ping development guidelines @@ -361,7 +361,7 @@ Rake tasks exist to export Service Ping data in different formats. - The Rake tasks export the Redis counter class or the line of the Redis block for `redis_usage_data`. - The Rake tasks calculate the `alt_usage_data` metrics. -In the home directory of your local GitLab installation run the following Rake tasks for the YAML and JSON versions respectively: +In the home directory of your local GitLab installation run the following Rake tasks for either the YAML or the JSON versions: ```shell # for YAML export of SQL queries diff --git a/doc/development/internal_analytics/service_ping/metrics_dictionary.md b/doc/development/internal_analytics/service_ping/metrics_dictionary.md deleted file mode 100644 index 9adcaaf431d..00000000000 --- a/doc/development/internal_analytics/service_ping/metrics_dictionary.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: '../metrics/metrics_dictionary.md' -remove_date: '2023-09-25' ---- - -This document was moved to [another location](../metrics/metrics_dictionary.md). - -<!-- This redirect file can be deleted after <2023-12-25>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/development/internal_analytics/service_ping/metrics_lifecycle.md b/doc/development/internal_analytics/service_ping/metrics_lifecycle.md deleted file mode 100644 index 126bc3988ed..00000000000 --- a/doc/development/internal_analytics/service_ping/metrics_lifecycle.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: '../metrics/metrics_lifecycle.md' -remove_date: '2023-09-25' ---- - -This document was moved to [another location](../metrics/metrics_lifecycle.md). - -<!-- This redirect file can be deleted after <2023-12-25>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/development/internal_analytics/service_ping/performance_indicator_metrics.md b/doc/development/internal_analytics/service_ping/performance_indicator_metrics.md deleted file mode 100644 index 7388415f384..00000000000 --- a/doc/development/internal_analytics/service_ping/performance_indicator_metrics.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: '../metrics/metrics_dictionary.md#performance-indicator-metrics' -remove_date: '2023-09-25' ---- - -This document was moved to [another location](../metrics/metrics_lifecycle.md). - -<!-- This redirect file can be deleted after <2023-12-25>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/development/internal_analytics/service_ping/troubleshooting.md b/doc/development/internal_analytics/service_ping/troubleshooting.md index ba56ee1e223..1b74921fb2f 100644 --- a/doc/development/internal_analytics/service_ping/troubleshooting.md +++ b/doc/development/internal_analytics/service_ping/troubleshooting.md @@ -1,7 +1,7 @@ --- -stage: Analyze +stage: Monitor group: Analytics Instrumentation -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Troubleshooting Service Ping @@ -69,7 +69,7 @@ and run a local container instance: 1. In the downstream pipeline, wait for the `gitlab-docker` job to finish. 1. Open the job logs and locate the full container name including the version. It takes the following form: `registry.gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/gitlab-ee:<VERSION>`. 1. On your local machine, make sure you are signed in to the GitLab Docker registry. You can find the instructions for this in - [Authenticate to the GitLab Container Registry](../../../user/packages/container_registry/authenticate_with_container_registry.md). + [Authenticate to the GitLab container registry](../../../user/packages/container_registry/authenticate_with_container_registry.md). 1. Once signed in, download the new image by using `docker pull registry.gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/gitlab-ee:<VERSION>` 1. For more information about working with and running Omnibus GitLab containers in Docker, refer to [GitLab Docker images](../../../install/docker.md) documentation. @@ -142,8 +142,7 @@ checking the configuration file of your GitLab instance: - Using the Admin Area: - 1. On the left sidebar, select **Search or go to**. - 1. Select **Admin Area**. + 1. On the left sidebar, at the bottom, select **Admin Area**. 1. On the left sidebar, select **Settings > Metrics and profiling**. 1. Expand **Usage statistics**. 1. Are you able to check or uncheck the checkbox to disable Service Ping? @@ -200,8 +199,7 @@ To work around this bug, you have two options: sudo gitlab-ctl reconfigure ``` - 1. On the left sidebar, select **Search or go to**. - 1. Select **Admin Area**. + 1. On the left sidebar, at the bottom, select **Admin Area**. 1. On the left sidebar, select **Settings > Metrics and profiling**. 1. Expand **Usage statistics**. 1. Clear the **Enable Service Ping** checkbox. diff --git a/doc/development/internal_api/index.md b/doc/development/internal_api/index.md index 9b5bafaad8f..84f5d09418f 100644 --- a/doc/development/internal_api/index.md +++ b/doc/development/internal_api/index.md @@ -1,8 +1,7 @@ --- stage: Create group: Source Code -info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" -type: reference, api +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments" --- # Internal API @@ -467,8 +466,7 @@ Example response: ## GitLab agent endpoints > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/41045) in GitLab 13.4. -> - This feature is not deployed on GitLab.com -> - It's not recommended for production use. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/432773) in GitLab 16.7. The following endpoints are used by the GitLab agent server (`kas`) for various purposes. @@ -556,6 +554,46 @@ curl --request POST --header "Gitlab-Kas-Api-Request: <JWT token>" --header "Con --data '{"counters": {"gitops_sync":1}}' "http://localhost:3000/api/v4/internal/kubernetes/usage_metrics" ``` +### GitLab agent events + +Called from GitLab agent server (`kas`) to track events. + +| Attribute | Type | Required | Description | +|:------------------------------------------------------------------------------|:--------------|:---------|:--------------------------------------------------------------------------| +| `events` | hash | no | Hash of events | +| `events["k8s_api_proxy_requests_unique_users_via_ci_access"]` | hash array | no | Array of events for `k8s_api_proxy_requests_unique_users_via_ci_access` | +| `events["k8s_api_proxy_requests_unique_users_via_ci_access"]["user_id"]` | integer | no | The user ID for the event | +| `events["k8s_api_proxy_requests_unique_users_via_ci_access"]["project_id"]` | integer | no | The project ID for the event | +| `events["k8s_api_proxy_requests_unique_users_via_user_access"]` | hash array | no | Array of events for `k8s_api_proxy_requests_unique_users_via_user_access` | +| `events["k8s_api_proxy_requests_unique_users_via_user_access"]["user_id"]` | integer | no | The user ID for the event | +| `events["k8s_api_proxy_requests_unique_users_via_user_access"]["project_id"]` | integer | no | The project ID for the event | +| `events["k8s_api_proxy_requests_unique_users_via_pat_access"]` | hash array | no | Array of events for `k8s_api_proxy_requests_unique_users_via_pat_access` | +| `events["k8s_api_proxy_requests_unique_users_via_pat_access"]["user_id"]` | integer | no | The user ID for the event | +| `events["k8s_api_proxy_requests_unique_users_via_pat_access"]["project_id"]` | integer | no | The project ID for the event | + +```plaintext +POST /internal/kubernetes/agent_events +``` + +Example Request: + +```shell +curl --request POST \ + --url "http://localhost:3000/api/v4/internal/kubernetes/agent_events" \ + --header "Gitlab-Kas-Api-Request: <JWT token>" \ + --header "Content-Type: application/json" \ + --data '{ + "events": { + "k8s_api_proxy_requests_unique_users_via_ci_access": [ + { + "user_id": 1, + "project_id": 1 + } + ] + } + }' +``` + ### Create Starboard vulnerability Called from the GitLab agent server (`kas`) to create a security vulnerability @@ -1236,10 +1274,12 @@ This group SCIM API is different to the [SCIM API](../../api/scim.md). The SCIM - Does not implement the [RFC7644 protocol](https://www.rfc-editor.org/rfc/rfc7644). - Gets, checks, updates, and deletes SCIM identities within groups. +NOTE: +This API does not require the `Gitlab-Shell-Api-Request` header. + ### Get a list of SCIM provisioned users -This endpoint is used as part of the SCIM syncing mechanism. It only returns -a single user based on a unique ID which should match the `extern_uid` of the user. +This endpoint is used as part of the SCIM syncing mechanism. It returns a list of users depending on the filter used. ```plaintext GET /api/scim/v2/groups/:group_path/Users @@ -1257,7 +1297,7 @@ Parameters: NOTE: Pagination follows the [SCIM spec](https://www.rfc-editor.org/rfc/rfc7644#section-3.4.2.4) rather than GitLab pagination as used elsewhere. If records change between requests it is possible for a page to either be missing records that have moved to a different page or repeat records from a previous request. -Example request: +Example request filtering on a specific identifier: ```shell curl "https://gitlab.example.com/api/scim/v2/groups/test_group/Users?filter=id%20eq%20%220b1d561c-21ff-4092-beab-8154b17f82f2%22" \ @@ -1414,7 +1454,7 @@ Parameters: | `group_path` | string | yes | Full path to the group. | | `Operations` | JSON string | yes | An [operations](#available-operations) expression. | -Example request: +Example request to update the user's `id`: ```shell curl --verbose --request PATCH "https://gitlab.example.com/api/scim/v2/groups/test_group/Users/f0b1d561c-21ff-4092-beab-8154b17f82f2" \ @@ -1424,6 +1464,16 @@ curl --verbose --request PATCH "https://gitlab.example.com/api/scim/v2/groups/te Returns an empty response with a `204` status code if successful. +Example request to set the user's `active` state: + +```shell +curl --verbose --request PATCH "https://gitlab.example.com/api/scim/v2/groups/test_group/Users/f0b1d561c-21ff-4092-beab-8154b17f82f2" \ + --data '{ "Operations": [{"op":"replace","path":"active","value":"true"}] }' \ + --header "Authorization: Bearer <your_scim_token>" --header "Content-Type: application/scim+json" +``` + +Returns an empty response with a `204` status code if successful. + ### Remove a single SCIM provisioned user Removes the user's SSO identity and group membership. @@ -1472,10 +1522,12 @@ This instance SCIM API is different to the [SCIM API](../../api/scim.md). The SC - Does not implement the [RFC7644 protocol](https://www.rfc-editor.org/rfc/rfc7644). - Gets, checks, updates, and deletes SCIM identities within groups. +NOTE: +This API does not require the `Gitlab-Shell-Api-Request` header. + ### Get a list of SCIM provisioned users -This endpoint is used as part of the SCIM syncing mechanism. It only returns -a single user based on a unique ID which should match the `extern_uid` of the user. +This endpoint is used as part of the SCIM syncing mechanism. It returns a list of users depending on the filter used. ```plaintext GET /api/scim/v2/application/Users @@ -1651,9 +1703,9 @@ curl --verbose --request PATCH "https://gitlab.example.com/api/scim/v2/applicati Returns an empty response with a `204` status code if successful. -### Remove a single SCIM provisioned user +### Block a single SCIM provisioned user -The user is placed in an `ldap_blocked` status and signed out. This means +The user is placed in a `blocked` state and signed out. This means the user cannot sign in or push or pull code. ```plaintext diff --git a/doc/development/internal_api/internal_api_allowed.md b/doc/development/internal_api/internal_api_allowed.md index d5e4e8cf63d..8af6a85712f 100644 --- a/doc/development/internal_api/internal_api_allowed.md +++ b/doc/development/internal_api/internal_api_allowed.md @@ -1,8 +1,7 @@ --- stage: Create group: Source Code -info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" -type: reference, api +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments" --- # Internal allowed API diff --git a/doc/development/internal_users.md b/doc/development/internal_users.md index 2b809137906..4af6245a64c 100644 --- a/doc/development/internal_users.md +++ b/doc/development/internal_users.md @@ -1,9 +1,8 @@ --- description: "Internal users documentation." -type: concepts, reference, dev stage: none group: unassigned -info: "See the Technical Writers assigned to Development Guidelines: https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-development-guidelines" +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Internal users diff --git a/doc/development/issuable-like-models.md b/doc/development/issuable-like-models.md index 0b48883fe58..3e98dde6a2e 100644 --- a/doc/development/issuable-like-models.md +++ b/doc/development/issuable-like-models.md @@ -1,7 +1,7 @@ --- stage: Plan group: Project Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Issuable-like Rails models utilities diff --git a/doc/development/issue_types.md b/doc/development/issue_types.md index d0d513af781..8bacdc06c37 100644 --- a/doc/development/issue_types.md +++ b/doc/development/issue_types.md @@ -1,7 +1,7 @@ --- stage: Plan group: Project Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Issue Types (deprecated) diff --git a/doc/development/json.md b/doc/development/json.md index bdb7f73ab62..2292e102fa3 100644 --- a/doc/development/json.md +++ b/doc/development/json.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # JSON development guidelines diff --git a/doc/development/kubernetes.md b/doc/development/kubernetes.md index e537b4c4c76..43341483d86 100644 --- a/doc/development/kubernetes.md +++ b/doc/development/kubernetes.md @@ -1,7 +1,7 @@ --- stage: Deploy group: Environments -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Kubernetes integration development guidelines diff --git a/doc/development/labels/index.md b/doc/development/labels/index.md index c2caabda109..4288408baf6 100644 --- a/doc/development/labels/index.md +++ b/doc/development/labels/index.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Labels @@ -24,9 +24,9 @@ Most issues will have labels for at least one of the following: - Priority: `~"priority::1"`, `~"priority::2"`, `~"priority::3"`, `~"priority::4"` - Severity: `~"severity::1"`, `~"severity::2"`, `~"severity::3"`, `~"severity::4"` -Please add `~"breaking change"` label if the issue can be considered as a [breaking change](../deprecation_guidelines/index.md). +Add `~"breaking change"` label if the issue can be considered as a [breaking change](../deprecation_guidelines/index.md). -Please add `~security` label if the issue is related to application security. +Add `~security` label if the issue is related to application security. All labels, their meaning and priority are defined on the [labels page](https://gitlab.com/gitlab-org/gitlab/-/labels). @@ -252,7 +252,7 @@ We have the following priority labels: - `~"priority::3"` - `~"priority::4"` -Please refer to the issue triage [priority label](https://about.gitlab.com/handbook/engineering/quality/issue-triage/#priority) section in our handbook to see how it's used. +Refer to the issue triage [priority label](https://about.gitlab.com/handbook/engineering/quality/issue-triage/#priority) section in our handbook to see how it's used. ## Severity labels @@ -263,7 +263,7 @@ We have the following severity labels: - `~"severity::3"` - `~"severity::4"` -Please refer to the issue triage [severity label](https://about.gitlab.com/handbook/engineering/quality/issue-triage/#severity) section in our handbook to see how it's used. +Refer to the issue triage [severity label](https://about.gitlab.com/handbook/engineering/quality/issue-triage/#severity) section in our handbook to see how it's used. ## Label for community contributors @@ -301,7 +301,7 @@ with the [label `~"Community Challenge"`](https://gitlab.com/gitlab-org/gitlab/- If your MR for the `~"Community Challenge"` issue gets merged, you will also have a chance to win a custom GitLab merchandise. -If you've decided that you would like to work on an issue, please @-mention +If you've decided that you would like to work on an issue, @-mention the [appropriate product manager](https://about.gitlab.com/handbook/product/#who-to-talk-to-for-what) as soon as possible. The product manager will then pull in appropriate GitLab team members to further discuss scope, design, and technical considerations. This will diff --git a/doc/development/lfs.md b/doc/development/lfs.md index bb327bf5d04..289c258fafe 100644 --- a/doc/development/lfs.md +++ b/doc/development/lfs.md @@ -1,7 +1,7 @@ --- stage: Create group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Git LFS development guidelines diff --git a/doc/development/licensing.md b/doc/development/licensing.md index de2df3c8ca1..c84f42270c3 100644 --- a/doc/development/licensing.md +++ b/doc/development/licensing.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # GitLab Licensing and Compatibility @@ -44,7 +44,7 @@ To tell License Finder about a dependency's license if it isn't auto-detected: license_finder licenses add my_unknown_dependency MIT ``` -For all of the above, please include `--why "Reason"` and `--who "My Name"` so the `decisions.yml` file can keep track of when, why, and who approved of a dependency. +For all of the above, include `--why "Reason"` and `--who "My Name"` so the `decisions.yml` file can keep track of when, why, and who approved of a dependency. More detailed information on how the gem and its commands work is available in the [License Finder README](https://github.com/pivotal/LicenseFinder). @@ -77,4 +77,4 @@ Those projects are set to use a test license encryption key by default. ## Additional information -Please see the [Open Source](https://about.gitlab.com/handbook/engineering/open-source/#using-open-source-libraries) page for more information on licensing. +See the [Open Source](https://about.gitlab.com/handbook/engineering/open-source/#using-open-source-libraries) page for more information on licensing. diff --git a/doc/development/logging.md b/doc/development/logging.md index 908fa16d3f8..2af914d76ef 100644 --- a/doc/development/logging.md +++ b/doc/development/logging.md @@ -1,7 +1,7 @@ --- -stage: Monitor +stage: Service Management group: Respond -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Logging development guidelines @@ -361,7 +361,7 @@ class MyExampleWorker end ``` -Please see [this example](https://gitlab.com/gitlab-org/gitlab/-/blob/16ecc33341a3f6b6bebdf78d863c5bce76b040d3/app/workers/ci/pipeline_artifacts/expire_artifacts_worker.rb#L20-21) +See [this example](https://gitlab.com/gitlab-org/gitlab/-/blob/16ecc33341a3f6b6bebdf78d863c5bce76b040d3/app/workers/ci/pipeline_artifacts/expire_artifacts_worker.rb#L20-21) which logs a count of how many artifacts are destroyed per run of the `ExpireArtifactsWorker`. ## Exception Handling diff --git a/doc/development/maintenance_mode.md b/doc/development/maintenance_mode.md index 486519715ab..ad0893d8605 100644 --- a/doc/development/maintenance_mode.md +++ b/doc/development/maintenance_mode.md @@ -1,7 +1,7 @@ --- stage: Systems group: Geo -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Internal workings of GitLab Maintenance Mode diff --git a/doc/development/mass_insert.md b/doc/development/mass_insert.md index 24f6127a3de..412639927a7 100644 --- a/doc/development/mass_insert.md +++ b/doc/development/mass_insert.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Mass inserting Rails models @@ -15,5 +15,5 @@ the following snippet in the rails console. ```ruby u = User.find(1) -Project.last(100).each { |p| p.set_timestamps_for_create && p.add_maintainer(u, current_user: u) } # Change 100 to whatever number of projects you need access to +Project.last(100).each { |p| p.send(:set_timestamps_for_create) && p.add_maintainer(u, current_user: u) } # Change 100 to whatever number of projects you need access to ``` diff --git a/doc/development/merge_request_concepts/approval_rules.md b/doc/development/merge_request_concepts/approval_rules.md index bba61a71543..44b53e2feb8 100644 --- a/doc/development/merge_request_concepts/approval_rules.md +++ b/doc/development/merge_request_concepts/approval_rules.md @@ -1,7 +1,7 @@ --- stage: Create group: Code Review -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Approval rules development guidelines diff --git a/doc/development/merge_request_concepts/diffs/development.md b/doc/development/merge_request_concepts/diffs/development.md index 42f6c2dc16e..0aef8d03e51 100644 --- a/doc/development/merge_request_concepts/diffs/development.md +++ b/doc/development/merge_request_concepts/diffs/development.md @@ -1,7 +1,7 @@ --- stage: Create group: Code Review -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Merge request diffs development guide diff --git a/doc/development/merge_request_concepts/diffs/frontend.md b/doc/development/merge_request_concepts/diffs/frontend.md index 0625c32377f..aeb1ba452a3 100644 --- a/doc/development/merge_request_concepts/diffs/frontend.md +++ b/doc/development/merge_request_concepts/diffs/frontend.md @@ -1,7 +1,7 @@ --- stage: Create group: Code Review -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Merge request diffs frontend overview diff --git a/doc/development/merge_request_concepts/diffs/index.md b/doc/development/merge_request_concepts/diffs/index.md index c2dec5c4753..ad0e8603983 100644 --- a/doc/development/merge_request_concepts/diffs/index.md +++ b/doc/development/merge_request_concepts/diffs/index.md @@ -1,11 +1,14 @@ --- stage: Create group: Code Review -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Working with diffs +This page contains developer documentation for diffs. For the user documentation, +see [Diffs in merge requests](../../../user/project/merge_requests/versions.md). + We rely on different sources to present diffs. These include: - Gitaly service @@ -18,7 +21,7 @@ We rely on different sources to present diffs. These include: In January 2019, Oswaldo Ferreira hosted a Deep Dive (GitLab team members only: `https://gitlab.com/gitlab-org/create-stage/-/issues/1`) on GitLab Diffs and Commenting on Diffs -functionality to share domain-specific knowledge with anyone who may work in this part of the +functionality to share domain-specific knowledge with anyone who works in this part of the codebase in the future: <!-- vale gitlab.Spelling = YES --> @@ -28,7 +31,7 @@ codebase in the future: - Slides on [Google Slides](https://docs.google.com/presentation/d/1bGutFH2AT3bxOPZuLMGl1ANWHqFnrxwQwjiwAZkF-TU/edit) - [PDF slides](https://gitlab.com/gitlab-org/create-stage/uploads/b5ad2f336e0afcfe0f99db0af0ccc71a/) -Everything covered in this deep dive was accurate as of GitLab 11.7, and while specific details may +Everything covered in this deep dive was accurate as of GitLab 11.7, and while specific details might have changed since then, it should still serve as a good introduction. ## Architecture overview diff --git a/doc/development/merge_request_concepts/index.md b/doc/development/merge_request_concepts/index.md index 8bec5e0c0b0..d04dff9f1be 100644 --- a/doc/development/merge_request_concepts/index.md +++ b/doc/development/merge_request_concepts/index.md @@ -1,8 +1,7 @@ --- -type: reference, dev stage: Create group: Code Review -info: "See the Technical Writers assigned to Development Guidelines: https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-development-guidelines" +info: "See the Technical Writers assigned to Development Guidelines: https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-development-guidelines" --- # Merge request concepts @@ -28,7 +27,7 @@ The merge widget is the component of the merge request where the `merge` button  -This area of the merge request is where all of the options and commit messages are defined prior to merging. It also contains information about what is in the merge request, what issues may be closed, and other important information to the merging process. +This area of the merge request is where all of the options and commit messages are defined prior to merging. It also contains information about what is in the merge request, what issues are closed, and other information important to the merging process. ## Report widgets diff --git a/doc/development/merge_request_concepts/mergeability_framework.md b/doc/development/merge_request_concepts/mergeability_framework.md new file mode 100644 index 00000000000..a755615c885 --- /dev/null +++ b/doc/development/merge_request_concepts/mergeability_framework.md @@ -0,0 +1,85 @@ +--- +stage: Create +group: Code Review +info: Detailing the process to add a new mergeability check +--- + +# Mergeability framework + +The initial work started with the [better defined mergeability framework](https://gitlab.com/groups/gitlab-org/-/epics/5598) + +Originally, the mergeability knowledge was spread throughout the backend and frontend. +This work was to consolidate some of the mergeability criteria into the same location +in the backend. This allows the frontend to simply consume the API and display the error. + +## Add a new check + +The mergeability checks live under `app/services/merge_requests/mergeability/`. + +1. To create a new check, we can use this as a base: + + ```ruby + # frozen_string_literal: true + module MergeRequests + module Mergeability + class CheckCiStatusService < CheckBaseService + identifier :ci_must_pass + description 'Checks whether CI has passed' + + def execute + # If the merge check is behind a setting, we return inactive if the setting is false + return inactive unless merge_request.only_allow_merge_if_pipeline_succeeds? + + if merge_request.mergeable_ci_state? + success + else + failure + end + end + + def skip? + # Here we can check for the param or return false if its not skippable + # Skippablility of an MR is related to merge when checks pass functionality + params[:skip_ci_check].present? + end + + def cacheable? + false + end + end + end + end + ``` + +1. Add the new check in the `def mergeable_state_checks` method. +1. Add the new check to the GraphQL enum `app/graphql/types/merge_requests/detailed_merge_status_enum.rb`. +1. Update the GraphQL documentation with `bundle exec rake gitlab:graphql:compile_docs`. +1. Update the API documentation in `doc/api/merge_requests.md`. +1. Update the frontend to support the new message: `app/assets/javascripts/vue_merge_request_widget/components/checks/message.vue`. + +## Considerations + +1. **Should it be skippable?** If it is part of the merge when checks pass work, + then we should add the skippable check. Otherwise, you should return `false`. +1. **Performance**: These mergeability checks are run very frequently, and therefore + performance is a big consideration here. It is critical to check how the new + mergeability check performs. In general, we are expecting around 10-20 ms. +1. **Caching is an option too.** We can set the `def cacheable?` method to return `true`, + and in that case, we need to create another method `def cache_key` to set the + cache key for the particular check. Cache invalidation can often be tricky, + and we must consider all the edge cases in the cache key. If we keep the timing + around 10-20 ms, then caching is not needed. +1. **Time the checks.** We time each check through the `app/services/merge_requests/mergeability/logger.rb` + class, which can then be viewed in Kibana. + +## How the classes work together + +1. The main methods that call the mergeability framework are: `def mergeable?`, and `DetailedMergeStatusService`. +1. These methods call the `RunChecksService` class which handles the iterating + of the mergeability checks, caching and instrumentation. + +## Future work + +1. At the moment, the slow performance of the approval check is the main area of + concern. We have attempted to make this check cacheable, but there are a lot of + edge cases to consider in regard to when it is invalid. diff --git a/doc/development/merge_request_concepts/performance.md b/doc/development/merge_request_concepts/performance.md index 7e26bf982b2..7a1e33494d0 100644 --- a/doc/development/merge_request_concepts/performance.md +++ b/doc/development/merge_request_concepts/performance.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Merge Request Performance Guidelines @@ -286,7 +286,7 @@ be clearly mentioned in the merge request description. **Summary:** Iterating a single process to external services (for example, PostgreSQL, Redis, Object Storage) should be executed in a **batch-style** to reduce connection overheads. -For fetching rows from various tables in a batch-style, please see [Eager Loading](#eager-loading) section. +For fetching rows from various tables in a batch-style, see [Eager Loading](#eager-loading) section. ### Example: Delete multiple files from Object Storage @@ -323,7 +323,7 @@ Using [`ReactiveCaching`](../utilities.md#reactivecaching) is one of the best so transactions, otherwise it leads to severe contention problems as an open transaction basically blocks the release of a PostgreSQL backend connection. -For keeping transaction as minimal as possible, please consider using `AfterCommitQueue` +For keeping transaction as minimal as possible, consider using `AfterCommitQueue` module or `after_commit` AR hook. Here is [an example](https://gitlab.com/gitlab-org/gitlab/-/issues/36154#note_247228859) diff --git a/doc/development/merge_request_concepts/rate_limits.md b/doc/development/merge_request_concepts/rate_limits.md index 62c50e3d044..9544908b2a3 100644 --- a/doc/development/merge_request_concepts/rate_limits.md +++ b/doc/development/merge_request_concepts/rate_limits.md @@ -1,7 +1,7 @@ --- stage: Create group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Application and rate limit guidelines diff --git a/doc/development/migration_style_guide.md b/doc/development/migration_style_guide.md index afb36519b8d..30f598ef736 100644 --- a/doc/development/migration_style_guide.md +++ b/doc/development/migration_style_guide.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Migration Style Guide @@ -27,7 +27,7 @@ When writing your migrations, also consider that databases might have stale data or inconsistencies and guard for that. Try to make as few assumptions as possible about the state of the database. -Please don't depend on GitLab-specific code since it can change in future +Don't depend on GitLab-specific code since it can change in future versions. If needed copy-paste GitLab code into the migration to make it forward compatible. @@ -140,7 +140,7 @@ Changes to the schema should be committed to `db/structure.sql`. This file is automatically generated by Rails when you run `bundle exec rails db:migrate`, so you typically should not edit this file by hand. If your migration is adding a column to a -table, that column is added at the bottom. Please do not reorder +table, that column is added at the bottom. Do not reorder columns manually for existing tables as this causes confusion to other people using `db/structure.sql` generated by Rails. @@ -1019,9 +1019,13 @@ end ## Dropping a database table +NOTE: +After a table has been dropped, it should be added to the database dictionary, following the +steps in the [database dictionary guide](database/database_dictionary.md#dropping-tables). + Dropping a database table is uncommon, and the `drop_table` method provided by Rails is generally considered safe. Before dropping the table, -please consider the following: +consider the following: If your table has foreign keys on a [high-traffic table](#high-traffic-tables) (like `projects`), then the `DROP TABLE` statement is likely to stall concurrent traffic until it fails with **statement timeout** error. @@ -1105,9 +1109,6 @@ class DroppingTableMigrationClass < Gitlab::Database::Migration[2.1] end ``` -After a table has been dropped, it should be added to the database dictionary, following the -steps in the [database dictionary guide](database/database_dictionary.md#dropping-tables). - ## Dropping a sequence > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/88387) in GitLab 15.1. @@ -1369,7 +1370,7 @@ See the [Testing Rails migrations](testing_guide/testing_migrations_guide.md) st ## Data migration -Please prefer Arel and plain SQL over usual ActiveRecord syntax. In case of +Prefer Arel and plain SQL over usual ActiveRecord syntax. In case of using plain SQL, you need to quote all input manually with `quote_string` helper. Example with Arel: diff --git a/doc/development/module_with_instance_variables.md b/doc/development/module_with_instance_variables.md index 733823d7f6e..36e6639857b 100644 --- a/doc/development/module_with_instance_variables.md +++ b/doc/development/module_with_instance_variables.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Modules with instance variables could be considered harmful diff --git a/doc/development/multi_version_compatibility.md b/doc/development/multi_version_compatibility.md index 36b392a0037..40a30aa4926 100644 --- a/doc/development/multi_version_compatibility.md +++ b/doc/development/multi_version_compatibility.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Backwards compatibility across updates @@ -243,7 +243,7 @@ With all those details in mind, let's imagine we need to replace a query, and th 1. **contract**: from `Schema B` to `Schema C` (post-deployment migration). Nothing uses the old index anymore, we can safely remove it. This is only an example. More complex migrations, especially when background migrations are needed may -require more than one milestone. For details please refer to our [migration style guide](migration_style_guide.md). +require more than one milestone. For details refer to our [migration style guide](migration_style_guide.md). ## Examples of previous incidents diff --git a/doc/development/namespaces.md b/doc/development/namespaces.md index e25b0f57f08..8a094541c27 100644 --- a/doc/development/namespaces.md +++ b/doc/development/namespaces.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Namespaces diff --git a/doc/development/navigation_sidebar.md b/doc/development/navigation_sidebar.md index f7ab75b62f4..fb6385dd3ff 100644 --- a/doc/development/navigation_sidebar.md +++ b/doc/development/navigation_sidebar.md @@ -1,7 +1,7 @@ --- stage: Manage group: Foundations -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Navigation sidebar diff --git a/doc/development/omnibus.md b/doc/development/omnibus.md index a0ba2751a06..1d9706c7245 100644 --- a/doc/development/omnibus.md +++ b/doc/development/omnibus.md @@ -1,7 +1,7 @@ --- stage: Systems group: Distribution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # What you should know about Omnibus packages diff --git a/doc/development/organization/index.md b/doc/development/organization/index.md index 3a23e50caf9..3882000dc8b 100644 --- a/doc/development/organization/index.md +++ b/doc/development/organization/index.md @@ -1,8 +1,7 @@ --- -type: index, dev stage: Data Stores group: Tenant Scale -info: "See the Technical Writers assigned to Development Guidelines: https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-development-guidelines" +info: "See the Technical Writers assigned to Development Guidelines: https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-development-guidelines" description: "Development Guidelines: learn about organization when developing GitLab." --- diff --git a/doc/development/packages/cleanup_policies.md b/doc/development/packages/cleanup_policies.md index aa60cde450b..bbec6ce0623 100644 --- a/doc/development/packages/cleanup_policies.md +++ b/doc/development/packages/cleanup_policies.md @@ -1,7 +1,7 @@ --- stage: Package group: Container Registry -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Cleanup policies @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w Cleanup policies are recurrent background processes that automatically remove objects according to some parameters set by users. -## Container Registry +## Container registry Cleanup policies for the container registry work on all the container repositories hosted in a single project. All tags that match the cleanup parameters are removed. diff --git a/doc/development/packages/debian_repository.md b/doc/development/packages/debian_repository.md index 2d8ba98f9ad..deb5eb05118 100644 --- a/doc/development/packages/debian_repository.md +++ b/doc/development/packages/debian_repository.md @@ -1,7 +1,7 @@ --- stage: Package group: Package Registry -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Debian Repository diff --git a/doc/development/packages/dependency_proxy.md b/doc/development/packages/dependency_proxy.md index e9568699c7e..ad6459e4916 100644 --- a/doc/development/packages/dependency_proxy.md +++ b/doc/development/packages/dependency_proxy.md @@ -1,7 +1,7 @@ --- stage: Package group: Container Registry -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Dependency Proxy diff --git a/doc/development/packages/harbor_registry_development.md b/doc/development/packages/harbor_registry_development.md index 609aba9251c..0ed1378ad65 100644 --- a/doc/development/packages/harbor_registry_development.md +++ b/doc/development/packages/harbor_registry_development.md @@ -1,7 +1,7 @@ --- stage: Package group: Container Registry -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Harbor Registry diff --git a/doc/development/packages/index.md b/doc/development/packages/index.md index 4f027825422..91351d3ef5a 100644 --- a/doc/development/packages/index.md +++ b/doc/development/packages/index.md @@ -1,7 +1,7 @@ --- stage: Package group: Package Registry -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Package and container registry development guidelines diff --git a/doc/development/packages/new_format_development.md b/doc/development/packages/new_format_development.md index 0af0b8ad480..018d9c0cde9 100644 --- a/doc/development/packages/new_format_development.md +++ b/doc/development/packages/new_format_development.md @@ -1,7 +1,7 @@ --- stage: Package group: Package Registry -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Developing support for a new package format @@ -229,7 +229,7 @@ in your local development environment. #### File size limits -Files uploaded to the GitLab Package Registry are [limited by format](../../administration/instance_limits.md#package-registry-limits). +Files uploaded to the GitLab package registry are [limited by format](../../administration/instance_limits.md#package-registry-limits). On GitLab.com, these are typically set to 5GB to help prevent timeout issues and abuse. When a new package type is added to the `Packages::Package` model, a size limit must be added diff --git a/doc/development/packages/settings.md b/doc/development/packages/settings.md index 0fc49c4eb5d..89f91f41f4c 100644 --- a/doc/development/packages/settings.md +++ b/doc/development/packages/settings.md @@ -1,7 +1,7 @@ --- stage: Package group: Package Registry -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Package Settings @@ -10,7 +10,7 @@ This page includes an exhaustive list of settings related to and maintained by t ## Instance Settings -### Package Registry +### Package registry Setting | Table | Description ------- | ----- | ----------- @@ -36,7 +36,7 @@ Setting | Table | Description `terraform_module_max_file_size` | `plan_limits` | Maximum file size for a Terraform package file. `helm_max_file_size` | `plan_limits` | Maximum file size for a Helm package file. -### Container Registry +### Container registry Setting | Table | Description ------- | ----- | ----------- @@ -51,7 +51,7 @@ Setting | Table | Description `container_registry_expiration_policies_caching` | `application_settings` | Enable or disable tag creation timestamp caching during execution of cleanup policies. `container_registry_import_max_tags_count` | `application_settings` | Defines what is a the maximum amount of tags that we accept to migrate. `container_registry_import_max_retries` | `application_settings` | The maximum amount of retries done on a migration that is aborted. -`container_registry_import_start_max_retries` | `application_settings` | The maximum amount of requests to start an import step that is sent to the Container Registry API. +`container_registry_import_start_max_retries` | `application_settings` | The maximum amount of requests to start an import step that is sent to the container registry API. `container_registry_import_max_step_duration` | `application_settings` | The maximum amount of seconds before an ongoing migration is considered as stale. `container_registry_import_target_plan` | `application_settings` | The target subscription plan on which we're intend to pick container repositories. `container_registry_import_created_before` | `application_settings` | Only image repositories created before this timestamp are eligible for the migration. @@ -69,6 +69,7 @@ Setting | Table | Description `generic_duplicate_exception_regex` | `namespace_package_settings` | Regex defining generic packages that are allowed to be duplicate when duplicates are not allowed. `nuget_duplicates_allowed` | `namespace_package_settings` | Allow or prevent duplicate NuGet packages. `nuget_duplicate_exception_regex` | `namespace_package_settings` | Regex defining NuGet packages that are allowed to be duplicate when duplicates are not allowed. +`nuget_symbol_server_enabled` | `namespace_package_settings` | Enable or disable the NuGet symbol server. Dependency Proxy Cleanup Policies - `ttl` | `dependency_proxy_image_ttl_group_policies` | Number of days to retain an unused Dependency Proxy file before it is removed. Dependency Proxy - `enabled` | `dependency_proxy_image_ttl_group_policies` | Enable or disable the Dependency Proxy cleanup policy. diff --git a/doc/development/packages/structure.md b/doc/development/packages/structure.md index 281ec2e6c39..371f33712f6 100644 --- a/doc/development/packages/structure.md +++ b/doc/development/packages/structure.md @@ -1,12 +1,12 @@ --- stage: Package group: Package Registry -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Package Structure -## Package Registry +## Package registry ```mermaid erDiagram @@ -58,7 +58,7 @@ erDiagram packages_debian_file_metadata |o--|| packages_package_files : "" ``` -## Container Registry +## Container registry ```mermaid erDiagram diff --git a/doc/development/pages/dnsmasq.md b/doc/development/pages/dnsmasq.md index a14e215ccd7..1506527c3f1 100644 --- a/doc/development/pages/dnsmasq.md +++ b/doc/development/pages/dnsmasq.md @@ -1,8 +1,7 @@ --- -type: reference, dev stage: Plan group: Knowledge -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. description: "dnsmasq configuration guidelines for GitLab Pages" --- diff --git a/doc/development/pages/index.md b/doc/development/pages/index.md index 25c1ea7a864..f4710c951ed 100644 --- a/doc/development/pages/index.md +++ b/doc/development/pages/index.md @@ -1,8 +1,7 @@ --- -type: reference, dev stage: Plan group: Knowledge -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. description: "GitLab's development guidelines for GitLab Pages" --- diff --git a/doc/development/performance.md b/doc/development/performance.md index 428d5637aa9..9bdb65aacba 100644 --- a/doc/development/performance.md +++ b/doc/development/performance.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Performance Guidelines diff --git a/doc/development/permissions.md b/doc/development/permissions.md index 32d5ccfcdf1..9b0035f17ec 100644 --- a/doc/development/permissions.md +++ b/doc/development/permissions.md @@ -1,7 +1,7 @@ --- stage: Govern group: Authentication -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Permission development guidelines diff --git a/doc/development/permissions/authorizations.md b/doc/development/permissions/authorizations.md index 20b39f4093c..97eef8a648b 100644 --- a/doc/development/permissions/authorizations.md +++ b/doc/development/permissions/authorizations.md @@ -1,7 +1,7 @@ --- stage: Govern group: Authentication -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Authorization diff --git a/doc/development/permissions/custom_roles.md b/doc/development/permissions/custom_roles.md index 1630ea7b9ab..9d02c0c6f39 100644 --- a/doc/development/permissions/custom_roles.md +++ b/doc/development/permissions/custom_roles.md @@ -1,7 +1,7 @@ --- stage: Govern group: Authorization -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Custom Roles @@ -182,6 +182,7 @@ security dashboard. To add a new ability to a custom role: +- Generate YAML file by running `./ee/bin/custom-ability` generator - Add a new column to `member_roles` table, for example in [this change in merge request 114734](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114734/diffs#diff-content-5c53d6f1c29a272a87eecea3f62d017ab6635275). - Add the ability to the `MemberRole` model, `ALL_CUSTOMIZABLE_PERMISSIONS` hash, for example in [this change in merge request 121534](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121534/diffs#ce5ec769500a53ce2b603467d9984fc2b33ca71d_8_8). There are following possible keys in the `ALL_CUSTOMIZABLE_PERMISSIONS` hash: @@ -200,6 +201,33 @@ Examples of merge requests adding new abilities to custom roles: You should make sure a new custom roles ability is under a feature flag. +## Custom abilities definition + +All new custom abilities must have a type definition stored in `ee/config/custom_abilities` that contains a single source of truth for every ability that is part of custom roles feature. + +### Add a new custom ability definition + +To add a new custom ability: + +1. Create the YAML definition. You can either: + - Use the `ee/bin/custom-ability` CLI to create the YAML definition automatically. + - Perform manual steps to create a new file in `ee/config/custom_abilities/` with the filename matching the name of the ability name. +1. Add contents to the file that conform to the [schema](#schema) defined in `ee/config/custom_abilities/types/type_schema.json`. + +### Schema + +| Field | Required | Description | +| ----- | -------- |--------------| +| `name` | yes | Unique, lowercase and underscored name describing the custom ability. Must match the filename. | +| `description` | yes | Human-readable description of the custom ability. | +| `feature_category` | yes | Name of the feature category. For example, `vulnerability_management`. | +| `introduced_by_issue` | yes | Issue URL that proposed the addition of this custom ability. | +| `introduced_by_mr` | yes | MR URL that added this custom ability. | +| `milestone` | yes | Milestone in which this custom ability was added. | +| `group_ability` | yes | Indicate whether this ability is checked on group level. | +| `project_ability` | yes | Indicate whether this ability is checked on project level. | +| `skip_seat_consumption` | yes | Indicate wheter this ability should be skiped when counting licensed users. | + ### Privilege escalation consideration A base role typically has permissions that allow creation or management of artifacts corresponding to the base role when interacting with that artifact. For example, when a `Developer` creates an access token for a project, it is created with `Developer` access encoded into that credential. It is important to keep in mind that as new custom permissions are created, there might be a risk of elevated privileges when interacting with GitLab artifacts, and appropriate safeguards or base role checks should be added. diff --git a/doc/development/permissions/predefined_roles.md b/doc/development/permissions/predefined_roles.md index 0edbcb7b962..7cb00977e1f 100644 --- a/doc/development/permissions/predefined_roles.md +++ b/doc/development/permissions/predefined_roles.md @@ -1,7 +1,7 @@ --- stage: Govern group: Authentication -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Predefined system of user roles diff --git a/doc/development/pipelines/index.md b/doc/development/pipelines/index.md index 77f91300a57..c4dfda9466a 100644 --- a/doc/development/pipelines/index.md +++ b/doc/development/pipelines/index.md @@ -1,7 +1,7 @@ --- stage: none group: Engineering Productivity -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Pipelines for the GitLab project @@ -105,7 +105,7 @@ In addition, there are a few circumstances where we would always run the full RS #### Have you encountered a problem with backend predictive tests? -If so, please have a look at [the Engineering Productivity RUNBOOK on predictive tests](https://gitlab.com/gitlab-org/quality/engineering-productivity/team/-/blob/main/runbooks/predictive-tests.md) for instructions on how to act upon predictive tests issues. Additionally, if you identified any test selection gaps, please let `@gl-quality/eng-prod` know so that we can take the necessary steps to optimize test selections. +If so, have a look at [the Engineering Productivity RUNBOOK on predictive tests](https://gitlab.com/gitlab-org/quality/engineering-productivity/team/-/blob/main/runbooks/predictive-tests.md) for instructions on how to act upon predictive tests issues. Additionally, if you identified any test selection gaps, let `@gl-quality/eng-prod` know so that we can take the necessary steps to optimize test selections. ### Jest predictive jobs @@ -130,7 +130,7 @@ The `rules` definitions for full Jest tests are defined at `.frontend:rules:jest #### Have you encountered a problem with frontend predictive tests? -If so, please have a look at [the Engineering Productivity RUNBOOK on predictive tests](https://gitlab.com/gitlab-org/quality/engineering-productivity/team/-/blob/main/runbooks/predictive-tests.md) for instructions on how to act upon predictive tests issues. +If so, have a look at [the Engineering Productivity RUNBOOK on predictive tests](https://gitlab.com/gitlab-org/quality/engineering-productivity/team/-/blob/main/runbooks/predictive-tests.md) for instructions on how to act upon predictive tests issues. ### Fork pipelines @@ -302,7 +302,7 @@ The intent is to ensure that a change doesn't introduce a failure after `gitlab- #### What it is This pipeline is also called [JiHu validation pipeline](https://about.gitlab.com/handbook/ceo/chief-of-staff-team/jihu-support/jihu-validation-pipelines.html), -and it's currently allowed to fail. When that happens, please follow +and it's currently allowed to fail. When that happens, follow [What to do when the validation pipeline fails](https://about.gitlab.com/handbook/ceo/chief-of-staff-team/jihu-support/jihu-validation-pipelines.html#what-to-do-when-the-validation-pipeline-failed). #### How we run it diff --git a/doc/development/pipelines/internals.md b/doc/development/pipelines/internals.md index 0e2c1c991fd..1fdb2014c1f 100644 --- a/doc/development/pipelines/internals.md +++ b/doc/development/pipelines/internals.md @@ -1,7 +1,7 @@ --- stage: none group: Engineering Productivity -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # CI configuration internals diff --git a/doc/development/pipelines/performance.md b/doc/development/pipelines/performance.md index 7db498adc02..0cd4f4270fe 100644 --- a/doc/development/pipelines/performance.md +++ b/doc/development/pipelines/performance.md @@ -1,7 +1,7 @@ --- stage: none group: Engineering Productivity -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # CI configuration performance diff --git a/doc/development/policies.md b/doc/development/policies.md index d2a4fcef81f..7e4909c1eb9 100644 --- a/doc/development/policies.md +++ b/doc/development/policies.md @@ -1,7 +1,7 @@ --- stage: Govern group: Authentication -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # `DeclarativePolicy` framework diff --git a/doc/development/polling.md b/doc/development/polling.md index ff239effb80..fa295d6d2df 100644 --- a/doc/development/polling.md +++ b/doc/development/polling.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Polling with ETag caching diff --git a/doc/development/product_qualified_lead_guide/index.md b/doc/development/product_qualified_lead_guide/index.md index 9ebe607f66b..fae0225b52c 100644 --- a/doc/development/product_qualified_lead_guide/index.md +++ b/doc/development/product_qualified_lead_guide/index.md @@ -1,7 +1,7 @@ --- stage: Growth group: Acquisition -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Product Qualified Lead (PQL) development guidelines diff --git a/doc/development/profiling.md b/doc/development/profiling.md index efee6ff3cd5..a7a5f993ca9 100644 --- a/doc/development/profiling.md +++ b/doc/development/profiling.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Profiling diff --git a/doc/development/project_templates.md b/doc/development/project_templates.md index 11102f5825e..7200927858c 100644 --- a/doc/development/project_templates.md +++ b/doc/development/project_templates.md @@ -1,18 +1,18 @@ --- stage: Create group: Source Code -info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments" --- # Contribute to built-in project templates ## Adding a new built-in project template -If you'd like to contribute a new built-in project template to be distributed with GitLab, please do the following: +If you'd like to contribute a new built-in project template to be distributed with GitLab, do the following: 1. Create a new public project with the project content you'd like to contribute in a namespace of your choosing. You can view a working example [here](https://gitlab.com/gitlab-org/project-templates/dotnetcore). - Projects should be as simple as possible and free of any unnecessary assets or dependencies. -1. When the project is ready for review, please create a new issue in [GitLab](https://gitlab.com/gitlab-org/gitlab/issues) with a link to your project. +1. When the project is ready for review, create a new issue in [GitLab](https://gitlab.com/gitlab-org/gitlab/issues) with a link to your project. - In your issue, `@` mention the relevant Backend Engineering Manager and Product Manager for the [Create:Source Code group](https://about.gitlab.com/handbook/product/categories/#source-code-group). To make the project template available when creating a new project, the vendoring process will have to be completed: @@ -40,7 +40,7 @@ To make the project template available when creating a new project, the vendorin 1. Run the `bundle_repo` script. Make sure to pass the correct arguments, or the script may damage the folder structure. 1. Add exported project (`$name.tar.gz`) to `gitlab/vendor/project_templates` and remove the resulting build folders `tar-base` and `project`. -1. Run `bin/rake gettext:regenerate` in the `gitlab` project and commit new `.pot` file. +1. Run `tooling/bin/gettext_extractor locale/gitlab.pot` in the `gitlab` project and commit new `.pot` file. 1. Add a changelog entry in the commit message (for example, `Changelog: added`). For more information, see [Changelog entries](changelog.md). 1. Add an icon to [`gitlab-svgs`](https://gitlab.com/gitlab-org/gitlab-svgs), as shown in @@ -59,7 +59,7 @@ To make the project template available when creating a new project, the vendorin Existing templates are available in the [project-templates](https://gitlab.com/gitlab-org/project-templates) group. -To contribute a change, please open a merge request in the relevant project +To contribute a change, open a merge request in the relevant project and mention `@gitlab-org/manage/import/backend` when you are ready for a review. Then, if your merge request gets accepted, either open an issue on @@ -79,7 +79,7 @@ Complete the following steps to test the project template in your own GitLab Dev ## For GitLab team members -Please ensure the merge request has been reviewed by the Security Counterpart before merging. +Ensure the merge request has been reviewed by the Security Counterpart before merging. To review a merge request which changes a vendored project template, run the `check-template-changes` script: diff --git a/doc/development/project_templates/index.md b/doc/development/project_templates/index.md index b6ecf3b3f00..4f93c58fa23 100644 --- a/doc/development/project_templates/index.md +++ b/doc/development/project_templates/index.md @@ -1,7 +1,7 @@ --- stage: Create group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Custom group-level project templates development guidelines diff --git a/doc/development/projections.md b/doc/development/projections.md index caa54e7b912..a9a06d91d2c 100644 --- a/doc/development/projections.md +++ b/doc/development/projections.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Projections diff --git a/doc/development/prometheus_metrics.md b/doc/development/prometheus_metrics.md index f72805e0fe9..6a4a85f14ff 100644 --- a/doc/development/prometheus_metrics.md +++ b/doc/development/prometheus_metrics.md @@ -1,7 +1,7 @@ --- -stage: Monitor +stage: Service Management group: Respond -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Prometheus metrics development guidelines diff --git a/doc/development/pry_debugging.md b/doc/development/pry_debugging.md index 8ad584a8edc..890e6b490ae 100644 --- a/doc/development/pry_debugging.md +++ b/doc/development/pry_debugging.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Pry debugging diff --git a/doc/development/push_rules/index.md b/doc/development/push_rules/index.md new file mode 100644 index 00000000000..343b199e613 --- /dev/null +++ b/doc/development/push_rules/index.md @@ -0,0 +1,93 @@ +--- +stage: Create +group: Source Code +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. +--- + +# Push Rules development guidelines + +This document was created to help contributors understand the code design of +[Push Rules](../../user/project/repository/push_rules.md). You should read this +document before making changes to the code for this feature. + +This document is intentionally limited to an overview of how the code is +designed, as code can change often. To understand how a specific part of the +feature works, view the code and the specs. The details here explain how the +major components of the Push Rules feature work. + +NOTE: +This document should be updated when parts of the codebase referenced in this +document are updated, removed, or new parts are added. + +## Business logic + +The business logic is contained in two main places. The `PushRule` model stores +the settings for a rule and then we have checks that use those settings to +change the push behavior. + +- `PushRule`: the main model used to store the configuration of each push rule. + - Defined in `ee/app/models/push_rule.rb`. +- `EE::Gitlab::Checks::DiffCheck`: Diff check prevents filenames matching the + push rule's `file_name_regex` and also files with names matching known secret + files, for example `id_rsa`. + - Defined in `ee/lib/ee/gitlab/checks/diff_check.rb`. +- `EE::Gitlab::Checks::PushRuleCheck`: Executes various push rule checks. + - Defined in `ee/lib/ee/gitlab/checks/push_rule_check.rb`. +- `EE::Gitlab::Checks::PushRules::BranchCheck`: Executes push rule checks + related to branch rules. + - Defined in `ee/lib/ee/gitlab/checks/push_rules/branch_check.rb`. +- `EE::Gitlab::Checks::PushRules::CommitCheck`: Executes push rule checks + related to commit rules. + - Defined in `ee/lib/ee/gitlab/checks/push_rules/commit_check.rb`. +- `EE::Gitlab::Checks::PushRules::FileSizeCheck`: Executes push rule checks + related to file size rules. + - Defined in `ee/lib/ee/gitlab/checks/push_rules/file_size_check.rb`. +- `EE::Gitlab::Checks::PushRules::SecretsCheck`: Executes push rule checks + related to secrets rules. + - Defined in `ee/lib/ee/gitlab/checks/push_rules/secrets_check.rb`. +- `EE::Gitlab::Checks::PushRules::TagCheck`: Executes push rule checks + related to tag rules. + - Defined in `ee/lib/ee/gitlab/checks/push_rules/tag_check.rb`. + +## Entrypoints + +The following controllers and APIs are all entrypoints into the push rules logic: + +- `Admin::PushRulesController`: This controller is used to manage the global push rule. +- `Group::PushRulesController`: This controller is used to manage the group-level push rule. +- `Project::PushRulesController`: This controller is used to manage the project-level push rule. +- `Api::Internal::Base`: This `/internal/allowed` endpoint is called when pushing to GitLab over SSH to + ensure the user is allowed to push. The `/internal/allowed` endpoint performs a + `Gitlab::Checks::DiffCheck`. In EE, this includes push rules checks. + - Defined in `lib/api/internal/base.rb`. +- `Repositories::GitHttpController`: When changes are pushed to GitLab over HTTP, the controller performs an access + check to ensure the user is allowed to push. The checks perform a + `Gitlab::Checks::DiffCheck`. In EE, this includes push rules checks. + - Defined in `app/controllers/repositories/git_http_controller.rb`. + +## Flow + +These flowcharts should help explain the flow from the controllers down to the +models for different features. + +### Git push over SSH + +```mermaid +graph TD + Repositories::GitHttpController --> Gitlab::GitAccess + Api::Internal::Base --> Gitlab::GitAccess + Gitlab::GitAccess --> Gitlab::Checks::ChangesAccess + Gitlab::Checks::ChangesAccess --> Gitlab::Checks::SingleChangeAccess + Gitlab::Checks::ChangesAccess --> EE::Gitlab::Checks::PushRuleCheck + Gitlab::Checks::SingleChangeAccess --> Gitlab::Checks::DiffCheck + EE::Gitlab::Checks::PushRuleCheck -->|Only if pushing to a tag| EE::Gitlab::Checks::PushRules::TagCheck + EE::Gitlab::Checks::PushRuleCheck -->|Only if pushing to a branch| EE::Gitlab::Checks::PushRules::BranchCheck + EE::Gitlab::Checks::PushRuleCheck --> EE::Gitlab::Checks::PushRules::FileSizeCheck + EE::Gitlab::Checks::PushRuleCheck --> EE::Gitlab::Checks::PushRules::SecretsCheck + EE::Gitlab::Checks::PushRuleCheck --> EE::Gitlab::Checks::PushRules::SecretsCheck +``` + +NOTE: +The `PushRuleCheck` only triggers checks in parallel if the +`parallel_push_checks` feature flag is enabled. Otherwise tag or branch check +runs first, then file size, then secrets. diff --git a/doc/development/python_guide/index.md b/doc/development/python_guide/index.md index e9b52b81c0e..d1d9fc03d8f 100644 --- a/doc/development/python_guide/index.md +++ b/doc/development/python_guide/index.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Python development guidelines diff --git a/doc/development/rails_endpoints/index.md b/doc/development/rails_endpoints/index.md index c5a166dd4be..adc17fde1d0 100644 --- a/doc/development/rails_endpoints/index.md +++ b/doc/development/rails_endpoints/index.md @@ -1,8 +1,7 @@ --- stage: Create group: Source Code -info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" -type: reference, api +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments" --- # Rails Endpoints @@ -20,7 +19,7 @@ These Rails Endpoints: ## Proof of concept period: Feedback Request -We are currently evaluating a new approach for documenting Rails endpoints. Please [check out the Feedback Issue](https://gitlab.com/gitlab-org/gitlab/-/issues/411605) and feel free to share your thoughts, suggestions, or concerns. We appreciate your participation in helping us improve the documentation! +We are currently evaluating a new approach for documenting Rails endpoints. [Check out the Feedback Issue](https://gitlab.com/gitlab-org/gitlab/-/issues/411605) and feel free to share your thoughts, suggestions, or concerns. We appreciate your participation in helping us improve the documentation! ## SAST Scanners diff --git a/doc/development/rails_initializers.md b/doc/development/rails_initializers.md index 93a7b568974..e3faf1accb1 100644 --- a/doc/development/rails_initializers.md +++ b/doc/development/rails_initializers.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Rails initializers diff --git a/doc/development/rails_update.md b/doc/development/rails_update.md index 772206c2d73..d00a1681e76 100644 --- a/doc/development/rails_update.md +++ b/doc/development/rails_update.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Rails upgrade guidelines diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md index d879668649d..25b0a42db3a 100644 --- a/doc/development/rake_tasks.md +++ b/doc/development/rake_tasks.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Rake tasks for developers diff --git a/doc/development/reactive_caching.md b/doc/development/reactive_caching.md index d0652c85c6d..00110d21dc0 100644 --- a/doc/development/reactive_caching.md +++ b/doc/development/reactive_caching.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # `ReactiveCaching` diff --git a/doc/development/real_time.md b/doc/development/real_time.md index 1770760ac9b..9f735a9419d 100644 --- a/doc/development/real_time.md +++ b/doc/development/real_time.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Cloud Connector -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Build and deploy real-time view components diff --git a/doc/development/redis.md b/doc/development/redis.md index 63a778ec1c1..612d2bbcf2a 100644 --- a/doc/development/redis.md +++ b/doc/development/redis.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Redis development guidelines diff --git a/doc/development/redis/new_redis_instance.md b/doc/development/redis/new_redis_instance.md index f67f00967e5..ff5394cef8f 100644 --- a/doc/development/redis/new_redis_instance.md +++ b/doc/development/redis/new_redis_instance.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Add a new Redis instance diff --git a/doc/development/refactoring_guide/index.md b/doc/development/refactoring_guide/index.md index 5dd505ff5c6..ac9b9692a04 100644 --- a/doc/development/refactoring_guide/index.md +++ b/doc/development/refactoring_guide/index.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Refactoring guide diff --git a/doc/development/reference_processing.md b/doc/development/reference_processing.md index 5a026e0a2a0..044c1b01b30 100644 --- a/doc/development/reference_processing.md +++ b/doc/development/reference_processing.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. description: 'An introduction to reference parsers and reference filters, and a guide to their implementation.' --- diff --git a/doc/development/renaming_features.md b/doc/development/renaming_features.md index ee759efd7e2..94004177126 100644 --- a/doc/development/renaming_features.md +++ b/doc/development/renaming_features.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Renaming features diff --git a/doc/development/repository_mirroring.md b/doc/development/repository_mirroring.md index 6d95dec823b..d65df403fb9 100644 --- a/doc/development/repository_mirroring.md +++ b/doc/development/repository_mirroring.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Repository mirroring diff --git a/doc/development/repository_storage_moves/index.md b/doc/development/repository_storage_moves/index.md index 578bc1eabee..a6a2efebc50 100644 --- a/doc/development/repository_storage_moves/index.md +++ b/doc/development/repository_storage_moves/index.md @@ -1,7 +1,7 @@ --- stage: Create group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Project Repository Storage Moves diff --git a/doc/development/reusing_abstractions.md b/doc/development/reusing_abstractions.md index 4a964f7d3c9..ba5998da844 100644 --- a/doc/development/reusing_abstractions.md +++ b/doc/development/reusing_abstractions.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Guidelines for reusing abstractions diff --git a/doc/development/routing.md b/doc/development/routing.md index 8f475674c39..78c540683ce 100644 --- a/doc/development/routing.md +++ b/doc/development/routing.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Routing @@ -83,7 +83,8 @@ gitlab-org/serverless/runtimes/-/settings/repository Don't change a URL to an existing page, unless it's necessary. If you must make a change, make it unnoticeable for users, because we don't want them to receive `404 Not Found` -if we can avoid it. This table should help: +if we can avoid it. This table describes the minimum required in different +cases: | URL description | Example | What to do | |---|---|---| @@ -91,6 +92,11 @@ if we can avoid it. This table should help: | Likely to be saved or shared | `issue#show` | Add a redirect from an old URL to a new URL until the next major release. | | Limited use, unlikely to be shared | `admin#labels` | No extra steps required. | +In all cases, an old route should only be removed once traffic to it has +dropped sufficiently (e.g., according to logs or BigQuery). Otherwise, more +effort may be required to inform users about its deprecation before it can be +considered again for removal. + ## Migrating unscoped routes Currently, the majority of routes are placed under the `/-/` scope. However, diff --git a/doc/development/rubocop_development_guide.md b/doc/development/rubocop_development_guide.md index 807544b71d4..e56443d5653 100644 --- a/doc/development/rubocop_development_guide.md +++ b/doc/development/rubocop_development_guide.md @@ -1,8 +1,7 @@ --- -type: reference, dev stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # RuboCop rule development guidelines @@ -45,6 +44,9 @@ More context can go into code comments above this inline disable comment. To reduce verbose code comments link a resource (issue, epic, ...) to provide detailed context. +For temporary inline disables use `rubocop:todo` and link the follow-up issue +or epic. + For example: ```ruby @@ -53,6 +55,10 @@ module Types module Domain # rubocop:disable Graphql/AuthorizeTypes class SomeType < BaseObject + if condition # rubocop:disable Style/GuardClause + # more logic... + end + object.public_send(action) # rubocop:disable GitlabSecurity/PublicSend end # rubocop:enable Graphql/AuthorizeTypes @@ -64,6 +70,10 @@ module Types module Domain # rubocop:disable Graphql/AuthorizeTypes -- already authroized in parent entity class SomeType < BaseObject + if condition # rubocop:todo Style/GuardClause -- Cleanup via https://gitlab.com/gitlab-org/gitlab/-/issues/1234567890 + # more logic... + end + # At this point `action` is safe to be used in `public_send`. # See https://gitlab.com/gitlab-org/gitlab/-/issues/123457890. object.public_send(action) # rubocop:disable GitlabSecurity/PublicSend -- User input verified diff --git a/doc/development/ruby3_gotchas.md b/doc/development/ruby3_gotchas.md index 0d000bc68df..d32f71948d7 100644 --- a/doc/development/ruby3_gotchas.md +++ b/doc/development/ruby3_gotchas.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Ruby 3 gotchas @@ -150,7 +150,7 @@ We run automated detection for this warning in tests via `deprecation_toolkit`, but it relies on the fact that `Kernel#warn` emits a warning, so stubbing out this call will effectively remove the call to warn, which means `deprecation_toolkit` will never see the deprecation warnings. Stubbing out the implementation removes that warning, and we never pick it up, so the build is green. -Please refer to [issue 364099](https://gitlab.com/gitlab-org/gitlab/-/issues/364099) for more context. +Refer to [issue 364099](https://gitlab.com/gitlab-org/gitlab/-/issues/364099) for more context. ## Testing in `irb` and `rails console` diff --git a/doc/development/ruby_upgrade.md b/doc/development/ruby_upgrade.md index 61bc629e8c8..110bc6076b0 100644 --- a/doc/development/ruby_upgrade.md +++ b/doc/development/ruby_upgrade.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Ruby upgrade guidelines diff --git a/doc/development/runner_fleet_dashboard.md b/doc/development/runner_fleet_dashboard.md index 2a7c7d05453..70499e5a087 100644 --- a/doc/development/runner_fleet_dashboard.md +++ b/doc/development/runner_fleet_dashboard.md @@ -4,13 +4,13 @@ group: Runner info: >- To determine the technical writer assigned to the Stage/Group associated with this page, see - https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments + https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Runner Fleet Dashboard **(ULTIMATE BETA)** +# Runner Fleet Dashboard **(ULTIMATE EXPERIMENT)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/424495) in GitLab 16.6 behind several [feature flags](#enable-feature-flags). -This feature is in [BETA](../policy/experiment-beta-support.md). +This feature is an [Experiment](../policy/experiment-beta-support.md). To join the list of users testing this feature, contact us in [epic 11180](https://gitlab.com/groups/gitlab-org/-/epics/11180). @@ -33,8 +33,7 @@ Prerequisites: To view the runner fleet dashboard: -1. On the left sidebar, select **Search or go to**. -1. Select **Admin Area**. +1. On the left sidebar, at the bottom, select **Admin Area**. 1. Select **Runners**. 1. Click **Fleet dashboard**. @@ -49,7 +48,7 @@ for some customers to try this feature. To test the Runner Fleet Dashboard as part of the early adopters program, you must: -- Run GitLab 16.6 or above. +- Run GitLab 16.7 or above. - Have an [Ultimate license](https://about.gitlab.com/pricing/). - Be able to run ClickHouse database. We recommend using [ClickHouse Cloud](https://clickhouse.cloud/). @@ -59,6 +58,7 @@ To setup ClickHouse as the GitLab data storage: 1. [Run ClickHouse Cluster and configure database](#run-and-configure-clickhouse). 1. [Configure GitLab connection to Clickhouse](#configure-the-gitlab-connection-to-clickhouse). +1. [Run ClickHouse migrations](#run-clickhouse-migrations). 1. [Enable the feature flags](#enable-feature-flags). ### Run and configure ClickHouse @@ -89,62 +89,6 @@ To create necessary user and database objects: GRANT gitlab_app TO gitlab; ``` -1. Connect to the `gitlab_clickhouse_main_production` database (or just switch it in the ClickHouse Cloud UI). - -1. To create the required database objects, execute: - - ```sql - CREATE TABLE ci_finished_builds - ( - id UInt64 DEFAULT 0, - project_id UInt64 DEFAULT 0, - pipeline_id UInt64 DEFAULT 0, - status LowCardinality(String) DEFAULT '', - created_at DateTime64(6, 'UTC') DEFAULT now(), - queued_at DateTime64(6, 'UTC') DEFAULT now(), - finished_at DateTime64(6, 'UTC') DEFAULT now(), - started_at DateTime64(6, 'UTC') DEFAULT now(), - runner_id UInt64 DEFAULT 0, - runner_manager_system_xid String DEFAULT '', - runner_run_untagged Boolean DEFAULT FALSE, - runner_type UInt8 DEFAULT 0, - runner_manager_version LowCardinality(String) DEFAULT '', - runner_manager_revision LowCardinality(String) DEFAULT '', - runner_manager_platform LowCardinality(String) DEFAULT '', - runner_manager_architecture LowCardinality(String) DEFAULT '', - duration Int64 MATERIALIZED age('ms', started_at, finished_at), - queueing_duration Int64 MATERIALIZED age('ms', queued_at, started_at) - ) - ENGINE = ReplacingMergeTree - ORDER BY (status, runner_type, project_id, finished_at, id) - PARTITION BY toYear(finished_at); - - CREATE TABLE ci_finished_builds_aggregated_queueing_delay_percentiles - ( - status LowCardinality(String) DEFAULT '', - runner_type UInt8 DEFAULT 0, - started_at_bucket DateTime64(6, 'UTC') DEFAULT now(), - - count_builds AggregateFunction(count), - queueing_duration_quantile AggregateFunction(quantile, Int64) - ) - ENGINE = AggregatingMergeTree() - ORDER BY (started_at_bucket, status, runner_type); - - CREATE MATERIALIZED VIEW ci_finished_builds_aggregated_queueing_delay_percentiles_mv - TO ci_finished_builds_aggregated_queueing_delay_percentiles - AS - SELECT - status, - runner_type, - toStartOfInterval(started_at, INTERVAL 5 minute) AS started_at_bucket, - - countState(*) as count_builds, - quantileState(queueing_duration) AS queueing_duration_quantile - FROM ci_finished_builds - GROUP BY status, runner_type, started_at_bucket; - ``` - ### Configure the GitLab connection to ClickHouse ::Tabs @@ -216,6 +160,14 @@ To verify that your connection is set up successfully: If successful, the command returns `[{"1"=>1}]` +### Run ClickHouse migrations + +To create the required database objects execute: + +```shell +sudo gitlab-rake gitlab:clickhouse:migrate +``` + ### Enable feature flags Features that use ClickHouse are currently under development and are disabled by feature flags. diff --git a/doc/development/scalability.md b/doc/development/scalability.md index 733e94cb5a7..ada7adc8bdd 100644 --- a/doc/development/scalability.md +++ b/doc/development/scalability.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # GitLab scalability diff --git a/doc/development/search/advanced_search_migration_styleguide.md b/doc/development/search/advanced_search_migration_styleguide.md index 10c4fa3dcc6..87083d1a36f 100644 --- a/doc/development/search/advanced_search_migration_styleguide.md +++ b/doc/development/search/advanced_search_migration_styleguide.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Global Search -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Advanced search migration style guide @@ -66,17 +66,15 @@ The following migration helpers are available in `ee/app/workers/concerns/elasti Backfills a specific field in an index. In most cases, the mapping for the field should already be added. -Requires the `index_name` and `field_name` methods to backfill a single field. +Requires the `field_name` method and `DOCUMENT_TYPE` constant to backfill a single field. ```ruby class MigrationName < Elastic::Migration include Elastic::MigrationBackfillHelper - private + DOCUMENT_TYPE = Issue - def index_name - Issue.__elasticsearch__.index_name - end + private def field_name :schema_version @@ -84,17 +82,15 @@ class MigrationName < Elastic::Migration end ``` -Requires the `index_name` and `field_names` methods to backfill multiple fields if any field is null. +Requires the `field_names` method and `DOCUMENT_TYPE` constant to backfill multiple fields if any field is null. ```ruby class MigrationName < Elastic::Migration include Elastic::MigrationBackfillHelper - private + DOCUMENT_TYPE = Issue - def index_name - Issue.__elasticsearch__.index_name - end + private def field_names %w[schema_version visibility_level] @@ -106,17 +102,15 @@ end Updates a mapping in an index by calling `put_mapping` with the mapping specified. -Requires the `index_name` and `new_mappings` methods. +Requires the `new_mappings` method and `DOCUMENT_TYPE` constant. ```ruby class MigrationName < Elastic::Migration include Elastic::MigrationUpdateMappingsHelper - private + DOCUMENT_TYPE = Issue - def index_name - Issue.__elasticsearch__.index_name - end + private def new_mappings { @@ -402,7 +396,7 @@ actually want to hold up GitLab.com deployments on advanced search migrations, as they may still have another week to go, and that's too long to block deployments. -### Process for removing migrations +### Process for marking migrations as obsolete For every migration that was created 2 minor versions before the major version being upgraded to, we do the following: @@ -415,8 +409,17 @@ being upgraded to, we do the following: ``` 1. Delete any spec files to support this migration. +1. Verify that there are no references of the migration in the `.rubocop_todo/` directory. 1. Remove any logic handling backwards compatibility for this migration. You can find this by looking for `Elastic::DataMigrationService.migration_has_finished?(:migration_name_in_lowercase)`. 1. Create a merge request with these changes. Noting that we should not accidentally merge this before the major release is started. + +### Process for removing migrations + +1. Select migrations that were marked as obsolete before the current major release +1. If the step above includes all obsolete migrations, keep one last migration as a safeguard for customers with unapplied migrations +1. Delete migration files and spec files for those migrations +1. Verify that there are no references of the migrations in the `.rubocop_todo/` directory. +1. Create a merge request and assign it to a team member from the global search team. diff --git a/doc/development/sec/analyzer_development_guide.md b/doc/development/sec/analyzer_development_guide.md index c76b7f5e55f..eb59d8fcaf5 100644 --- a/doc/development/sec/analyzer_development_guide.md +++ b/doc/development/sec/analyzer_development_guide.md @@ -1,7 +1,7 @@ --- stage: Secure group: Static Analysis -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Sec section analyzer development @@ -226,7 +226,7 @@ After the above steps have been completed, the automatic release process execute ### Steps to perform after releasing an analyzer -1. After a new version of the analyzer Docker image has been tagged and deployed, please test it with the corresponding test project. +1. After a new version of the analyzer Docker image has been tagged and deployed, test it with the corresponding test project. 1. Announce the release on the relevant group Slack channel. Example message: > FYI I've just released `ANALYZER_NAME` `ANALYZER_VERSION`. `LINK_TO_RELEASE` diff --git a/doc/development/sec/gemnasium_analyzer_data.md b/doc/development/sec/gemnasium_analyzer_data.md index 2da787a277a..bc3a6aac78b 100644 --- a/doc/development/sec/gemnasium_analyzer_data.md +++ b/doc/development/sec/gemnasium_analyzer_data.md @@ -1,7 +1,7 @@ --- stage: Secure group: Composition Analysis -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Gemnasium analyzer data diff --git a/doc/development/sec/generate_test_vulnerabilities.md b/doc/development/sec/generate_test_vulnerabilities.md index 75fb7edd68e..d862ee91c83 100644 --- a/doc/development/sec/generate_test_vulnerabilities.md +++ b/doc/development/sec/generate_test_vulnerabilities.md @@ -1,8 +1,7 @@ --- -type: reference, howto stage: Govern group: Threat Insights -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Generate test vulnerabilities @@ -11,7 +10,7 @@ You can generate test vulnerabilities for the [Vulnerability Report](../../user/ vulnerability management features without running a pipeline. 1. Log in to GitLab. -1. Go to `/-/profile/personal_access_tokens` and generate a personal access token with `api` permissions. +1. Go to `/-/user_settings/personal_access_tokens` and generate a personal access token with `api` permissions. 1. Go to your project page and find the project ID. You can find the project ID below the project title. 1. [Clone the GitLab repository](../../gitlab-basics/start-using-git.md#clone-a-repository) to your local machine. 1. Open a terminal and go to `gitlab/qa` directory. diff --git a/doc/development/sec/index.md b/doc/development/sec/index.md index b887d13c267..9e8486b26fa 100644 --- a/doc/development/sec/index.md +++ b/doc/development/sec/index.md @@ -1,8 +1,7 @@ --- stage: Secure group: Static Analysis -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: index, concepts, howto +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Sec section development guidelines @@ -72,7 +71,7 @@ For details on how GitLab processes the reports generated by the scanners, see ## CI/CD template development While CI/CD templates are the responsibility of the Verify section, many are critical to the Sec Section's feature usage. -If you are working with CI/CD templates, please read the [development guide for GitLab CI/CD templates](../cicd/templates.md). +If you are working with CI/CD templates, read the [development guide for GitLab CI/CD templates](../cicd/templates.md). ## Importance of the primary identifier diff --git a/doc/development/sec/security_report_ingestion_overview.md b/doc/development/sec/security_report_ingestion_overview.md index 0408e47a1dd..de0c1861cfd 100644 --- a/doc/development/sec/security_report_ingestion_overview.md +++ b/doc/development/sec/security_report_ingestion_overview.md @@ -2,7 +2,6 @@ stage: Secure group: Threat Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: concepts --- # Security report ingestion overview diff --git a/doc/development/secure_coding_guidelines.md b/doc/development/secure_coding_guidelines.md index 180d35e04fe..a575d1ff890 100644 --- a/doc/development/secure_coding_guidelines.md +++ b/doc/development/secure_coding_guidelines.md @@ -1,8 +1,7 @@ --- -type: reference, dev stage: none group: unassigned -info: "See the Technical Writers assigned to Development Guidelines: https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-development-guidelines" +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Secure coding development guidelines @@ -15,17 +14,16 @@ goal of reducing the number of vulnerabilities released over time. **Contributing** If you would like to contribute to one of the existing documents, or add -guidelines for a new vulnerability type, please open an MR! Please try to +guidelines for a new vulnerability type, open an MR! Try to include links to examples of the vulnerability found, and link to any resources -used in defined mitigations. If you have questions or when ready for a review, -please ping `gitlab-com/gl-security/appsec`. +used in defined mitigations. If you have questions or when ready for a review, ping `gitlab-com/gl-security/appsec`. ## Permissions ### Description Application permissions are used to determine who can access what and what actions they can perform. -For more information about the permission model at GitLab, please see [the GitLab permissions guide](permissions.md) or the [EE docs on permissions](../../ee/user/permissions.md). +For more information about the permission model at GitLab, see [the GitLab permissions guide](permissions.md) or the [EE docs on permissions](../../ee/user/permissions.md). ### Impact @@ -341,7 +339,7 @@ The injected client-side code is executed on the victim's browser in the context - potentially <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> [obtain the victim's session tokens](https://youtu.be/2VFavqfDS6w?t=739) - perform actions that lead to data loss/theft or account takeover -Much of the impact is contingent upon the function of the application and the capabilities of the victim's session. For further impact possibilities, please check out [the beef project](https://beefproject.com/). +Much of the impact is contingent upon the function of the application and the capabilities of the victim's session. For further impact possibilities, check out [the beef project](https://beefproject.com/). For a demonstration of the impact on GitLab with a realistic attack scenario, see [this video on the GitLab Unfiltered channel](https://www.youtube.com/watch?v=t4PzHNycoKo) (internal, it requires being logged in with the GitLab Unfiltered account). @@ -1308,6 +1306,24 @@ This sensitive data must be handled carefully to avoid leaks which could lead to In the event of credential leak through an MR, issue, or any other medium, [reach out to SIRT team](https://about.gitlab.com/handbook/security/security-operations/sirt/#-engaging-sirt). +### Token prefixes + +User error or software bugs can lead to tokens leaking. Consider prepending a static prefix to the beginning of secrets and adding that prefix to our secrets detection capabilities. For example, GitLab Personal Access Tokens have a prefix so that the plaintext is `glpat-1234567890abcdefghij`. + +The prefix pattern should be: + +1. `gl` for GitLab +1. lowercase letters abbreviating the token class name +1. a hyphen (`-`) + +Add the new prefix to: + +- [`gitlab/app/assets/javascripts/lib/utils/secret_detection.js`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/lib/utils/secret_detection.js) +- The [GitLab Secret Detection gem](https://gitlab.com/gitlab-org/gitlab/-/tree/master/gems/gitlab-secret_detection) +- GitLab [secrets SAST analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/secrets) +- [Tokinator](https://gitlab.com/gitlab-com/gl-security/appsec/tokinator/-/blob/main/CONTRIBUTING.md?ref_type=heads) (internal tool / team members only) +- [Token Overview](../security/token_overview.md#gitlab-tokens) documentation + ### Examples Encrypting a token with `attr_encrypted` so that the plaintext can be retrieved @@ -1339,6 +1355,19 @@ class WebHookLog < ApplicationRecord end ``` +Using the `TokenAuthenticatable` class helper to create a prefixed token. + +```ruby +class User + FEED_TOKEN_PREFIX = 'glft-' + + add_authentication_token_field :feed_token, format_with_prefix: :prefix_for_feed_token + + def prefix_for_feed_token + FEED_TOKEN_PREFIX + end +``` + ## Serialization Serialization of active record models can leak sensitive attributes if they are not protected. @@ -1464,11 +1493,43 @@ Logging helps track events for debugging. Logging also allows the application to - An audit trail for log edits must be available. - To avoid data loss, logs must be saved on different storage. -### Who to contact if you have questions +## URL Spoofing + +We want to protect our users from bad actors who might try to use GitLab +features to redirect other users to malicious sites. + +Many features in GitLab allow users to post links to external websites. It is +important that the destination of any user-specified link is made very clear +to the user. + +### `external_redirect_path` + +When presenting links provided by users, if the actual URL is hidden, use the `external_redirect_path` +helper method to redirect the user to a warning page first. For example: + +```ruby +# Bad :( +# This URL comes from User-Land and may not be safe... +# We need the user to *see* where they are going. +link_to foo_social_url(@user), title: "Foo Social" do + sprite_icon('question-o') +end + +# Good :) +# The external_redirect "leaving GitLab" page will show the URL to the user +# before they leave. +link_to external_redirect_path(url: foo_social_url(@user)), title: "Foo" do + sprite_icon('question-o') +end +``` + +Also see this [real-life usage](https://gitlab.com/gitlab-org/gitlab/-/blob/bdba5446903ff634fb12ba695b2de99b6d6881b5/app/helpers/application_helper.rb#L378) as an example. + +## Who to contact if you have questions For general guidance, contact the [Application Security](https://about.gitlab.com/handbook/security/security-engineering/application-security/) team. -### Related topics +## Related topics - [Log system in GitLab](../administration/logs/index.md) - [Audit event development guidelines](../development/audit_event_guide/index.md)) diff --git a/doc/development/service_measurement.md b/doc/development/service_measurement.md index 9d22e9c376c..f193035560f 100644 --- a/doc/development/service_measurement.md +++ b/doc/development/service_measurement.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # GitLab Developers Guide to service measurement diff --git a/doc/development/session.md b/doc/development/session.md index c30364c27f8..ecf9bee836a 100644 --- a/doc/development/session.md +++ b/doc/development/session.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Accessing session data diff --git a/doc/development/shared_files.md b/doc/development/shared_files.md index 3bf18a4d410..f39d3ee98bb 100644 --- a/doc/development/shared_files.md +++ b/doc/development/shared_files.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Shared files diff --git a/doc/development/shell_commands.md b/doc/development/shell_commands.md index 321bd7aeadd..39bcf7a875f 100644 --- a/doc/development/shell_commands.md +++ b/doc/development/shell_commands.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Shell command development guidelines diff --git a/doc/development/shell_scripting_guide/index.md b/doc/development/shell_scripting_guide/index.md index d3d9304446e..25c7a50bdcf 100644 --- a/doc/development/shell_scripting_guide/index.md +++ b/doc/development/shell_scripting_guide/index.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Shell scripting standards and style guidelines diff --git a/doc/development/sidekiq/compatibility_across_updates.md b/doc/development/sidekiq/compatibility_across_updates.md index 5ca6bf773fc..d6f61efbdb1 100644 --- a/doc/development/sidekiq/compatibility_across_updates.md +++ b/doc/development/sidekiq/compatibility_across_updates.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Sidekiq Compatibility across Updates diff --git a/doc/development/sidekiq/idempotent_jobs.md b/doc/development/sidekiq/idempotent_jobs.md index 8ccbeee283d..029b18adb46 100644 --- a/doc/development/sidekiq/idempotent_jobs.md +++ b/doc/development/sidekiq/idempotent_jobs.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Sidekiq idempotent jobs @@ -30,7 +30,7 @@ an unstarted job with the same arguments is already in the queue. Make sure the worker tests pass using the following shared example: ```ruby -include_examples 'an idempotent worker' do +it_behaves_like 'an idempotent worker' do it 'marks the MR as merged' do # Using subject inside this block will process the job multiple times subject @@ -80,7 +80,7 @@ GitLab supports two deduplication strategies: More [deduplication strategies have been suggested](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/195). If you are implementing a worker that could benefit from a different -strategy, please comment in the issue. +strategy, comment in the issue. #### Until Executing diff --git a/doc/development/sidekiq/index.md b/doc/development/sidekiq/index.md index 508f19a5be7..62e9e9f6b04 100644 --- a/doc/development/sidekiq/index.md +++ b/doc/development/sidekiq/index.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Sidekiq development guidelines diff --git a/doc/development/sidekiq/limited_capacity_worker.md b/doc/development/sidekiq/limited_capacity_worker.md index b1aff829d4d..e8561cb936d 100644 --- a/doc/development/sidekiq/limited_capacity_worker.md +++ b/doc/development/sidekiq/limited_capacity_worker.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Sidekiq limited capacity worker diff --git a/doc/development/sidekiq/logging.md b/doc/development/sidekiq/logging.md index 8a2a77b5226..45599e68d81 100644 --- a/doc/development/sidekiq/logging.md +++ b/doc/development/sidekiq/logging.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Sidekiq logging diff --git a/doc/development/sidekiq/worker_attributes.md b/doc/development/sidekiq/worker_attributes.md index 814b61c746b..016bf0b6634 100644 --- a/doc/development/sidekiq/worker_attributes.md +++ b/doc/development/sidekiq/worker_attributes.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Sidekiq worker attributes @@ -336,7 +336,7 @@ worker that checks if any paused jobs must be restarted. To use `pause_control`, you can: - Use one of the strategies defined in `lib/gitlab/sidekiq_middleware/pause_control/strategies/`. -- Define a custom strategy in `lib/gitlab/sidekiq_middleware/pause_control/strategies/` and add the strategy to `lib/gitlab/sidekiq_middleware/pause_control/strategies.rb`. +- Define a custom strategy in `lib/gitlab/sidekiq_middleware/pause_control/strategies/` and add the strategy to `lib/gitlab/sidekiq_middleware/pause_control.rb`. For example: diff --git a/doc/development/software_design.md b/doc/development/software_design.md index f3a2c8eee0b..a771e7a9e61 100644 --- a/doc/development/software_design.md +++ b/doc/development/software_design.md @@ -1,7 +1,7 @@ --- stage: none group: Engineering Productivity -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Software design guides diff --git a/doc/development/spam_protection_and_captcha/exploratory_testing.md b/doc/development/spam_protection_and_captcha/exploratory_testing.md index 188c168fca5..788b971c4b9 100644 --- a/doc/development/spam_protection_and_captcha/exploratory_testing.md +++ b/doc/development/spam_protection_and_captcha/exploratory_testing.md @@ -1,7 +1,7 @@ --- stage: Govern group: Anti-Abuse -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Exploratory testing of CAPTCHAs diff --git a/doc/development/spam_protection_and_captcha/graphql_api.md b/doc/development/spam_protection_and_captcha/graphql_api.md index 346648aefbb..dca01aff3c7 100644 --- a/doc/development/spam_protection_and_captcha/graphql_api.md +++ b/doc/development/spam_protection_and_captcha/graphql_api.md @@ -1,7 +1,7 @@ --- stage: Govern group: Anti-Abuse -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # GraphQL API spam protection and CAPTCHA support diff --git a/doc/development/spam_protection_and_captcha/index.md b/doc/development/spam_protection_and_captcha/index.md index 732d324f185..f48dc8e9189 100644 --- a/doc/development/spam_protection_and_captcha/index.md +++ b/doc/development/spam_protection_and_captcha/index.md @@ -1,7 +1,7 @@ --- stage: Govern group: Anti-Abuse -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Spam protection and CAPTCHA diff --git a/doc/development/spam_protection_and_captcha/model_and_services.md b/doc/development/spam_protection_and_captcha/model_and_services.md index 17d90ed7f1e..28d28e75c52 100644 --- a/doc/development/spam_protection_and_captcha/model_and_services.md +++ b/doc/development/spam_protection_and_captcha/model_and_services.md @@ -1,7 +1,7 @@ --- stage: Govern group: Anti-Abuse -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Model and services spam protection and CAPTCHA support diff --git a/doc/development/spam_protection_and_captcha/rest_api.md b/doc/development/spam_protection_and_captcha/rest_api.md index 2a43b585b94..4e5cd2ce3cb 100644 --- a/doc/development/spam_protection_and_captcha/rest_api.md +++ b/doc/development/spam_protection_and_captcha/rest_api.md @@ -1,7 +1,7 @@ --- stage: Govern group: Anti-Abuse -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # REST API spam protection and CAPTCHA support diff --git a/doc/development/spam_protection_and_captcha/web_ui.md b/doc/development/spam_protection_and_captcha/web_ui.md index c134f5e6683..f6e417a6647 100644 --- a/doc/development/spam_protection_and_captcha/web_ui.md +++ b/doc/development/spam_protection_and_captcha/web_ui.md @@ -1,7 +1,7 @@ --- stage: Govern group: Anti-Abuse -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Web UI spam protection and CAPTCHA support diff --git a/doc/development/sql.md b/doc/development/sql.md index d7a5923efce..86892f86e0c 100644 --- a/doc/development/sql.md +++ b/doc/development/sql.md @@ -1,7 +1,7 @@ --- stage: Data Stores group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # SQL Query Guidelines @@ -86,7 +86,7 @@ _can_ be used by `ILIKE` / `LIKE` and can lead to greatly improved performance. One downside of these indexes is that they can easily get quite large (depending on the amount of data indexed). -To keep naming of these indexes consistent please use the following naming +To keep naming of these indexes consistent, use the following naming pattern: ```plaintext diff --git a/doc/development/stage_group_observability/dashboards/error_budget_detail.md b/doc/development/stage_group_observability/dashboards/error_budget_detail.md index c45c31c5bd1..64f158ec098 100644 --- a/doc/development/stage_group_observability/dashboards/error_budget_detail.md +++ b/doc/development/stage_group_observability/dashboards/error_budget_detail.md @@ -1,7 +1,7 @@ --- stage: Platforms group: Scalability -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Error budget detail dashboard diff --git a/doc/development/stage_group_observability/dashboards/index.md b/doc/development/stage_group_observability/dashboards/index.md index c2da46bde7d..581ab0370cd 100644 --- a/doc/development/stage_group_observability/dashboards/index.md +++ b/doc/development/stage_group_observability/dashboards/index.md @@ -1,7 +1,7 @@ --- stage: Platforms group: Scalability -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Dashboards for stage groups diff --git a/doc/development/stage_group_observability/dashboards/stage_group_dashboard.md b/doc/development/stage_group_observability/dashboards/stage_group_dashboard.md index 7c596e544b5..948ad4340a1 100644 --- a/doc/development/stage_group_observability/dashboards/stage_group_dashboard.md +++ b/doc/development/stage_group_observability/dashboards/stage_group_dashboard.md @@ -1,7 +1,7 @@ --- stage: Platforms group: Scalability -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Stage group dashboard diff --git a/doc/development/stage_group_observability/index.md b/doc/development/stage_group_observability/index.md index ba17b4cc73a..441e3268044 100644 --- a/doc/development/stage_group_observability/index.md +++ b/doc/development/stage_group_observability/index.md @@ -1,7 +1,7 @@ --- stage: Platforms group: Scalability -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Observability for stage groups diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md index dbee7acac69..458a5efea42 100644 --- a/doc/development/testing_guide/best_practices.md +++ b/doc/development/testing_guide/best_practices.md @@ -1,8 +1,7 @@ --- -type: reference, dev stage: none group: unassigned -info: "See the Technical Writers assigned to Development Guidelines: https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-development-guidelines" +info: "See the Technical Writers assigned to Development Guidelines: https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-development-guidelines" description: "GitLab development guidelines - testing best practices." --- @@ -386,7 +385,7 @@ NOTE: `stub_method` does not support method existence and method arity checks. WARNING: -`stub_method` is supposed to be used in factories only. It's strongly discouraged to be used elsewhere. Please consider using [RSpec mocks](https://rspec.info/features/3-12/rspec-mocks/) if available. +`stub_method` is supposed to be used in factories only. It's strongly discouraged to be used elsewhere. Consider using [RSpec mocks](https://rspec.info/features/3-12/rspec-mocks/) if available. #### Stubbing member access level @@ -583,7 +582,7 @@ Use the coverage reports to ensure your tests cover 100% of your code. NOTE: Before writing a new system test, -[please consider **not** writing one](testing_levels.md#consider-not-writing-a-system-test)! +[consider **not** writing one](testing_levels.md#consider-not-writing-a-system-test)! - Feature specs should be named `ROLE_ACTION_spec.rb`, such as `user_changes_password_spec.rb`. @@ -1298,7 +1297,7 @@ creates and deletes indices before and after all examples. The `:elastic_delete_by_query` trait was added to reduce runtime for pipelines by creating and deleting indices at the start and end of each context only. The [Elasticsearch delete by query API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-delete-by-query.html) -is used to delete data in all indices between examples to ensure a clean index. +is used to delete data in all indices (except the migrations index) between examples to ensure a clean index. The `:elastic_clean` trait creates and deletes indices between examples to ensure a clean index. This way, tests are not polluted with non-essential data. If using the `:elastic` or `:elastic_delete_by_query` trait @@ -1336,6 +1335,12 @@ It uses the [Elasticsearch Refresh API](https://www.elastic.co/guide/en/elastics to make sure all operations performed on an index since the last refresh are available for search. This method is typically called after loading data into PostgreSQL to ensure the data is indexed and searchable. +You can use the `SEARCH_SPEC_BENCHMARK` environment variable to benchmark test setup steps: + +```console +SEARCH_SPEC_BENCHMARK=1 bundle exec rspec ee/spec/lib/elastic/latest/merge_request_class_proxy_spec.rb +``` + #### Test Snowplow events WARNING: diff --git a/doc/development/testing_guide/contract/consumer_tests.md b/doc/development/testing_guide/contract/consumer_tests.md index 4a84dafdcf2..de2ef53949b 100644 --- a/doc/development/testing_guide/contract/consumer_tests.md +++ b/doc/development/testing_guide/contract/consumer_tests.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Writing consumer tests diff --git a/doc/development/testing_guide/contract/index.md b/doc/development/testing_guide/contract/index.md index 45933e9cb4f..ece0b6de420 100644 --- a/doc/development/testing_guide/contract/index.md +++ b/doc/development/testing_guide/contract/index.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Contract testing diff --git a/doc/development/testing_guide/contract/provider_tests.md b/doc/development/testing_guide/contract/provider_tests.md index 86ac6518d2f..22f3d3487ba 100644 --- a/doc/development/testing_guide/contract/provider_tests.md +++ b/doc/development/testing_guide/contract/provider_tests.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Writing provider tests diff --git a/doc/development/testing_guide/end_to_end/beginners_guide.md b/doc/development/testing_guide/end_to_end/beginners_guide.md index 4a3aec97d29..b57757c5aac 100644 --- a/doc/development/testing_guide/end_to_end/beginners_guide.md +++ b/doc/development/testing_guide/end_to_end/beginners_guide.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Beginner's guide to writing end-to-end tests diff --git a/doc/development/testing_guide/end_to_end/best_practices.md b/doc/development/testing_guide/end_to_end/best_practices.md index 8cf1a46d5d2..cd92f8656e9 100644 --- a/doc/development/testing_guide/end_to_end/best_practices.md +++ b/doc/development/testing_guide/end_to_end/best_practices.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # End-to-end testing Best Practices @@ -128,7 +128,7 @@ end 1. Every `describe`, `context`, and `it` blocks should have a short description attached 1. Keep descriptions as concise as possible. 1. Long descriptions or multiple conditionals could be a sign it should be split up (additional `context` blocks). - 1. The [Documentation Style Guide](../../documentation/styleguide/index.md) gives recommendations on how to write concisely and with [active voice](../../documentation/styleguide/word_list.md#active-voice). + 1. The [Documentation Style Guide](../../documentation/styleguide/index.md) gives recommendations on how to write concisely and with [active voice](../../documentation/styleguide/index.md#active-voice). 1. The outermost `Rspec.describe` block should be [the DevOps stage name](https://about.gitlab.com/handbook/product/categories/#devops-stages) 1. Inside the `Rspec.describe` block is a `describe` block with the name of the feature being tested 1. Optional `context` blocks define what the conditions being tested are @@ -361,7 +361,7 @@ When you add a new test that requires administrator access, apply the RSpec meta When running tests locally or configuring a pipeline, the environment variable `QA_CAN_TEST_ADMIN_FEATURES` can be set to `false` to skip tests that have the `:requires_admin` tag. NOTE: -If the _only_ action in the test that requires administrator access is to toggle a feature flag, please use the `feature_flag` tag instead. More details can be found in [testing with feature flags](feature_flags.md). +If the _only_ action in the test that requires administrator access is to toggle a feature flag, use the `feature_flag` tag instead. More details can be found in [testing with feature flags](feature_flags.md). ## Prefer `Commit` resource over `ProjectPush` diff --git a/doc/development/testing_guide/end_to_end/capybara_to_chemlab_migration_guide.md b/doc/development/testing_guide/end_to_end/capybara_to_chemlab_migration_guide.md index 025f998c0c9..64bb5df5db1 100644 --- a/doc/development/testing_guide/end_to_end/capybara_to_chemlab_migration_guide.md +++ b/doc/development/testing_guide/end_to_end/capybara_to_chemlab_migration_guide.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Migration Guide Capybara → Chemlab diff --git a/doc/development/testing_guide/end_to_end/dynamic_element_validation.md b/doc/development/testing_guide/end_to_end/dynamic_element_validation.md index 5cc82916cdd..f78b7208bbf 100644 --- a/doc/development/testing_guide/end_to_end/dynamic_element_validation.md +++ b/doc/development/testing_guide/end_to_end/dynamic_element_validation.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Dynamic Element Validation diff --git a/doc/development/testing_guide/end_to_end/execution_context_selection.md b/doc/development/testing_guide/end_to_end/execution_context_selection.md index 0082bb6ee23..f625dc466b9 100644 --- a/doc/development/testing_guide/end_to_end/execution_context_selection.md +++ b/doc/development/testing_guide/end_to_end/execution_context_selection.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Execution context selection diff --git a/doc/development/testing_guide/end_to_end/feature_flags.md b/doc/development/testing_guide/end_to_end/feature_flags.md index 228f63d2354..e11119d2c0b 100644 --- a/doc/development/testing_guide/end_to_end/feature_flags.md +++ b/doc/development/testing_guide/end_to_end/feature_flags.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Testing with feature flags @@ -16,7 +16,7 @@ and `GITLAB_ADMIN_PASSWORD`. ## `feature_flag` RSpec tag -Please be sure to include the `feature_flag` tag so that the test can be skipped on the appropriate environments. +Be sure to include the `feature_flag` tag so that the test can be skipped on the appropriate environments. **Optional metadata:** @@ -181,7 +181,7 @@ active feature flag. To circumvent this behavior, add a wait for elements behind It's also possible to run an entire scenario with a feature flag enabled, without having to edit existing tests or write new ones. -Please see the [QA README](https://gitlab.com/gitlab-org/gitlab/-/tree/master/qa#running-tests-with-a-feature-flag-enabled) +See the [QA README](https://gitlab.com/gitlab-org/gitlab/-/tree/master/qa#running-tests-with-a-feature-flag-enabled) for details. ## Confirming that end-to-end tests pass with a feature flag enabled @@ -219,4 +219,4 @@ pass on the default branch. The end-to-end tests run on the default branch every If the relevant tests do not enable the feature flag themselves, you can check if the tests will need to be updated by opening a draft merge request that enables the flag by default via a [feature flag definition file](../../feature_flags/index.md#feature-flag-definition-and-validation). That will [automatically execute the end-to-end test suite](#automatic-test-execution-when-a-feature-flag-definition-changes). -The merge request can be closed once the tests pass. If you need assistance to update the tests, please contact the relevant [stable counterpart in the Quality department](https://about.gitlab.com/handbook/engineering/quality/#individual-contributors), or any Software Engineer in Test if there is no stable counterpart for your group. +The merge request can be closed once the tests pass. If you need assistance to update the tests, contact the relevant [stable counterpart in the Quality department](https://about.gitlab.com/handbook/engineering/quality/#individual-contributors), or any Software Engineer in Test if there is no stable counterpart for your group. diff --git a/doc/development/testing_guide/end_to_end/flows.md b/doc/development/testing_guide/end_to_end/flows.md index 58702ceb861..c1d9f935da5 100644 --- a/doc/development/testing_guide/end_to_end/flows.md +++ b/doc/development/testing_guide/end_to_end/flows.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Flows in GitLab QA diff --git a/doc/development/testing_guide/end_to_end/index.md b/doc/development/testing_guide/end_to_end/index.md index 30fb1abd7df..33465816ec1 100644 --- a/doc/development/testing_guide/end_to_end/index.md +++ b/doc/development/testing_guide/end_to_end/index.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # End-to-end Testing @@ -78,7 +78,7 @@ subgraph " `gitlab-org/gitlab-qa-mirror` pipeline" and polls for the resulting status. We call this a _status attribution_. 1. In the [`gitlab-org/build/omnibus-gitlab-mirror` pipeline](https://gitlab.com/gitlab-org/build/omnibus-gitlab-mirror): - 1. Docker image is being built and pushed to its Container Registry. + 1. Docker image is being built and pushed to its container registry. 1. Once Docker images are built and pushed jobs in `test` stage are started 1. In the `Test` stage: @@ -266,7 +266,7 @@ use the [GitLab QA orchestrator](https://gitlab.com/gitlab-org/gitlab-qa/tree/ma On the other hand, if you would like to run against a local development GitLab environment, you can use the [GitLab Development Kit (GDK)](https://gitlab.com/gitlab-org/gitlab-development-kit/). -Please refer to the instructions in the [QA README](https://gitlab.com/gitlab-org/gitlab/-/tree/master/qa/README.md#how-can-i-use-it) +Refer to the instructions in the [QA README](https://gitlab.com/gitlab-org/gitlab/-/tree/master/qa/README.md#how-can-i-use-it) and the section below. ### Running tests that require special setup @@ -302,7 +302,7 @@ Continued reading: ## Where can you ask for help? -You can ask question in the `#quality` channel on Slack (GitLab internal) or +You can ask question in the `#test-platform` channel on Slack (GitLab internal) or you can find an issue you would like to work on in [the `gitlab` issue tracker](https://gitlab.com/gitlab-org/gitlab/-/issues?label_name%5B%5D=QA&label_name%5B%5D=test), or [the `gitlab-qa` issue tracker](https://gitlab.com/gitlab-org/gitlab-qa/-/issues?label_name%5B%5D=new+scenario). diff --git a/doc/development/testing_guide/end_to_end/package_and_test_pipeline.md b/doc/development/testing_guide/end_to_end/package_and_test_pipeline.md deleted file mode 100644 index 240db2cbfe5..00000000000 --- a/doc/development/testing_guide/end_to_end/package_and_test_pipeline.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'test_pipelines.md' -remove_date: '2023-11-08' ---- - -This document was moved to [another location](test_pipelines.md). - -<!-- This redirect file can be deleted after <2023-11-08>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/development/testing_guide/end_to_end/page_objects.md b/doc/development/testing_guide/end_to_end/page_objects.md index 3e13aaa4f02..812d2724b72 100644 --- a/doc/development/testing_guide/end_to_end/page_objects.md +++ b/doc/development/testing_guide/end_to_end/page_objects.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Page objects in GitLab QA @@ -303,8 +303,8 @@ from within the `qa` directory. ## Where to ask for help? -If you need more information, ask for help on `#quality` channel on Slack +If you need more information, ask for help on `#test-platform` channel on Slack (internal, GitLab Team only). -If you are not a Team Member, and you still need help to contribute, please +If you are not a Team Member, and you still need help to contribute, open an issue in GitLab CE issue tracker with the `~QA` label. diff --git a/doc/development/testing_guide/end_to_end/resources.md b/doc/development/testing_guide/end_to_end/resources.md index 735bab2fa0a..7ff44f8cddb 100644 --- a/doc/development/testing_guide/end_to_end/resources.md +++ b/doc/development/testing_guide/end_to_end/resources.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Resource classes in GitLab QA @@ -494,14 +494,14 @@ We have a mechanism to [collect](https://gitlab.com/gitlab-org/gitlab/-/blob/443 all resources created during test executions, and another to [handle](https://gitlab.com/gitlab-org/gitlab/-/blob/44345381e89d6bbd440f7b4c680d03e8b75b86de/qa/qa/tools/test_resources_handler.rb#L44) these resources. On [dotcom environments](https://about.gitlab.com/handbook/engineering/infrastructure/environments/#environments), after a test suite finishes in the [QA pipelines](https://about.gitlab.com/handbook/engineering/quality/quality-engineering/debugging-qa-test-failures/#scheduled-qa-test-pipelines), resources from all passing test are automatically deleted in the same pipeline run. Resources from all failed tests are reserved for investigation, -and won't be deleted until the following Saturday by a scheduled pipeline. When introducing new resources, please +and won't be deleted until the following Saturday by a scheduled pipeline. When introducing new resources, also make sure to add any resource that cannot be deleted to the [IGNORED_RESOURCES](https://gitlab.com/gitlab-org/gitlab/-/blob/44345381e89d6bbd440f7b4c680d03e8b75b86de/qa/qa/tools/test_resources_handler.rb#L29) list. ## Where to ask for help? -If you need more information, ask for help on `#quality` channel on Slack +If you need more information, ask for help on `#test-platform` channel on Slack (internal, GitLab Team only). -If you are not a Team Member, and you still need help to contribute, please +If you are not a Team Member, and you still need help to contribute, open an issue in GitLab CE issue tracker with the `~QA` label. diff --git a/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md b/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md index 93adc492d5a..5d7897331a2 100644 --- a/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md +++ b/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # RSpec metadata for end-to-end tests diff --git a/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md b/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md index 0544b331b1a..aaea3f8958d 100644 --- a/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md +++ b/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Running tests that require special setup @@ -10,7 +10,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w The [`jenkins_build_status_spec`](https://gitlab.com/gitlab-org/gitlab/-/blob/24a86debf49f3aed6f2ecfd6e8f9233b3a214181/qa/qa/specs/features/browser_ui/3_create/jenkins/jenkins_build_status_spec.rb) spins up a Jenkins instance in a Docker container with the Jenkins GitLab plugin pre-installed. Due to a license restriction we are unable to distribute this image. -To build a QA compatible image, please visit the [third party images project](https://gitlab.com/gitlab-org/quality/third-party-docker-public), where third party Dockerfiles can be found. +To build a QA compatible image, visit the [third party images project](https://gitlab.com/gitlab-org/quality/third-party-docker-public), where third party Dockerfiles can be found. The project also has instructions for forking and building the images automatically in CI. Some extra environment variables for the location of the forked repository are also needed. @@ -47,7 +47,7 @@ bin/qa Test::Instance::All http://localhost -- qa/specs/features/ee/browser_ui/3 The test automatically spins up a Docker container for Jenkins and tear down once the test completes. -If you need to run Jenkins manually outside of the tests, please refer to the README for the +If you need to run Jenkins manually outside of the tests, refer to the README for the [third party images project](https://gitlab.com/gitlab-org/quality/third-party-docker-public/-/blob/main/jenkins/README.md) ### Troubleshooting @@ -420,7 +420,7 @@ To run these tests locally against the GDK: QA_LOG_LEVEL=debug WEBDRIVER_HEADLESS=false bin/qa Test::Instance::All http://localhost:3000 qa/specs/features/browser_ui/2_plan/email/trigger_email_notification_spec.rb -- --tag orchestrated ``` -For instructions on how to run these tests using the `gitlab-qa` gem, please refer to [the GitLab QA documentation](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/master/docs/what_tests_can_be_run.md#testintegrationsmtp-ceeefull-image-address). +For instructions on how to run these tests using the `gitlab-qa` gem, refer to [the GitLab QA documentation](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/master/docs/what_tests_can_be_run.md#testintegrationsmtp-ceeefull-image-address). ## Guide to the mobile suite @@ -600,3 +600,23 @@ To run the [`login_via_oauth_and_oidc_with_gitlab_as_idp_spec`](https://gitlab.c RELEASE_REGISTRY_URL='registry.gitlab.com' RELEASE_REGISTRY_USERNAME='<your_gitlab_username>' RELEASE_REGISTRY_PASSWORD='<your_gitlab_personal_access_token>' RELEASE='registry.gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/gitlab-ee:c0ae46db6b31ea231b2de88961cd687acf634179' GITLAB_QA_ADMIN_ACCESS_TOKEN="<your_gdk_admin_personal_access_token>" QA_DEBUG=true CHROME_HEADLESS=false bundle exec bin/qa Test::Instance::All http://gdk.test:3000 qa/specs/features/browser_ui/1_manage/login/login_via_oauth_and_oidc_with_gitlab_as_idp_spec.rb ``` + +## Product Analytics tests + +Product Analytics e2e tests require Product Analytics services running and connected to your GDK. + +In order to run Product Analytics services, devkit can be used. Instructions to set it up and connect to your GDK can be found in the [devkit project's `README.md`](https://gitlab.com/gitlab-org/analytics-section/product-analytics/devkit). + +Additionally, the following setup is required on the GDK: + +- Ultimate license applied. + - [How to request the license](https://about.gitlab.com/handbook/developer-onboarding/#working-on-gitlab-ee-developer-licenses). + - [How to activate GitLab EE with a license file or key](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/administration/license_file.md#activate-gitlab-ee-with-a-license-file-or-key). +- Product Analytics feature flags enabled. The list of feature flags can be [found here](https://gitlab.com/gitlab-org/gitlab/-/tree/master/doc/user/product_analytics#enable-product-analytics). +- Simulate SaaS enabled. Instructions can be [found here](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/ee_features.md#simulate-a-saas-instance). + +Once Product Analytics services are running and are connected to your GDK, the tests can be executed with: + +```shell +bundle exec rspec qa/specs/features/ee/browser_ui/8_monitor/product_analytics/onboarding_spec.rb +``` diff --git a/doc/development/testing_guide/end_to_end/style_guide.md b/doc/development/testing_guide/end_to_end/style_guide.md index f5a3fa2fc51..966ed851115 100644 --- a/doc/development/testing_guide/end_to_end/style_guide.md +++ b/doc/development/testing_guide/end_to_end/style_guide.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Style guide for writing end-to-end tests @@ -19,10 +19,8 @@ When selecting a single link to navigate, use `click_`. For example: ```ruby -def click_ci_cd_pipelines - within_sidebar do - click_element(:link_pipelines) - end +def click_add_badge_button + click_element :add_badge_button end ``` @@ -38,12 +36,8 @@ When interacting with multiple elements to go to a page, use `go_to_`. For example: ```ruby -def go_to_operations_environments - hover_operations do - within_submenu do - click_element(:operations_environments_link) - end - end +def go_to_applications + click_element('nav-item-link', submenu_item: 'Applications') end ``` @@ -77,7 +71,7 @@ We follow a simple formula roughly based on Hungarian notation. - `_menu_item` NOTE: -If none of the listed types are suitable, please open a merge request to add an appropriate type to the list. +If none of the listed types are suitable, open a merge request to add an appropriate type to the list. ### Examples diff --git a/doc/development/testing_guide/end_to_end/test_infrastructure.md b/doc/development/testing_guide/end_to_end/test_infrastructure.md index 00d4507d35e..3f00885ef96 100644 --- a/doc/development/testing_guide/end_to_end/test_infrastructure.md +++ b/doc/development/testing_guide/end_to_end/test_infrastructure.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # End-to-end Testing Infrastructure for Cloud Integrations diff --git a/doc/development/testing_guide/end_to_end/test_pipelines.md b/doc/development/testing_guide/end_to_end/test_pipelines.md index b47b75e398a..12a0728c65d 100644 --- a/doc/development/testing_guide/end_to_end/test_pipelines.md +++ b/doc/development/testing_guide/end_to_end/test_pipelines.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # End-to-end test pipelines @@ -161,7 +161,7 @@ The pipeline setup consists of several jobs in the main GitLab pipeline: #### `build-gdk-image` [This job](https://gitlab.com/gitlab-org/gitlab/-/blob/07504c34b28ac656537cd60810992aa15e9e91b8/.gitlab/ci/build-images.gitlab-ci.yml#L32) -uses the code from the merge request to build a Docker image that can be used in test jobs to launch a GDK instance in a container. The image is pushed to the Container Registry. +uses the code from the merge request to build a Docker image that can be used in test jobs to launch a GDK instance in a container. The image is pushed to the container registry. The job also runs in pipelines on the default branch to build a base image that includes the GDK and GitLab components. This avoids building the entire image from scratch in merge requests. However, if the merge request includes changes to diff --git a/doc/development/testing_guide/end_to_end/troubleshooting.md b/doc/development/testing_guide/end_to_end/troubleshooting.md index b4c47b90f0b..4b9a2137424 100644 --- a/doc/development/testing_guide/end_to_end/troubleshooting.md +++ b/doc/development/testing_guide/end_to_end/troubleshooting.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Troubleshooting end-to-end tests diff --git a/doc/development/testing_guide/flaky_tests.md b/doc/development/testing_guide/flaky_tests.md index 019e0654456..1895b9bdb39 100644 --- a/doc/development/testing_guide/flaky_tests.md +++ b/doc/development/testing_guide/flaky_tests.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Flaky tests @@ -233,7 +233,7 @@ Once a test is in quarantine, there are 3 choices: On our CI, we use [`RSpec::Retry`](https://github.com/NoRedInk/rspec-retry) to automatically retry a failing example a few times (see [`spec/spec_helper.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/spec_helper.rb) for the precise retries count). -We also use a custom [`RspecFlaky::Listener`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/gems/rspec_flaky/lib/rspec_flaky/listener.rb). +We also use a custom [`Gitlab::RspecFlaky::Listener`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/gems/gitlab-rspec_flaky/lib/gitlab/rspec_flaky/listener.rb). This listener runs in the `update-tests-metadata` job in `maintenance` scheduled pipelines on the `master` branch, and saves flaky examples to `rspec/flaky/report-suite.json`. The report file is then retrieved by the `retrieve-tests-metadata` job in all pipelines. @@ -333,6 +333,17 @@ If a spec hangs, it might be caused by a [bug in Rails](https://github.com/rails It could help to split the large RSpec files in multiple files in order to narrow down the context and identify the problematic tests. +### Recreate job failure in CI by forcing the job to run the same set of test files + +Reproducing a job failure in CI always helps with troubleshooting why and how a test fails. This require us running the same test files with the same spec order. Since we use Knapsack to distribute tests across parallelized jobs, and files can be distributed differently between two pipelines, we can hardcode this job distribution through the following steps: + +1. Find a job that you want to reproduce, identify the commit that it ran against, set your local `gitlab-org/gitlab` branch to the same commit to ensure we are running with the same copy of the project. +1. In the job log, locate the list of spec files that were distributed by Knapsack - you can search for `Running command: bundle exec rspec`, the last argument of this command should contain a list of file names. Copy this list. +1. Go to `tooling/lib/tooling/parallel_rspec_runner.rb` where the test file distribution happens. Have a look at [this merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/137924/diffs) as an example, store the file list you copied from step 2 into a `TEST_FILES` constant and have RSpec run this list by updating the `rspec_command` method as done in the example MR. +1. Skip the tests in `spec/tooling/lib/tooling/parallel_rspec_runner_spec.rb` so it doesn't cause your pipeline to fail early. +1. Since we want to force the pipeline to run against a specific version, we do not want to run a merged results pipeline. We can introduce a merge conflict into the MR to achieve this. +1. To preserve spec ordering, update the `spec/support/rspec_order.rb` file by hard coding `Kernel.srand` with the value shown in the originally failing job, as done [here](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/128428/diffs#32f6fa4961481518204e227252552dba7483c3b0_62_62). You can fine the srand value in the job log by searching `Randomized with seed` which is followed by this value. + ## Resources - [Flaky Tests: Are You Sure You Want to Rerun Them?](https://semaphoreci.com/blog/2017/04/20/flaky-tests.html) diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index 7e79080bbd1..d14ca15991c 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Frontend testing standards and style guidelines diff --git a/doc/development/testing_guide/index.md b/doc/development/testing_guide/index.md index 68d30ff6656..2136a686489 100644 --- a/doc/development/testing_guide/index.md +++ b/doc/development/testing_guide/index.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Testing standards and style guidelines diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md index 9da63d0d1d7..d0a6fa8da5f 100644 --- a/doc/development/testing_guide/review_apps.md +++ b/doc/development/testing_guide/review_apps.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Using review apps in the development of GitLab @@ -203,16 +203,16 @@ subgraph "CNG-mirror pipeline" **Additional notes:** - If the `review-deploy` job keeps failing (and a manual retry didn't help), - please post a message in the `#g_qe_engineering_productivity` channel and/or create a `~"Engineering Productivity"` `~"ep::review apps"` `~"type::bug"` + post a message in the `#g_qe_engineering_productivity` channel and/or create a `~"Engineering Productivity"` `~"ep::review apps"` `~"type::bug"` issue with a link to your merge request. The deployment failure can reveal an actual problem introduced in your merge request (that is, this isn't necessarily a transient failure)! - If the `review-qa-smoke` or `review-qa-reliable` job keeps failing (we already retry them once), - please check the job's logs: you could discover an actual problem introduced in + check the job's logs: you could discover an actual problem introduced in your merge request. You can also download the artifacts to see screenshots of the page at the time the failures occurred. If you don't find the cause of the - failure or if it seems unrelated to your change, please post a message in the - `#quality` channel and/or create a ~Quality ~"type::bug" issue with a link to your + failure or if it seems unrelated to your change, post a message in the + `#test-platform` channel and/or create a ~Quality ~"type::bug" issue with a link to your merge request. - The manual `review-stop` can be used to stop a review app manually, and is also started by GitLab once a merge diff --git a/doc/development/testing_guide/smoke.md b/doc/development/testing_guide/smoke.md index 62a09ca864b..0c7cb9219ad 100644 --- a/doc/development/testing_guide/smoke.md +++ b/doc/development/testing_guide/smoke.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Smoke Tests diff --git a/doc/development/testing_guide/test_results_tracking.md b/doc/development/testing_guide/test_results_tracking.md index 92c1e0917c7..4e40d47dab0 100644 --- a/doc/development/testing_guide/test_results_tracking.md +++ b/doc/development/testing_guide/test_results_tracking.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Test results tracking diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md index 4e4dc671c03..81399c7ce05 100644 --- a/doc/development/testing_guide/testing_levels.md +++ b/doc/development/testing_guide/testing_levels.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Testing levels diff --git a/doc/development/testing_guide/testing_migrations_guide.md b/doc/development/testing_guide/testing_migrations_guide.md index abe04274b72..ce6ba082f3b 100644 --- a/doc/development/testing_guide/testing_migrations_guide.md +++ b/doc/development/testing_guide/testing_migrations_guide.md @@ -1,8 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: reference +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Testing Rails migrations at GitLab @@ -84,8 +83,7 @@ end ``` In some cases, you must require multiple migration files to use them in your specs. Here, there's no -pattern between your spec file and the other migration file. You can provide the migration filename -like so: +pattern between your spec file and the other migration file. You can provide the migration file name like so: ```ruby # frozen_string_literal: true @@ -371,7 +369,7 @@ end ## Testing a non-`ActiveRecord::Migration` class To test a non-`ActiveRecord::Migration` test (a background migration), -you must manually provide a required schema version. Please add a +you must manually provide a required schema version. Add a `schema` tag to a context that you want to switch the database schema within. If not set, `schema` defaults to `:latest`. diff --git a/doc/development/testing_guide/testing_rake_tasks.md b/doc/development/testing_guide/testing_rake_tasks.md index 07332f8708b..38d852616b6 100644 --- a/doc/development/testing_guide/testing_rake_tasks.md +++ b/doc/development/testing_guide/testing_rake_tasks.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Testing Rake tasks diff --git a/doc/development/transient/prevention-patterns.md b/doc/development/transient/prevention-patterns.md index 730cc757396..de15d8eba95 100644 --- a/doc/development/transient/prevention-patterns.md +++ b/doc/development/transient/prevention-patterns.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Preventing Transient Bugs diff --git a/doc/development/uploads/index.md b/doc/development/uploads/index.md index 92b52d2b59c..13ef5c34d7e 100644 --- a/doc/development/uploads/index.md +++ b/doc/development/uploads/index.md @@ -1,7 +1,7 @@ --- stage: SaaS Platforms group: Scalability -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Uploads development guidelines diff --git a/doc/development/uploads/working_with_uploads.md b/doc/development/uploads/working_with_uploads.md index 6be6e472555..6fa9ed25257 100644 --- a/doc/development/uploads/working_with_uploads.md +++ b/doc/development/uploads/working_with_uploads.md @@ -1,7 +1,7 @@ --- stage: SaaS Platforms group: Scalability -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Uploads guide: Adding new uploads diff --git a/doc/development/utilities.md b/doc/development/utilities.md index 83b87d6d289..dbdf13feef8 100644 --- a/doc/development/utilities.md +++ b/doc/development/utilities.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # GitLab utilities @@ -284,3 +284,54 @@ end ## `ReactiveCaching` Read the documentation on [`ReactiveCaching`](reactive_caching.md). + +## `CircuitBreaker` + +The `Gitlab::CircuitBreaker` can be wrapped around any class that needs to run code with circuit breaker protection. It provides a `run_with_circuit` method that wraps a code block with circuit breaker functionality, which helps prevent cascading failures and improves system resilience. For more information about the circuit breaker pattern, see: + +- [What is Circuit breaker](https://martinfowler.com/bliki/CircuitBreaker.html). +- [The Hystrix documentation on CircuitBreaker](https://github.com/Netflix/Hystrix/wiki/How-it-Works#circuit-breaker). + +### Use CircuitBreaker + +To use the CircuitBreaker wrapper: + +```ruby +class MyService + def call_external_service + Gitlab::CircuitBreaker.run_with_circuit('ServiceName') do + # Code that interacts with external service goes here + + raise Gitlab::CircuitBreaker::InternalServerError # if there is an issue + end + end +end +``` + +The `call_external_service` method is an example method that interacts with an external service. +By wrapping the code that interacts with the external service with `run_with_circuit`, the method is executed within the circuit breaker. + +The method should raise an `InternalServerError` error, which will be counted towards the error threshold if raised during the execution of the code block. +The circuit breaker tracks the number of errors and the rate of requests, +and opens the circuit if it reaches the configured error threshold or volume threshold. +If the circuit is open, subsequent requests fail fast without executing the code block, and the circuit breaker periodically allows a small number of requests through to test the service's availability before closing the circuit again. + +### Configuration + +You need to specify a service name for each unique circuit that is used as the cache key. This should be a `CamelCase` string which identifies the circuit. + +The circuit breaker has defaults that can be overridden per circuit, for example: + +```ruby +Gitlab::CircuitBreaker.run_with_circuit('ServiceName', options = { volume_threshold: 5 }) do + ... +end +``` + +The defaults are: + +- `exceptions`: `[Gitlab::CircuitBreaker::InternalServerError]` +- `error_threshold`: `50` +- `volume_threshold`: `10` +- `sleep_window`: `90` +- `time_window`: `60` diff --git a/doc/development/ux/index.md b/doc/development/ux/index.md index 784a59a3a4a..ab6cd4c64f5 100644 --- a/doc/development/ux/index.md +++ b/doc/development/ux/index.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Contribute to UX design diff --git a/doc/development/value_stream_analytics.md b/doc/development/value_stream_analytics.md index 5aa6aecd9db..725c8aa45d2 100644 --- a/doc/development/value_stream_analytics.md +++ b/doc/development/value_stream_analytics.md @@ -1,7 +1,7 @@ --- stage: Plan group: Optimize -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Value stream analytics development guidelines diff --git a/doc/development/value_stream_analytics/value_stream_analytics_aggregated_backend.md b/doc/development/value_stream_analytics/value_stream_analytics_aggregated_backend.md index 4d2a751e6c6..53c6721b01f 100644 --- a/doc/development/value_stream_analytics/value_stream_analytics_aggregated_backend.md +++ b/doc/development/value_stream_analytics/value_stream_analytics_aggregated_backend.md @@ -1,7 +1,7 @@ --- stage: Plan group: Optimize -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Aggregated Value Stream Analytics diff --git a/doc/development/vs_code_debugging.md b/doc/development/vs_code_debugging.md index 66f69a74c86..457b3e5f792 100644 --- a/doc/development/vs_code_debugging.md +++ b/doc/development/vs_code_debugging.md @@ -1,7 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # VS Code debugging diff --git a/doc/development/wikis.md b/doc/development/wikis.md index eca43f6df03..8c9064bf7a9 100644 --- a/doc/development/wikis.md +++ b/doc/development/wikis.md @@ -1,7 +1,7 @@ --- stage: Plan group: Knowledge -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. description: "GitLab's development guidelines for Wikis" --- diff --git a/doc/development/windows.md b/doc/development/windows.md index 99085b4b5f8..a36e094cbcb 100644 --- a/doc/development/windows.md +++ b/doc/development/windows.md @@ -1,8 +1,7 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: reference, howto +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Windows Development diff --git a/doc/development/work_items.md b/doc/development/work_items.md index 73993b1d9ee..0c3bc4611f5 100644 --- a/doc/development/work_items.md +++ b/doc/development/work_items.md @@ -1,7 +1,7 @@ --- stage: Plan group: Project Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Work items and work item types @@ -190,7 +190,7 @@ and incidents) into work items. Eventually (when these resources become regular work items), `base_type` will be removed. Until the architecture of WIT widgets is finalized, we are holding off on the creation of new work item -types. If a new work item type is absolutely necessary, please reach out to a +types. If a new work item type is absolutely necessary, reach out to a member of the [Project Management Engineering Team](https://gitlab.com/gitlab-org/gitlab/-/issues/370599). ### Creating a new work item type in the database diff --git a/doc/development/work_items_widgets.md b/doc/development/work_items_widgets.md index 2e5a09af911..332dfde8a68 100644 --- a/doc/development/work_items_widgets.md +++ b/doc/development/work_items_widgets.md @@ -1,7 +1,7 @@ --- stage: Plan group: Project Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Work items widgets diff --git a/doc/development/workhorse/channel.md b/doc/development/workhorse/channel.md index 2c28cea42a3..667040e20c9 100644 --- a/doc/development/workhorse/channel.md +++ b/doc/development/workhorse/channel.md @@ -1,7 +1,7 @@ --- stage: Create group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Websocket channel support for Workhorse diff --git a/doc/development/workhorse/configuration.md b/doc/development/workhorse/configuration.md index d19e85f3f7a..becd65387e8 100644 --- a/doc/development/workhorse/configuration.md +++ b/doc/development/workhorse/configuration.md @@ -1,7 +1,7 @@ --- stage: Create group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Workhorse configuration diff --git a/doc/development/workhorse/gitlab_features.md b/doc/development/workhorse/gitlab_features.md index b4146e8b62c..73ee22929ab 100644 --- a/doc/development/workhorse/gitlab_features.md +++ b/doc/development/workhorse/gitlab_features.md @@ -1,7 +1,7 @@ --- stage: Create group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Features that rely on Workhorse diff --git a/doc/development/workhorse/index.md b/doc/development/workhorse/index.md index 680dd0c205b..8fe7f027cfd 100644 --- a/doc/development/workhorse/index.md +++ b/doc/development/workhorse/index.md @@ -1,7 +1,7 @@ --- stage: Create group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # GitLab Workhorse diff --git a/doc/development/workhorse/new_features.md b/doc/development/workhorse/new_features.md index dbde06ddee4..0f6c5b763ce 100644 --- a/doc/development/workhorse/new_features.md +++ b/doc/development/workhorse/new_features.md @@ -1,7 +1,7 @@ --- stage: Create group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review. --- # Adding new features to Workhorse |
