diff options
Diffstat (limited to 'doc/development/integrations/index.md')
| -rw-r--r-- | doc/development/integrations/index.md | 332 |
1 files changed, 332 insertions, 0 deletions
diff --git a/doc/development/integrations/index.md b/doc/development/integrations/index.md new file mode 100644 index 00000000000..34ac307c98a --- /dev/null +++ b/doc/development/integrations/index.md @@ -0,0 +1,332 @@ +--- +stage: Ecosystem +group: Integrations +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +description: "GitLab's development guidelines for Integrations" +--- + +# Integrations development guide **(FREE)** + +This page provides development guidelines for implementing [GitLab integrations](../../user/project/integrations/index.md), +which are part of our [main Rails project](https://gitlab.com/gitlab-org/gitlab). + +Also see our [direction page](https://about.gitlab.com/direction/ecosystem/integrations/) for an overview of our strategy around integrations. + +This guide is a work in progress. You're welcome to ping `@gitlab-org/ecosystem-stage/integrations` +if you need clarification or spot any outdated information. + +## Add a new integration + +### Define the integration + +1. Add a new model in `app/models/integrations` extending from `Integration`. + - For example, `Integrations::FooBar` in `app/models/integrations/foo_bar.rb`. + - For certain types of integrations, you can also build on these base classes: + - `Integrations::BaseChatNotification` + - `Integrations::BaseIssueTracker` + - `Integrations::BaseMonitoring` + - `Integrations::BaseSlashCommands` + - For integrations that primarily trigger HTTP calls to external services, you can + also use the `Integrations::HasWebHook` concern. This reuses the [webhook functionality](../../user/project/integrations/webhooks.md) + in GitLab through an associated `ServiceHook` model, and automatically records request logs + which can be viewed in the integration settings. +1. Add the integration's underscored name (`'foo_bar'`) to `Integration::INTEGRATION_NAMES`. +1. Add the integration as an association on `Project`: + + ```ruby + has_one :foo_bar_integration, class_name: 'Integrations::FooBar' + ``` + +1. TEMPORARY: Accommodate the current migration to [rename "services" to "integrations"](#rename-services-to-integrations): + - Add the integration's camel-cased name (`'FooBar'`) to `Gitlab::Integrations::StiType::NAMESPACED_INTEGRATIONS`. + +### Define properties + +Integrations can define arbitrary properties to store their configuration with the class method `Integration.prop_accessor`. +The values are stored as a serialized JSON hash in the `integrations.properties` column. + +For example: + +```ruby +module Integrations + class FooBar < Integration + prop_accessor :url + prop_accessor :tags + end +end +``` + +`Integration.prop_accessor` installs accessor methods on the class. Here we would have `#url`, `#url=` and `#url_changed?`, to manage the `url` field. Fields stored in `Integration#properties` should be accessed by these accessors directly on the model, just like other ActiveRecord attributes. + +You should always access the properties through their getters, and not interact with the `properties` hash directly. +You **must not** write to the `properties` hash, you **must** use the generated setter method instead. Direct writes to this +hash are not persisted. + +You should also define validations for all your properties. + +Also refer to the section [Customize the frontend form](#customize-the-frontend-form) below to see how these properties +are exposed in the frontend form for the integration. + +There is an alternative approach using `Integration.data_field`, which you may see in other integrations. +With data fields the values are stored in a separate table per integration. At the moment we don't recommend using this for new integrations. + +### Define trigger events + +Integrations are triggered by calling their `#execute` method in response to events in GitLab, +which gets passed a payload hash with details about the event. + +The supported events have some overlap with [webhook events](../../user/project/integrations/webhook_events.md), +and receive the same payload. You can specify the events you're interested in by overriding +the class method `Integration.supported_events` in your model. + +The following events are supported for integrations: + +| Event type | Default | Value | Trigger +|:-----------------------------------------------------------------------------------------------|:--------|:---------------------|:-- +| Alert event | | `alert` | A a new, unique alert is recorded. +| Commit event | ✓ | `commit` | A commit is created or updated. +| [Deployment event](../../user/project/integrations/webhook_events.md#deployment-events) | | `deployment` | A deployment starts or finishes. +| [Issue event](../../user/project/integrations/webhook_events.md#issue-events) | ✓ | `issue` | An issue is created, updated, or closed. +| [Confidential issue event](../../user/project/integrations/webhook_events.md#issue-events) | ✓ | `confidential_issue` | A confidential issue is created, updated, or closed. +| [Job event](../../user/project/integrations/webhook_events.md#job-events) | | `job` +| [Merge request event](../../user/project/integrations/webhook_events.md#merge-request-events) | ✓ | `merge_request` | A merge request is created, updated, or merged. +| [Comment event](../../user/project/integrations/webhook_events.md#comment-events) | | `comment` | A new comment is added. +| [Confidential comment event](../../user/project/integrations/webhook_events.md#comment-events) | | `confidential_note` | A new comment on a confidential issue is added. +| [Pipeline event](../../user/project/integrations/webhook_events.md#pipeline-events) | | `pipeline` | A pipeline status changes. +| [Push event](../../user/project/integrations/webhook_events.md#push-events) | ✓ | `push` | A push is made to the repository. +| [Tag push event](../../user/project/integrations/webhook_events.md#tag-events) | ✓ | `tag_push` | New tags are pushed to the repository. +| Vulnerability event **(ULTIMATE)** | | `vulnerability` | A new, unique vulnerability is recorded. +| [Wiki page event](../../user/project/integrations/webhook_events.md#wiki-page-events) | ✓ | `wiki_page` | A wiki page is created or updated. + +#### Event examples + +This example defines an integration that responds to `commit` and `merge_request` events: + +```ruby +module Integrations + class FooBar < Integration + def self.supported_events + %w[commit merge_request] + end + end +end +``` + +An integration can also not respond to events, and implement custom functionality some other way: + +```ruby +module Integrations + class FooBar < Integration + def self.supported_events + [] + end + end +end +``` + +### Customize the frontend form + +The frontend form is generated dynamically based on metadata defined in the model. + +By default, the integration form provides: + +- A checkbox to enable or disable the integration. +- Checkboxes for each of the trigger events returned from `Integration#configurable_events`. + +You can also add help text at the top of the form by either overriding `Integration#help`, +or providing a template in `app/views/projects/services/$INTEGRATION_NAME/_help.html.haml`. + +To add your custom properties to the form, you can define the metadata for them in `Integration#fields`. + +This method should return an array of hashes for each field, where the keys can be: + +| Key | Type | Required | Default | Description +|:---------------|:--------|:---------|:-----------------------------|:-- +| `type:` | string | true | | The type of the form field. Can be `text`, `textarea`, `password`, `checkbox`, or `select`. +| `name:` | string | true | | The property name for the form field. This must match a `prop_accessor` [defined on the class](#define-properties). +| `required:` | boolean | false | `false` | Specify if the form field is required or optional. +| `title:` | string | false | Capitalized value of `name:` | The label for the form field. +| `placeholder:` | string | false | | A placeholder for the form field. +| `help:` | string | false | | A help text that displays below the form field. + +#### Additional keys for `type: 'checkbox'` + +| Key | Type | Required | Default | Description +|:------------------|:-------|:---------|:------------------|:-- +| `checkbox_label:` | string | false | Value of `title:` | A custom label that displays next to the checkbox. + +#### Additional keys for `type: 'select'` + +| Key | Type | Required | Default | Description +|:-----------|:------|:---------|:--------|:-- +| `choices:` | array | true | | A nested array of `[label, value]` tuples. + +#### Additional keys for `type: 'password'` + +| Key | Type | Required | Default | Description +|:----------------------------|:-------|:---------|:------------------|:-- +| `non_empty_password_title:` | string | false | Value of `title:` | An alternative label that displays when a value is already stored. +| `non_empty_password_help:` | string | false | Value of `help:` | An alternative help text that displays when a value is already stored. + +#### Frontend form examples + +This example defines a required `url` field, and optional `username` and `password` fields: + +```ruby +module Integrations + class FooBar < Integration + prop_accessor :url, :username, :password + + def fields + [ + { + type: 'text', + name: 'url', + title: s_('FooBarIntegration|Server URL'), + placeholder: 'https://example.com/', + required: true + }, + { + type: 'text', + name: 'username', + title: s_('FooBarIntegration|Username'), + }, + { + type: 'password', + name: 'password', + title: s_('FoobarIntegration|Password' + non_empty_password_title: s_('FooBarIntegration|Enter new password') + } + ] + end + end +end +``` + +### Expose the integration in the API + +#### REST API + +To expose the integration in the [REST API](../../api/integrations.md): + +1. Add the integration's class (`::Integrations::FooBar`) to `API::Helpers::IntegrationsHelpers.integration_classes`. +1. Add all properties that should be exposed to `API::Helpers::IntegrationsHelpers.integrations`. +1. Update the reference documentation in `doc/api/integrations.md`, add a new section for your integration, and document all properties. + +You can also refer to our [REST API style guide](../api_styleguide.md). + +#### GraphQL API + +Integrations use the `Types::Projects::ServiceType` type by default, +which only exposes the `type` and `active` properties. + +To expose additional properties, you can write a class implementing `ServiceType`: + +```ruby +# in app/graphql/types/project/services/foo_bar_service_type.rb +module Types + module Projects + module Services + class FooBarServiceType < BaseObject + graphql_name 'FooBarService' + implements(Types::Projects::ServiceType) + authorize :read_project + + field :frobinity, + GraphQL::Types::Float, + null: true, + description: 'The level of frobinity.' + + field :foo_label, + GraphQL::Types::String, + null: true, + description: 'The foo label to apply.' + end + end + end +end +``` + +Each property you want to expose should have a field defined for it. You can also expose any public instance method of the integration. + +Contact a member of the Integrations team to discuss the best authorization. + +Reference documentation for GraphQL is automatically generated. + +You can also refer to our [GraphQL API style guide](../api_graphql_styleguide.md). + +## Availability of integrations + +By default, integrations are available on the project, group, and instance level. +Most integrations only act in a project context, but can be still configured +from the group and instance levels. + +For some integrations it can make sense to only make it available on the project level. +To do that, the integration must be removed from `Integration::INTEGRATION_NAMES` and +added to `Integration::PROJECT_SPECIFIC_INTEGRATION_NAMES` instead. + +When developing a new integration, we also recommend you gate the availability behind a +[feature flag](../feature_flags/index.md) in `Integration.available_integration_names`. + +## Documentation + +You can provide help text in the integration form, including links to off-site documentation, +as described above in [Customize the frontend form](#customize-the-frontend-form). Refer to +our [usability guidelines](https://design.gitlab.com/usability/helping-users) for help text. + +For more detailed documentation, provide a page in `doc/user/project/integrations`, +and link it from the [Integrations overview](../../user/project/integrations/overview.md). + +You can also refer to our general [documentation guidelines](../documentation/index.md). + +## Testing + +It is often sufficient to add tests for the integration model in `spec/models/integrations`, +and a factory with example settings in `spec/factories/integrations.rb`. + +Each integration is also tested as part of generalized tests. For example, there are feature specs +that verify that the settings form is rendering correctly for all integrations. + +If your integration implements any custom behavior, especially in the frontend, this should be +covered by additional tests. + +You can also refer to our general [testing guidelines](../testing_guide/index.md). + +## Internationalization + +All UI strings should be prepared for translation by following our [internationalization guidelines](../i18n/externalization.md). + +The strings should use the integration name as [namespace](../i18n/externalization.md#namespaces), for example, `s_('FooBarIntegration|My string')`. + +## Ongoing migrations and refactorings + +The Integrations team is in the process of some larger migrations that developers should be aware of. + +### [Rename "services" to "integrations"](https://gitlab.com/groups/gitlab-org/-/epics/2504) + +The "integrations" in GitLab were historically called "services", which frequently caused +confusion with our "service" classes in `app/services`. We sometimes also called +them "project services" because they were initially only available on projects, which is +not the case anymore. + +We decided to change the naming from "services" and "project services" to "integrations". +This refactoring is an ongoing effort, and there are still references to the old names in some places. + +Developers should be especially aware that we still use the old class names for the STI column +`integrations.type`. For example, a class `Integrations::FooBar` still stores +the old name `FooBarService` in the database. This mapping is handled via `Gitlab::Integrations::StiType` +and should be mostly transparent to the rest of the app. + +### [Consolidate integration settings](https://gitlab.com/groups/gitlab-org/-/epics/3955) + +We want to unify the way integration properties are defined. + +## Integration examples + +You can refer to these issues for examples of adding new integrations: + +- [Datadog](https://gitlab.com/gitlab-org/gitlab/-/issues/270123): Metrics collector, similar to the Prometheus integration. +- [EWM/RTC](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/36662): External issue tracker. +- [Shimo](https://gitlab.com/gitlab-org/gitlab/-/issues/343386): External wiki, similar to the Confluence and External Wiki integrations. +- [Webex Teams](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31543): Chat notifications. +- [ZenTao](https://gitlab.com/gitlab-org/gitlab/-/issues/338178): External issue tracker with custom issue views, similar to the Jira integration. |