diff options
Diffstat (limited to 'doc/development/real_time.md')
| -rw-r--r-- | doc/development/real_time.md | 538 |
1 files changed, 519 insertions, 19 deletions
diff --git a/doc/development/real_time.md b/doc/development/real_time.md index 2aa48fed8eb..017e308fc03 100644 --- a/doc/development/real_time.md +++ b/doc/development/real_time.md @@ -1,29 +1,347 @@ --- -stage: Plan -group: Project Management +stage: Data Stores +group: Application Performance info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Real-Time Features +# Build and deploy real-time view components -This guide contains instructions on how to safely roll out new real-time -features. +GitLab provides an interactive user experience through individual view components that accept +user input and reflect state changes back to the user. For example, on the Merge Request +page, users can approve, leave comments, interact with the CI/CD pipeline, and more. -Real-time features are implemented using GraphQL Subscriptions. -[Developer documentation](api_graphql_styleguide.md#subscriptions) is available. +However, GitLab often does not reflect state updates in a timely manner. +This means parts of the page display stale data that only update after users reload the page. + +To address this, GitLab has introduced technology and programming APIs that allow view components +to receive state updates in real-time over a WebSocket. + +The following documentation tells you how to build and deploy view components that +receive updates in real-time from the GitLab Ruby on Rails server. + +NOTE: +Action Cable and GraphQL subscriptions are a work-in-progress and under active development. +Developers must evaluate their use case to check if these are the right tools to use. +If you are not sure, ask for help in the [`#f_real-time` internal Slack channel](https://gitlab.slack.com/archives/CUX9Z2N66). + +## Build real-time view components + +Prerequisites: + +Read the: + +- [GraphQL development guide](fe_guide/graphql.md). +- [Vue development guide](fe_guide/vue.md). + +To build a real-time view component on GitLab, you must: + +- Integrate a Vue component with Apollo subscriptions in the GitLab frontend. +- Add and trigger GraphQL subscriptions from the GitLab Ruby on Rails backend. + +### Integrate a Vue component with Apollo subscriptions + +NOTE: +Our current real-time stack assumes that client code is built using Vue as the rendering layer and +Apollo as the state and networking layer. If you are working with a part of +the GitLab frontend that has not been migrated to Vue + Apollo yet, complete that task first. + +Consider a hypothetical `IssueView` Vue component that observes and renders GitLab `Issue` data. +For simplicity, we assume here that all it does is render an issue's title and description: + +```javascript +import issueQuery from '~/issues/queries/issue_view.query.graqhql'; + +export default { + props: { + issueId: { + type: Number, + required: false, + default: null, + }, + }, + apollo: { + // Name of the Apollo query object. Must match the field name bound by `data`. + issue: { + // Query used for the initial fetch. + query: issueQuery, + // Bind arguments used for the initial fetch query. + variables() { + return { + iid: this.issueId, + }; + }, + // Map response data to view properties. + update(data) { + return data.project?.issue || {}; + }, + }, + }, + // Reactive Vue component data. Apollo updates these when queries return or subscriptions fire. + data() { + return { + issue: {}, // It is good practice to return initial state here while the view is loading. + }; + }, +}; + +// The <template> code is omitted for brevity as it is not relevant to this discussion. +``` + +The query should: + +- Be defined at `app/assets/javascripts/issues/queries/issue_view.query.graqhql`. +- Contain the following GraphQL operation: + + ```plaintext + query gitlabIssue($iid: String!) { + # We hard-code the path here only for illustration. Don't do this in practice. + project(fullPath: "gitlab-org/gitlab") { + issue(iid: $iid) { + title + description + } + } + } + ``` + +So far this view component only defines the initial fetch query to populate itself with data. +This is an ordinary GraphQL `query` operation sent as an HTTP POST request, initiated by the view. +Any subsequent updates on the server would make this view stale. For it to receive updates from the server, you must: + +1. Add a GraphQL subscription definition. +1. Define an Apollo subscription hook. + +#### Add a GraphQL subscription definition + +A subscription defines a GraphQL query as well, but it is wrapped inside a GraphQL `subscription` operation. +This query is initiated by the backend and its results pushed over a WebSocket into the view component. + +Similar to the initial fetch query, you must: + +- Define the subscription file at `app/assets/javascripts/issues/queries/issue_updated.subscription.graqhql`. +- Include the following GraphQL operation in the file: + + ```plaintext + subscription issueUpdatedSubscription($iid: String!) { + issueUpdated($issueId: IssueID!) { + issue(issueId: $issueId) { + title + description + } + } + } + ``` + +When adding new subscriptions, use the following naming guidelines: + +- End the subscription's operation name with `Subscription`, or `SubscriptionEE` if it's exclusive to GitLab EE. + For example, `issueUpdatedSubscription`, or `issueUpdatedSubscriptionEE`. +- Use a "has happened" action verb in the subscription's event name. For example, `issueUpdated`. + +While subscription definitions look similar to ordinary queries, there are some key differences that are important to understand: + +- The `query`: + - Originates from the frontend. + - Uses an internal ID (`iid`, numeric), which is how entities are usually referenced in URLs. + Because the internal ID is relative to the enclosing namespace (in this example, the `project`), you must nest the query under the `fullPath`. +- The `subscription`: + - Is a request from the frontend to the backend to receive future updates. + - Consists of: + - The operation name describing the subscription itself (`issueUpdatedSubscription` in this example). + - A nested event query (`issueUpdated` in this example). The nested event query: + - Executes when running the [GraphQL trigger](#trigger-graphql-subscriptions) of the same name, + so the event name used in the subscription must match the trigger field used in the backend. + - Uses a global ID string instead of a numeric internal ID, which is the preferred way to identify resources in GraphQL. + For more information, see [GraphQL global IDs](fe_guide/graphql.md#global-ids). + +#### Define an Apollo subscription hook + +After defining the subscription, add it to the view component using Apollo's `subscribeToMore` property: + +```javascript +import issueQuery from '~/issues/queries/issue_view.query.graqhql'; +import issueUpdatedSubscription from '~/issues/queries/issue_updated.subscription.graqhql'; + +export default { + // As before. + // ... + apollo: { + issue: { + // As before. + // ... + // This Apollo hook enables real-time pushes. + subscribeToMore: { + // Subscription operation that returns future updates. + document: issueUpdatedSubscription, + // Bind arguments used for the subscription operation. + variables() { + return { + iid: this.issueId, + }; + }, + // Implement this to return true|false if subscriptions should be disabled. + // Useful when using feature-flags. + skip() { + return this.shouldSkipRealTimeUpdates; + }, + }, + }, + }, + // As before. + // ... + computed: { + shouldSkipRealTimeUpdates() { + return false; // Might check a feature flag here. + }, + }, +}; +``` + +Now you can enable the view component to receive updates over a WebSocket connection through Apollo. +Next, we cover how events are triggered from the backend to initiate a push update to the frontend. + +### Trigger GraphQL subscriptions + +Writing a view component that can receive updates from a WebSocket is only half the story. +In the GitLab Rails application, we need to perform the following steps: + +1. Implement a `GraphQL::Schema::Subscription` class. This class: + - Is used by `graphql-ruby` to resolve the `subscription` operation sent by the frontend. + - Defines the arguments a subscription takes and the payload returned to the caller, if any. + - Runs any necessary business logic to ensure that the caller is authorized to create this subscription. +1. Add a new `field` to the `Types::SubscriptionType` class. This field maps the event name used + [when integrating the Vue component](#integrate-a-vue-component-with-apollo-subscriptions) to the + `GraphQL::Schema::Subscription` class. +1. Add a method matching the event name to `GraphqlTriggers` that runs the corresponding GraphQL trigger. +1. Use a service or Active Record model class to execute the new trigger as part of your domain logic. + +#### Implement the subscription + +If you subscribe to a an event that is already implemented as a `GraphQL::Schema::Subscription`, this step is optional. +Otherwise, create a new class under `app/graphql/subscriptions/` +that implements the new subscription. For the example of an `issueUpdated` event happening in response to an `Issue` being updated, +the subscription implementation is as follows: + +```ruby +module Subscriptions + class IssueUpdated < BaseSubscription + include Gitlab::Graphql::Laziness + + payload_type Types::IssueType + + argument :issue_id, Types::GlobalIDType[Issue], + required: true, + description: 'ID of the issue.' + + def authorized?(issue_id:) + issue = force(GitlabSchema.find_by_gid(issue_id)) + + unauthorized! unless issue && Ability.allowed?(current_user, :read_issue, issue) + + true + end + end +end +``` + +When creating this new class: + +- Make sure every subscription type inherits from `Subscriptions::BaseSubscription`. +- Use an appropriate `payload_type` to indicate what data subscribed queries may access, + or define the individual `field`s you want to expose. +- You may also define custom `subscribe` and `update` hooks that are called each time a client subscribes or + an event fires. Refer to the [official documentation](https://graphql-ruby.org/subscriptions/subscription_classes) + for how to use these methods. +- Implement `authorized?` to perform any necessary permission checks. These checks execute for each call + to `subscribe` or `update`. + +Read more about GraphQL subscription classes [in the official documentation](https://graphql-ruby.org/subscriptions/subscription_classes). + +#### Hook up the subscription + +Skip this step if you did not implement a new subscription class. + +After you implement a new subscription class, you must map that class to a `field` on the `SubscriptionType` before +it can execute. Open the `Types::SubscriptionType` class and add the new field: + +```ruby +module Types + class SubscriptionType < ::Types::BaseObject + graphql_name 'Subscription' + + # Existing fields + # ... + + field :issue_updated, + subscription: Subscriptions::IssueUpdated, null: true, + description: 'Triggered when an issue is updated.' + end +end +``` + +NOTE: +If you are connecting an EE subscription, update `EE::Types::SubscriptionType` instead. + +Make sure the `:issue_updated` argument matches the name used in the `subscription` request sent by the frontend in camel-case (`issueUpdated`), or `graphql-ruby` does not know which subscribers to inform. The event can now trigger. + +#### Add the new trigger + +Skip this step if you can reuse an existing trigger. + +We use a facade around `GitlabSchema.subscriptions.trigger` to make it simpler to trigger an event. +Add the new trigger to `GraphqlTriggers`: + +```ruby +module GraphqlTriggers + # Existing triggers + # ... + + def self.issue_updated(issue) + GitlabSchema.subscriptions.trigger(:issue_updated, { issue_id: issue.to_gid }, issue) + end +end +``` + +NOTE: +If the trigger is for an EE subscription, update `EE::GraphqlTriggers` instead. + +- The first argument, `:issue_updated`, must match the `field` name used in the previous + step. +- The argument hash specifies the issue for which we publish the event. + GraphQL uses this hash to identify the topic it should publish the event to. + +The final step is to call into this trigger function. + +#### Execute the trigger + +The implementation of this step depends on what exactly it is you are building. In the example +of the issue's fields changing, we could extend `Issues::UpdateService` to call `GraphqlTriggers.issue_updated`. + +The real-time view component is now functional. Updates to an issue should now propagate immediately into the GitLab UI. + +## Deploy real-time view components WebSockets are a relatively new technology at GitLab, and supporting them at scale introduces some challenges. For that reason, new features should be rolled out using the instructions below. -## Reuse an existing WebSocket connection +### Shipping a real-time component + +You can work on the frontend and backend at the same time, because updates over WebSockets +are difficult to simulate without the necessary backend code in place. + +However, it is safer to send changes in separate Merge Requests and deploy the backend changes first. +This ensures that when the frontend starts subscribing to events, the backend is already prepared +to service them. + +### Reuse an existing WebSocket connection Features reusing an existing connection incur minimal risk. Feature flag rollout is recommended to give more control to self-hosting customers. However, it is not necessary to roll out in percentages, or to estimate new connections for GitLab.com. -## Introduce a new WebSocket connection +### Introduce a new WebSocket connection Any change that introduces a WebSocket connection to part of the GitLab application incurs some scalability risk, both to nodes responsible for maintaining open @@ -70,7 +388,7 @@ of the feature flag ensures that effects can be observed on the 1. Copy in a member of the Plan and Scalability teams to estimate a percentage-based roll-out plan. -## Backward compatibility +### Backward compatibility For the duration of the feature flag roll-out and indefinitely thereafter, real-time features must be backward-compatible, or at least degrade @@ -80,18 +398,200 @@ needs to be done before Action Cable can be enabled by default. Making real-time a requirement represents a breaking change, so the next opportunity to do this is version 15.0. -## Enable Real-Time by default - -Mounting the Action Cable library adds minimal memory footprint. However, -serving WebSocket requests introduces additional memory requirements. For this -reason, enabling Action Cable by default requires additional work; perhaps -to reduce overall memory usage, including a known issue with Workhorse, but at -least to revise Reference Architectures. - -## Real-time infrastructure on GitLab.com +### Real-time infrastructure on GitLab.com On GitLab.com, WebSocket connections are served from dedicated infrastructure, entirely separate from the regular Web fleet and deployed with Kubernetes. This limits risk to nodes handling requests but not to shared services. For more information on the WebSockets Kubernetes deployment see [this epic](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/355). + +## The GitLab real-time stack in depth + +Because a push initiated by the server needs to propagate over the network and trigger a view update +in the client without any user interaction whatsoever, real-time features can only be understood +by looking at the entire stack including frontend and backend. + +NOTE: +For historic reasons, the controller routes that service updates in response to clients polling +for changes are called `realtime_changes`. They use conditional GET requests and are unrelated +to the real-time behavior covered in this guide. + +Any real-time update pushed into a client originates from the GitLab Rails application. We use the following +technologies to initiate and service these updates: + +In the GitLab Rails backend: + +- Redis PubSub to handle subscription state. +- Action Cable to handle WebSocket connections and data transport. +- `graphql-ruby` to implement GraphQL subscriptions and triggers. + +In the GitLab frontend: + +- Apollo Client to handle GraphQL requests, routing and caching. +- Vue.js to define and render view components that update in real-time. + +The following figure illustrates how data propagates between these layers. + +```mermaid +sequenceDiagram + participant V as Vue Component + participant AP as Apollo Client + participant P as Rails/GraphQL + participant AC as Action Cable/GraphQL + participant R as Redis PubSub + AP-->>V: injected + AP->>P: HTTP GET /-/cable + AC-->>P: Hijack TCP connection + AC->>+R: SUBSCRIBE(client) + R-->>-AC: channel subscription + AC-->>AP: HTTP 101: Switching Protocols + par + V->>AP: query(gql) + Note over AP,P: Fetch initial data for this view + AP->>+P: HTTP POST /api/graphql (initial query) + P-->>-AP: initial query response + AP->>AP: cache and/or transform response + AP->>V: trigger update + V->>V: re-render + and + Note over AP,AC: Subscribe to future updates for this view + V->>AP: subscribeToMore(event, gql) + AP->>+AC: WS: subscribe(event, query) + AC->>+R: SUBSCRIBE(event) + R-->>-AC: event subscription + AC-->>-AP: confirm_subscription + end + Note over V,R: time passes + P->>+AC: trigger event + AC->>+R: PUBLISH(event) + R-->>-AC: subscriptions + loop For each subscriber + AC->>AC: run GQL query + AC->>+R: PUBLISH(client, query_result) + R-->>-AC: callback + AC->>-AP: WS: push query result + end + AP->>AP: cache and/or transform response + AP->>V: trigger update + V->>V: re-render +``` + +In the subsequent sections we explain each element of this stack in detail. + +### Action Cable and WebSockets + +[Action Cable](https://guides.rubyonrails.org/action_cable_overview.html) is a library that adds +[WebSocket](https://www.rfc-editor.org/rfc/rfc6455) support to Ruby on Rails. +WebSockets were developed as an HTTP-friendly solution to enhance existing HTTP-based servers and +applications with bidirectional communication over a single TCP connection. +A client first sends an ordinary HTTP request to the server, asking it to upgrade the connection +to a WebSocket instead. When successful, the same TCP connection can then be used by both client +and server to send and receive data in either direction. + +Because the WebSocket protocol does not prescribe how the transmitted data is encoded or structured, +we need libraries like Action Cable that take care of these concerns. Action Cable: + +- Handles the initial connection upgrade from HTTP to the WebSocket protocol. Subsequent requests using + the `ws://` scheme are then handled by the Action Cable server and not Action Pack. +- Defines how data transmitted over the WebSocket is encoded. Action Cable specifies this to be JSON. This allows the + application to provide data as a Ruby Hash and Action Cable (de)serializes it from and to JSON. +- Provides callback hooks to handle clients connecting or disconnecting and client authentication. +- Provides `ActionCable::Channel` as a developer abstraction to implement publish/subscribe and remote procedure calls. + +Action Cable supports different implementations to track which client is subscribed to which +`ActionCable::Channel`. At GitLab we use the Redis adapter, which uses +[Redis PubSub](https://redis.io/docs/manual/pubsub/) channels as a distributed message bus. +Shared storage is necessary because different clients might connect to the same Action Cable channel +from different Puma instances. + +NOTE: +Do not confuse Action Cable channels with Redis PubSub channels. An Action Cable `Channel` object is a +programming abstraction to classify and handle the various kinds of data going over the WebSocket connection. +In Action Cable, the underlying PubSub channel is referred to as a broadcasting instead and the association +between a client and a broadcasting is called a subscription. In particular, there can be many broadcastings +(PubSub channels) and subscriptions for each Action Cable `Channel`. + +Because Action Cable allows us to express different kinds of behavior through its `Channel` API, and because +updates to any `Channel` can use the same WebSocket connection, we only require a single WebSocket connection +to be established for each GitLab page to enhance a view component on that page with real-time behavior. + +To implement real-time updates on a GitLab page, we do not write individual `Channel` implementations. +Instead, we provide the `GraphqlChannel` to which all pages that require push-based updates on GitLab +subscribe. + +### GraphQL subscriptions: Backend + +GitLab supports [GraphQL](https://graphql.org) for clients to request structured data from the server +using GraphQL queries. Refer to the [GitLab GraphQL overview](../api/graphql/index.md) to learn about why we adopted GraphQL. +GraphQL support in the GitLab backend is provided by the [`graphql-ruby`](https://graphql-ruby.org) gem. + +Ordinarily, GraphQL queries are client-initiated HTTP POST requests that follow the standard request-response cycle. +For real-time functionality, we use GraphQL subscriptions instead, which are an implementation of the publish/subscribe pattern. +In this approach the client first sends a subscription request to the `GraphqlChannel` with the: + +- Name of the subscription `field` (the event name). +- GraphQL query to run when this event triggers. + +This information is used by the server to create a `topic` that represents this event stream. The topic is a unique name +derived from the subscription arguments and event name and is used to identify all subscribers +that need to be informed if the event triggers. More than one client can subscribe to the +same topic. For example, `issuableAssigneesUpdated:issuableId:<hashed_id>` might serve as the topic +that clients subscribe to if they wish to be updated whenever the assignees for the issue with the +given ID change. + +The backend is responsible for triggering a subscription, typically in response to a domain +event such as "issue added to epic" or "user assigned to issue". At GitLab, this could be a service object +or an ActiveRecord model object. +A trigger is executed by calling into [`GitlabSchema.subscriptions.trigger`](https://gitlab.com/gitlab-org/gitlab/-/blob/5e3c334116178eec5f50fc5fee2ec0b3841a2504/app/graphql/graphql_triggers.rb) with the respective event name and arguments, +from which `graphql-ruby` derives the topic. It then finds all subscribers for this topic, executes the query for +each subscriber, and pushes the result back to all topic subscribers. + +Because we use Action Cable as the underlying transport for GraphQL subscriptions, topics are implemented +as Action Cable broadcastings, which as mentioned above represent Redis PubSub channels. +This means that for each subscriber, two PubSub channels are used: + +- One `graphql-event:<namespace>:<topic>` channel per each topic. This channel is used to track which client is subscribed + to which event and is shared among all potential clients. The use of a `namespace` is optional and it can be blank. +- One `graphql-subscription:<subscription-id>` channel per each client. This channel is used to transmit the query result + back to the respective client and hence cannot be shared between different clients. + +The next section describes how the GitLab frontend uses GraphQL subscriptions to implement real-time updates. + +### GraphQL subscriptions: Frontend + +Because the GitLab frontend executes JavaScript, not Ruby, we need a different GraphQL implementation +to send GraphQL queries, mutations, and subscriptions from the client to the server. +We use [Apollo](https://www.apollographql.com) to do this. + +Apollo is a comprehensive implementation of GraphQL in JavaScript and is split into `apollo-server` and `apollo-client` +as well as additional utility modules. Because we run a Ruby backend, we use `apollo-client` instead of `apollo-server`. + +It simplifies: + +- Networking, connection management and request routing. +- Client-side state management and response caching. +- Integrating GraphQL with view components using a bridge module. + +NOTE: +When reading the Apollo Client documentation, it assumes that React.js is used for view rendering. We do not use React.js +at GitLab. We use Vue.js, which integrates with Apollo using the [Vue.js adapter](https://apollo.vuejs.org/). + +Apollo provides functions and hooks with which you define how: + +- Views send queries, mutations or subscriptions. +- Responses should be dealt with. +- Response data is cached. + +The entry point is `ApolloClient`, which is a GraphQL client object that: + +- Is shared between all view components on a single page. +- All view components use internally to communicate with the server. + +To decide how different types of requests should be routed, Apollo uses the `ApolloLink` abstraction. Specifically, +it splits real-time server subscriptions from other GraphQL requests using the `ActionCableLink`. This: + +- Establishes the WebSocket connection to Action Cable. +- Maps server pushes to an `Observable` event stream in the client that views can subscribe to in order to update themselves. + +For more information about Apollo and Vue.js, see the [GitLab GraphQL development guide](fe_guide/graphql.md). |