diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2020-03-18 06:09:43 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2020-03-18 06:09:43 +0300 |
commit | b4b9b3854eddd2a4829113ebfc1812c3a332a7d9 (patch) | |
tree | 6a21e491917e1606d81329af710459b0217eb1a4 /doc/development | |
parent | 2e31c85a97183814ffa7ba5cc58f7bbad668fb2b (diff) |
Add latest changes from gitlab-org/gitlab@master
Diffstat (limited to 'doc/development')
-rw-r--r-- | doc/development/contributing/issue_workflow.md | 2 | ||||
-rw-r--r-- | doc/development/insert_into_tables_in_batches.md | 50 |
2 files changed, 46 insertions, 6 deletions
diff --git a/doc/development/contributing/issue_workflow.md b/doc/development/contributing/issue_workflow.md index 46d1d4c2414..94cf1c223dd 100644 --- a/doc/development/contributing/issue_workflow.md +++ b/doc/development/contributing/issue_workflow.md @@ -81,7 +81,7 @@ already reserved for category labels). The descriptions on the [labels page](https://gitlab.com/groups/gitlab-org/-/labels) explain what falls under each type label. -The GitLab handbook documents [when something is a bug and when it is a feature request.](https://about.gitlab.com/handbook/product/product-management/process/feature-or-bug.html) +The GitLab handbook documents [when something is a bug and when it is a feature request](https://about.gitlab.com/handbook/product/product-management/process/feature-or-bug.html). ### Facet labels diff --git a/doc/development/insert_into_tables_in_batches.md b/doc/development/insert_into_tables_in_batches.md index 763185013c9..de62d2cca52 100644 --- a/doc/development/insert_into_tables_in_batches.md +++ b/doc/development/insert_into_tables_in_batches.md @@ -32,12 +32,12 @@ The `BulkInsertSafe` concern has two functions: - It performs checks against your model class to ensure that it does not use ActiveRecord APIs that are not safe to use with respect to bulk insertions (more on that below). -- It adds a new class method `bulk_insert!`, which you can use to insert many records at once. +- It adds new class methods `bulk_insert!` and `bulk_upsert!`, which you can use to insert many records at once. -## Insert records via `bulk_insert!` +## Insert records with `bulk_insert!` and `bulk_upsert!` -If the target class passes the checks performed by `BulkInsertSafe`, you can proceed to use -the `bulk_insert!` class method as follows: +If the target class passes the checks performed by `BulkInsertSafe`, you can insert an array of +ActiveRecord model objects as follows: ```ruby records = [MyModel.new, ...] @@ -45,6 +45,28 @@ records = [MyModel.new, ...] MyModel.bulk_insert!(records) ``` +Note that calls to `bulk_insert!` will always attempt to insert _new records_. If instead +you would like to replace existing records with new values, while still inserting those +that do not already exist, then you can use `bulk_upsert!`: + +```ruby +records = [MyModel.new, existing_model, ...] + +MyModel.bulk_upsert!(records, unique_by: [:name]) +``` + +In this example, `unique_by` specifies the columns by which records are considered to be +unique and as such will be updated if they existed prior to insertion. For example, if +`existing_model` has a `name` attribute, and if a record with the same `name` value already +exists, its fields will be updated with those of `existing_model`. + +The `unique_by` parameter can also be passed as a `Symbol`, in which case it specifies +a database index by which a column is considered unique: + +```ruby +MyModel.bulk_insert!(records, unique_by: :index_on_name) +``` + ### Record validation The `bulk_insert!` method guarantees that `records` will be inserted transactionally, and @@ -74,6 +96,23 @@ Since this will also affect the number of `INSERT`s that occur, make sure you me performance impact this might have on your code. There is a trade-off between the number of `INSERT` statements the database has to process and the size and cost of each `INSERT`. +### Handling duplicate records + +NOTE: **Note:** +This parameter applies only to `bulk_insert!`. If you intend to update existing +records, use `bulk_upsert!` instead. + +It may happen that some records you are trying to insert already exist, which would result in +primary key conflicts. There are two ways to address this problem: failing fast by raising an +error or skipping duplicate records. The default behavior of `bulk_insert!` is to fail fast +and raise an `ActiveRecord::RecordNotUnique` error. + +If this is undesirable, you can instead skip duplicate records with the `skip_duplicates` flag: + +```ruby +MyModel.bulk_insert!(records, skip_duplicates: true) +``` + ### Requirements for safe bulk insertions Large parts of ActiveRecord's persistence API are built around the notion of callbacks. Many @@ -145,11 +184,12 @@ simply be treated as if you had invoked `save` from outside the block. There are a few restrictions to how these APIs can be used: -- Bulk inserts only work for new records; `UPDATE`s or "upserts" are not supported yet. - `ON CONFLICT` behavior cannot currently be configured; an error will be raised on primary key conflicts. - `BulkInsertableAssociations` furthermore has the following restrictions: - only compatible with `has_many` relations. - does not support `has_many through: ...` relations. +- Writing [`jsonb`](https://www.postgresql.org/docs/current/datatype-json.html) content is +[not currently supported](https://gitlab.com/gitlab-org/gitlab/-/issues/210560). Moreover, input data should either be limited to around 1000 records at most, or already batched prior to calling bulk insert. The `INSERT` statement will run in a single |