| Age | Commit message (Collapse) | Author |
|---|
|
The numbers in
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/22650#note_117645395
are quite promising with that target.
|
|
This improves readability quite a bit.
|
|
|
|
|
|
|
|
|
|
A tablesample count executes in two phases:
* Estimate table sizes based on reltuples.
* Based on the estimate:
* If the table is considered 'small', execute an exact relation count.
* Otherwise, count on a sample of the table using TABLESAMPLE.
The size of the sample is chosen in a way that we always roughly scan
the same amount of rows (see TABLESAMPLE_ROW_TARGET).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Enables frozens string for the following:
* lib/gitlab/conflict/**/*.rb
* lib/gitlab/cross_project_access/**/*.rb
* lib/gitlab/cycle_analytics/**/*.rb
* lib/gitlab/data_builder/**/*.rb
* lib/gitlab/database/**/*.rb
* lib/gitlab/dependency_linker/**/*.rb
* lib/gitlab/diff/**/*.rb
* lib/gitlab/downtime_check/**/*.rb
* lib/gitlab/email/**/*.rb
* lib/gitlab/etag_caching/**/*.rb
Partially addresses gitlab-org/gitlab-ce#47424.
|
|
|
|
Add dbname to MySQL migration GRANT command
See merge request gitlab-org/gitlab-ce!21719
|
|
Add mysql_compatible_index_length to migration helpers
Closes #50253
See merge request gitlab-org/gitlab-ce!21852
|
|
The original code caused Rails to generate invalid SQL. The problem
lays in the `.arel` method in `ActiveRecord::Relation`. When there was
a `limit` on the relation, the `LIMIT` statement was taken over to
Arel, but the value wasn't.
```ruby
relation = Event.limit(2)
relation.to_sql
#=> "SELECT `events`.* FROM `events` LIMIT 2"
relation.arel.to_sql
#=> "SELECT `events`.* FROM `events` LIMIT ?"
```
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/51729
|
|
Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>
|
|
`delete_all` doesn't support limit, so you'd need to subquery
that. And instead of subquerying with `where(id: query)`, it's better
to use an `INNER JOIN`. This method also works with MySQL, while
subquerying doesn't (without another layer of subquerying)
Reference:
https://stackoverflow.com/questions/17892762/mysql-this-version-of-mysql-doesnt-yet-support-limit-in-all-any-some-subqu/17892886#17892886
|
|
This commit adds the module `FromUnion`, which provides the class method
`from_union`. This simplifies the process of selecting data from the
result of a UNION, and reduces the likelihood of making mistakes. As a
result, instead of this:
union = Gitlab::SQL::Union.new([foo, bar])
Foo.from("(#{union.to_sql}) #{Foo.table_name}")
We can now write this instead:
Foo.from_union([foo, bar])
This commit also includes some changes to make this new setup work
properly. For example, a bug in Rails 4
(https://github.com/rails/rails/issues/24193) would break the use of
`from("sub-query-here").includes(:relation)` in certain cases. There was
also a CI query which appeared to repeat a lot of conditions from an
outer query on an inner query, which isn't necessary.
Finally, we include a RuboCop cop to ensure developers use this new
module, instead of using Gitlab::SQL::Union directly.
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/51307
|
|
|
|
it will decide the method for disable statement_timeout upon
per transaction or per session, based on how it's called.
When calling with a block, block will be executed and it will use
session based statement_timeout, otherwise will default to existing
behavior.
|
|
By default statement_timeout will only be enabled during transaction
lifetime, therefore not leaking outside of it.
With `transaction: false` it will set for entire session, but requires
a block to passed. It yields control and cleans up session after block
finishes, also preventing leaking outside of it.
|
|
This changes the BackgroundMigration worker so it checks for the health
of the DB before performing a background migration. This in turn allows
us to reduce the minimum interval, without having to worry about blowing
things up if we schedule too many migrations.
In this setup, the BackgroundMigration worker will reschedule jobs as
long as the database is considered to be in an unhealthy state. Once the
database has recovered, the migration can be performed.
To determine if the database is in a healthy state, we look at the
replication lag of any replication slots defined on the primary. If the
lag is deemed to great (100 MB by default) for too many slots, the
migration is rescheduled for a later point in time.
The health checking code is hidden behind a feature flag, allowing us to
disable it if necessary.
|
|
Add a helper to rename a column using a background migration
Closes #47591
See merge request gitlab-org/gitlab-ce!20180
|
|
This works the same way as change_column_type_using_background_migration, but
for renaming a column. It takes a table, not a relation, to match its concurrent
counterpart.
Also, generalise the cleanup migrations to reduce code duplication.
|
|
|
|
Gitlab::Database::RenameReservedPathsMigration::V1::MigrationClasses::Namespace
|
|
|
|
This commit does a number of things:
1. Reduces the number of queries needed by perform a single query to get all
the tuples for the relevant rows.
2. Uses a transaction to query the tuple counts to ensure that the data
is retrieved from the primary.
Closes #46742
|
|
Uses PostgreSQL tuple estimates to provide a much faster yet approximate
count. See https://wiki.postgresql.org/wiki/Slow_Counting for more details.
We only use this fast method if the table has been analyzed or vacuumed
within the last hour.
Closes #46255
|
|
In Arel 7.0.0 (Arel 7.1.4 is used in Rails 5.0) the `engine` parameter
of `Arel::UpdateManager#initializer` was removed.
This commit makes the gitlab database helpers work both in rails 4 and
rails 5.
Fixes errors like this one:
```
1) Gitlab::Database::MigrationHelpers#update_column_in_batches when running outside of a transaction updates all the rows in a table
Failure/Error:
update_arel = Arel::UpdateManager.new(ActiveRecord::Base)
.table(table)
.set([[table[column], value]])
.where(table[:id].gteq(start_id))
ArgumentError:
wrong number of arguments (given 1, expected 0)
# ./lib/gitlab/database/migration_helpers.rb:317:in `new'
# ./lib/gitlab/database/migration_helpers.rb:317:in `block in update_column_in_batches'
# ./lib/gitlab/database/migration_helpers.rb:307:in `loop'
# ./lib/gitlab/database/migration_helpers.rb:307:in `update_column_in_batches'
# ./spec/lib/gitlab/database/migration_helpers_spec.rb:367:in `block (4 levels) in <top (required)>'
```
|
|
Direct disk access is done through Gitaly now, so the legacy path was
deprecated. This path was used in Gitlab::Shell however. This required
the refactoring in this commit.
Added is the removal of direct path access on the project model, as that
lookup wasn't needed anymore is most cases.
Closes https://gitlab.com/gitlab-org/gitaly/issues/1111
|
|
|
|
'security-10-6'
[10.6] Prevent notes on confidential issues from being sent to chat
See merge request gitlab/gitlabhq!2366
# Conflicts:
# app/helpers/services_helper.rb
|
|
This will use the same query as `\di` to find an index on postgresql.
|
|
Route path lookups through legacy_disk_path
See merge request gitlab-org/gitlab-ce!17743
|
|
|
|
Closes #43887.
|
|
Index creation does not have an effect if the index is present already.
Index removal does not have an affect if the index is not present.
This helps to avoid patterns like this in migrations:
```
if index_exists?(...)
remove_concurrent_index(...)
end
```
|
|
The concurrency issue originates from inserts on
`user_interacted_projects` from the app while running the post-deploy
migration.
This change comes with a strategy to lock the table while removing
duplicates and creating the unique index (and similar for FK
constraints).
Also, we'll have a non-unique index until the post-deploy migration is
finished to speed up queries during that time.
Closes #44205.
|
|
|
|
|
|
|
|
|
|
This fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/38634.
|
|
Prior to this commit we would essentially update all rows in a table,
even those where the source column (e.g. `issues.closed_at`) was NULL.
This in turn could lead to statement timeouts when using the default
batch size of 10 000 rows per job.
To work around this we don't schedule jobs for rows where the source
value is NULL. We also don't update rows where the source column is NULL
(as an extra precaution) or the target column already has a non-NULL
value. Using this approach it should be possible to update 10 000 rows
in the "issues" table in about 7.5 - 8 seconds.
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/42158
|
|
|
|
This adds a minimum interval to BackgroundMigrationWorker, ensuring
background migrations of the same class only run once every 5 minutes.
This prevents a thundering herd problem where scheduled migrations all
run at once due to their delays having been expired (e.g. as the result
of a queue being paused for a long time).
If a job was recently executed it's rescheduled with a delay that equals
the remaining time of the job's lease. This means that if the lease
expires in two minutes we only need to wait two minutes, instead of
five.
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/41624
|
