Welcome to mirror list, hosted at ThFree Co, Russian Federation.

gitlab.com/gitlab-org/gitaly.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorSami Hiltunen <shiltunen@gitlab.com>2022-10-14 15:29:16 +0300
committerSami Hiltunen <shiltunen@gitlab.com>2022-11-09 14:03:52 +0300
commitb85412239fb23b92e7a0514c21e5efb2db27669a (patch)
treec54f351366b04ec05c385f89c65f01905a00f631
parentb32121facdc4cdd4939400dafeeb919bb2360004 (diff)
Implement basic transaction management with write-ahead loggingsmh-log-worker
Gitaly is currently lacking in transaction control. Each write coming in launches their own Git commands which operate on the repository concurrently. This makes transaction management difficult. It's difficult to optimize the writes as they are being done from multiple locations without synchronization. The concurrent writers may step on each others toes and surface lock conflicts to the users. Recovering from crashes is also difficult as Gitaly is not logging the modifications it is about to perform and thus loses the transaction state on crashes. There's also no clear notion of ordering which further complicates replication related matters. It's not easy to say which writes a repository is missing and which not. We've recently designed a new replication architecture for Gitaly. The new architecture relies on a replicated write-ahead log. The write-ahead log defines a clear order of writes, aids in crash recovery. A single writer will be operating on a repository's log which makes further optimization easier such as write batching. This commit implements the first steps towards the new architecture by implementing the TransactionManager. The TransactionManager will be responsible for transaction management of a single repository. It will be the single goroutine that writes into a repository and is invoked by all other locations in the code that wish to write. It will also be responsible for synchronizing reads by ensuring they see the changes they are supposed to see. TransactionManager implementation introduced here does not contain the full implementation but aims to provide a basis for future iteration. For now, it implements basic write processing with a write-ahead log. It processes writes one-by-one by verifying references, logging the changes and finally applying the changes to the repository. It also supports recovering from the write-ahead log should the log processing be interrupted. The reference verification behavior can be tuned on a per transaction level to match behavior Git's `--atomic` and `--force` push flags. The TransactionManager stores the state related to the write-ahead log in BadgerDB, which is a key-value store local that will be local to each Gitaly storage. The values are marshaled protocol buffer messages. The write-ahead log does not yet contain pack files. This iteration is mostly concerned with the reference updating logic. Along with tests for the newly implemented behavior, a benchmark is introduced to test the reference updating throughput with various levels of concurrency. The throughput for now is roughly 30 reference updates per second. We'll further iterate on the performance later. There are a lot of low hanging fruits around like using long-lived git processes, batching multiple transactions into a shared write calls and pipelining the various steps of execution.
-rw-r--r--NOTICE1265
-rw-r--r--go.mod10
-rw-r--r--go.sum39
-rw-r--r--internal/gitaly/database.go18
-rw-r--r--internal/gitaly/testhelper_test.go89
-rw-r--r--internal/gitaly/transaction_manager.go427
-rw-r--r--internal/gitaly/transaction_manager_test.go1302
-rw-r--r--internal/testhelper/leakage.go4
-rw-r--r--proto/go/gitalypb/log.pb.go303
-rw-r--r--proto/go/gitalypb/protolist.go1
-rw-r--r--proto/log.proto37
11 files changed, 3492 insertions, 3 deletions
diff --git a/NOTICE b/NOTICE
index 8dde106cc..b72e0fdb3 100644
--- a/NOTICE
+++ b/NOTICE
@@ -6735,6 +6735,31 @@ LICENSE - github.com/census-instrumentation/opencensus-proto/gen-go
limitations under the License.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+LICENSE.txt - github.com/cespare/xxhash
+Copyright (c) 2016 Caleb Spare
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
LICENSE.txt - github.com/cespare/xxhash/v2
Copyright (c) 2016 Caleb Spare
@@ -7283,6 +7308,431 @@ ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+LICENSE - github.com/dgraph-io/badger/v3
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+LICENSE - github.com/dgraph-io/ristretto
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+LICENSE - github.com/dgraph-io/ristretto/z
+bbloom.go
+
+// The MIT License (MIT)
+// Copyright (c) 2014 Andreas Briese, eduToolbox@Bri-C GmbH, Sarstedt
+
+// Permission is hereby granted, free of charge, to any person obtaining a copy of
+// this software and associated documentation files (the "Software"), to deal in
+// the Software without restriction, including without limitation the rights to
+// use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
+// the Software, and to permit persons to whom the Software is furnished to do so,
+// subject to the following conditions:
+
+// The above copyright notice and this permission notice shall be included in all
+// copies or substantial portions of the Software.
+
+// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
+// FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
+// COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+// IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+// CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+rtutil.go
+
+// MIT License
+
+// Copyright (c) 2019 Ewan Chou
+
+// Permission is hereby granted, free of charge, to any person obtaining a copy
+// of this software and associated documentation files (the "Software"), to deal
+// in the Software without restriction, including without limitation the rights
+// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+// copies of the Software, and to permit persons to whom the Software is
+// furnished to do so, subject to the following conditions:
+
+// The above copyright notice and this permission notice shall be included in all
+// copies or substantial portions of the Software.
+
+// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+// SOFTWARE.
+
+Modifications:
+
+/*
+ * Copyright 2019 Dgraph Labs, Inc. and Contributors
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
LICENSE - github.com/dgryski/go-minhash
The MIT License (MIT)
@@ -7524,6 +7974,30 @@ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+LICENSE - github.com/dustin/go-humanize
+Copyright (c) 2005-2008 Dustin Sallings <dustin@spy.net>
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+<http://www.opensource.org/licenses/mit-license.php>
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
LICENSE - github.com/ekzhu/minhash-lsh
MIT License
@@ -8691,6 +9165,200 @@ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+LICENSE - github.com/golang/glog
+Apache License
+Version 2.0, January 2004
+http://www.apache.org/licenses/
+
+TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+1. Definitions.
+
+"License" shall mean the terms and conditions for use, reproduction, and
+distribution as defined by Sections 1 through 9 of this document.
+
+"Licensor" shall mean the copyright owner or entity authorized by the copyright
+owner that is granting the License.
+
+"Legal Entity" shall mean the union of the acting entity and all other entities
+that control, are controlled by, or are under common control with that entity.
+For the purposes of this definition, "control" means (i) the power, direct or
+indirect, to cause the direction or management of such entity, whether by
+contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the
+outstanding shares, or (iii) beneficial ownership of such entity.
+
+"You" (or "Your") shall mean an individual or Legal Entity exercising
+permissions granted by this License.
+
+"Source" form shall mean the preferred form for making modifications, including
+but not limited to software source code, documentation source, and configuration
+files.
+
+"Object" form shall mean any form resulting from mechanical transformation or
+translation of a Source form, including but not limited to compiled object code,
+generated documentation, and conversions to other media types.
+
+"Work" shall mean the work of authorship, whether in Source or Object form, made
+available under the License, as indicated by a copyright notice that is included
+in or attached to the work (an example is provided in the Appendix below).
+
+"Derivative Works" shall mean any work, whether in Source or Object form, that
+is based on (or derived from) the Work and for which the editorial revisions,
+annotations, elaborations, or other modifications represent, as a whole, an
+original work of authorship. For the purposes of this License, Derivative Works
+shall not include works that remain separable from, or merely link (or bind by
+name) to the interfaces of, the Work and Derivative Works thereof.
+
+"Contribution" shall mean any work of authorship, including the original version
+of the Work and any modifications or additions to that Work or Derivative Works
+thereof, that is intentionally submitted to Licensor for inclusion in the Work
+by the copyright owner or by an individual or Legal Entity authorized to submit
+on behalf of the copyright owner. For the purposes of this definition,
+"submitted" means any form of electronic, verbal, or written communication sent
+to the Licensor or its representatives, including but not limited to
+communication on electronic mailing lists, source code control systems, and
+issue tracking systems that are managed by, or on behalf of, the Licensor for
+the purpose of discussing and improving the Work, but excluding communication
+that is conspicuously marked or otherwise designated in writing by the copyright
+owner as "Not a Contribution."
+
+"Contributor" shall mean Licensor and any individual or Legal Entity on behalf
+of whom a Contribution has been received by Licensor and subsequently
+incorporated within the Work.
+
+2. Grant of Copyright License.
+
+Subject to the terms and conditions of this License, each Contributor hereby
+grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free,
+irrevocable copyright license to reproduce, prepare Derivative Works of,
+publicly display, publicly perform, sublicense, and distribute the Work and such
+Derivative Works in Source or Object form.
+
+3. Grant of Patent License.
+
+Subject to the terms and conditions of this License, each Contributor hereby
+grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free,
+irrevocable (except as stated in this section) patent license to make, have
+made, use, offer to sell, sell, import, and otherwise transfer the Work, where
+such license applies only to those patent claims licensable by such Contributor
+that are necessarily infringed by their Contribution(s) alone or by combination
+of their Contribution(s) with the Work to which such Contribution(s) was
+submitted. If You institute patent litigation against any entity (including a
+cross-claim or counterclaim in a lawsuit) alleging that the Work or a
+Contribution incorporated within the Work constitutes direct or contributory
+patent infringement, then any patent licenses granted to You under this License
+for that Work shall terminate as of the date such litigation is filed.
+
+4. Redistribution.
+
+You may reproduce and distribute copies of the Work or Derivative Works thereof
+in any medium, with or without modifications, and in Source or Object form,
+provided that You meet the following conditions:
+
+You must give any other recipients of the Work or Derivative Works a copy of
+this License; and
+You must cause any modified files to carry prominent notices stating that You
+changed the files; and
+You must retain, in the Source form of any Derivative Works that You distribute,
+all copyright, patent, trademark, and attribution notices from the Source form
+of the Work, excluding those notices that do not pertain to any part of the
+Derivative Works; and
+If the Work includes a "NOTICE" text file as part of its distribution, then any
+Derivative Works that You distribute must include a readable copy of the
+attribution notices contained within such NOTICE file, excluding those notices
+that do not pertain to any part of the Derivative Works, in at least one of the
+following places: within a NOTICE text file distributed as part of the
+Derivative Works; within the Source form or documentation, if provided along
+with the Derivative Works; or, within a display generated by the Derivative
+Works, if and wherever such third-party notices normally appear. The contents of
+the NOTICE file are for informational purposes only and do not modify the
+License. You may add Your own attribution notices within Derivative Works that
+You distribute, alongside or as an addendum to the NOTICE text from the Work,
+provided that such additional attribution notices cannot be construed as
+modifying the License.
+You may add Your own copyright statement to Your modifications and may provide
+additional or different license terms and conditions for use, reproduction, or
+distribution of Your modifications, or for any such Derivative Works as a whole,
+provided Your use, reproduction, and distribution of the Work otherwise complies
+with the conditions stated in this License.
+
+5. Submission of Contributions.
+
+Unless You explicitly state otherwise, any Contribution intentionally submitted
+for inclusion in the Work by You to the Licensor shall be under the terms and
+conditions of this License, without any additional terms or conditions.
+Notwithstanding the above, nothing herein shall supersede or modify the terms of
+any separate license agreement you may have executed with Licensor regarding
+such Contributions.
+
+6. Trademarks.
+
+This License does not grant permission to use the trade names, trademarks,
+service marks, or product names of the Licensor, except as required for
+reasonable and customary use in describing the origin of the Work and
+reproducing the content of the NOTICE file.
+
+7. Disclaimer of Warranty.
+
+Unless required by applicable law or agreed to in writing, Licensor provides the
+Work (and each Contributor provides its Contributions) on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied,
+including, without limitation, any warranties or conditions of TITLE,
+NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are
+solely responsible for determining the appropriateness of using or
+redistributing the Work and assume any risks associated with Your exercise of
+permissions under this License.
+
+8. Limitation of Liability.
+
+In no event and under no legal theory, whether in tort (including negligence),
+contract, or otherwise, unless required by applicable law (such as deliberate
+and grossly negligent acts) or agreed to in writing, shall any Contributor be
+liable to You for damages, including any direct, indirect, special, incidental,
+or consequential damages of any character arising as a result of this License or
+out of the use or inability to use the Work (including but not limited to
+damages for loss of goodwill, work stoppage, computer failure or malfunction, or
+any and all other commercial damages or losses), even if such Contributor has
+been advised of the possibility of such damages.
+
+9. Accepting Warranty or Additional Liability.
+
+While redistributing the Work or Derivative Works thereof, You may choose to
+offer, and charge a fee for, acceptance of support, warranty, indemnity, or
+other liability obligations and/or rights consistent with this License. However,
+in accepting such obligations, You may act only on Your own behalf and on Your
+sole responsibility, not on behalf of any other Contributor, and only if You
+agree to indemnify, defend, and hold each Contributor harmless for any liability
+incurred by, or claims asserted against, such Contributor by reason of your
+accepting any such warranty or additional liability.
+
+END OF TERMS AND CONDITIONS
+
+APPENDIX: How to apply the Apache License to your work
+
+To apply the Apache License to your work, attach the following boilerplate
+notice, with the fields enclosed by brackets "[]" replaced with your own
+identifying information. (Don't include the brackets!) The text should be
+enclosed in the appropriate comment syntax for the file format. We also
+recommend that a file or class name and description of purpose be included on
+the same "printed page" as the copyright notice for easier identification within
+third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
LICENSE - github.com/golang/groupcache/lru
Apache License
Version 2.0, January 2004
@@ -8916,6 +9584,36 @@ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+LICENSE - github.com/golang/snappy
+Copyright (c) 2011 The Snappy-Go Authors. All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+
+ * Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above
+copyright notice, this list of conditions and the following disclaimer
+in the documentation and/or other materials provided with the
+distribution.
+ * Neither the name of Google Inc. nor the names of its
+contributors may be used to endorse or promote products derived from
+this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
LICENSE - github.com/golang-jwt/jwt/v4
Copyright (c) 2012 Dave Grijalva
Copyright (c) 2021 golang-jwt maintainers
@@ -8928,6 +9626,211 @@ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLI
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+LICENSE.txt - github.com/google/flatbuffers/go
+
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
LICENSE - github.com/google/go-cmp/cmp
Copyright (c) 2017 The Go Authors. All rights reserved.
@@ -18671,6 +19574,368 @@ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+LICENSE - github.com/klauspost/compress
+Copyright (c) 2012 The Go Authors. All rights reserved.
+Copyright (c) 2019 Klaus Post. All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+
+ * Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above
+copyright notice, this list of conditions and the following disclaimer
+in the documentation and/or other materials provided with the
+distribution.
+ * Neither the name of Google Inc. nor the names of its
+contributors may be used to endorse or promote products derived from
+this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+------------------
+
+Files: gzhttp/*
+
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright 2016-2017 The New York Times Company
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+------------------
+
+Files: s2/cmd/internal/readahead/*
+
+The MIT License (MIT)
+
+Copyright (c) 2015 Klaus Post
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+---------------------
+Files: snappy/*
+Files: internal/snapref/*
+
+Copyright (c) 2011 The Snappy-Go Authors. All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+
+ * Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above
+copyright notice, this list of conditions and the following disclaimer
+in the documentation and/or other materials provided with the
+distribution.
+ * Neither the name of Google Inc. nor the names of its
+contributors may be used to endorse or promote products derived from
+this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+-----------------
+
+Files: s2/cmd/internal/filepathx/*
+
+Copyright 2016 The filepathx Authors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+LICENSE - github.com/klauspost/compress/internal/snapref
+Copyright (c) 2011 The Snappy-Go Authors. All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+
+ * Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above
+copyright notice, this list of conditions and the following disclaimer
+in the documentation and/or other materials provided with the
+distribution.
+ * Neither the name of Google Inc. nor the names of its
+contributors may be used to endorse or promote products derived from
+this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+LICENSE.txt - github.com/klauspost/compress/zstd/internal/xxhash
+Copyright (c) 2016 Caleb Spare
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
LICENSE - github.com/leonelquinteros/gotext
The MIT License (MIT)
diff --git a/go.mod b/go.mod
index 9d73ee2c8..710a727f3 100644
--- a/go.mod
+++ b/go.mod
@@ -10,6 +10,7 @@ require (
github.com/beevik/ntp v0.3.0
github.com/cloudflare/tableflip v1.2.3
github.com/containerd/cgroups v1.0.4
+ github.com/dgraph-io/badger/v3 v3.2103.3
github.com/getsentry/sentry-go v0.13.0
github.com/git-lfs/git-lfs/v3 v3.2.0
github.com/go-enry/go-enry/v2 v2.8.3
@@ -40,7 +41,7 @@ require (
go.uber.org/goleak v1.2.0
gocloud.dev v0.26.0
golang.org/x/sync v0.0.0-20220601150217-0de741cfad7f
- golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f
+ golang.org/x/sys v0.0.0-20221010170243-090e33056c14
golang.org/x/time v0.0.0-20220609170525-579cf78fd858
google.golang.org/grpc v1.49.0
google.golang.org/protobuf v1.28.1
@@ -90,14 +91,17 @@ require (
github.com/aws/smithy-go v1.11.2 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/census-instrumentation/opencensus-proto v0.3.0 // indirect
+ github.com/cespare/xxhash v1.1.0 // indirect
github.com/cespare/xxhash/v2 v2.1.2 // indirect
github.com/client9/reopen v1.0.0 // indirect
github.com/cloudflare/circl v1.1.0 // indirect
github.com/coreos/go-systemd/v22 v22.3.2 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
+ github.com/dgraph-io/ristretto v0.1.1 // indirect
github.com/dgryski/go-minhash v0.0.0-20170608043002-7fe510aff544 // indirect
github.com/docker/go-units v0.4.0 // indirect
github.com/dpotapov/go-spnego v0.0.0-20220426193508-b7f82e4507db // indirect
+ github.com/dustin/go-humanize v1.0.0 // indirect
github.com/ekzhu/minhash-lsh v0.0.0-20171225071031-5c06ee8586a1 // indirect
github.com/emirpasic/gods v1.12.0 // indirect
github.com/git-lfs/gitobj/v2 v2.1.0 // indirect
@@ -112,8 +116,11 @@ require (
github.com/go-ole/go-ole v1.2.4 // indirect
github.com/godbus/dbus/v5 v5.0.4 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
+ github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.2 // indirect
+ github.com/golang/snappy v0.0.3 // indirect
+ github.com/google/flatbuffers v1.12.1 // indirect
github.com/google/pprof v0.0.0-20210804190019-f964ff605595 // indirect
github.com/google/wire v0.5.0 // indirect
github.com/googleapis/gax-go/v2 v2.2.0 // indirect
@@ -135,6 +142,7 @@ require (
github.com/jdkato/prose v1.1.0 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/kevinburke/ssh_config v0.0.0-20201106050909-4977a11b4351 // indirect
+ github.com/klauspost/compress v1.15.1 // indirect
github.com/leonelquinteros/gotext v1.5.0 // indirect
github.com/lightstep/lightstep-tracer-common/golang/gogo v0.0.0-20210210170715-a8dfcb80d3a7 // indirect
github.com/lightstep/lightstep-tracer-go v0.25.0 // indirect
diff --git a/go.sum b/go.sum
index 93248c545..cc6011906 100644
--- a/go.sum
+++ b/go.sum
@@ -134,6 +134,7 @@ github.com/Microsoft/go-winio v0.4.14/go.mod h1:qXqCSQ3Xa7+6tgxaGTIe4Kpcdsi+P8jB
github.com/Microsoft/go-winio v0.4.16/go.mod h1:XB6nPKklQyQ7GC9LdcBEcBl8PF76WugXOPRXwdLnMv0=
github.com/Microsoft/go-winio v0.5.0 h1:Elr9Wn+sGKPlkaBvwu4mTrxtmOp3F3yV9qhaHbXGjwU=
github.com/Microsoft/go-winio v0.5.0/go.mod h1:JPGBdM1cNvN/6ISo+n8V5iA4v8pBzdOpzfwIujj1a84=
+github.com/OneOfOne/xxhash v1.2.2 h1:KMrpdQIwFcEqXDklaen+P1axHaj9BSKzvpUUfnHldSE=
github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
github.com/ProtonMail/go-crypto v0.0.0-20210428141323-04723f9f07d7/go.mod h1:z4/9nQmJSSwwds7ejkxaJwO37dru3geImFUdJlaLzQo=
github.com/ProtonMail/go-crypto v0.0.0-20221026131551-cf6655e29de4 h1:ra2OtmuW0AE5csawV4YXMNGNQQXvLRps3z2Z59OPO+I=
@@ -155,6 +156,7 @@ github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239 h1:kFOfPq6dUM1hTo
github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239/go.mod h1:2FmKhYUyUczH0OGQWaF5ceTx0UBShxjsH6f8oGKYe2c=
github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY=
github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o=
+github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=
github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY=
github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8=
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPdPJAN/hZIm0C4OItdklCFmMRWYpio=
@@ -218,6 +220,7 @@ github.com/bwesterb/go-ristretto v1.2.0/go.mod h1:fUIoIZaG73pV5biE2Blr2xEzDoMj7N
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/census-instrumentation/opencensus-proto v0.3.0 h1:t/LhUZLVitR1Ow2YOnduCsavhwFUklBMoGVYUCqmCqk=
github.com/census-instrumentation/opencensus-proto v0.3.0/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
+github.com/cespare/xxhash v1.1.0 h1:a6HrQnmkObjyL+Gs60czilIUGqrzKutQD6XZog3p+ko=
github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=
github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/cespare/xxhash/v2 v2.1.2 h1:YRXhKfTDauu4ajMg1TPgFO5jnlC2HCbmLXMcTG5cbYE=
@@ -244,11 +247,15 @@ github.com/cockroachdb/apd v1.1.0 h1:3LFP3629v+1aKXU5Q37mxmRxX/pIu1nijXydLShEq5I
github.com/cockroachdb/apd v1.1.0/go.mod h1:8Sl8LxpKi29FqWXR16WEFZRNSz3SoPzUzeMeY4+DwBQ=
github.com/containerd/cgroups v1.0.4 h1:jN/mbWBEaz+T1pi5OFtnkQ+8qnmEbAr1Oo1FRm5B0dA=
github.com/containerd/cgroups v1.0.4/go.mod h1:nLNQtsF7Sl2HxNebu77i1R0oDlhiTG+kO4JTrUzo6IA=
+github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
+github.com/coreos/go-etcd v2.0.0+incompatible/go.mod h1:Jez6KQU2B/sWsbdaef3ED8NzMklzPG4d5KIOhIy30Tk=
+github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/coreos/go-systemd v0.0.0-20190719114852-fd7a80b32e1f/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/coreos/go-systemd/v22 v22.3.2 h1:D9/bQk5vlXQFZ6Kwuu6zaiXJ9oTPe68++AzAJc1DzSI=
github.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
+github.com/cpuguy83/go-md2man v1.0.10/go.mod h1:SmD6nW6nTyfqj6ABTjUi3V3JVMnlJmwcJI5acqYI6dE=
github.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
github.com/creack/pty v1.1.7/go.mod h1:lj5s0c3V2DBrqTV7llrYr5NG6My20zk30Fl46Y7DoTY=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
@@ -258,6 +265,12 @@ github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs
github.com/denisenkom/go-mssqldb v0.9.0/go.mod h1:xbL0rPBG9cCiLr28tMa8zpbdarY27NDyej4t/EjAShU=
github.com/denisenkom/go-mssqldb v0.12.0/go.mod h1:iiK0YP1ZeepvmBQk/QpLEhhTNJgfzrpArPY/aFvc9yU=
github.com/devigned/tab v0.1.1/go.mod h1:XG9mPq0dFghrYvoBF3xdRrJzSTX1b7IQrvaL9mzjeJY=
+github.com/dgraph-io/badger/v3 v3.2103.3 h1:s63J1pisDhKpzWslXFe+ChuthuZptpwTE6qEKoczPb4=
+github.com/dgraph-io/badger/v3 v3.2103.3/go.mod h1:4MPiseMeDQ3FNCYwRbbcBOGJLf5jsE0PPFzRiKjtcdw=
+github.com/dgraph-io/ristretto v0.1.1 h1:6CWw5tJNgpegArSHpNHJKldNeq03FQCwYvfMVWajOK8=
+github.com/dgraph-io/ristretto v0.1.1/go.mod h1:S1GPSBCYCIhmVNfcth17y2zZtQT6wzkzgwUve0VDWWA=
+github.com/dgryski/go-farm v0.0.0-20190423205320-6a90982ecee2 h1:tdlZCpZ/P9DhczCTSixgIKmwPv6+wP5DGjqLYw5SUiA=
+github.com/dgryski/go-farm v0.0.0-20190423205320-6a90982ecee2/go.mod h1:SqUrOPUnsFjfmXRMNPybcSiG0BgUW2AuFH8PAnS2iTw=
github.com/dgryski/go-metro v0.0.0-20180109044635-280f6062b5bc h1:8WFBn63wegobsYAX0YjD+8suexZDga5CctH4CCTx2+8=
github.com/dgryski/go-metro v0.0.0-20180109044635-280f6062b5bc/go.mod h1:c9O8+fpSOX1DM8cPNSkX/qsBWdkD4yd2dpciOWQjpBw=
github.com/dgryski/go-minhash v0.0.0-20170608043002-7fe510aff544 h1:54Y/2GF52MSJ4n63HWvNDFRtztgm6tq2UrOX61sjGKc=
@@ -273,6 +286,8 @@ github.com/docker/go-units v0.4.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDD
github.com/dpotapov/go-spnego v0.0.0-20210315154721-298b63a54430/go.mod h1:AVSs/gZKt1bOd2AhkhbS7Qh56Hv7klde22yXVbwYJhc=
github.com/dpotapov/go-spnego v0.0.0-20220426193508-b7f82e4507db h1:3EIvol92cLWG5m13Me3/LfEtQqr23xzwZCxiWoT5gGE=
github.com/dpotapov/go-spnego v0.0.0-20220426193508-b7f82e4507db/go.mod h1:AVSs/gZKt1bOd2AhkhbS7Qh56Hv7klde22yXVbwYJhc=
+github.com/dustin/go-humanize v1.0.0 h1:VSnTsYCnlFHaM2/igO1h6X3HA71jcobQuxemgkq4zYo=
+github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
github.com/ekzhu/minhash-lsh v0.0.0-20171225071031-5c06ee8586a1 h1:/7G7q8SDJdrah5jDYqZI8pGFjSqiCzfSEO+NgqKCYX0=
github.com/ekzhu/minhash-lsh v0.0.0-20171225071031-5c06ee8586a1/go.mod h1:yEtCVi+QamvzjEH4U/m6ZGkALIkF2xfQnFp0BcKmIOk=
github.com/emirpasic/gods v1.12.0 h1:QAUIPSaCu4G+POclxeqb3F+WPpdKqFGlw36+yOzGlrg=
@@ -377,6 +392,7 @@ github.com/golang-jwt/jwt/v4 v4.4.2/go.mod h1:m21LjoU+eqJr34lmDMbreY2eSTRJ1cv77w
github.com/golang-sql/civil v0.0.0-20190719163853-cb61b32ac6fe/go.mod h1:8vg3r2VgvsThLBIFL93Qb5yWzgyZWhEmBwUJWevAkK0=
github.com/golang-sql/sqlexp v0.0.0-20170517235910-f1bb20e5a188/go.mod h1:vXjM/+wXQnTPR4KqTKDgJukSZ6amVRtWMPEjE6sQoK8=
github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0/go.mod h1:E/TSTwGwJL78qG/PmXZO1EjYhfJinVAhrmmHX6Z8B9k=
+github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b h1:VKtxabqXZkF25pY9ekfRL6a582T4P37/31XEstQ5p58=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
@@ -411,9 +427,12 @@ github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaS
github.com/golang/protobuf v1.5.1/go.mod h1:DopwsBzvsk0Fs44TXzsVbJyPhcCPeIwnvohx4u74HPM=
github.com/golang/protobuf v1.5.2 h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw=
github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
+github.com/golang/snappy v0.0.3 h1:fHPg5GQYlCeLIPB9BZqMVR5nR9A+IM5zcgeTdjMYmLA=
github.com/golang/snappy v0.0.3/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
+github.com/google/flatbuffers v1.12.1 h1:MVlul7pQNoDzWRLTw5imwYsl+usrS1TXG2H4jg6ImGw=
+github.com/google/flatbuffers v1.12.1/go.mod h1:1AeVuKshWv4vARoZatz6mlQ0JxURH0Kv5+zNeJKJCa8=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
@@ -630,6 +649,8 @@ github.com/kevinburke/ssh_config v0.0.0-20201106050909-4977a11b4351/go.mod h1:CT
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/compress v1.10.3/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
+github.com/klauspost/compress v1.12.3/go.mod h1:8dP1Hq4DHOhN9w426knH3Rhby4rFm6D8eO+e+Dq5Gzg=
+github.com/klauspost/compress v1.15.1 h1:y9FcTHGyrebwfP0ZZqFiaxTaiDnUrGkJkI+f583BL1A=
github.com/klauspost/compress v1.15.1/go.mod h1:/3/Vjq9QcHkK5uEr5lBEmyoZ1iFhe47etQ6QUkpK6sk=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/konsorten/go-windows-terminal-sequences v1.0.2/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
@@ -662,6 +683,7 @@ github.com/lightstep/lightstep-tracer-common/golang/gogo v0.0.0-20210210170715-a
github.com/lightstep/lightstep-tracer-common/golang/gogo v0.0.0-20210210170715-a8dfcb80d3a7/go.mod h1:Spd59icnvRxSKuyijbbwe5AemzvcyXAUBgApa7VybMw=
github.com/lightstep/lightstep-tracer-go v0.25.0 h1:sGVnz8h3jTQuHKMbUe2949nXm3Sg09N1UcR3VoQNN5E=
github.com/lightstep/lightstep-tracer-go v0.25.0/go.mod h1:G1ZAEaqTHFPWpWunnbUn1ADEY/Jvzz7jIOaXwAfD6A8=
+github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
github.com/magiconair/properties v1.8.5/go.mod h1:y3VJvCyxH9uVvJTWEGAELF3aiYNyPKd5NZ3oSwXrF60=
github.com/markbates/errx v1.1.0 h1:QDFeR+UP95dO12JgW+tgi2UVfo0V8YBHiUIOaeBPiEI=
github.com/markbates/errx v1.1.0/go.mod h1:PLa46Oex9KNbVDZhKel8v1OT7hD5JZ2eI7AHhA0wswc=
@@ -742,6 +764,7 @@ github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FI
github.com/pavelmemory/go-license-detector/v4 v4.3.1-0.20220801101717-a7c9e28533cf h1:0hsdCeyyCHec4CFciP7tlz2t7fc4XeS1kXF6DXPWFxQ=
github.com/pavelmemory/go-license-detector/v4 v4.3.1-0.20220801101717-a7c9e28533cf/go.mod h1:3cTg6OLuDbqzstQeL1OUysZZGudU9yLAE+NmURSwmic=
github.com/pborman/getopt v0.0.0-20170112200414-7148bc3a4c30/go.mod h1:85jBQOZwpVEaDAr341tbn15RS4fCAsIst0qp7i8ex1o=
+github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
github.com/pelletier/go-toml v1.9.3/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c=
github.com/pelletier/go-toml/v2 v2.0.5 h1:ipoSadvV8oGUjnUbMub59IDPPwfxF694nG/jwbMiyQg=
github.com/pelletier/go-toml/v2 v2.0.5/go.mod h1:OMHamSCAODeSsVrwwvcJOaoN0LIUIaFVNZzmWyNfXas=
@@ -797,6 +820,7 @@ github.com/rubenv/sql-migrate v1.2.0 h1:fOXMPLMd41sK7Tg75SXDec15k3zg5WNV6SjuDRiN
github.com/rubenv/sql-migrate v1.2.0/go.mod h1:Z5uVnq7vrIrPmHbVFfR4YLHRZquxeHpckCnRq0P/K9Y=
github.com/rubyist/tracerx v0.0.0-20170927163412-787959303086 h1:mncRSDOqYCng7jOD+Y6+IivdRI6Kzv2BLWYkWkdQfu0=
github.com/rubyist/tracerx v0.0.0-20170927163412-787959303086/go.mod h1:YpdgDXpumPB/+EGmGTYHeiW/0QVFRzBYTNFaxWfPDk4=
+github.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g=
github.com/russross/blackfriday/v2 v2.0.1 h1:lPqVAte+HuHNfhJ/0LC98ESWRz8afy9tM/0RK8m9o+Q=
github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts=
@@ -827,13 +851,20 @@ github.com/sirupsen/logrus v1.9.0/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVs
github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc=
github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA=
github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
+github.com/spaolacci/murmur3 v1.1.0 h1:7c1g84S4BPRrfL5Xrdp6fOJ206sU9y293DDHaoy0bLI=
+github.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
+github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=
github.com/spf13/afero v1.6.0/go.mod h1:Ai8FlHk4v/PARR026UzYexafAt9roJ7LcLMAmO6Z93I=
+github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
github.com/spf13/cast v1.3.1/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
github.com/spf13/cobra v0.0.3/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ=
+github.com/spf13/cobra v0.0.5/go.mod h1:3K3wKZymM7VvHMDS9+Akkh4K60UwM26emMESw8tLCHU=
github.com/spf13/cobra v1.2.1/go.mod h1:ExllRjgxM/piMAM+3tAZvg8fsklGAf3tPfi+i8t68Nk=
+github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo=
github.com/spf13/jwalterweatherman v1.1.0/go.mod h1:aNWZUN0dPAAO/Ljvb5BEdw96iTZ0EXowPYD95IqWIGo=
github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
+github.com/spf13/viper v1.3.2/go.mod h1:ZiWeW+zYFKm7srdB9IoDzzZXaJaI5eL9QjNiN/DMA2s=
github.com/spf13/viper v1.8.1/go.mod h1:o0Pch8wJ9BVSWGQMbra6iw0oQ5oktSIBaujf1rJH9Ns=
github.com/ssgelm/cookiejarparser v1.0.1 h1:cRdXauUbOTFzTPJFaeiWbHnQ+tRGlpKKzvIK9PUekE4=
github.com/ssgelm/cookiejarparser v1.0.1/go.mod h1:DUfC0mpjIzlDN7DzKjXpHj0qMI5m9VrZuz3wSlI+OEI=
@@ -864,6 +895,7 @@ github.com/uber/jaeger-client-go v2.30.0+incompatible/go.mod h1:WVhlPFC8FDjOFMMW
github.com/uber/jaeger-lib v2.4.1+incompatible h1:td4jdvLcExb4cBISKIpHuGoVXh+dVKhn2Um6rjCsSsg=
github.com/uber/jaeger-lib v2.4.1+incompatible/go.mod h1:ComeNDZlWwrWnDv8aPp0Ba6+uUTzImX/AauajbLI56U=
github.com/ugorji/go v1.1.7/go.mod h1:kZn38zHttfInRq0xu/PH0az30d+z6vm202qpg1oXVMw=
+github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8/go.mod h1:VFNgLljTbGfSG7qAOspJ7OScBnGdDN/yBr0sguwnwf0=
github.com/ugorji/go/codec v1.1.7/go.mod h1:Ax+UKWsSmolVDwsd+7N3ZtXu+yMGCf907BLYF3GoBXY=
github.com/xanzy/ssh-agent v0.2.1/go.mod h1:mLlQY/MoOhWBj+gOGMQkOeiEvkx+8pJSI+0Bx9h2kr4=
github.com/xanzy/ssh-agent v0.3.0 h1:wUMzuKtKilRgBAD1sUb8gOwwRr2FGoBVumcjoOACClI=
@@ -874,6 +906,7 @@ github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 h1:EzJWgHo
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415/go.mod h1:GwrjFmJcFw6At/Gs6z4yjiIwzuJ1/+UwLxMQDVQXShQ=
github.com/xeipuuv/gojsonschema v0.0.0-20170210233622-6b67b3fab74d/go.mod h1:5yf86TLmAcydyeJq5YvxkGPE2fm/u4myDekKRoLuqhs=
github.com/xeipuuv/gojsonschema v1.2.0 h1:LhYJRs+L4fBtjZUfuSZIKGeVu0QRy8e5Xi7D17UxZ74=
+github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=
github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
@@ -923,6 +956,7 @@ gocloud.dev v0.26.0 h1:4rM/SVL0lLs+rhC0Gmc+gt/82DBpb7nbpIZKXXnfMXg=
gocloud.dev v0.26.0/go.mod h1:mkUgejbnbLotorqDyvedJO20XcZNTynmSeVSQS9btVg=
golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
+golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190219172222-a4c6cb3142f2/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190325154230-a5d413f7728c/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
@@ -1089,6 +1123,7 @@ golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5h
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181026203630-95b1ffbd15a5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190221075227-b4e8571b14e0/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@@ -1172,8 +1207,8 @@ golang.org/x/sys v0.0.0-20220227234510-4e6760a101f9/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220328115105-d36c6a25d886/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220330033206-e17cdc41300f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f h1:v4INt8xihDGvnrfjMDVXGxw9wrfxYyCjk0KbXjhR55s=
-golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20221010170243-090e33056c14 h1:k5II8e6QD8mITdi+okbbmR/cIyEbeXLBhy5Ha4nevyc=
+golang.org/x/sys v0.0.0-20221010170243-090e33056c14/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 h1:JGgROgKl9N8DuW20oFS5gxc+lE67/N3FcwmBPMe7ArY=
diff --git a/internal/gitaly/database.go b/internal/gitaly/database.go
new file mode 100644
index 000000000..592e227b5
--- /dev/null
+++ b/internal/gitaly/database.go
@@ -0,0 +1,18 @@
+package gitaly
+
+import (
+ "github.com/dgraph-io/badger/v3"
+)
+
+// OpenDatabase opens a new database handle to a database at the given path.
+func OpenDatabase(databasePath string) (*badger.DB, error) {
+ dbOptions := badger.DefaultOptions(databasePath)
+ // Enable SyncWrites to ensure all writes are persisted to disk before considering
+ // them committed.
+ dbOptions.SyncWrites = true
+ // Badger by default logs fairly verbose statistics when opening a database. Disable the
+ // logging for now.
+ dbOptions.Logger = nil
+
+ return badger.Open(dbOptions)
+}
diff --git a/internal/gitaly/testhelper_test.go b/internal/gitaly/testhelper_test.go
new file mode 100644
index 000000000..e83cd79d7
--- /dev/null
+++ b/internal/gitaly/testhelper_test.go
@@ -0,0 +1,89 @@
+package gitaly
+
+import (
+ "bytes"
+ "context"
+ "fmt"
+ "reflect"
+ "sort"
+ "testing"
+
+ "github.com/dgraph-io/badger/v3"
+ "github.com/stretchr/testify/require"
+ "gitlab.com/gitlab-org/gitaly/v15/internal/git"
+ "gitlab.com/gitlab-org/gitaly/v15/internal/git/localrepo"
+ "gitlab.com/gitlab-org/gitaly/v15/internal/testhelper"
+ "gitlab.com/gitlab-org/gitaly/v15/proto/go/gitalypb"
+ "google.golang.org/protobuf/proto"
+)
+
+func TestMain(m *testing.M) {
+ testhelper.Run(m)
+}
+
+// DatabaseState describes the expected state of the key-value store. The keys in the map are the expected keys
+// in the database and the values are the expected unmarshaled values.
+type DatabaseState map[string]proto.Message
+
+// RequireDatabase asserts the actual database state matches the expected database state. The actual values in the
+// database are unmarshaled to the same type the values have in the expected database state.
+func RequireDatabase(tb testing.TB, ctx context.Context, database *badger.DB, expectedDatabase DatabaseState) {
+ tb.Helper()
+
+ actualDatabase := DatabaseState{}
+ unexpectedKeys := []string{}
+ require.NoError(tb, database.View(func(txn *badger.Txn) error {
+ iterator := txn.NewIterator(badger.DefaultIteratorOptions)
+ defer iterator.Close()
+
+ for iterator.Rewind(); iterator.Valid(); iterator.Next() {
+ key := iterator.Item().Key()
+ expectedValue, ok := expectedDatabase[string(key)]
+ if !ok {
+ // Print the keys out escaped as otherwise the non-printing characters are not visible in the assertion failure.
+ unexpectedKeys = append(unexpectedKeys, fmt.Sprintf("%q", key))
+ continue
+ }
+
+ require.NoError(tb, iterator.Item().Value(func(value []byte) error {
+ // Unmarshal the actual value to the same type as the expected value has.
+ actualValue := reflect.New(reflect.TypeOf(expectedValue).Elem()).Interface().(proto.Message)
+ require.NoError(tb, proto.Unmarshal(value, actualValue))
+
+ if logEntry, ok := actualValue.(*gitalypb.LogEntry); ok {
+ // The are write-ahead logged in undeterministic order. They are input as a map and the map
+ // iteration order is random. Sort them so we can assert their equality.
+ sort.Slice(logEntry.ReferenceUpdates, func(i, j int) bool {
+ return bytes.Compare(
+ logEntry.ReferenceUpdates[i].ReferenceName,
+ logEntry.ReferenceUpdates[j].ReferenceName,
+ ) == -1
+ })
+ }
+
+ actualDatabase[string(key)] = actualValue
+
+ return nil
+ }))
+ }
+
+ return nil
+ }))
+
+ require.Empty(tb, unexpectedKeys, "database contains unexpected keys")
+
+ if expectedDatabase == nil {
+ expectedDatabase = DatabaseState{}
+ }
+
+ testhelper.ProtoEqual(tb, expectedDatabase, actualDatabase)
+}
+
+// RequireReferences asserts that the actual state of the references in the repository match the expected.
+func RequireReferences(tb testing.TB, ctx context.Context, repo *localrepo.Repo, expectedReferences []git.Reference) {
+ tb.Helper()
+
+ actualReferences, err := repo.GetReferences(ctx)
+ require.NoError(tb, err)
+ require.ElementsMatch(tb, expectedReferences, actualReferences)
+}
diff --git a/internal/gitaly/transaction_manager.go b/internal/gitaly/transaction_manager.go
new file mode 100644
index 000000000..42226503d
--- /dev/null
+++ b/internal/gitaly/transaction_manager.go
@@ -0,0 +1,427 @@
+package gitaly
+
+import (
+ "bytes"
+ "context"
+ "encoding/binary"
+ "errors"
+ "fmt"
+
+ "github.com/dgraph-io/badger/v3"
+ "gitlab.com/gitlab-org/gitaly/v15/internal/git"
+ "gitlab.com/gitlab-org/gitaly/v15/internal/git/localrepo"
+ "gitlab.com/gitlab-org/gitaly/v15/internal/git/repository"
+ "gitlab.com/gitlab-org/gitaly/v15/internal/git/updateref"
+ "gitlab.com/gitlab-org/gitaly/v15/proto/go/gitalypb"
+ "google.golang.org/protobuf/proto"
+)
+
+// LogIndex points to a specific position in a repository's write-ahead log.
+type LogIndex uint64
+
+// Primitive returns the primitive representation of LogIndex for serialization purposes.
+func (index LogIndex) Primitive() uint64 { return uint64(index) }
+
+// ReferenceTips describes the state of a reference's old and new tip in an update.
+type ReferenceTips struct {
+ // OldOID is the old OID the reference is expected to point to prior to updating it.
+ // If the reference does not point to the old value, the reference verification fails.
+ OldOID git.ObjectID
+ // NewOID is the new desired OID to point the reference to.
+ NewOID git.ObjectID
+}
+
+// ReferenceUpdates contains references to update. Reference name is used as the key and the value
+// is the expected old tip and the desired new tip.
+type ReferenceUpdates map[git.ReferenceName]ReferenceTips
+
+// ReferenceVerificationStrategy determines the reference verification behavior for a transaction.
+type ReferenceVerificationStrategy byte
+
+const (
+ // ReferenceVerificationStrategyStrict is the default strategy. The entire transaction is aborted
+ // if any reference fails the verification.
+ //
+ // This maps to `git push` with the `--atomic` flag.
+ ReferenceVerificationStrategyStrict ReferenceVerificationStrategy = iota
+ // ReferenceVerificationStrategySkipFailures allows for completing a transaction even if there are
+ // some references verification failures. The references that failed verification are dropped from
+ // the transaction. The references that were successfully verified are committed.
+ //
+ // This maps to `git push` without the `--atomic` flag.
+ ReferenceVerificationStrategySkipFailures
+ // ReferenceVerificationStrategySkip skips the entire reference verification process. The references
+ // are updated to point to the new tips regardless of their current state.
+ //
+ // This maps to `git push` with the `--force` flag.
+ ReferenceVerificationStrategySkip
+)
+
+// Transaction is a unit-of-work that contains reference changes to perform on the repository.
+type Transaction struct {
+ // ReferenceVerificationStrategy determines the reference verification strategy for this transaction.
+ ReferenceVerificationStrategy
+ // ReferenceUpdates contains the reference updates to be performed.
+ ReferenceUpdates
+}
+
+// TransactionManager is responsible for transaction management of a single repository. Each repository has
+// a single TransactionManager; it is the repository's single-writer. It accepts writes one at a time from
+// the admissionQueue. Each admitted write is processed in three steps:
+//
+// 1. The references being updated are verified by ensuring the expected old tips match what the references
+// actually point to prior to update. The entire transaction is by default aborted if a single reference
+// fails the verification step. The reference verification behavior can be controlled on a per-transaction
+// level by setting:
+// - The reference verification failures can be ignored instead of aborting the entire transaction.
+// If done, the references that failed verification are dropped from the transaction but the updates
+// that passed verification are still performed.
+// - The reference verification may also be skipped if the write is force updating references. If
+// done, the current state of the references is ignored and they are directly updated to point
+// to the new tips.
+// 2. The transaction is appended to the write-ahead log. Once the write has been logged, it is effectively
+// committed and will be applied to the repository even after restarting.
+// 3. The transaction is applied from the write-ahead log to the repository by actually performing the reference
+// changes.
+//
+// The goroutine that issued the transaction is waiting for the result while these steps are being performed. As
+// there is no transaction control for readers yet, the issuer is only notified of a successful write after the
+// write has been applied to the repository.
+//
+// TransactionManager recovers transactions after interruptions by applying the write-ahead logged transactions to
+// the repository on start up.
+//
+// TransactionManager maintains the write-ahead log in a key-value store. It maintains the following key spaces:
+// - `repository/<repository_id:string>/log/index/applied`
+// - This key stores the index of the log entry that has been applied to the repository. This allows for
+// determining how far a repository is in processing the log and which log entries need to be applied
+// after starting up. Repository starts from log index 0 if there are no log entries recorded to have
+// been applied.
+// - `repository/<repository_id:string>/log/entry/<log_index:uint64>`
+// - These keys hold the actual write-ahead log entries. A repository's first log entry starts at index 1
+// and the log index keeps monotonically increasing from there on without gaps. The write-ahead log
+// entries are processed in ascending order.
+//
+// The values in the database are marshaled protocol buffer messages. Numbers in the keys are encoded as big
+// endian to preserve the sort order of the numbers also in lexicographical order.
+type TransactionManager struct {
+ // repository is the repository this TransactionManager is acting on.
+ repository *localrepo.Repo
+ // db is the handle to the key-value store used for storing the write-ahead log related state.
+ db *badger.DB
+ // admissionQueue is where the incoming writes are waiting to be admitted to the transaction
+ // manager.
+ admissionQueue chan transactionFuture
+
+ // appendedLogIndex holds the index of the last log entry appended to the log.
+ appendedLogIndex LogIndex
+ // appliedLogIndex holds the index of the last log entry applied to the repository
+ appliedLogIndex LogIndex
+
+ // testingKnobs are used by tests to gain more control over the Run to test more complex interactions.
+ // These should be never set in production.
+ testingKnobs struct {
+ // noLogApplication prevents the applying log entries to the repository when set.
+ noLogApplication bool
+ }
+}
+
+// NewTransactionManager returns a new TransactionManager for the given repository.
+func NewTransactionManager(db *badger.DB, repo *localrepo.Repo) *TransactionManager {
+ return &TransactionManager{
+ repository: repo,
+ db: db,
+ admissionQueue: make(chan transactionFuture),
+ }
+}
+
+// resultFuture represents a future that will yield the result of a transaction once its
+// outcome has been decided.
+type resultFuture chan error
+
+// transactionFuture holds a transaction and the resultFuture the transaction queuer is waiting
+// for the result on.
+type transactionFuture struct {
+ transaction Transaction
+ result resultFuture
+}
+
+// Propose queues a transaction for the TransactionManager to process. Propose returns once the transaction
+// has been successfully applied to the repository unless it errors. If Propose returns an error, the
+// transaction may or may not have been committed.
+func (mgr *TransactionManager) Propose(ctx context.Context, transaction Transaction) error {
+ queuedTransaction := transactionFuture{transaction: transaction, result: make(resultFuture, 1)}
+
+ select {
+ case mgr.admissionQueue <- queuedTransaction:
+ select {
+ case err := <-queuedTransaction.result:
+ return err
+ case <-ctx.Done():
+ return ctx.Err()
+ }
+ case <-ctx.Done():
+ return ctx.Err()
+ }
+}
+
+// Run starts the transaction processing. On start up Run loads the indexes of the last appended and applied
+// log entries from the database. It will then apply any transactions that have been logged but not applied
+// to the repository. Once the recovery is completed, Run starts processing new transactions by verifying
+// the reference changes apply according to the reference verification strategy, logging the transaction and
+// finally applying it to the repository. The transactions are acknowledged once they've been applied to the
+// repository.
+//
+// Run keeps running until Stop is called. Run does not synchronize closing with on-going Propose call
+// as that is going to be a responsibility of a higher layer later.
+func (mgr *TransactionManager) Run() error {
+ ctx, cancel := context.WithCancel(context.Background())
+ defer cancel()
+
+ if err := mgr.initialize(); err != nil {
+ return fmt.Errorf("initialize: %w", err)
+ }
+
+ // awaitingTransactions contains transactions waiting for their log entry to be applied to
+ // the repository. It's keyed by the log index the tranaction is waiting to be applied and the
+ // value is the resultFuture that is waiting the result.
+ awaitingTransactions := make(map[LogIndex]resultFuture)
+
+ for {
+ if mgr.appliedLogIndex < mgr.appendedLogIndex && !mgr.testingKnobs.noLogApplication {
+ logIndex := mgr.appliedLogIndex + 1
+
+ if err := mgr.applyLogEntry(ctx, logIndex); err != nil {
+ return fmt.Errorf("apply log entry: %w", err)
+ }
+
+ // There is no awaiter for a transaction if the transaction manager is recovering
+ // transactions from the log after starting up.
+ if resultFuture, ok := awaitingTransactions[logIndex]; ok {
+ resultFuture <- nil
+ delete(awaitingTransactions, logIndex)
+ }
+
+ continue
+ }
+
+ transaction, ok := <-mgr.admissionQueue
+ if !ok {
+ return nil
+ }
+
+ if err := func() error {
+ failedVerification, err := mgr.verifyReferences(ctx, transaction.transaction)
+ if err != nil {
+ return fmt.Errorf("verify references: %w", err)
+ }
+
+ if err := mgr.appendLogEntry(transaction.transaction, failedVerification); err != nil {
+ return fmt.Errorf("append log entry: %w", err)
+ }
+
+ return nil
+ }(); err != nil {
+ transaction.result <- err
+ }
+
+ if mgr.testingKnobs.noLogApplication {
+ // Yield the transaction immediately if log application has been disabled.
+ transaction.result <- nil
+ continue
+ }
+
+ awaitingTransactions[mgr.appendedLogIndex] = transaction.result
+ }
+}
+
+// Stop stops the transaction processing. Caller is responsible for ensuring there are no
+// active Propose calls when calling Stop.
+func (mgr *TransactionManager) Stop() { close(mgr.admissionQueue) }
+
+// initialize initializes the TransactionManager by loading the applied log index and determining the
+// appended log index from the database.
+func (mgr *TransactionManager) initialize() error {
+ var appliedLogIndex gitalypb.LogIndex
+ if err := mgr.readKey(keyAppliedLogIndex(getRepositoryID(mgr.repository)), &appliedLogIndex); err != nil && !errors.Is(err, badger.ErrKeyNotFound) {
+ return fmt.Errorf("read applied log index: %w", err)
+ }
+
+ mgr.appliedLogIndex = LogIndex(appliedLogIndex.LogIndex)
+
+ // The index of the last appended log entry is determined from the indexes of the latest entry in the log and
+ // the latest applied log entry. If there is a log entry, it is the latest appended log entry. If there are no
+ // log entries, the latest log entry must have been applied to the repository and pruned away, meaning the index
+ // of the last appended log entry is the same as the index if the last applied log entry.
+ //
+ // As the log indexes in the keys are encoded in big endian, the latest log entry can be found by taking
+ // the first key when iterating the log entry key space in reverse.
+ return mgr.db.View(func(txn *badger.Txn) error {
+ logPrefix := keyPrefixLogEntries(getRepositoryID(mgr.repository))
+
+ iterator := txn.NewIterator(badger.IteratorOptions{Reverse: true, Prefix: logPrefix})
+ defer iterator.Close()
+
+ mgr.appendedLogIndex = mgr.appliedLogIndex
+
+ // The iterator seeks to a key that is greater than or equal than seeked key. Since we are doing a reverse
+ // seek, we need to add 0xff to the prefix so the first iterated key is the latest log entry.
+ if iterator.Seek(append(logPrefix, 0xff)); iterator.Valid() {
+ mgr.appendedLogIndex = LogIndex(binary.BigEndian.Uint64(bytes.TrimPrefix(iterator.Item().Key(), logPrefix)))
+ }
+
+ return nil
+ })
+}
+
+// verifyReferences verifies the references in the transaction apply on top of the already accepted reference changes by
+// comparing the old tips in the transaction to the actual current tips. The verification behavior can be controlled by
+// setting the reference verification strategy of a transaction. If the reference verification strategy is to skip
+// references which failed verification, the failed references are returned.
+func (mgr *TransactionManager) verifyReferences(ctx context.Context, transaction Transaction) (map[git.ReferenceName]struct{}, error) {
+ if transaction.ReferenceVerificationStrategy == ReferenceVerificationStrategySkip {
+ return nil, nil
+ }
+
+ failedVerification := make(map[git.ReferenceName]struct{})
+ for referenceName, expectedTips := range transaction.ReferenceUpdates {
+ actualOldTip, err := mgr.repository.ResolveRevision(ctx, referenceName.Revision())
+ if errors.Is(err, git.ErrReferenceNotFound) {
+ objectHash, err := mgr.repository.ObjectHash(ctx)
+ if err != nil {
+ return nil, fmt.Errorf("object hash: %w", err)
+ }
+
+ actualOldTip = objectHash.ZeroOID
+ } else if err != nil {
+ return nil, fmt.Errorf("resolve revision: %w", err)
+ }
+
+ if expectedTips.OldOID != actualOldTip {
+ if transaction.ReferenceVerificationStrategy == ReferenceVerificationStrategySkipFailures {
+ failedVerification[referenceName] = struct{}{}
+ continue
+ }
+
+ return nil, fmt.Errorf("expected old tip of %q to be %q but it was %q", referenceName, expectedTips.OldOID, actualOldTip)
+ }
+ }
+
+ return failedVerification, nil
+}
+
+// appendLogEntry appends the transaction to the write-ahead log. References that failed verification are skipped and thus not
+// logged nor applied later.
+func (mgr *TransactionManager) appendLogEntry(transaction Transaction, failedVerification map[git.ReferenceName]struct{}) error {
+ var logEntry gitalypb.LogEntry
+ for referenceName, tips := range transaction.ReferenceUpdates {
+ if _, ok := failedVerification[referenceName]; ok {
+ continue
+ }
+
+ logEntry.ReferenceUpdates = append(logEntry.ReferenceUpdates, &gitalypb.LogEntry_ReferenceUpdate{
+ ReferenceName: []byte(referenceName),
+ NewOid: []byte(tips.NewOID),
+ })
+ }
+
+ nextLogIndex := mgr.appendedLogIndex + 1
+
+ if err := mgr.setKey(keyLogEntry(getRepositoryID(mgr.repository), nextLogIndex), &logEntry); err != nil {
+ return fmt.Errorf("set log entry: %w", err)
+ }
+
+ mgr.appendedLogIndex = nextLogIndex
+
+ return nil
+}
+
+// applyLogEntry reads a log entry at the given index and applies it to the repository.
+func (mgr *TransactionManager) applyLogEntry(ctx context.Context, logIndex LogIndex) (returnedErr error) {
+ var logEntry gitalypb.LogEntry
+ if err := mgr.readKey(keyLogEntry(getRepositoryID(mgr.repository), logIndex), &logEntry); err != nil {
+ return fmt.Errorf("read log entry: %w", err)
+ }
+
+ updater, err := updateref.New(ctx, mgr.repository, updateref.WithDisabledTransactions())
+ if err != nil {
+ return fmt.Errorf("start updateref: %w", err)
+ }
+ defer func() {
+ if err := updater.Cancel(); err != nil && returnedErr == nil {
+ returnedErr = fmt.Errorf("cancel updated: %w ", err)
+ }
+ }()
+
+ for _, update := range logEntry.ReferenceUpdates {
+ if err := updater.Update(git.ReferenceName(update.ReferenceName), git.ObjectID(update.NewOid), ""); err != nil {
+ return fmt.Errorf("update reference: %w", err)
+ }
+ }
+
+ if err := updater.Commit(); err != nil {
+ return fmt.Errorf("commit transaction: %w", err)
+ }
+
+ if err := mgr.setKey(keyAppliedLogIndex(getRepositoryID(mgr.repository)), &gitalypb.LogIndex{LogIndex: logIndex.Primitive()}); err != nil {
+ return fmt.Errorf("set applied log index: %w", err)
+ }
+
+ mgr.appliedLogIndex = logIndex
+
+ return nil
+}
+
+// setKey marshals and stores a given protocol buffer message into the database under the given key.
+func (mgr *TransactionManager) setKey(key []byte, value proto.Message) error {
+ marshaledValue, err := proto.Marshal(value)
+ if err != nil {
+ return fmt.Errorf("marshal value: %w", err)
+ }
+
+ writeBatch := mgr.db.NewWriteBatch()
+ defer writeBatch.Cancel()
+
+ if err := writeBatch.Set(key, marshaledValue); err != nil {
+ return fmt.Errorf("set: %w", err)
+ }
+
+ return writeBatch.Flush()
+}
+
+// readKey reads a key from the database and unmarshals its value in to the destination protocol
+// buffer message.
+func (mgr *TransactionManager) readKey(key []byte, destination proto.Message) error {
+ return mgr.db.View(func(txn *badger.Txn) error {
+ item, err := txn.Get(key)
+ if err != nil {
+ return fmt.Errorf("get: %w", err)
+ }
+
+ return item.Value(func(value []byte) error { return proto.Unmarshal(value, destination) })
+ })
+}
+
+// getRepositoryID returns a repository's ID. The ID should never change as it is used in the database
+// keys. Gitaly does not have a permanent ID to use yet so the repository's storage name and relative
+// path are used as a composite key.
+func getRepositoryID(repository repository.GitRepo) string {
+ return repository.GetStorageName() + ":" + repository.GetRelativePath()
+}
+
+// keyAppliedLogIndex returns the database key storing a repository's last applied log entry's index.
+func keyAppliedLogIndex(repositoryID string) []byte {
+ return []byte(fmt.Sprintf("repository/%s/log/index/applied", repositoryID))
+}
+
+// keyLogEntry returns the database key storing a repository's log entry at a given index.
+func keyLogEntry(repositoryID string, index LogIndex) []byte {
+ marshaledIndex := make([]byte, binary.Size(index))
+ binary.BigEndian.PutUint64(marshaledIndex, index.Primitive())
+ return []byte(fmt.Sprintf("%s%s", keyPrefixLogEntries(repositoryID), marshaledIndex))
+}
+
+// keyPrefixLogEntries returns the key prefix holding repository's write-ahead log entries.
+func keyPrefixLogEntries(repositoryID string) []byte {
+ return []byte(fmt.Sprintf("repository/%s/log/entry/", repositoryID))
+}
diff --git a/internal/gitaly/transaction_manager_test.go b/internal/gitaly/transaction_manager_test.go
new file mode 100644
index 000000000..d9ad0e484
--- /dev/null
+++ b/internal/gitaly/transaction_manager_test.go
@@ -0,0 +1,1302 @@
+package gitaly
+
+import (
+ "fmt"
+ "sync"
+ "testing"
+
+ "github.com/dgraph-io/badger/v3"
+ "github.com/stretchr/testify/require"
+ "gitlab.com/gitlab-org/gitaly/v15/internal/git"
+ "gitlab.com/gitlab-org/gitaly/v15/internal/git/catfile"
+ "gitlab.com/gitlab-org/gitaly/v15/internal/git/gittest"
+ "gitlab.com/gitlab-org/gitaly/v15/internal/git/localrepo"
+ "gitlab.com/gitlab-org/gitaly/v15/internal/gitaly/config"
+ "gitlab.com/gitlab-org/gitaly/v15/internal/testhelper"
+ "gitlab.com/gitlab-org/gitaly/v15/internal/testhelper/testcfg"
+ "gitlab.com/gitlab-org/gitaly/v15/proto/go/gitalypb"
+)
+
+func TestTransactionManager(t *testing.T) {
+ t.Parallel()
+
+ ctx := testhelper.Context(t)
+
+ // A clean repository is setup for each test. We build a repository ahead of the tests here once to
+ // get deterministic commit IDs, relative path and object hash we can use to build the declarative
+ // test cases.
+ relativePath := gittest.NewRepositoryName(t, true)
+ setupRepository := func(tb testing.TB) (*localrepo.Repo, git.ObjectID, git.ObjectID, git.ObjectID) {
+ tb.Helper()
+
+ cfg := testcfg.Build(tb)
+
+ repo, repoPath := gittest.CreateRepository(tb, ctx, cfg, gittest.CreateRepositoryConfig{
+ SkipCreationViaService: true,
+ RelativePath: relativePath,
+ })
+
+ rootCommitOID := gittest.WriteCommit(tb, cfg, repoPath, gittest.WithParents())
+ secondCommitOID := gittest.WriteCommit(tb, cfg, repoPath, gittest.WithParents(rootCommitOID))
+ thirdCommitOID := gittest.WriteCommit(tb, cfg, repoPath, gittest.WithParents(secondCommitOID))
+
+ cmdFactory, clean, err := git.NewExecCommandFactory(cfg)
+ require.NoError(tb, err)
+ tb.Cleanup(clean)
+
+ catfileCache := catfile.NewCache(cfg)
+ tb.Cleanup(catfileCache.Stop)
+
+ localRepo := localrepo.New(
+ config.NewLocator(cfg),
+ cmdFactory,
+ catfileCache,
+ repo,
+ )
+
+ return localRepo, rootCommitOID, secondCommitOID, thirdCommitOID
+ }
+
+ // Collect commit OIDs and the object has so we can define the test cases with them.
+ repo, rootCommitOID, secondCommitOID, thirdCommitOID := setupRepository(t)
+ objectHash, err := repo.ObjectHash(ctx)
+ require.NoError(t, err)
+
+ type restartManager struct {
+ // noLogApplication causes the manager to skip the write-ahead log application.
+ noLogApplication bool
+ }
+
+ // Step defines a single execution step in a test. Each test case can define multiple steps to setup exercise
+ // more complex behavior and to assert the state after each step.
+ type steps []struct {
+ // RestartManager restarts the TransactionManager at the beginning of the step if restartManager is set. The options
+ // in restartManager are applied to the manager and are effective in later steps until the manager is restarted again.
+ RestartManager *restartManager
+ // Transaction is the transaction that is proposed in this step.
+ Transaction Transaction
+ // ExpectedError is the error that is expected to be returned when proposing the transaction in this step.
+ ExpectedError error
+ // ExpectedReferences is the expected state of references at the end of this step.
+ ExpectedReferences []git.Reference
+ // ExpectedDatabase is the expected state of the database at the end of this step.
+ ExpectedDatabase DatabaseState
+ }
+
+ for _, tc := range []struct {
+ desc string
+ steps steps
+ }{
+ {
+ desc: "create reference",
+ steps: steps{
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: objectHash.ZeroOID, NewOID: rootCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{{Name: "refs/heads/main", Target: rootCommitOID.String()}},
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 1},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ {
+ desc: "create reference ignoring verification failure",
+ steps: steps{
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: objectHash.ZeroOID, NewOID: rootCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{{Name: "refs/heads/main", Target: rootCommitOID.String()}},
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 1},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ },
+ },
+ {
+ Transaction: Transaction{
+ ReferenceVerificationStrategy: ReferenceVerificationStrategySkipFailures,
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: objectHash.ZeroOID, NewOID: secondCommitOID},
+ "refs/heads/non-conflicting": {OldOID: objectHash.ZeroOID, NewOID: secondCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{
+ {Name: "refs/heads/main", Target: rootCommitOID.String()},
+ {Name: "refs/heads/non-conflicting", Target: secondCommitOID.String()},
+ },
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 2},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ string(keyLogEntry(getRepositoryID(repo), 2)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/non-conflicting"),
+ NewOid: []byte(secondCommitOID),
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ {
+ desc: "create reference that already exists",
+ steps: steps{
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: objectHash.ZeroOID, NewOID: rootCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{{Name: "refs/heads/main", Target: rootCommitOID.String()}},
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 1},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ },
+ },
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: objectHash.ZeroOID, NewOID: secondCommitOID},
+ "refs/heads/non-conflicting": {OldOID: objectHash.ZeroOID, NewOID: secondCommitOID},
+ },
+ },
+ ExpectedError: fmt.Errorf("verify references: %w", fmt.Errorf("expected old tip of %q to be %q but it was %q", "refs/heads/main", objectHash.ZeroOID, rootCommitOID)),
+ ExpectedReferences: []git.Reference{{Name: "refs/heads/main", Target: rootCommitOID.String()}},
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 1},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ {
+ desc: "create reference no-op",
+ steps: steps{
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: objectHash.ZeroOID, NewOID: rootCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{{Name: "refs/heads/main", Target: rootCommitOID.String()}},
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 1},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ },
+ },
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: objectHash.ZeroOID, NewOID: rootCommitOID},
+ },
+ },
+ ExpectedError: fmt.Errorf("verify references: %w", fmt.Errorf("expected old tip of %q to be %q but it was %q", "refs/heads/main", objectHash.ZeroOID, rootCommitOID)),
+ ExpectedReferences: []git.Reference{{Name: "refs/heads/main", Target: rootCommitOID.String()}},
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 1},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ {
+ desc: "update reference",
+ steps: steps{
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: objectHash.ZeroOID, NewOID: rootCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{{Name: "refs/heads/main", Target: rootCommitOID.String()}},
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 1},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ },
+ },
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: rootCommitOID, NewOID: secondCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{{Name: "refs/heads/main", Target: secondCommitOID.String()}},
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 2},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ string(keyLogEntry(getRepositoryID(repo), 2)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(secondCommitOID),
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ {
+ desc: "force update reference",
+ steps: steps{
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: objectHash.ZeroOID, NewOID: rootCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{{Name: "refs/heads/main", Target: rootCommitOID.String()}},
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 1},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ },
+ },
+ {
+ Transaction: Transaction{
+ ReferenceVerificationStrategy: ReferenceVerificationStrategySkip,
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {NewOID: secondCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{{Name: "refs/heads/main", Target: secondCommitOID.String()}},
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 2},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ string(keyLogEntry(getRepositoryID(repo), 2)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(secondCommitOID),
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ {
+ desc: "update reference ignoring verification failures",
+ steps: steps{
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: objectHash.ZeroOID, NewOID: rootCommitOID},
+ "refs/heads/non-conflicting": {OldOID: objectHash.ZeroOID, NewOID: rootCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{
+ {Name: "refs/heads/main", Target: rootCommitOID.String()},
+ {Name: "refs/heads/non-conflicting", Target: rootCommitOID.String()},
+ },
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 1},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ {
+ ReferenceName: []byte("refs/heads/non-conflicting"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ },
+ },
+ {
+ Transaction: Transaction{
+ ReferenceVerificationStrategy: ReferenceVerificationStrategySkipFailures,
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: secondCommitOID, NewOID: thirdCommitOID},
+ "refs/heads/non-conflicting": {OldOID: rootCommitOID, NewOID: thirdCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{
+ {Name: "refs/heads/main", Target: rootCommitOID.String()},
+ {Name: "refs/heads/non-conflicting", Target: thirdCommitOID.String()},
+ },
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 2},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ {
+ ReferenceName: []byte("refs/heads/non-conflicting"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ string(keyLogEntry(getRepositoryID(repo), 2)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/non-conflicting"),
+ NewOid: []byte(thirdCommitOID),
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ {
+ desc: "update reference with incorrect old tip",
+ steps: steps{
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: objectHash.ZeroOID, NewOID: rootCommitOID},
+ "refs/heads/non-conflicting": {OldOID: objectHash.ZeroOID, NewOID: rootCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{
+ {Name: "refs/heads/main", Target: rootCommitOID.String()},
+ {Name: "refs/heads/non-conflicting", Target: rootCommitOID.String()},
+ },
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 1},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ {
+ ReferenceName: []byte("refs/heads/non-conflicting"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ },
+ },
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: secondCommitOID, NewOID: thirdCommitOID},
+ "refs/heads/non-conflicting": {OldOID: rootCommitOID, NewOID: thirdCommitOID},
+ },
+ },
+ ExpectedError: fmt.Errorf("verify references: %w", fmt.Errorf("expected old tip of %q to be %q but it was %q", "refs/heads/main", secondCommitOID, rootCommitOID)),
+ ExpectedReferences: []git.Reference{
+ {Name: "refs/heads/main", Target: rootCommitOID.String()},
+ {Name: "refs/heads/non-conflicting", Target: rootCommitOID.String()},
+ },
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 1},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ {
+ ReferenceName: []byte("refs/heads/non-conflicting"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ {
+ desc: "update non-existent reference",
+ steps: steps{
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: secondCommitOID, NewOID: thirdCommitOID},
+ },
+ },
+ ExpectedError: fmt.Errorf("verify references: %w", fmt.Errorf("expected old tip of %q to be %q but it was %q", "refs/heads/main", secondCommitOID, objectHash.ZeroOID)),
+ },
+ },
+ },
+ {
+ desc: "update reference no-op",
+ steps: steps{
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: objectHash.ZeroOID, NewOID: rootCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{{Name: "refs/heads/main", Target: rootCommitOID.String()}},
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 1},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ },
+ },
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: rootCommitOID, NewOID: rootCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{{Name: "refs/heads/main", Target: rootCommitOID.String()}},
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 2},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ string(keyLogEntry(getRepositoryID(repo), 2)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ {
+ desc: "delete reference",
+ steps: steps{
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: objectHash.ZeroOID, NewOID: rootCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{{Name: "refs/heads/main", Target: rootCommitOID.String()}},
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 1},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ },
+ },
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: rootCommitOID, NewOID: objectHash.ZeroOID},
+ },
+ },
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 2},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ string(keyLogEntry(getRepositoryID(repo), 2)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(objectHash.ZeroOID),
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ {
+ desc: "force delete reference",
+ steps: steps{
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: objectHash.ZeroOID, NewOID: rootCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{{Name: "refs/heads/main", Target: rootCommitOID.String()}},
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 1},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ },
+ },
+ {
+ Transaction: Transaction{
+ ReferenceVerificationStrategy: ReferenceVerificationStrategySkip,
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {NewOID: objectHash.ZeroOID},
+ },
+ },
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 2},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ string(keyLogEntry(getRepositoryID(repo), 2)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(objectHash.ZeroOID),
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ {
+ desc: "delete reference ignoring verification failures",
+ steps: steps{
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: objectHash.ZeroOID, NewOID: rootCommitOID},
+ "refs/heads/non-conflicting": {OldOID: objectHash.ZeroOID, NewOID: rootCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{
+ {Name: "refs/heads/main", Target: rootCommitOID.String()},
+ {Name: "refs/heads/non-conflicting", Target: rootCommitOID.String()},
+ },
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 1},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ {
+ ReferenceName: []byte("refs/heads/non-conflicting"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ },
+ },
+ {
+ Transaction: Transaction{
+ ReferenceVerificationStrategy: ReferenceVerificationStrategySkipFailures,
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: secondCommitOID, NewOID: objectHash.ZeroOID},
+ "refs/heads/non-conflicting": {OldOID: rootCommitOID, NewOID: objectHash.ZeroOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{{Name: "refs/heads/main", Target: rootCommitOID.String()}},
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 2},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ {
+ ReferenceName: []byte("refs/heads/non-conflicting"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ string(keyLogEntry(getRepositoryID(repo), 2)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/non-conflicting"),
+ NewOid: []byte(objectHash.ZeroOID),
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ {
+ desc: "delete reference with incorrect old tip",
+ steps: steps{
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: objectHash.ZeroOID, NewOID: rootCommitOID},
+ "refs/heads/non-conflicting": {OldOID: objectHash.ZeroOID, NewOID: rootCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{
+ {Name: "refs/heads/main", Target: rootCommitOID.String()},
+ {Name: "refs/heads/non-conflicting", Target: rootCommitOID.String()},
+ },
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 1},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ {
+ ReferenceName: []byte("refs/heads/non-conflicting"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ },
+ },
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: secondCommitOID, NewOID: objectHash.ZeroOID},
+ "refs/heads/non-conflicting": {OldOID: rootCommitOID, NewOID: objectHash.ZeroOID},
+ },
+ },
+ ExpectedError: fmt.Errorf("verify references: %w", fmt.Errorf("expected old tip of %q to be %q but it was %q", "refs/heads/main", secondCommitOID, rootCommitOID)),
+ ExpectedReferences: []git.Reference{
+ {Name: "refs/heads/main", Target: rootCommitOID.String()},
+ {Name: "refs/heads/non-conflicting", Target: rootCommitOID.String()},
+ },
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 1},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ {
+ ReferenceName: []byte("refs/heads/non-conflicting"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ {
+ desc: "delete non-existent reference",
+ steps: steps{
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: rootCommitOID, NewOID: objectHash.ZeroOID},
+ },
+ },
+ ExpectedError: fmt.Errorf("verify references: %w", fmt.Errorf("expected old tip of %q to be %q but it was %q", "refs/heads/main", rootCommitOID, objectHash.ZeroOID)),
+ },
+ },
+ },
+ {
+ desc: "delete reference no-op",
+ steps: steps{
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: objectHash.ZeroOID, NewOID: objectHash.ZeroOID},
+ },
+ },
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 1},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(objectHash.ZeroOID),
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ {
+ desc: "continues processing after reference verification failure",
+ steps: steps{
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: rootCommitOID, NewOID: secondCommitOID},
+ },
+ },
+ ExpectedError: fmt.Errorf("verify references: %w", fmt.Errorf("expected old tip of %q to be %q but it was %q", "refs/heads/main", rootCommitOID, objectHash.ZeroOID)),
+ },
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: objectHash.ZeroOID, NewOID: secondCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{{Name: "refs/heads/main", Target: secondCommitOID.String()}},
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 1},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(secondCommitOID),
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ {
+ desc: "continues processing after a restart",
+ steps: steps{
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: objectHash.ZeroOID, NewOID: rootCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{{Name: "refs/heads/main", Target: rootCommitOID.String()}},
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 1},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ },
+ },
+ {
+ RestartManager: &restartManager{},
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: rootCommitOID, NewOID: secondCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{{Name: "refs/heads/main", Target: secondCommitOID.String()}},
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 2},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ string(keyLogEntry(getRepositoryID(repo), 2)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(secondCommitOID),
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ {
+ desc: "continues processing after restarting after a reference verification failure",
+ steps: steps{
+ {
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: rootCommitOID, NewOID: secondCommitOID},
+ },
+ },
+ ExpectedError: fmt.Errorf("verify references: %w", fmt.Errorf("expected old tip of %q to be %q but it was %q", "refs/heads/main", rootCommitOID, objectHash.ZeroOID)),
+ },
+ {
+ RestartManager: &restartManager{},
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: objectHash.ZeroOID, NewOID: secondCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{{Name: "refs/heads/main", Target: secondCommitOID.String()}},
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 1},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(secondCommitOID),
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ {
+ desc: "recovers from the write-ahead log on start up",
+ steps: steps{
+ {
+ RestartManager: &restartManager{noLogApplication: true},
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: objectHash.ZeroOID, NewOID: rootCommitOID},
+ },
+ },
+ ExpectedDatabase: DatabaseState{
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ },
+ },
+ {
+ Transaction: Transaction{
+ ReferenceVerificationStrategy: ReferenceVerificationStrategySkip,
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {NewOID: secondCommitOID},
+ },
+ },
+ ExpectedDatabase: DatabaseState{
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ string(keyLogEntry(getRepositoryID(repo), 2)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(secondCommitOID),
+ },
+ },
+ },
+ },
+ },
+ {
+ RestartManager: &restartManager{},
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: secondCommitOID, NewOID: thirdCommitOID},
+ },
+ },
+ ExpectedReferences: []git.Reference{{Name: "refs/heads/main", Target: thirdCommitOID.String()}},
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 3},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ string(keyLogEntry(getRepositoryID(repo), 2)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(secondCommitOID),
+ },
+ },
+ },
+ string(keyLogEntry(getRepositoryID(repo), 3)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(thirdCommitOID),
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ {
+ desc: "reference verification fails after recovering logged writes",
+ steps: steps{
+ {
+ RestartManager: &restartManager{noLogApplication: true},
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: objectHash.ZeroOID, NewOID: rootCommitOID},
+ },
+ },
+ ExpectedDatabase: DatabaseState{
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ },
+ },
+ {
+ RestartManager: &restartManager{},
+ Transaction: Transaction{
+ ReferenceUpdates: ReferenceUpdates{
+ "refs/heads/main": {OldOID: secondCommitOID, NewOID: rootCommitOID},
+ },
+ },
+ ExpectedError: fmt.Errorf("verify references: %w", fmt.Errorf("expected old tip of %q to be %q but it was %q", "refs/heads/main", secondCommitOID, rootCommitOID)),
+ ExpectedReferences: []git.Reference{{Name: "refs/heads/main", Target: rootCommitOID.String()}},
+ ExpectedDatabase: DatabaseState{
+ string(keyAppliedLogIndex(getRepositoryID(repo))): &gitalypb.LogIndex{LogIndex: 1},
+ string(keyLogEntry(getRepositoryID(repo), 1)): &gitalypb.LogEntry{
+ ReferenceUpdates: []*gitalypb.LogEntry_ReferenceUpdate{
+ {
+ ReferenceName: []byte("refs/heads/main"),
+ NewOid: []byte(rootCommitOID),
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ } {
+ tc := tc
+ t.Run(tc.desc, func(t *testing.T) {
+ t.Parallel()
+
+ // Setup the repository with the exact same state as what was used to build the test cases.
+ repo, _, _, _ := setupRepository(t)
+ RequireReferences(t, ctx, repo, []git.Reference{})
+
+ databaseDir := t.TempDir()
+
+ var (
+ // managerErr is used to synchronize the startManager and stopManager functions to
+ // ensure there is only ever a single manager running and to return the error from Run.
+ managerErr = make(chan error)
+ // database is the currently open database handle.
+ database *badger.DB
+ // transactionManager is the currently running TransactionManager
+ transactionManager *TransactionManager
+ )
+
+ // stopManager stops the manager and closes the database. It waits until the manager's Run
+ // method has exited and the database handle is closed.
+ stopManager := func() {
+ transactionManager.Stop()
+ require.NoError(t, <-managerErr)
+ require.NoError(t, database.Close())
+ }
+
+ // startManager starts fresh manager and applies the testing knobs to it. Database is always
+ // reinstantiated but the state persists between steps within a test.
+ startManager := func(restartManager *restartManager) {
+ database = retryOpenDatabase(t, databaseDir)
+ transactionManager = NewTransactionManager(database, repo)
+ transactionManager.testingKnobs.noLogApplication = restartManager.noLogApplication
+ go func() { managerErr <- transactionManager.Run() }()
+ }
+
+ // Start the manager by default with all testing knobs disabled.
+ defer stopManager()
+ startManager(&restartManager{})
+
+ for _, step := range tc.steps {
+ // Restart the manager if the test step indicates the need to do so.
+ if step.RestartManager != nil {
+ stopManager()
+ startManager(step.RestartManager)
+ }
+
+ require.Equal(t, step.ExpectedError, transactionManager.Propose(ctx, step.Transaction))
+ RequireReferences(t, ctx, repo, step.ExpectedReferences)
+ RequireDatabase(t, ctx, database, step.ExpectedDatabase)
+ }
+ })
+ }
+}
+
+// retryOpenDatabase retries database opening until it succeeds or fails due to an error not related to flock(2).
+//
+// Badger uses flock(2) on the database dircetory to ensure it has exclusive access to it. When the database is
+// closed, Badger doesn't explicitly unlock with `LOCK_UN` but just closes the file. This doesn't seem to always
+// release the lock immediately and the following sequence may happen:
+//
+// OpenDatabase(databaseDir)
+// CloseDatabase()
+// OpenDatabase(databaseDir) -- Errors due to failing to grab the exclusive lock
+//
+// The second OpenDatabase call will succeed after a while if retried. This seems to hint at a race in closing
+// the file that is holding the lock.
+func retryOpenDatabase(tb testing.TB, databaseDir string) *badger.DB {
+ for {
+ database, err := OpenDatabase(databaseDir)
+ if err != nil {
+ if err.Error() == fmt.Sprintf("Cannot acquire directory lock on %q. Another process is using this Badger database. error: resource temporarily unavailable", databaseDir) {
+ continue
+ }
+
+ require.NoError(tb, err)
+ }
+
+ return database
+ }
+}
+
+// BenchmarkTransactionManager benchmarks the transaction throughput of the TransactionManager at various levels
+// of concurrency and transaction sizes.
+func BenchmarkTransactionManager(b *testing.B) {
+ for _, tc := range []struct {
+ // numberOfRepositories sets the number of repositories that are updating the references. Each repository has
+ // its own TransactionManager. Setting this to 1 allows for testing throughput of a single repository while
+ // setting this higher allows for testing parallel throughput of multiple repositories. This mostly serves
+ // to determine the impact of the shared resources such as the database.
+ numberOfRepositories int
+ // concurrentUpdaters sets the number of goroutines that are calling Propose for a repository. Each of the
+ // updaters work on their own references so they don't block each other. Setting this to 1 allows for testing
+ // sequential update throughput of a repository. Setting this higher allows for testing reference update
+ // throughput when multiple references are being updated concurrently.
+ concurrentUpdaters int
+ // transactionSize sets the number of references that are updated in each transaction.
+ transactionSize int
+ }{
+ {
+ numberOfRepositories: 1,
+ concurrentUpdaters: 1,
+ transactionSize: 1,
+ },
+ {
+ numberOfRepositories: 1,
+ concurrentUpdaters: 10,
+ transactionSize: 1,
+ },
+ {
+ numberOfRepositories: 10,
+ concurrentUpdaters: 1,
+ transactionSize: 1,
+ },
+ {
+ numberOfRepositories: 10,
+ concurrentUpdaters: 10,
+ transactionSize: 1,
+ },
+ } {
+ desc := fmt.Sprintf("%d repositories/%d updaters/%d transaction size",
+ tc.numberOfRepositories,
+ tc.concurrentUpdaters,
+ tc.transactionSize,
+ )
+ b.Run(desc, func(b *testing.B) {
+ ctx := testhelper.Context(b)
+
+ cfg := testcfg.Build(b)
+
+ cmdFactory, clean, err := git.NewExecCommandFactory(cfg)
+ require.NoError(b, err)
+ defer clean()
+
+ cache := catfile.NewCache(cfg)
+ defer cache.Stop()
+
+ db, err := OpenDatabase(b.TempDir())
+ require.NoError(b, err)
+ defer func() { require.NoError(b, db.Close()) }()
+
+ var (
+ // managerWG records the running TransactionManager.Run goroutines.
+ managerWG sync.WaitGroup
+ managers []*TransactionManager
+
+ // The references are updated back and forth between commit1 and commit2.
+ commit1 git.ObjectID
+ commit2 git.ObjectID
+ )
+
+ // Set up the repositories and start their TransactionManagers.
+ for i := 0; i < tc.numberOfRepositories; i++ {
+ repo, repoPath := gittest.CreateRepository(b, ctx, cfg, gittest.CreateRepositoryConfig{
+ SkipCreationViaService: true,
+ })
+
+ localRepo := localrepo.New(
+ config.NewLocator(cfg),
+ cmdFactory,
+ cache,
+ repo,
+ )
+
+ // Set up two commits that the updaters update their references back and forth.
+ commit1 = gittest.WriteCommit(b, cfg, repoPath, gittest.WithParents())
+ commit2 = gittest.WriteCommit(b, cfg, repoPath, gittest.WithParents(commit1))
+
+ manager := NewTransactionManager(db, localRepo)
+ managers = append(managers, manager)
+
+ managerWG.Add(1)
+ go func() {
+ defer managerWG.Done()
+ require.NoError(b, manager.Run())
+ }()
+ }
+
+ // getReferenceName builds a ReferenceUpdates with unique branches for the updater.
+ getReferenceUpdates := func(updaterID int, old, new git.ObjectID) ReferenceUpdates {
+ referenceUpdates := make(ReferenceUpdates, tc.transactionSize)
+ for i := 0; i < tc.transactionSize; i++ {
+ referenceUpdates[git.ReferenceName(fmt.Sprintf("refs/heads/branch-%d-%d", updaterID, i))] = ReferenceTips{
+ OldOID: old,
+ NewOID: new,
+ }
+ }
+
+ return referenceUpdates
+ }
+
+ // proposeWG tracks the number of of on going Propose calls.
+ var proposeWG sync.WaitGroup
+ transactionChan := make(chan struct{})
+
+ for _, manager := range managers {
+ manager := manager
+ for i := 0; i < tc.concurrentUpdaters; i++ {
+
+ // Build the reference updates that this updater will go back and forth with.
+ currentReferences := getReferenceUpdates(i, commit1, commit2)
+ nextReferences := getReferenceUpdates(i, commit2, commit1)
+
+ // Setup the starting state so the references start at the expected old tip.
+ require.NoError(b, manager.Propose(ctx, Transaction{
+ ReferenceVerificationStrategy: ReferenceVerificationStrategySkip,
+ ReferenceUpdates: currentReferences,
+ }))
+
+ proposeWG.Add(1)
+ go func() {
+ defer proposeWG.Done()
+
+ for range transactionChan {
+ require.NoError(b, manager.Propose(ctx, Transaction{ReferenceUpdates: nextReferences}))
+ currentReferences, nextReferences = nextReferences, currentReferences
+ }
+ }()
+ }
+ }
+
+ b.ReportAllocs()
+ b.ResetTimer()
+ for n := 0; n < b.N; n++ {
+ transactionChan <- struct{}{}
+ }
+ close(transactionChan)
+
+ proposeWG.Wait()
+ b.StopTimer()
+
+ for _, manager := range managers {
+ manager.Stop()
+ }
+
+ managerWG.Wait()
+ })
+ }
+}
diff --git a/internal/testhelper/leakage.go b/internal/testhelper/leakage.go
index ffec6c7ae..5dc9cae20 100644
--- a/internal/testhelper/leakage.go
+++ b/internal/testhelper/leakage.go
@@ -16,6 +16,10 @@ import (
// mustHaveNoGoroutines panics if it finds any Goroutines running.
func mustHaveNoGoroutines() {
if err := goleak.Find(
+ // BadgerDB depends on Ristretto which uses glog for logging. glog initializes
+ // on import a log flushing goroutine that keeps running in the background.
+ // Ignore this goroutine as there is no way to stop it.
+ goleak.IgnoreTopFunction("github.com/golang/glog.(*loggingT).flushDaemon"),
// opencensus has a "defaultWorker" which is started by the package's
// `init()` function. There is no way to stop this worker, so it will leak
// whenever we import the package.
diff --git a/proto/go/gitalypb/log.pb.go b/proto/go/gitalypb/log.pb.go
new file mode 100644
index 000000000..11b96bf13
--- /dev/null
+++ b/proto/go/gitalypb/log.pb.go
@@ -0,0 +1,303 @@
+// Code generated by protoc-gen-go. DO NOT EDIT.
+// versions:
+// protoc-gen-go v1.28.1
+// protoc v3.21.7
+// source: log.proto
+
+package gitalypb
+
+import (
+ protoreflect "google.golang.org/protobuf/reflect/protoreflect"
+ protoimpl "google.golang.org/protobuf/runtime/protoimpl"
+ reflect "reflect"
+ sync "sync"
+)
+
+const (
+ // Verify that this generated code is sufficiently up-to-date.
+ _ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)
+ // Verify that runtime/protoimpl is sufficiently up-to-date.
+ _ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
+)
+
+// LogEntry is a single entry in a repository's write-ahead log.
+//
+// Schema for :
+// - `repository/<repository_id>/log/entry/<log_index>`.
+type LogEntry struct {
+ state protoimpl.MessageState
+ sizeCache protoimpl.SizeCache
+ unknownFields protoimpl.UnknownFields
+
+ // reference_updates contains the reference updates this log
+ // entry records. The logged reference updates have already passed
+ // through verification and are applied without any further checks.
+ ReferenceUpdates []*LogEntry_ReferenceUpdate `protobuf:"bytes,1,rep,name=reference_updates,json=referenceUpdates,proto3" json:"reference_updates,omitempty"`
+}
+
+func (x *LogEntry) Reset() {
+ *x = LogEntry{}
+ if protoimpl.UnsafeEnabled {
+ mi := &file_log_proto_msgTypes[0]
+ ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
+ ms.StoreMessageInfo(mi)
+ }
+}
+
+func (x *LogEntry) String() string {
+ return protoimpl.X.MessageStringOf(x)
+}
+
+func (*LogEntry) ProtoMessage() {}
+
+func (x *LogEntry) ProtoReflect() protoreflect.Message {
+ mi := &file_log_proto_msgTypes[0]
+ if protoimpl.UnsafeEnabled && x != nil {
+ ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
+ if ms.LoadMessageInfo() == nil {
+ ms.StoreMessageInfo(mi)
+ }
+ return ms
+ }
+ return mi.MessageOf(x)
+}
+
+// Deprecated: Use LogEntry.ProtoReflect.Descriptor instead.
+func (*LogEntry) Descriptor() ([]byte, []int) {
+ return file_log_proto_rawDescGZIP(), []int{0}
+}
+
+func (x *LogEntry) GetReferenceUpdates() []*LogEntry_ReferenceUpdate {
+ if x != nil {
+ return x.ReferenceUpdates
+ }
+ return nil
+}
+
+// LogIndex serializes a log index. It's used for storing a repository's
+// applied log index in the database.
+//
+// Schema for:
+// - `repository/<repository_id>/log/index/applied`
+type LogIndex struct {
+ state protoimpl.MessageState
+ sizeCache protoimpl.SizeCache
+ unknownFields protoimpl.UnknownFields
+
+ // log_index is an index pointing to a position in the log.
+ LogIndex uint64 `protobuf:"varint,1,opt,name=log_index,json=logIndex,proto3" json:"log_index,omitempty"`
+}
+
+func (x *LogIndex) Reset() {
+ *x = LogIndex{}
+ if protoimpl.UnsafeEnabled {
+ mi := &file_log_proto_msgTypes[1]
+ ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
+ ms.StoreMessageInfo(mi)
+ }
+}
+
+func (x *LogIndex) String() string {
+ return protoimpl.X.MessageStringOf(x)
+}
+
+func (*LogIndex) ProtoMessage() {}
+
+func (x *LogIndex) ProtoReflect() protoreflect.Message {
+ mi := &file_log_proto_msgTypes[1]
+ if protoimpl.UnsafeEnabled && x != nil {
+ ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
+ if ms.LoadMessageInfo() == nil {
+ ms.StoreMessageInfo(mi)
+ }
+ return ms
+ }
+ return mi.MessageOf(x)
+}
+
+// Deprecated: Use LogIndex.ProtoReflect.Descriptor instead.
+func (*LogIndex) Descriptor() ([]byte, []int) {
+ return file_log_proto_rawDescGZIP(), []int{1}
+}
+
+func (x *LogIndex) GetLogIndex() uint64 {
+ if x != nil {
+ return x.LogIndex
+ }
+ return 0
+}
+
+// ReferenceUpdate models a single reference update.
+type LogEntry_ReferenceUpdate struct {
+ state protoimpl.MessageState
+ sizeCache protoimpl.SizeCache
+ unknownFields protoimpl.UnknownFields
+
+ // reference_name is the fully qualified name of the reference
+ // to update.
+ ReferenceName []byte `protobuf:"bytes,1,opt,name=reference_name,json=referenceName,proto3" json:"reference_name,omitempty"`
+ // new_oid is the new oid to point the reference to. Deletions
+ // are denoted as the SHA1 or SHA256 zero OID depending on the
+ // hash type used in the repository.
+ NewOid []byte `protobuf:"bytes,2,opt,name=new_oid,json=newOid,proto3" json:"new_oid,omitempty"`
+}
+
+func (x *LogEntry_ReferenceUpdate) Reset() {
+ *x = LogEntry_ReferenceUpdate{}
+ if protoimpl.UnsafeEnabled {
+ mi := &file_log_proto_msgTypes[2]
+ ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
+ ms.StoreMessageInfo(mi)
+ }
+}
+
+func (x *LogEntry_ReferenceUpdate) String() string {
+ return protoimpl.X.MessageStringOf(x)
+}
+
+func (*LogEntry_ReferenceUpdate) ProtoMessage() {}
+
+func (x *LogEntry_ReferenceUpdate) ProtoReflect() protoreflect.Message {
+ mi := &file_log_proto_msgTypes[2]
+ if protoimpl.UnsafeEnabled && x != nil {
+ ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
+ if ms.LoadMessageInfo() == nil {
+ ms.StoreMessageInfo(mi)
+ }
+ return ms
+ }
+ return mi.MessageOf(x)
+}
+
+// Deprecated: Use LogEntry_ReferenceUpdate.ProtoReflect.Descriptor instead.
+func (*LogEntry_ReferenceUpdate) Descriptor() ([]byte, []int) {
+ return file_log_proto_rawDescGZIP(), []int{0, 0}
+}
+
+func (x *LogEntry_ReferenceUpdate) GetReferenceName() []byte {
+ if x != nil {
+ return x.ReferenceName
+ }
+ return nil
+}
+
+func (x *LogEntry_ReferenceUpdate) GetNewOid() []byte {
+ if x != nil {
+ return x.NewOid
+ }
+ return nil
+}
+
+var File_log_proto protoreflect.FileDescriptor
+
+var file_log_proto_rawDesc = []byte{
+ 0x0a, 0x09, 0x6c, 0x6f, 0x67, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x06, 0x67, 0x69, 0x74,
+ 0x61, 0x6c, 0x79, 0x22, 0xac, 0x01, 0x0a, 0x08, 0x4c, 0x6f, 0x67, 0x45, 0x6e, 0x74, 0x72, 0x79,
+ 0x12, 0x4d, 0x0a, 0x11, 0x72, 0x65, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x5f, 0x75, 0x70,
+ 0x64, 0x61, 0x74, 0x65, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x20, 0x2e, 0x67, 0x69,
+ 0x74, 0x61, 0x6c, 0x79, 0x2e, 0x4c, 0x6f, 0x67, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x2e, 0x52, 0x65,
+ 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x52, 0x10, 0x72,
+ 0x65, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x73, 0x1a,
+ 0x51, 0x0a, 0x0f, 0x52, 0x65, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x55, 0x70, 0x64, 0x61,
+ 0x74, 0x65, 0x12, 0x25, 0x0a, 0x0e, 0x72, 0x65, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x5f,
+ 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x0d, 0x72, 0x65, 0x66, 0x65,
+ 0x72, 0x65, 0x6e, 0x63, 0x65, 0x4e, 0x61, 0x6d, 0x65, 0x12, 0x17, 0x0a, 0x07, 0x6e, 0x65, 0x77,
+ 0x5f, 0x6f, 0x69, 0x64, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x06, 0x6e, 0x65, 0x77, 0x4f,
+ 0x69, 0x64, 0x22, 0x27, 0x0a, 0x08, 0x4c, 0x6f, 0x67, 0x49, 0x6e, 0x64, 0x65, 0x78, 0x12, 0x1b,
+ 0x0a, 0x09, 0x6c, 0x6f, 0x67, 0x5f, 0x69, 0x6e, 0x64, 0x65, 0x78, 0x18, 0x01, 0x20, 0x01, 0x28,
+ 0x04, 0x52, 0x08, 0x6c, 0x6f, 0x67, 0x49, 0x6e, 0x64, 0x65, 0x78, 0x42, 0x34, 0x5a, 0x32, 0x67,
+ 0x69, 0x74, 0x6c, 0x61, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x67, 0x69, 0x74, 0x6c, 0x61, 0x62,
+ 0x2d, 0x6f, 0x72, 0x67, 0x2f, 0x67, 0x69, 0x74, 0x61, 0x6c, 0x79, 0x2f, 0x76, 0x31, 0x35, 0x2f,
+ 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2f, 0x67, 0x6f, 0x2f, 0x67, 0x69, 0x74, 0x61, 0x6c, 0x79, 0x70,
+ 0x62, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
+}
+
+var (
+ file_log_proto_rawDescOnce sync.Once
+ file_log_proto_rawDescData = file_log_proto_rawDesc
+)
+
+func file_log_proto_rawDescGZIP() []byte {
+ file_log_proto_rawDescOnce.Do(func() {
+ file_log_proto_rawDescData = protoimpl.X.CompressGZIP(file_log_proto_rawDescData)
+ })
+ return file_log_proto_rawDescData
+}
+
+var file_log_proto_msgTypes = make([]protoimpl.MessageInfo, 3)
+var file_log_proto_goTypes = []interface{}{
+ (*LogEntry)(nil), // 0: gitaly.LogEntry
+ (*LogIndex)(nil), // 1: gitaly.LogIndex
+ (*LogEntry_ReferenceUpdate)(nil), // 2: gitaly.LogEntry.ReferenceUpdate
+}
+var file_log_proto_depIdxs = []int32{
+ 2, // 0: gitaly.LogEntry.reference_updates:type_name -> gitaly.LogEntry.ReferenceUpdate
+ 1, // [1:1] is the sub-list for method output_type
+ 1, // [1:1] is the sub-list for method input_type
+ 1, // [1:1] is the sub-list for extension type_name
+ 1, // [1:1] is the sub-list for extension extendee
+ 0, // [0:1] is the sub-list for field type_name
+}
+
+func init() { file_log_proto_init() }
+func file_log_proto_init() {
+ if File_log_proto != nil {
+ return
+ }
+ if !protoimpl.UnsafeEnabled {
+ file_log_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} {
+ switch v := v.(*LogEntry); i {
+ case 0:
+ return &v.state
+ case 1:
+ return &v.sizeCache
+ case 2:
+ return &v.unknownFields
+ default:
+ return nil
+ }
+ }
+ file_log_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} {
+ switch v := v.(*LogIndex); i {
+ case 0:
+ return &v.state
+ case 1:
+ return &v.sizeCache
+ case 2:
+ return &v.unknownFields
+ default:
+ return nil
+ }
+ }
+ file_log_proto_msgTypes[2].Exporter = func(v interface{}, i int) interface{} {
+ switch v := v.(*LogEntry_ReferenceUpdate); i {
+ case 0:
+ return &v.state
+ case 1:
+ return &v.sizeCache
+ case 2:
+ return &v.unknownFields
+ default:
+ return nil
+ }
+ }
+ }
+ type x struct{}
+ out := protoimpl.TypeBuilder{
+ File: protoimpl.DescBuilder{
+ GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
+ RawDescriptor: file_log_proto_rawDesc,
+ NumEnums: 0,
+ NumMessages: 3,
+ NumExtensions: 0,
+ NumServices: 0,
+ },
+ GoTypes: file_log_proto_goTypes,
+ DependencyIndexes: file_log_proto_depIdxs,
+ MessageInfos: file_log_proto_msgTypes,
+ }.Build()
+ File_log_proto = out.File
+ file_log_proto_rawDesc = nil
+ file_log_proto_goTypes = nil
+ file_log_proto_depIdxs = nil
+}
diff --git a/proto/go/gitalypb/protolist.go b/proto/go/gitalypb/protolist.go
index cec39714e..ed2f064f9 100644
--- a/proto/go/gitalypb/protolist.go
+++ b/proto/go/gitalypb/protolist.go
@@ -13,6 +13,7 @@ var GitalyProtos = []string{
"hook.proto",
"internal.proto",
"lint.proto",
+ "log.proto",
"namespace.proto",
"objectpool.proto",
"operations.proto",
diff --git a/proto/log.proto b/proto/log.proto
new file mode 100644
index 000000000..6231c1b5e
--- /dev/null
+++ b/proto/log.proto
@@ -0,0 +1,37 @@
+syntax = "proto3";
+
+package gitaly;
+
+option go_package = "gitlab.com/gitlab-org/gitaly/v15/proto/go/gitalypb";
+
+// LogEntry is a single entry in a repository's write-ahead log.
+//
+// Schema for :
+// - `repository/<repository_id>/log/entry/<log_index>`.
+message LogEntry {
+ // ReferenceUpdate models a single reference update.
+ message ReferenceUpdate {
+ // reference_name is the fully qualified name of the reference
+ // to update.
+ bytes reference_name = 1;
+ // new_oid is the new oid to point the reference to. Deletions
+ // are denoted as the SHA1 or SHA256 zero OID depending on the
+ // hash type used in the repository.
+ bytes new_oid = 2;
+ }
+
+ // reference_updates contains the reference updates this log
+ // entry records. The logged reference updates have already passed
+ // through verification and are applied without any further checks.
+ repeated ReferenceUpdate reference_updates = 1;
+}
+
+// LogIndex serializes a log index. It's used for storing a repository's
+// applied log index in the database.
+//
+// Schema for:
+// - `repository/<repository_id>/log/index/applied`
+message LogIndex {
+ // log_index is an index pointing to a position in the log.
+ uint64 log_index = 1;
+}