diff options
| author | diosmosis <diosmosis@users.noreply.github.com> | 2020-05-07 19:11:06 +0300 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2020-05-07 19:11:06 +0300 |
| commit | 6d744db7f3f7a55fc4bb1ca573e937f0832c642b (patch) | |
| tree | 148d845d4ed521641f66f6bb2640697b14da6b26 /core/Db | |
| parent | d829c23496c9f1c53bd4144009d67f8a287d97e4 (diff) | |
Rewrite cron archiving process for easier maintenance and performance (#15499)
* Adding initial new code for cron archive rewrite.
* first pass at removing unused CronArchive code.
* unfinished commit
* fill out archiveinvalidator code
* getting some tests to pass
* unfinished commit
* fixing part of test
* Another test fix.
* another sql change
* fix broken merge or something else that went wrong
* Couple more fixes and extra logs.
* Fixing enough issues to get core archive command to run completely.
* Fix and log change.
* Fixed more segment/test related issues for CronArchiveTest. Includes optimization for no visits for period + segment process from handling.
* another optimization and possible build fix
* no visit optimization
* test fix
* Implement archiving_custom_ranges logic w/ queue based implementation
* fixes to get archivecrontest to work
* add logic to invalidate today period
* fix optimization and some tests
* Fixing more tests.
* Fixing more tests
* debug travis failure
* more test fixes
* more test fixes, removing more unneeded code, handling some TODOs
* Handle more TODOs including creating ArchiveFilter class for some cli options.
* tests and todos
* idarchives are specific to table + start on archivefilter tests
* one test
* more TODOs and tests
* more tests and todo taken care of
* handle more todos
* fixing more tests
* fix comment
* make sure autoarchiving is enabled for segments when cron archive picks them up
* Fixing test.
* apply more pr feedback
* order by date1 asc
* quick refactor
* use batch insert instead of createDummyArchives
* apply rest of pr feedback
* add removed events, add new test, fix an issue (when deleting idarchives older than do not lump all segments together).
* re-add fixed/shared siteids
* fix tests
* incomplete commit
* Insert archive entries into archive_invalidations table.
* Use invalidations table in core:archive and get ArchiveCronTest to pass.
* fixing some tests
* debugging travis
* fix more tests & remove DONE_IN_PROGRESS which is no longer used.
* fix more tests
* apply review feedback
* fix tests
* try fixing alltests build
* try again
* try again
* Move archive_invalidations update to new beta since omnifixture was updated w/o it.
* Another fix.
Diffstat (limited to 'core/Db')
| -rw-r--r-- | core/Db/Schema/Mysql.php | 15 |
1 files changed, 15 insertions, 0 deletions
diff --git a/core/Db/Schema/Mysql.php b/core/Db/Schema/Mysql.php index be739fc509..e26d0e68f8 100644 --- a/core/Db/Schema/Mysql.php +++ b/core/Db/Schema/Mysql.php @@ -299,6 +299,21 @@ class Mysql implements SchemaInterface ) ENGINE=$engine DEFAULT CHARSET=utf8 ", + 'archive_invalidations' => "CREATE TABLE `{$prefixTables}archive_invalidations` ( + idinvalidation BIGINT UNSIGNED NOT NULL AUTO_INCREMENT, + idarchive INTEGER UNSIGNED NULL, + name VARCHAR(255) NOT NULL, + idsite INTEGER UNSIGNED NOT NULL, + date1 DATE NOT NULL, + date2 DATE NOT NULL, + period TINYINT UNSIGNED NOT NULL, + ts_invalidated DATETIME NULL, + status TINYINT(1) UNSIGNED DEFAULT 0, + PRIMARY KEY(idinvalidation), + INDEX index_idsite_dates_period_name(idsite, date1, period, name) + ) ENGINE=$engine DEFAULT CHARSET=utf8 + ", + 'sequence' => "CREATE TABLE {$prefixTables}sequence ( `name` VARCHAR(120) NOT NULL, `value` BIGINT(20) UNSIGNED NOT NULL , |