diff options
| author | diosmosis <diosmosis@users.noreply.github.com> | 2020-05-07 19:11:06 +0300 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2020-05-07 19:11:06 +0300 |
| commit | 6d744db7f3f7a55fc4bb1ca573e937f0832c642b (patch) | |
| tree | 148d845d4ed521641f66f6bb2640697b14da6b26 /plugins/Diagnostics | |
| parent | d829c23496c9f1c53bd4144009d67f8a287d97e4 (diff) | |
Rewrite cron archiving process for easier maintenance and performance (#15499)
* Adding initial new code for cron archive rewrite.
* first pass at removing unused CronArchive code.
* unfinished commit
* fill out archiveinvalidator code
* getting some tests to pass
* unfinished commit
* fixing part of test
* Another test fix.
* another sql change
* fix broken merge or something else that went wrong
* Couple more fixes and extra logs.
* Fixing enough issues to get core archive command to run completely.
* Fix and log change.
* Fixed more segment/test related issues for CronArchiveTest. Includes optimization for no visits for period + segment process from handling.
* another optimization and possible build fix
* no visit optimization
* test fix
* Implement archiving_custom_ranges logic w/ queue based implementation
* fixes to get archivecrontest to work
* add logic to invalidate today period
* fix optimization and some tests
* Fixing more tests.
* Fixing more tests
* debug travis failure
* more test fixes
* more test fixes, removing more unneeded code, handling some TODOs
* Handle more TODOs including creating ArchiveFilter class for some cli options.
* tests and todos
* idarchives are specific to table + start on archivefilter tests
* one test
* more TODOs and tests
* more tests and todo taken care of
* handle more todos
* fixing more tests
* fix comment
* make sure autoarchiving is enabled for segments when cron archive picks them up
* Fixing test.
* apply more pr feedback
* order by date1 asc
* quick refactor
* use batch insert instead of createDummyArchives
* apply rest of pr feedback
* add removed events, add new test, fix an issue (when deleting idarchives older than do not lump all segments together).
* re-add fixed/shared siteids
* fix tests
* incomplete commit
* Insert archive entries into archive_invalidations table.
* Use invalidations table in core:archive and get ArchiveCronTest to pass.
* fixing some tests
* debugging travis
* fix more tests & remove DONE_IN_PROGRESS which is no longer used.
* fix more tests
* apply review feedback
* fix tests
* try fixing alltests build
* try again
* try again
* Move archive_invalidations update to new beta since omnifixture was updated w/o it.
* Another fix.
Diffstat (limited to 'plugins/Diagnostics')
| -rw-r--r-- | plugins/Diagnostics/tests/Integration/Commands/AnalyzeArchiveTableTest.php | 37 |
1 files changed, 3 insertions, 34 deletions
diff --git a/plugins/Diagnostics/tests/Integration/Commands/AnalyzeArchiveTableTest.php b/plugins/Diagnostics/tests/Integration/Commands/AnalyzeArchiveTableTest.php index faaff14579..9e3fbd224a 100644 --- a/plugins/Diagnostics/tests/Integration/Commands/AnalyzeArchiveTableTest.php +++ b/plugins/Diagnostics/tests/Integration/Commands/AnalyzeArchiveTableTest.php @@ -43,46 +43,15 @@ Statistics for the archive_numeric_2010_03 and archive_blob_2010_03 tables: +-------------------------------------------+------------+---------------+-------------+---------+-----------+----------------+-------------+-------------+ | week[2010-03-01 - 2010-03-07] idSite = 1 | 7 | 0 | 0 | 0 | 6 | 73 | 97 | %d | | month[2010-03-01 - 2010-03-31] idSite = 1 | 7 | 0 | 0 | 0 | 6 | 73 | 97 | %d | -| day[2010-03-03 - 2010-03-03] idSite = 1 | 7 | 0 | 0 | 0 | 6 | 0 | 0 | | -| day[2010-03-04 - 2010-03-04] idSite = 1 | 7 | 0 | 0 | 0 | 6 | 0 | 0 | | -| day[2010-03-05 - 2010-03-05] idSite = 1 | 7 | 0 | 0 | 0 | 6 | 0 | 0 | | | day[2010-03-06 - 2010-03-06] idSite = 1 | 7 | 0 | 0 | 0 | 6 | 73 | 73 | %d | -| day[2010-03-07 - 2010-03-07] idSite = 1 | 7 | 0 | 0 | 0 | 6 | 0 | 0 | | -| day[2010-03-08 - 2010-03-08] idSite = 1 | 5 | 0 | 0 | 0 | 4 | 0 | 0 | | -| week[2010-03-08 - 2010-03-14] idSite = 1 | 7 | 0 | 0 | 0 | 6 | 0 | 0 | | -| day[2010-03-09 - 2010-03-09] idSite = 1 | 5 | 0 | 0 | 0 | 4 | 0 | 0 | | -| day[2010-03-10 - 2010-03-10] idSite = 1 | 5 | 0 | 0 | 0 | 4 | 0 | 0 | | -| day[2010-03-11 - 2010-03-11] idSite = 1 | 5 | 0 | 0 | 0 | 4 | 0 | 0 | | -| day[2010-03-12 - 2010-03-12] idSite = 1 | 5 | 0 | 0 | 0 | 4 | 0 | 0 | | -| day[2010-03-13 - 2010-03-13] idSite = 1 | 5 | 0 | 0 | 0 | 4 | 0 | 0 | | -| day[2010-03-14 - 2010-03-14] idSite = 1 | 5 | 0 | 0 | 0 | 4 | 0 | 0 | | -| day[2010-03-15 - 2010-03-15] idSite = 1 | 5 | 0 | 0 | 0 | 4 | 0 | 0 | | -| week[2010-03-15 - 2010-03-21] idSite = 1 | 7 | 0 | 0 | 0 | 6 | 0 | 0 | | -| day[2010-03-16 - 2010-03-16] idSite = 1 | 5 | 0 | 0 | 0 | 4 | 0 | 0 | | -| day[2010-03-17 - 2010-03-17] idSite = 1 | 5 | 0 | 0 | 0 | 4 | 0 | 0 | | -| day[2010-03-18 - 2010-03-18] idSite = 1 | 5 | 0 | 0 | 0 | 4 | 0 | 0 | | -| day[2010-03-19 - 2010-03-19] idSite = 1 | 5 | 0 | 0 | 0 | 4 | 0 | 0 | | -| day[2010-03-20 - 2010-03-20] idSite = 1 | 5 | 0 | 0 | 0 | 4 | 0 | 0 | | -| day[2010-03-21 - 2010-03-21] idSite = 1 | 5 | 0 | 0 | 0 | 4 | 0 | 0 | | -| day[2010-03-22 - 2010-03-22] idSite = 1 | 5 | 0 | 0 | 0 | 4 | 0 | 0 | | -| week[2010-03-22 - 2010-03-28] idSite = 1 | 7 | 0 | 0 | 0 | 6 | 0 | 0 | | -| day[2010-03-23 - 2010-03-23] idSite = 1 | 5 | 0 | 0 | 0 | 4 | 0 | 0 | | -| day[2010-03-24 - 2010-03-24] idSite = 1 | 5 | 0 | 0 | 0 | 4 | 0 | 0 | | -| day[2010-03-25 - 2010-03-25] idSite = 1 | 5 | 0 | 0 | 0 | 4 | 0 | 0 | | -| day[2010-03-26 - 2010-03-26] idSite = 1 | 5 | 0 | 0 | 0 | 4 | 0 | 0 | | -| day[2010-03-27 - 2010-03-27] idSite = 1 | 5 | 0 | 0 | 0 | 4 | 0 | 0 | | -| day[2010-03-28 - 2010-03-28] idSite = 1 | 5 | 0 | 0 | 0 | 4 | 0 | 0 | | -| day[2010-03-29 - 2010-03-29] idSite = 1 | 7 | 0 | 0 | 0 | 6 | 0 | 0 | | -| day[2010-03-30 - 2010-03-30] idSite = 1 | 7 | 0 | 0 | 0 | 6 | 0 | 0 | | -| day[2010-03-31 - 2010-03-31] idSite = 1 | 7 | 0 | 0 | 0 | 6 | 0 | 0 | | +-------------------------------------------+------------+---------------+-------------+---------+-----------+----------------+-------------+-------------+ -Total # Archives: 196 +Total # Archives: 21 Total # Invalidated Archives: 0 Total # Temporary Archives: 0 Total # Error Archives: 0 -Total # Segment Archives: 162 -Total Size of Blobs: 30.%d K +Total # Segment Archives: 18 +Total Size of Blobs: 31.%d K OUTPUT; |