Rewrite cron archiving process for easier maintenance and performance (#15499)

* Adding initial new code for cron archive rewrite.

* first pass at removing unused CronArchive code.

* unfinished commit

* fill out archiveinvalidator code

* getting some tests to pass

* unfinished commit

* fixing part of test

* Another test fix.

* another sql change

* fix broken merge or something else that went wrong

* Couple more fixes and extra logs.

* Fixing enough issues to get core archive command to run completely.

* Fix and log change.

* Fixed more segment/test related issues for CronArchiveTest. Includes optimization for no visits for period + segment process from handling.

* another optimization and possible build fix

* no visit optimization

* test fix

* Implement archiving_custom_ranges logic w/ queue based implementation

* fixes to get archivecrontest to work

* add logic to invalidate today period

* fix optimization and some tests

* Fixing more tests.

* Fixing more tests

* debug travis failure

* more test fixes

* more test fixes, removing more unneeded code, handling some TODOs

* Handle more TODOs including creating ArchiveFilter class for some cli options.

* tests and todos

* idarchives are specific to table + start on archivefilter tests

* one test

* more TODOs and tests

* more tests and todo taken care of

* handle more todos

* fixing more tests

* fix comment

* make sure autoarchiving is enabled for segments when cron archive picks them up

* Fixing test.

* apply more pr feedback

* order by date1 asc

* quick refactor

* use batch insert instead of createDummyArchives

* apply rest of pr feedback

* add removed events, add new test, fix an issue (when deleting idarchives older than do not lump all segments together).

* re-add fixed/shared siteids

* fix tests

* incomplete commit

* Insert archive entries into archive_invalidations table.

* Use invalidations table in core:archive and get ArchiveCronTest to pass.

* fixing some tests

* debugging travis

* fix more tests & remove DONE_IN_PROGRESS which is no longer used.

* fix more tests

* apply review feedback

* fix tests

* try fixing alltests build

* try again

* try again

* Move archive_invalidations update to new beta since omnifixture was updated w/o it.

* Another fix.

1 files changed, 2 insertions, 0 deletions

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 8322e43485..46a8b27c72 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -46,6 +46,8 @@ The Product Changelog at **[matomo.org/changelog](https://matomo.org/changelog)*
 * GeoIP Legacy support has been fully removed. Users of GeoIP Legacy need to set up a new location provider like GeoIP2, otherwise the default location provider will be used.
 * Site search category and count are no longer stored as custom variables. That also means they will now have an extra field in action details and no longer appear in custom variables.
 * The parameter `alias` from the API methods `UsersManager.addUser` and `UsersManager.updateUser` has been removed.
+* The `$completed` parameter for the 'CronArchive.archiveSingleSite.finish' event has been removed. For both this event and the CronArchive.archiveSingleSite.start event, a new
+  parameter is added for the process' pid. Multiple processes can now trigger this event for the same site ID.
 * The dimension and `log_link_visit_action` column interaction_position has been renamed to pageview_position. If your database queries rely on the column you can simply replace the name.
 * The metric (avg.) page generation time has been deprecated. It is no longer possible to track it. Already tracked values will still be shown in old reports. More detailed performance metrics are now available in PagePerformance plugin.
 * The signature of `Dimension::configureSegments()` has been changed. Similar to configuring Metrics it now takes two parameters `SegmentsList $segmentsList` and `DimensionSegmentFactory $dimensionSegmentFactory`.