allow invalidating plugin archives only and archiving past data for plugins (#15889)

* Adding initial new code for cron archive rewrite.

* first pass at removing unused CronArchive code.

* unfinished commit

* fill out archiveinvalidator code

* getting some tests to pass

* unfinished commit

* fixing part of test

* Another test fix.

* another sql change

* fix broken merge or something else that went wrong

* Couple more fixes and extra logs.

* Fixing enough issues to get core archive command to run completely.

* Fix and log change.

* Fixed more segment/test related issues for CronArchiveTest. Includes optimization for no visits for period + segment process from handling.

* another optimization and possible build fix

* no visit optimization

* test fix

* Implement archiving_custom_ranges logic w/ queue based implementation

* fixes to get archivecrontest to work

* add logic to invalidate today period

* fix optimization and some tests

* Fixing more tests.

* Fixing more tests

* debug travis failure

* more test fixes

* more test fixes, removing more unneeded code, handling some TODOs

* Handle more TODOs including creating ArchiveFilter class for some cli options.

* tests and todos

* idarchives are specific to table + start on archivefilter tests

* one test

* more TODOs and tests

* more tests and todo taken care of

* handle more todos

* fixing more tests

* fix comment

* make sure autoarchiving is enabled for segments when cron archive picks them up

* Fixing test.

* apply more pr feedback

* order by date1 asc

* quick refactor

* use batch insert instead of createDummyArchives

* apply rest of pr feedback

* add removed events, add new test, fix an issue (when deleting idarchives older than do not lump all segments together).

* re-add fixed/shared siteids

* fix tests

* incomplete commit

* Insert archive entries into archive_invalidations table.

* Use invalidations table in core:archive and get ArchiveCronTest to pass.

* fixing some tests

* debugging travis

* fix more tests & remove DONE_IN_PROGRESS which is no longer used.

* fix more tests

* Allow forcing plugin specific archive in core:archive.

* When querying from archive data use all available archives including "all" archives and plugin specific archives.

* Adding some code for invalidating specific plugin archives.

* Get archive invalidation test to pass.

* add plugin capability to invalidate command

* Handle plugin only archives in core:archive.

* Add Archive test and get ArchiveCronTest to pass.

* update some expected files

* Fix some more tests.

* incomplete commit

* allow invalidating individual reports

* adding more API for DONE_PARTIAL support

* get archivecrontest to pass

* add archive processor tests

* fix some test randomnes

* when purging keep latest partial archives if there is no newer whole archive

* add rearchivereport method + some unfinished tests

* Add archiveReports API method, fix race condition in test, when archiving single report, always ignore inserting other reports.

* require archivers to handle partial archives themselves entirely instead of trying to do it automatically and allow requested report to be any string

* couple fixes

* Use core config option for last N montsh to invalidate.

* Add test for ArchiveSelector method.

* Ignore archives w/ deactivated plugins.

* Refactor queue looping into new QueueConsumer class.

* apply more review feedback + another fix

* invalidate segments too in reArchiveReport w/ etsts

* remove DONE_IN_PROGRESS, no longer used. use new status in query and add queue consumer test.

* forgot to add file

* delete old unneeded archives when finalizing a new one.

* tweak invalidation archive description

* add plugin archiving tests and get them to pass

* fix test

* many fixes

* fix another test

* update expected test files

* fix more tests

* last test fixes hopefully

* tweak log

* In case a column already exists, do not try to add it in an AddColumns migration or the entire migration will fail and no columns will be added.

* try to fix tests again

* fix again?

* apply some review feedback + fix test

* fix test

* fix another test

* couple fixes

* Remove extra param.

* apply pr feedback

* check for usable archive before invalidating and before initiating archiving

* fixing tests

* fixing tests

* fixing tests

* fix another test issue

* fix archiveinvalidator test

* fix one test and debug another

* more debugging

* fix test

* use twig

* remove no longer needed change

* add back previous logic

* fix tracking is not working

* apply pr feedback and add tests

* fixing tests

* update submodule

* debugging random travis failure

* update test

* more debugging

* more debugging

* another attempt at debugging

* Lets try this fix

* trying to fix the build

* debug

* simpler test

* fix test

* fix test

* fix test

* fix test

* fix test failure

* update screenshots

* update screenshots

Co-authored-by: Thomas Steur <tsteur@users.noreply.github.com>

2 files changed, 9 insertions, 1 deletions

diff --git a/plugins/GeoIp2/LocationProvider/GeoIp2/ServerModule.php b/plugins/GeoIp2/LocationProvider/GeoIp2/ServerModule.php
index 5ebe6114ce..f1baf19552 100644
--- a/plugins/GeoIp2/LocationProvider/GeoIp2/ServerModule.php
+++ b/plugins/GeoIp2/LocationProvider/GeoIp2/ServerModule.php
@@ -16,7 +16,9 @@ use Piwik\Piwik;
 use Piwik\Plugins\GeoIp2\LocationProvider\GeoIp2;
 use Piwik\Plugins\UserCountry\LocationProvider;
 use Piwik\Plugins\GeoIp2\SystemSettings;
+use Piwik\SettingsServer;
 use Piwik\Url;
+use Piwik\View;
 
 /**
  * A LocationProvider that uses an GeoIP 2 module installed in an HTTP Server.
@@ -250,7 +252,12 @@ class ServerModule extends GeoIp2
         $configUrl = Url::getCurrentQueryStringWithParametersModified(array(
             'module' => 'CoreAdminHome', 'action' => 'generalSettings'
         ));
-        $extraMessage .= '<br />'.Piwik::translate('GeoIp2_GeoIPVariablesConfigurationHere', ['<a href="'.$configUrl.'">', '</a>']);
+        if (!SettingsServer::isTrackerApiRequest()) {
+            // can't render in tracking mode as there is no theme
+            $view = new View('@GeoIp2/serverModule');
+            $view->configUrl = $configUrl;
+            $extraMessage .= $view->render();
+        }
 
         return array('id'            => self::ID,
             'title'         => $title,
diff --git a/plugins/GeoIp2/templates/serverModule.twig b/plugins/GeoIp2/templates/serverModule.twig
new file mode 100644
index 0000000000..66df4331fb
--- /dev/null
+++ b/plugins/GeoIp2/templates/serverModule.twig
@@ -0,0 +1 @@
+<br />{{ 'GeoIp2_GeoIPVariablesConfigurationHere'|translate('<a href="' ~ configUrl|e('html_attr') ~ '">', '</a>')|raw }}
\ No newline at end of file