Goals per page - new reports and metrics for tracking page conversions (#18221)

* Goals per page - added new reports and metrics for tracking page conversions

* Added top entry page statistic, added entry titles related report, adjusted percent formatting

* Added datatable filter to remove goal revenue columns if there are no values, refactored goal visualisation classes to minimise duplicate methods

* Fixed conversion query matching issue

* Added system tests for page goals reports, rework of metrics and SQL to match expected calculations

* Added UI tests

* Remove unnecessary code, merge system tests

* Fix for archiving query error on MySQL

* Fix for query to work with MySQL 5.7

* Update core/API/DocumentationGenerator.php

Co-authored-by: Stefan Giehl <stefan@matomo.org>

* Revert unnecessary DocumentationGenerator change

* Fix incorrect row limit config setting names

* Remove special handling of pages reports

* Update plugins/Goals/Visualizations/GoalsPages.php

Co-authored-by: Stefan Giehl <stefan@matomo.org>

* Update plugins/Goals/Visualizations/GoalsEntryPages.php

Co-authored-by: Stefan Giehl <stefan@matomo.org>

* Tidy up visualization classes

* Fix for sorting

* Remove unnecessary columns from reports, fix tests

* Add new visualizations to Javascript checks

* Add BasePages parent report class to deduplicate new reports

* Reworked to add per-goal metric columns to the Actions_action and Action_action_url archives instead of generating separate page goal archives

* Added a filter to remove goal columns from Actions datatables by default and an optional parameter to include goals columns

* Improved remove goals actions filter to recurse subtables, get goals list for site

* Test fixes

* Move page goal metrics to nested column structure on actions and actions_url archives

* Fixes and updates for apiGetReportMetadata tests

* Test fixes

* Test fixes and updates

* More test updates

* Bug fix for entry page incorrectly aggregating data from different goals

* Backwards compatibility test fixes

* Test fix

* Update submodule

* Updated tests

* Updated tests

* Updated test

* built vue files

* Force sorting of scheduled reports list by unique id to avoid sorting inconsistencies between PHP7 and PHP8 tests

* Updated tests after sorting scheduled reports list

* Updated tests

* Test fixes

* Skip specific tests that pass with PHP7 but fail with PHP8

* Test updates

* Test updates

* UI test screenshot updates

* Slightly increased reasonable release total filesize test from 55mb to 56mb

* Test fix

* Disable test for PHP8

* Update submodule

* Update submodule

* Test updates

* Revert test changes

* Predictably sort scheduled reports

* Disable failing test for PHP 8

* Updated tests after conflict fix

* Update tests after conflict fix

* Update submodule

* Revert unnecessary change

* Rework to use a single goals visualisation and show goal metrics directly on the action page reports

* Update UI tests

* Update tests, fix for goals menu ordering issue

* Update tests

* Revert goal overview menu item ordering

* Do not add goal metadata to actions reports when the includeGoals API parameter is set to false

* Updated unit and UI tests

* Update UI tests

* Update submodule

* Update submodules

* Code improvements, handle ecommerce metrics for page goals, remove obsolete test expected xml files

* Remove unnecessary row properties after use

* fix phpcs

* Improve & refactor code

* Update system test

* Tweaked release reasonable size test from 55mb to 58mb to prevent test failure

* Update submodule

* Update system test

* Update UI test screenshots

* Update UI test screenshots

* Update submodule

* Update UI test screenshot

* update submodule

* test improvements

* updates expected UI files

* fix removing unused revenue columns from UI

* applies some psr12 code formatting

* Minor query optimisations

* Rework the conversions by pageview query to remove subquery and all grouping, add aggregation in code

* Test fixes, null checks for revenue metrics

* Expanded goal page tests to cover multiple goals converted in a single visit

* Updated UI test screenshots

* Expanded test to be multi-day, multi-goal, multiple conversion per visit. Fixed summary logic for multi-goal visits. Fixed calculation of viewed before page rate to get conversion total via API call

* Update system and UI screenshot tests

* Added method return type hint, ensure request parameters are blank on Goals.get filter API call

* Retrieve conversion totals from numeric archives directly instead of via API call

* Move goal conversion totals lookup from the CalculateConversionPageRate filter to the Actions API and then pass to the filter

* Tidy up unused namespaces

* Revert move of goal conversion total retrieval from filter to actions api. Included segment in archive build for goal conversions totals.

* fixes: date might be manipulated too often

* updates expected UI files

* avoid building archive too often

* updates expected test files

* fix ui tests

Co-authored-by: sgiehl <stefan@matomo.org>
Co-authored-by: bx80 <bx80@users.noreply.github.com>

3 files changed, 6 insertions, 6 deletions

diff --git a/plugins/Actions/tests/UI/expected-screenshots/ActionsDataTable_column_sorted.png b/plugins/Actions/tests/UI/expected-screenshots/ActionsDataTable_column_sorted.png
index c2204eafed..15ad0a4947 100644
--- a/plugins/Actions/tests/UI/expected-screenshots/ActionsDataTable_column_sorted.png
+++ b/plugins/Actions/tests/UI/expected-screenshots/ActionsDataTable_column_sorted.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bf88c02a14c5de403939674beca872b1f13b05e0d0deec24b39daa6436cff2f6
-size 363615
+oid sha256:9894f0d2e70eb141d1c75a6c3a1009bba2dc7655cc9a1e276e5cef0750811bbe
+size 363759
diff --git a/plugins/Actions/tests/UI/expected-screenshots/ActionsDataTable_flattened.png b/plugins/Actions/tests/UI/expected-screenshots/ActionsDataTable_flattened.png
index da3003396f..c3553d406c 100644
--- a/plugins/Actions/tests/UI/expected-screenshots/ActionsDataTable_flattened.png
+++ b/plugins/Actions/tests/UI/expected-screenshots/ActionsDataTable_flattened.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3c8be898a1e002b1996e5c38ccb567fda47535f7eefe498be0232a70bec7613c
-size 466574
+oid sha256:ed07b21ea2b127eae2128fb7a32d8bd53a0f61a19ab57ab7934d79addc2c201f
+size 466560
diff --git a/plugins/Actions/tests/UI/expected-screenshots/ActionsDataTable_subtables_loaded.png b/plugins/Actions/tests/UI/expected-screenshots/ActionsDataTable_subtables_loaded.png
index 8bb3ecb49a..232bd4be8b 100644
--- a/plugins/Actions/tests/UI/expected-screenshots/ActionsDataTable_subtables_loaded.png
+++ b/plugins/Actions/tests/UI/expected-screenshots/ActionsDataTable_subtables_loaded.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:71e3e2238b43afca2a5b7fd52709871e13e174b853be160097282bd426f4a8d6
-size 379466
+oid sha256:fe045aa00aaadf72043af7cf6601eca9980ef0123a1b24e37a1ab5eedcd28945
+size 379628