Compare segments and periods (in API and UI) (#14365)

* Allow row metadata to be datatables in API output.

* Fill out initial DataComparisonFilter.

* fixing some issues

* couple more fixes

* couple more fixes + initial system test

* more fixes

* Finish up segment comparison system test and get to pass.

* Soft limit for number of segments/periods.

* Add supportsComparison method.

* Add UX code for adding/removing/seeing data comparisons + code to forward query parameters in AJAX requests + allow broadcast to handle multi-value query param values.

* Start showing comparison tables in html tables.

* Adding all comparison rows to html table visualization and adding "all visits" segment translation and add currently selected segment to comparisons table.

* Show totals ratio for comparison rows.

* finished poc html table visualization support for comparison

* start working on comparisons support in graph visualizations

* Some UX tweaks to htmltable and add comparisons to bar/pie graphs.

* Getting comparisons to work in evolution graphs.

* Get row evolution to work properly in comparison table.

* Get segmented visitor log to work in comparison tables.

* Fix regression in comparisons in evolution graph.

* Get comparisons to work in actions datatable, fix twig issue that results in 100% cpu usage (when reading dataTable param w/ many rows & comparison tables), get overlay/transitions icons to appear, overlay should work properly.

* Get transitions and overlay to work in comparison rows.

* Fixing some datatable API output issues, fixing tests, support comparisons in subtables by forcing idSubtables of comparisons to be sent in request (makes UI work, but not pracitcal for API).

* Remove typo.

* apply original change

* Allow All Visits default segment to be compared.

* working on disabling currently compared segments.

* Get currently compared segments code to work.

* starting on refactoring datacomparisonfilter

* Most of refactor done.

* Get tests to pass and fix a bunch of datatable metadata consistency issues.

* Modify evolution graph to modify compare parameters and show some sort of accurate comparison line graphs.

* Set xaxis labels correctly in tooltips and make sure series data for comparisons is set correctly.

* more fixes to displaying evolution comparisons where compared date ranges vary in length + make sure normal reports w/ no data display the no data message even when comparing

* Show period type in comparison card.

* Unsanitize compare segments.

* Get correct period count.

* Couple more fixes to evolution graph series labels, but still wonky.

* Include comparison series label in comparison output so evolution graph has an easier time of building series data.

* For multi period vs multi period show correct compareDate/comparePeriod for child tables.

* Redesign period selector comparison section and get to work.

* Allow plugins to disable comparisons for specific pages.

* Start supporting comparison in sparkline visualization.

* Get sparkline points & lengths to work correctly when comparing.

* Fix comparison enabling check.

* Pick series and shade colors.

* Rewriting comparison card to show individual serieses.

* Rewrite comparisons cards to only show segments as cards and individual serieses inside the cards.

* Use comparison colors and shades in evolution graph + fix a couple bugs.

* Tweak series colors and fix a couple regressions to comparison totals calculation.

* Add ratio tooltip suffix explaining comparison percent.

* fix typo

* Forward comparison params in report export.

* tweak series colors again + add tooltip with visitssummary metrics to comparison rows + fix a bug in using array query params in piwik-api + fix bug in formatting of comparison table metrics

* Tooltip fixes, start on sparklines supporting comparison, modify comparison filter to only calculate change metrics against periods since they are time related.

* Sparkline comparison support.

* Tweak line thickness and set metric index properly in jqplot data generator.

* In sparklines comparison, show evolution for compared period, rewrite top tooltip to be better, fix tooltip issues when multiple metrics used in evolution graph, and get comparison to previous period to work.

* Update submodule

* Make things look ok w/ a very long segment name, add numbers to compared datatable row labels, fix pie chart colors + a couple other regressions.

* more bug fixes

* Fix query param retrieval issue.

* Do not throw if no comparison params specified, just do nothing.

* try to fix a couple warnings

* Another query param get fix.

* Do not save comparison parameters.

* pass by reference

* fix JS error

* DO not set compare params if not set in URL for dashboard widgets.

* Fix comparison table styling in dashboard.

* Expand bar graph if there are too many bars when comparing.

* tweak comparison bar graph sizing

* make sure flatten works w/ comparison

* Apply compute processed metrics to comparison tables.

* Hack to get Goals.get to be formatted during comparison.

* Fix ordering of yunits in evolution graph.

* If rows are selected, incorporate into comparison series names.

* Format revenue properly in goals comparison sparklines.

* First working attempt at adding Referrers.get method for use w/ Sparklines visualization.

* get referrers sparklines to work w/ comparison

* Finish using new referrers API method and get referrers sparklines/evolution graph to play well w/ each other in comparison mode.

* Simplify table comparison view if only comparing periods, no segments.

* Take into account visible rows when calculating series metric index.

* Get comparison to work when totals rows are added to tables.

* Show series color in evolution graph tooltip.

* Fix error when loading row evolution/segmented visitor log for compared ranges.

* fix regression in normal subtable loading

* Fix row style

* Forward comparisonIdSubtables parameter if present so it is used when changing limit/offset

* Initialize the row index prefix to the filter offset.

* Do not show period header if only segments compared in table.

* Add UI tests and fix issues so they pass locally.

* quick tweak

* Fix PHP error

* Updating screenshots

* Fixing several bugs and updating expected screenshots.

* Fix comparison tests and clear some TODO.

* Prefix referrers metrics.

* Revert "apply original change"

This reverts commit 8f6ceb0430e5c7306a777498199ad7db21fd7175.

* Show period label if comparing two periods of same type.

* segment sanitization fixes

* More segment fixes.

* Another fix to the tooltip.

* Fix related reports when comparing + make totals tooltip clearer + store segment + pretty title in datatable metadata so it does not have to be looked up every time.

* Allow disabling comparisons for individual uses of visualizations.

* Remove limit on hover for actions tables + fix subtable expansion for normal actions tables.

* Make sure parameters are arrays.

* Stricter check for empty parameters.

* Allow first compared segment to be "removed".

* several more fixes

* Fixing table cell alignment and width and everything else that broke while making changes (hopefully).

* Several fixes, including xss fixes and test fixes and bug fixes for comparisons.

* more table css tweaks

* Correct workings of previous period/year comparison + always convert periods to ranges when comparing in evolution graph.

* Correct workings of previous period/year comparison + always convert periods to ranges when comparing in evolution graph + more css tweaks.

* fix more test regressions

* Forgot to add file

* fix several TODO as well ass get comparison sparklines to have right colors in widgets.

* Use DataTable metadata instead of getting available segments.

* When comparing periods that do not uniformly support unique visitors, do not display unique visitors metric.

* Small refactor and make sure sparklines shows over period w/o using lastN.

* more refactoring and fixes

* some more refactoring

* Move comparison index math to helper methods.

* Use piwikUrl.getSearchParam

* Process comparison tables like normal tables in API.getProcessedReport.

* remove some code redundancy

* use new format date method

* Add first working unit test for comparisons service.

* Finish writing unit test for comparison service.

* refactor comparisons service and fix a couple regressions

* Fix more TODO items and refactoring.

* Fill out more TODO.

* Remove more TODO.

* Fixing some tests.

* another test fix

* FIx some more tests.

* More test fixes and regression fixes.

* Do not add segments to summary rows in actions reports.

* more test fixes

* fix more tests

* more test fixes

* Fixing more tests.

* Fixing more tests + debugging failing one.

* Fix twig loop issue

* Make sure empty compare params are not used in URL.

* Remove cached request array.

* Support comparison rows in multirow evolution popover and LabelFilter.

* Tweak placement of some icons.

* Forward current segment in reporting menu links.

* Fix for split dimension view.

* tweak css

* Add more tests.

* Add year to xlabels in evolution graph (but not xaxis tick).

* applying review feedback

* Apply more PR feedback

* remove debugging code

* tweak event docs

* Fix test.

* fix some test

* fix a test and regression

* updating tes files + fixing test

* Fix regression

* Fix dropdown z-index issue (or workaround really).

* Fixing tests again.

* Update screenshots

* Fix bug and remove some debugging code

* Apply review feedback.

* Make sure ratio tooltips show in widgetized mode.

* Fix some UI tests.

* Fix tests

* Fix a couple more tests.

1 files changed, 5 insertions, 1 deletions

diff --git a/composer.json b/composer.json
index c0c84ff276..88a03cd148 100644
--- a/composer.json
+++ b/composer.json
@@ -47,7 +47,7 @@
         "composer/semver": "~1.3.0",
         "szymach/c-pchart": "^2.0",
         "geoip2/geoip2": "^2.8",
-        "davaxi/sparkline": "^1.1",
+        "davaxi/sparkline": "dev-multiple-series",
         "matomo-org/jshrink": "1.3.1"
     },
     "require-dev": {
@@ -113,6 +113,10 @@
                     "reference": "v1.3.1"
                 }
             }
+        },
+        {
+            "type": "git",
+            "url": "https://github.com/matomo-org/Sparkline.git"
         }
     ],
     "scripts": {