Compare segments and periods (in API and UI) (#14365)

* Allow row metadata to be datatables in API output.

* Fill out initial DataComparisonFilter.

* fixing some issues

* couple more fixes

* couple more fixes + initial system test

* more fixes

* Finish up segment comparison system test and get to pass.

* Soft limit for number of segments/periods.

* Add supportsComparison method.

* Add UX code for adding/removing/seeing data comparisons + code to forward query parameters in AJAX requests + allow broadcast to handle multi-value query param values.

* Start showing comparison tables in html tables.

* Adding all comparison rows to html table visualization and adding "all visits" segment translation and add currently selected segment to comparisons table.

* Show totals ratio for comparison rows.

* finished poc html table visualization support for comparison

* start working on comparisons support in graph visualizations

* Some UX tweaks to htmltable and add comparisons to bar/pie graphs.

* Getting comparisons to work in evolution graphs.

* Get row evolution to work properly in comparison table.

* Get segmented visitor log to work in comparison tables.

* Fix regression in comparisons in evolution graph.

* Get comparisons to work in actions datatable, fix twig issue that results in 100% cpu usage (when reading dataTable param w/ many rows & comparison tables), get overlay/transitions icons to appear, overlay should work properly.

* Get transitions and overlay to work in comparison rows.

* Fixing some datatable API output issues, fixing tests, support comparisons in subtables by forcing idSubtables of comparisons to be sent in request (makes UI work, but not pracitcal for API).

* Remove typo.

* apply original change

* Allow All Visits default segment to be compared.

* working on disabling currently compared segments.

* Get currently compared segments code to work.

* starting on refactoring datacomparisonfilter

* Most of refactor done.

* Get tests to pass and fix a bunch of datatable metadata consistency issues.

* Modify evolution graph to modify compare parameters and show some sort of accurate comparison line graphs.

* Set xaxis labels correctly in tooltips and make sure series data for comparisons is set correctly.

* more fixes to displaying evolution comparisons where compared date ranges vary in length + make sure normal reports w/ no data display the no data message even when comparing

* Show period type in comparison card.

* Unsanitize compare segments.

* Get correct period count.

* Couple more fixes to evolution graph series labels, but still wonky.

* Include comparison series label in comparison output so evolution graph has an easier time of building series data.

* For multi period vs multi period show correct compareDate/comparePeriod for child tables.

* Redesign period selector comparison section and get to work.

* Allow plugins to disable comparisons for specific pages.

* Start supporting comparison in sparkline visualization.

* Get sparkline points & lengths to work correctly when comparing.

* Fix comparison enabling check.

* Pick series and shade colors.

* Rewriting comparison card to show individual serieses.

* Rewrite comparisons cards to only show segments as cards and individual serieses inside the cards.

* Use comparison colors and shades in evolution graph + fix a couple bugs.

* Tweak series colors and fix a couple regressions to comparison totals calculation.

* Add ratio tooltip suffix explaining comparison percent.

* fix typo

* Forward comparison params in report export.

* tweak series colors again + add tooltip with visitssummary metrics to comparison rows + fix a bug in using array query params in piwik-api + fix bug in formatting of comparison table metrics

* Tooltip fixes, start on sparklines supporting comparison, modify comparison filter to only calculate change metrics against periods since they are time related.

* Sparkline comparison support.

* Tweak line thickness and set metric index properly in jqplot data generator.

* In sparklines comparison, show evolution for compared period, rewrite top tooltip to be better, fix tooltip issues when multiple metrics used in evolution graph, and get comparison to previous period to work.

* Update submodule

* Make things look ok w/ a very long segment name, add numbers to compared datatable row labels, fix pie chart colors + a couple other regressions.

* more bug fixes

* Fix query param retrieval issue.

* Do not throw if no comparison params specified, just do nothing.

* try to fix a couple warnings

* Another query param get fix.

* Do not save comparison parameters.

* pass by reference

* fix JS error

* DO not set compare params if not set in URL for dashboard widgets.

* Fix comparison table styling in dashboard.

* Expand bar graph if there are too many bars when comparing.

* tweak comparison bar graph sizing

* make sure flatten works w/ comparison

* Apply compute processed metrics to comparison tables.

* Hack to get Goals.get to be formatted during comparison.

* Fix ordering of yunits in evolution graph.

* If rows are selected, incorporate into comparison series names.

* Format revenue properly in goals comparison sparklines.

* First working attempt at adding Referrers.get method for use w/ Sparklines visualization.

* get referrers sparklines to work w/ comparison

* Finish using new referrers API method and get referrers sparklines/evolution graph to play well w/ each other in comparison mode.

* Simplify table comparison view if only comparing periods, no segments.

* Take into account visible rows when calculating series metric index.

* Get comparison to work when totals rows are added to tables.

* Show series color in evolution graph tooltip.

* Fix error when loading row evolution/segmented visitor log for compared ranges.

* fix regression in normal subtable loading

* Fix row style

* Forward comparisonIdSubtables parameter if present so it is used when changing limit/offset

* Initialize the row index prefix to the filter offset.

* Do not show period header if only segments compared in table.

* Add UI tests and fix issues so they pass locally.

* quick tweak

* Fix PHP error

* Updating screenshots

* Fixing several bugs and updating expected screenshots.

* Fix comparison tests and clear some TODO.

* Prefix referrers metrics.

* Revert "apply original change"

This reverts commit 8f6ceb0430e5c7306a777498199ad7db21fd7175.

* Show period label if comparing two periods of same type.

* segment sanitization fixes

* More segment fixes.

* Another fix to the tooltip.

* Fix related reports when comparing + make totals tooltip clearer + store segment + pretty title in datatable metadata so it does not have to be looked up every time.

* Allow disabling comparisons for individual uses of visualizations.

* Remove limit on hover for actions tables + fix subtable expansion for normal actions tables.

* Make sure parameters are arrays.

* Stricter check for empty parameters.

* Allow first compared segment to be "removed".

* several more fixes

* Fixing table cell alignment and width and everything else that broke while making changes (hopefully).

* Several fixes, including xss fixes and test fixes and bug fixes for comparisons.

* more table css tweaks

* Correct workings of previous period/year comparison + always convert periods to ranges when comparing in evolution graph.

* Correct workings of previous period/year comparison + always convert periods to ranges when comparing in evolution graph + more css tweaks.

* fix more test regressions

* Forgot to add file

* fix several TODO as well ass get comparison sparklines to have right colors in widgets.

* Use DataTable metadata instead of getting available segments.

* When comparing periods that do not uniformly support unique visitors, do not display unique visitors metric.

* Small refactor and make sure sparklines shows over period w/o using lastN.

* more refactoring and fixes

* some more refactoring

* Move comparison index math to helper methods.

* Use piwikUrl.getSearchParam

* Process comparison tables like normal tables in API.getProcessedReport.

* remove some code redundancy

* use new format date method

* Add first working unit test for comparisons service.

* Finish writing unit test for comparison service.

* refactor comparisons service and fix a couple regressions

* Fix more TODO items and refactoring.

* Fill out more TODO.

* Remove more TODO.

* Fixing some tests.

* another test fix

* FIx some more tests.

* More test fixes and regression fixes.

* Do not add segments to summary rows in actions reports.

* more test fixes

* fix more tests

* more test fixes

* Fixing more tests.

* Fixing more tests + debugging failing one.

* Fix twig loop issue

* Make sure empty compare params are not used in URL.

* Remove cached request array.

* Support comparison rows in multirow evolution popover and LabelFilter.

* Tweak placement of some icons.

* Forward current segment in reporting menu links.

* Fix for split dimension view.

* tweak css

* Add more tests.

* Add year to xlabels in evolution graph (but not xaxis tick).

* applying review feedback

* Apply more PR feedback

* remove debugging code

* tweak event docs

* Fix test.

* fix some test

* fix a test and regression

* updating tes files + fixing test

* Fix regression

* Fix dropdown z-index issue (or workaround really).

* Fixing tests again.

* Update screenshots

* Fix bug and remove some debugging code

* Apply review feedback.

* Make sure ratio tooltips show in widgetized mode.

* Fix some UI tests.

* Fix tests

* Fix a couple more tests.

5 files changed, 271 insertions, 101 deletions

diff --git a/plugins/CoreHome/javascripts/broadcast.js b/plugins/CoreHome/javascripts/broadcast.js
index 5e392b50dd..ef1b23ff28 100644
--- a/plugins/CoreHome/javascripts/broadcast.js
+++ b/plugins/CoreHome/javascripts/broadcast.js
@@ -242,9 +242,10 @@ var broadcast = {
         }
 
         if (disableHistory) {
-            var newLocation = window.location.href.split('#')[0] + '#?' + currentHashStr;
+            var $window = piwikHelper.getAngularDependency('$window');
+            var newLocation = $window.location.href.split('#')[0] + '#?' + currentHashStr;
             // window.location.replace changes the current url without pushing it on the browser's history stack
-            window.location.replace(newLocation);
+            $window.location.replace(newLocation);
         }
         else {
             // Let history know about this new Hash and load it.
@@ -306,20 +307,25 @@ var broadcast = {
      * @param {string} str  url with parameters to be updated
      * @param {boolean} [showAjaxLoading] whether to show the ajax loading gif or not.
      * @param {string} strHash additional parameters that should be updated on the hash
+     * @param {array} paramsToRemove Optional parameters to remove from the URL.
      * @return {void}
      */
-    propagateNewPage: function (str, showAjaxLoading, strHash) {
+    propagateNewPage: function (str, showAjaxLoading, strHash, paramsToRemove) {
         // abort all existing ajax requests
         globalAjaxQueue.abort();
 
+        paramsToRemove = paramsToRemove || [];
+
         if (typeof showAjaxLoading === 'undefined' || showAjaxLoading) {
             piwikHelper.showAjaxLoading();
         }
 
         var params_vals = str.split("&");
 
+        var $window = piwikHelper.getAngularDependency('$window');
+
         // available in global scope
-        var currentSearchStr = window.location.search;
+        var currentSearchStr = $window.location.search;
         var currentHashStr = broadcast.getHashFromUrl();
         
         if (!currentSearchStr) {
@@ -328,19 +334,41 @@ var broadcast = {
 
         var oldUrl = currentSearchStr + currentHashStr;
 
-        for (var i = 0; i < params_vals.length; i++) {
+        // remove all array query params that are currently set. if we don't do this the array parameters we add
+        // just get added to the existing parameters.
+        params_vals.forEach(function (param) {
+            if (/\[]=/.test(decodeURIComponent(param))) {
+                var paramName = decodeURIComponent(param).split('[]=')[0];
+                removeParam(paramName);
+            }
+        });
+
+        // remove parameters if needed
+        paramsToRemove.forEach(function (paramName) {
+            removeParam(paramName);
+        });
 
-            if(params_vals[i].length == 0) {
-                continue; // updating with empty string would destroy some values
+        // update/add parameters based on whether the parameter is an array param or not
+        params_vals.forEach(function (param) {
+            if(!param.length) {
+                return; // updating with empty string would destroy some values
             }
 
-            // update both the current search query and hash string
-            currentSearchStr = broadcast.updateParamValue(params_vals[i], currentSearchStr);
+            if (/\[]=/.test(decodeURIComponent(param))) { // array param value
+                currentSearchStr = broadcast.addArrayParamValue(param, currentSearchStr);
+
+                if (currentHashStr.length !== 0) {
+                    currentHashStr = broadcast.addArrayParamValue(param, currentHashStr);
+                }
+            } else {
+                // update both the current search query and hash string
+                currentSearchStr = broadcast.updateParamValue(param, currentSearchStr);
 
-            if (currentHashStr.length != 0) {
-                currentHashStr = broadcast.updateParamValue(params_vals[i], currentHashStr);
+                if (currentHashStr.length !== 0) {
+                    currentHashStr = broadcast.updateParamValue(param, currentHashStr);
+                }
             }
-        }
+        });
 
         var updatedUrl = new RegExp('&updated=([0-9]+)');
         var updatedCounter = updatedUrl.exec(currentSearchStr);
@@ -367,16 +395,22 @@ var broadcast = {
                 if (event) {
                     event.preventDefault();
                 }
-            })
+            });
         }
 
         if (oldUrl == newUrl) {
-            window.location.reload();
+            $window.location.reload();
         } else {
             this.forceReload = true;
-            window.location.href = newUrl;
+            $window.location.href = newUrl;
         }
         return false;
+
+        function removeParam(paramName) {
+            var paramRegex = new RegExp(paramName + '(\\[]|%5B%5D)?=[^&?#]*&?', 'gi');
+            currentSearchStr = currentSearchStr.replace(paramRegex, '');
+            currentHashStr = currentHashStr.replace(paramRegex, '');
+        }
     },
 
     /*************************************************
@@ -430,6 +464,21 @@ var broadcast = {
     },
 
     /**
+     * Adds a query param value. Use it to add an array parameter value where you don't want to remove an existing value first.
+     *
+     * @param newParamValue
+     * @param urlStr
+     */
+    addArrayParamValue: function (newParamValue, urlStr) {
+        if (urlStr.indexOf('?') === -1) {
+            urlStr += '?';
+        } else {
+            urlStr += '&';
+        }
+        return urlStr + newParamValue;
+    },
+
+    /**
      * Loads a popover by adding a 'popover' query parameter to the current URL and
      * indirectly executing the popover handler.
      *
@@ -446,7 +495,6 @@ var broadcast = {
      *                       handler.
      */
     propagateNewPopoverParameter: function (handlerName, value) {
-
         var $location = angular.element(document).injector().get('$location');
 
         var popover = '';
@@ -467,7 +515,11 @@ var broadcast = {
             }
         }
 
-        $location.search('popover', popover);
+        var $window = piwikHelper.getAngularDependency('$window');
+        var urlStr = $window.location.hash;
+        urlStr = broadcast.updateParamValue('popover=' + encodeURIComponent(popover), urlStr);
+        urlStr = urlStr.replace(/^[#?]+/, '');
+        $location.search(urlStr);
 
         setTimeout(function () {
             angular.element(document).injector().get('$rootScope').$apply();
@@ -726,11 +778,30 @@ var broadcast = {
         var startStr = url.indexOf(lookFor);
 
         if (startStr >= 0) {
-            var endStr = url.indexOf("&", startStr);
-            if (endStr == -1) {
+            return getSingleValue(startStr, url);
+        } else {
+            url = decodeURIComponent(url);
+
+            // try looking for multi value param
+            lookFor = param + '[]=';
+            startStr = url.indexOf(lookFor);
+            if (startStr >= 0) {
+                var result = [getSingleValue(startStr)];
+                while ((startStr = url.indexOf(lookFor, startStr + 1)) !== -1) {
+                    result.push(getSingleValue(startStr));
+                }
+                return result;
+            } else {
+                return '';
+            }
+        }
+
+        function getSingleValue(startPos) {
+            var endStr = url.indexOf("&", startPos);
+            if (endStr === -1) {
                 endStr = url.length;
             }
-            var value = url.substring(startStr + param.length + 1, endStr);
+            var value = url.substring(startPos + lookFor.length, endStr);
 
             // we sanitize values to add a protection layer against XSS
             // &segment= value is not sanitized, since segments are designed to accept any user input
@@ -738,8 +809,6 @@ var broadcast = {
                 value = value.replace(/[^_%~\*\+\-\<\>!@\$\.()=,;0-9a-zA-Z]/gi, '');
             }
             return value;
-        } else {
-            return '';
         }
     },
 
@@ -763,7 +832,7 @@ var broadcast = {
             var urlParts = url.split('#');
             searchString = urlParts[0];
         } else {
-            searchString = location.search;
+            searchString = window.location.search;
         }
         return searchString;
     }
diff --git a/plugins/CoreHome/javascripts/dataTable.js b/plugins/CoreHome/javascripts/dataTable.js
index 71dd8e78b7..dd00113ed7 100644
--- a/plugins/CoreHome/javascripts/dataTable.js
+++ b/plugins/CoreHome/javascripts/dataTable.js
@@ -107,7 +107,6 @@ $.extend(DataTable.prototype, UIControl.prototype, {
         this.workingDivId = this._createDivId();
         domElem.attr('id', this.workingDivId);
 
-        this.maxNumRowsToHandleEvents = 255;
         this.loadedSubDataTable = {};
         this.isEmpty = $('.pk-emptyDataTable', domElem).length > 0;
         this.bindEventsAndApplyStyle(domElem);
@@ -215,7 +214,7 @@ $.extend(DataTable.prototype, UIControl.prototype, {
     // The ajax request contains the function callback to trigger if the request is successful or failed
     // displayLoading = false When we don't want to display the Loading... DIV .loadingPiwik
     // for example when the script add a Loading... it self and doesn't want to display the generic Loading
-    reloadAjaxDataTable: function (displayLoading, callbackSuccess) {
+    reloadAjaxDataTable: function (displayLoading, callbackSuccess, extraParams) {
         var self = this;
 
         if (typeof displayLoading == "undefined") {
@@ -252,7 +251,7 @@ $.extend(DataTable.prototype, UIControl.prototype, {
 
         var params = {};
         for (var key in self.param) {
-            if (typeof self.param[key] != "undefined" && self.param[key] != '') {
+            if (typeof self.param[key] != "undefined" && self.param[key] !== null && self.param[key] !== '') {
                 if (key == 'filter_column' || key == 'filter_column_recursive' ) {
                     // search in (metadata) `combinedLabel` when dimensions are shown separately in flattened tables
                     // needs to be overwritten for each request as switching a searched table might return no results
@@ -270,6 +269,9 @@ $.extend(DataTable.prototype, UIControl.prototype, {
         }
 
         ajaxRequest.addParams(params, 'get');
+        if (extraParams) {
+            ajaxRequest.addParams(extraParams, 'post');
+        }
         ajaxRequest.withTokenInUrl();
 
         ajaxRequest.setCallback(
@@ -410,6 +412,14 @@ $.extend(DataTable.prototype, UIControl.prototype, {
             $domElem.width('');
             parentDataTable.width('');
 
+            var $table = $('table.dataTable', domElem);
+            if ($table.closest('.reportsByDimensionView').length) {
+                var requiredTableWidth = $table.width() - 40; // 40 is padding on card content
+                if (domElem.width() > requiredTableWidth) {
+                    domElem.css('max-width', requiredTableWidth + 'px');
+                }
+            }
+
             var tableWidth = getTableWidth(domElem);
 
             if (tableWidth <= maxTableWidth) {
@@ -443,7 +453,7 @@ $.extend(DataTable.prototype, UIControl.prototype, {
         {
             var labelWidth = minLabelWidth;
 
-            var columnsInFirstRow = $('tr:nth-child(1) td:not(.label)', domElem);
+            var columnsInFirstRow = $('tbody tr:not(.parentComparisonRow):not(.comparePeriod):eq(0) td:not(.label)', domElem);
 
             var widthOfAllColumns = 0;
             columnsInFirstRow.each(function (index, column) {
@@ -465,7 +475,8 @@ $.extend(DataTable.prototype, UIControl.prototype, {
             if (labelWidth > maxLabelWidth
                 && !self.isWidgetized()
                 && innerWidth !== domElem.width()
-                && !self.isDashboard()) {
+                && !self.isDashboard()
+            ) {
                 labelWidth = maxLabelWidth; // prevent for instance table in Actions-Pages is not too wide
             }
 
@@ -482,7 +493,6 @@ $.extend(DataTable.prototype, UIControl.prototype, {
             }
 
             var minWidthBody = $('tbody tr:nth-child(1) td.label', domElem).css('minWidth');
-
             if (minWidthBody) {
                 minWidthBody = parseInt(minWidthBody, 10);
                 if (minWidthBody && minWidthBody > minWidth) {
@@ -528,8 +538,6 @@ $.extend(DataTable.prototype, UIControl.prototype, {
             return labelWidth;
         }
 
-        setMaxTableWidthIfNeeded(domElem, 1200);
-
         var isTableVisualization = this.jsViewDataTable
             && typeof this.jsViewDataTable === 'string'
             && typeof this.jsViewDataTable.indexOf === 'function'
@@ -560,6 +568,8 @@ $.extend(DataTable.prototype, UIControl.prototype, {
             self.overflowContentIfNeeded(domElem);
         }
 
+        setMaxTableWidthIfNeeded(domElem, 1200);
+
         if (!self.windowResizeTableAttached) {
             self.windowResizeTableAttached = true;
 
@@ -1451,36 +1461,16 @@ $.extend(DataTable.prototype, UIControl.prototype, {
 
     //Apply some miscelleaneous style to the DataTable
     applyCosmetics: function (domElem) {
-        var self = this;
-
-        // Add some styles on the cells even/odd
-        // label (first column of a data row) or not
-        $("th:first-child", domElem).addClass('label');
-        $("td:first-child", domElem).addClass('label');
-
-        var metadata = this.getReportMetadata();
-
-        if (self.param.flat == "1" && self.param.show_dimensions == "1" && metadata.dimensions && Object.keys(metadata.dimensions).length > 1) {
-            for (var i = 1; i < Object.keys(metadata.dimensions).length; i++) {
-                $("th:nth-child("+(i+1)+")", domElem).addClass('label');
-                $("td:nth-child("+(i+1)+")", domElem).addClass('label');
-            }
-        }
-
-        $("tr td", domElem).addClass('column');
+        // empty
     },
 
     handleColumnHighlighting: function (domElem) {
-        if (!this.canHandleRowEvents(domElem)) {
-            return;
-        }
-
         var maxWidth = {};
         var currentNthChild = null;
         var self = this;
 
-        // higlight all columns on hover
-        $('td', domElem).hover(function() {
+        // give all values consistent width
+        $('td', domElem).each(function () {
             var $this = $(this);
             if ($this.hasClass('label')) {
                 return;
@@ -1492,16 +1482,29 @@ $.extend(DataTable.prototype, UIControl.prototype, {
 
             if (!maxWidth[nthChild]) {
                 maxWidth[nthChild] = 0;
-                rows.find("td:nth-child(" + (nthChild) + ").column .value").each(function (index, element) {
+                rows.find("td:nth-child(" + (nthChild) + ").column .value").add('> thead th:not(.label) .thDIV', table).each(function (index, element) {
                     var width = $(element).width();
                     if (width > maxWidth[nthChild]) {
                         maxWidth[nthChild] = width;
                     }
                 });
                 rows.find("td:nth-child(" + (nthChild) + ").column .value").each(function (index, element) {
-                    $(element).css({width: maxWidth[nthChild], display: 'inline-block'});
+                    $(element).closest('td').css({width: maxWidth[nthChild]});
                 });
             }
+        });
+
+        // higlight all columns on hover
+        $(domElem).on('mouseenter', 'td', function (e) {
+            e.stopPropagation();
+            var $this = $(e.target);
+            if ($this.hasClass('label')) {
+                return;
+            }
+
+            var table    = $this.closest('table');
+            var nthChild = $this.parent('tr').children().index($(e.target)) + 1;
+            var rows     = $('> tbody > tr', table);
 
             if (currentNthChild === nthChild) {
                 return;
@@ -1511,8 +1514,10 @@ $.extend(DataTable.prototype, UIControl.prototype, {
 
             rows.children("td:nth-child(" + (nthChild) + ")").addClass('highlight');
             self.repositionRowActions($this.parent('tr'));
-        }, function(event) {
-            var $this = $(this);
+        });
+
+        $(domElem).on('mouseleave', 'td', function(event) {
+            var $this = $(event.target);
             var table    = $this.closest('table');
             var $parentTr = $this.parent('tr');
             var tr       = $parentTr.children();
@@ -1531,6 +1536,21 @@ $.extend(DataTable.prototype, UIControl.prototype, {
         });
     },
 
+    getComparisonIdSubtables: function ($row) {
+        if ($row.is('.parentComparisonRow')) {
+            var comparisonRows = $row.nextUntil('.parentComparisonRow').filter('.comparisonRow');
+
+            var comparisonIdSubtables = {};
+            comparisonRows.each(function () {
+                var comparisonSeriesIndex = +$(this).data('comparison-series');
+                comparisonIdSubtables[comparisonSeriesIndex] = $(this).data('idsubtable');
+            });
+
+            return JSON.stringify(comparisonIdSubtables);
+        }
+        return undefined;
+    },
+
     //behaviour for 'nested DataTable' (DataTable loaded on a click on a row)
     handleSubDataTable: function (domElem) {
         var self = this;
@@ -1544,12 +1564,17 @@ $.extend(DataTable.prototype, UIControl.prototype, {
 
                 // if the subDataTable is not already loaded
                 if (typeof self.loadedSubDataTable[divIdToReplaceWithSubTable] == "undefined") {
-                    var numberOfColumns = $(this).children().length;
+                    var numberOfColumns = $(this).closest('table').find('thead tr').first().children().length;
+
+                    var $insertAfter = $(this).nextUntil(':not(.comparePeriod):not(.comparisonRow)').last();
+                    if (!$insertAfter.length) {
+                        $insertAfter = $(this);
+                    }
 
                     // at the end of the query it will replace the ID matching the new HTML table #ID
                     // we need to create this ID first
-                    $(this).after(
-                        '<tr>' +
+                    var newRow = $insertAfter.after(
+                        '<tr class="subDataTableContainer">' +
                             '<td colspan="' + numberOfColumns + '" class="cellSubDataTable">' +
                             '<div id="' + divIdToReplaceWithSubTable + '">' +
                             '<span class="loadingPiwik" style="display:inline"><img src="plugins/Morpheus/images/loading-blue.gif" />' + _pk_translate('General_Loading') + '</span>' +
@@ -1558,6 +1583,8 @@ $.extend(DataTable.prototype, UIControl.prototype, {
                             '</tr>'
                     );
 
+                    piwikHelper.lazyScrollTo(newRow);
+
                     var savedActionVariable = self.param.action;
 
                     // reset all the filters from the Parent table
@@ -1570,9 +1597,12 @@ $.extend(DataTable.prototype, UIControl.prototype, {
 
 					delete self.param.totalRows;
 
+                    var extraParams = {};
+                    extraParams.comparisonIdSubtables = self.getComparisonIdSubtables($(this));
+
                     self.reloadAjaxDataTable(false, function(response) {
                         self.dataTableLoaded(response, divIdToReplaceWithSubTable);
-                    });
+                    }, extraParams);
 
                     self.param.action = savedActionVariable;
                     delete self.param.idSubtable;
@@ -1580,14 +1610,15 @@ $.extend(DataTable.prototype, UIControl.prototype, {
 
                     self.loadedSubDataTable[divIdToReplaceWithSubTable] = true;
 
-                    $(this).next().toggle();
-
                     // when "loading..." is displayed, hide actions
                     // repositioning after loading is not easily possible
                     $(this).find('div.dataTableRowActions').hide();
+                } else {
+                    var $toToggle = $(this).nextUntil('.subDataTableContainer').last();
+                    $toToggle = $toToggle.length ? $toToggle : $(this);
+                    $toToggle.next().toggle();
                 }
 
-                $(this).next().toggle();
                 $(this).toggleClass('expanded');
                 self.repositionRowActions($(this));
             }
@@ -1644,10 +1675,6 @@ $.extend(DataTable.prototype, UIControl.prototype, {
         });
     },
 
-    canHandleRowEvents: function (domElem) {
-        return domElem.find('table > tbody > tr').length <= this.maxNumRowsToHandleEvents;
-    },
-
     handleRowActions: function (domElem) {
         this.doHandleRowActions(domElem.find('table > tbody > tr'));
     },
@@ -1663,6 +1690,15 @@ $.extend(DataTable.prototype, UIControl.prototype, {
 			hide: false,
 			tooltipClass: 'small'
 		});
+        domElem.find('span.ratio').tooltip({
+            track: true,
+            content: function() {
+                var title = $(this).attr('title');
+                return piwikHelper.escape(title.replace(/\n/g, '<br />'));
+            },
+            show: {delay: 700, duration: 200},
+            hide: false
+        })
 	},
 
     handleRelatedReports: function (domElem) {
@@ -1828,7 +1864,7 @@ $.extend(DataTable.prototype, UIControl.prototype, {
             listenEvent = 'click';
         }
 
-        parent.on(listenEvent, 'tr', function () {
+        parent.on(listenEvent, 'tr:not(.subDataTableContainer)', function () {
             var tr = this;
             var $tr = $(tr);
             var td = $tr.find('td.label:last');
@@ -1840,7 +1876,11 @@ $.extend(DataTable.prototype, UIControl.prototype, {
             }
 
             // if there are row actions, make sure the first column is not too narrow
-            td.css('minWidth', '145px');
+            td.css('minWidth', $tr.is('.comparisonRow') ? '117px' : '145px');
+
+            if ($(this).is('.parentComparisonRow,.comparePeriod').length) {
+                return;
+            }
 
             if (useTouchEvent && tr.actionsDom && tr.actionsDom.prop('rowActionsVisible')) {
                 tr.actionsDom.prop('rowActionsVisible', false);
diff --git a/plugins/CoreHome/javascripts/dataTable_rowactions.js b/plugins/CoreHome/javascripts/dataTable_rowactions.js
index 9d8751293b..a7272ad236 100644
--- a/plugins/CoreHome/javascripts/dataTable_rowactions.js
+++ b/plugins/CoreHome/javascripts/dataTable_rowactions.js
@@ -96,8 +96,8 @@ DataTable_RowActions_Registry.register({
     isAvailableOnReport: function (dataTableParams) {
         return (
             typeof dataTableParams.disable_row_evolution == 'undefined'
-                || dataTableParams.disable_row_evolution == "0"
-            );
+            || dataTableParams.disable_row_evolution == "0"
+        );
     },
 
     isAvailableOnRow: function (dataTableParams, tr) {
@@ -144,7 +144,7 @@ DataTable_RowAction.prototype.initTr = function (tr) {
     // API actions. For the label filter to work, we need to use the parent action.
     // We use jQuery events to let subtables access their parents.
     tr.bind(self.trEventName, function (e, params) {
-        self.trigger($(this), params.originalEvent, params.label);
+        self.trigger($(this), params.originalEvent, params.label, params.originalRow);
     });
 };
 
@@ -152,7 +152,7 @@ DataTable_RowAction.prototype.initTr = function (tr) {
  * This method is called from the click event and the tr event (see this.trEventName).
  * It derives the label and calls performAction.
  */
-DataTable_RowAction.prototype.trigger = function (tr, e, subTableLabel) {
+DataTable_RowAction.prototype.trigger = function (tr, e, subTableLabel, originalRow) {
     var label = this.getLabelFromTr(tr);
 
     // if we have received the event from the sub table, add the label
@@ -166,7 +166,8 @@ DataTable_RowAction.prototype.trigger = function (tr, e, subTableLabel) {
     if (subtable.is('.subDataTable')) {
         subtable.closest('tr').prev().trigger(this.trEventName, {
             label: label,
-            originalEvent: e
+            originalEvent: e,
+            originalRow: tr
         });
         return;
     }
@@ -186,20 +187,24 @@ DataTable_RowAction.prototype.trigger = function (tr, e, subTableLabel) {
                 }
                 ptr.trigger(this.trEventName, {
                     label: label,
-                    originalEvent: e
+                    originalEvent: e,
+                    originalRow: tr
                 });
                 return;
             }
         }
     }
 
-    this.performAction(label, tr, e);
+    this.performAction(label, tr, e, originalRow);
 };
 
 /** Get the label string from a tr dom element */
 DataTable_RowAction.prototype.getLabelFromTr = function (tr) {
-    var rowMetadata = this.getRowMetadata(tr);
+    if (tr.data('label')) {
+        return tr.data('label');
+    }
 
+    var rowMetadata = this.getRowMetadata(tr);
     if (rowMetadata.combinedLabel) {
         return '@' + rowMetadata.combinedLabel;
     }
@@ -264,6 +269,7 @@ function DataTable_RowActions_RowEvolution(dataTable) {
 
     /** The rows to be compared in multi row evolution */
     this.multiEvolutionRows = [];
+    this.multiEvolutionRowsSeries = [];
 }
 
 /** Static helper method to launch row evolution from anywhere */
@@ -274,20 +280,38 @@ DataTable_RowActions_RowEvolution.launch = function (apiMethod, label) {
 
 DataTable_RowActions_RowEvolution.prototype = new DataTable_RowAction;
 
-DataTable_RowActions_RowEvolution.prototype.performAction = function (label, tr, e) {
+DataTable_RowActions_RowEvolution.prototype.performAction = function (label, tr, e, originalRow) {
     if (e.shiftKey) {
         // only mark for multi row evolution if shift key is pressed
-        this.addMultiEvolutionRow(label);
+        this.addMultiEvolutionRow(label, $(originalRow || tr).data('comparison-series'));
         return;
     }
 
-    this.addMultiEvolutionRow(label);
+    this.addMultiEvolutionRow(label, $(originalRow || tr).data('comparison-series'));
 
     // check whether we have rows marked for multi row evolution
-    var extraParams = {};
+    var extraParams = $.extend({}, $(originalRow || tr).data('param-override'));
+    if (typeof extraParams !== 'object') {
+        extraParams = {};
+    }
+
     if (this.multiEvolutionRows.length > 1) {
         extraParams.action = 'getMultiRowEvolutionPopover';
         label = this.multiEvolutionRows.join(',');
+
+        if (this.multiEvolutionRowsSeries.length > 1) { // when comparison is active
+            var piwikUrl = piwikHelper.getAngularDependency('piwikUrl');
+            extraParams.compareDates = piwikUrl.getSearchParam('compareDates');
+            extraParams.comparePeriods = piwikUrl.getSearchParam('comparePeriods');
+            extraParams.compareSegments = piwikUrl.getSearchParam('compareSegments');
+            extraParams.labelSeries = this.multiEvolutionRowsSeries.join(',');
+
+            // remove override period/date/segment since we are sending compare params so we can have the whole set of comparison
+            // serieses for LabelFilter
+            delete extraParams.period;
+            delete extraParams.date;
+            delete extraParams.segment;
+        }
     }
 
     $.each(this.dataTable.param, function (index, value) {
@@ -310,9 +334,27 @@ DataTable_RowActions_RowEvolution.prototype.performAction = function (label, tr,
     this.openPopover(apiMethod, extraParams, label);
 };
 
-DataTable_RowActions_RowEvolution.prototype.addMultiEvolutionRow = function (label) {
-    if ($.inArray(label, this.multiEvolutionRows) == -1) {
+DataTable_RowActions_RowEvolution.prototype.addMultiEvolutionRow = function (label, seriesIndex) {
+    if (typeof seriesIndex !== 'undefined') {
+        var self = this;
+
+        var found = false;
+        this.multiEvolutionRows.forEach(function (rowLabel, index) {
+            var rowSeriesIndex = self.multiEvolutionRowsSeries[index];
+            if (label === rowLabel && seriesIndex === rowSeriesIndex) {
+                found = true;
+                return false;
+            }
+        });
+
+        if (!found) {
+            this.multiEvolutionRows.push(label);
+            this.multiEvolutionRowsSeries.push(seriesIndex);
+        }
+    } else if ($.inArray(label, this.multiEvolutionRows) === -1) {
         this.multiEvolutionRows.push(label);
+
+        this.multiEvolutionRowsSeries = []; // for safety, make sure state is consistent
     }
 };
 
@@ -374,6 +416,7 @@ DataTable_RowActions_RowEvolution.prototype.showRowEvolution = function (apiMeth
         Piwik_Popover.onClose(function () {
             // reset rows marked for multi row evolution on close
             self.multiEvolutionRows = [];
+            self.multiEvolutionRowsSeries = [];
         });
 
         if (self.dataTable !== null) {
diff --git a/plugins/CoreHome/javascripts/sparkline.js b/plugins/CoreHome/javascripts/sparkline.js
index 87ab401863..1a7f65622a 100644
--- a/plugins/CoreHome/javascripts/sparkline.js
+++ b/plugins/CoreHome/javascripts/sparkline.js
@@ -13,19 +13,36 @@ var sparklineDisplayHeight = 25;
 var sparklineDisplayWidth = 100;
 
 piwik.getSparklineColors = function () {
-    return piwik.ColorManager.getColors('sparkline-colors', sparklineColorNames);
+    var colors = piwik.ColorManager.getColors('sparkline-colors', sparklineColorNames);
+
+    var comparisonService = piwikHelper.getAngularDependency('piwikComparisonsService');
+    if (comparisonService.isComparing()) {
+        var comparisons = comparisonService.getAllComparisonSeries();
+        colors.lineColor = comparisons.map(function (comp) { return comp.color; });
+    }
+
+    return colors;
 };
 
 // initializes each sparkline so they use colors defined in CSS
 piwik.initSparklines = function() {
-    $('.sparkline > img').each(function () {
+    $('.sparkline img').each(function () {
         var $self = $(this);
 
         if ($self.attr('src')) {
             return;
         }
 
-        var colors = JSON.stringify(piwik.getSparklineColors());
+        var seriesIndices = $self.closest('.sparkline').data('series-indices');
+        var sparklineColors = piwik.getSparklineColors();
+
+        if (seriesIndices && sparklineColors.lineColor instanceof Array) {
+            sparklineColors.lineColor = sparklineColors.lineColor.filter(function (c, index) {
+                return seriesIndices.indexOf(index) !== -1;
+            });
+        }
+
+        var colors = JSON.stringify(sparklineColors);
         var appendToSparklineUrl = '&colors=' + encodeURIComponent(colors);
 
         // Append the token_auth to the URL if it was set (eg. embed dashboard)
@@ -40,8 +57,6 @@ piwik.initSparklines = function() {
 };
 
 window.initializeSparklines = function () {
-    var sparklineUrlParamsToIgnore = ['module', 'action', 'idSite', 'period', 'date', 'showtitle', 'viewDataTable', 'forceView', 'random'];
-
     $('.dataTableVizEvolution[data-report]').each(function () {
         var graph = $(this);
 
@@ -73,16 +88,15 @@ window.initializeSparklines = function () {
 
                 $this.addClass('linked');
 
-                var params = broadcast.getValuesFromUrl(sparklineUrl);
-                for (var i = 0; i != sparklineUrlParamsToIgnore.length; ++i) {
-                    delete params[sparklineUrlParamsToIgnore[i]];
-                }
-                for (var key in params) {
-                    if (typeof params[key] == 'undefined') {
-                        // this happens for example with an empty segment parameter
-                        delete params[key];
-                    } else {
-                        params[key] = decodeURIComponent(params[key]);
+                var params = $this.data('graph-params') || {};
+                if (!Object.keys(params).length) {
+                    var urlParams = broadcast.getValuesFromUrl(sparklineUrl);
+
+                    if (urlParams.columns) {
+                        params.columns = decodeURIComponent(urlParams.columns);
+                    }
+                    if (urlParams.rows) {
+                        params.rows = decodeURIComponent(urlParams.rows);
                     }
                 }
 
diff --git a/plugins/CoreHome/javascripts/uiControl.js b/plugins/CoreHome/javascripts/uiControl.js
index 29b8f08711..4ffa385123 100644
--- a/plugins/CoreHome/javascripts/uiControl.js
+++ b/plugins/CoreHome/javascripts/uiControl.js
@@ -11,6 +11,8 @@
 
     var exports = require('piwik/UI');
 
+    var ARRAY_PARAM_NAMES = ['compareDates', 'comparePeriods', 'compareSegments'];
+
     /**
      * Base type for Piwik UI controls. Provides functionality that all controls need (such as
      * cleanup on destruction).
@@ -30,7 +32,9 @@
 
         var params = JSON.parse($element.attr('data-params') || '{}');
         for (var key in params) { // convert values in params that are arrays to comma separated string lists
-            if (params[key] instanceof Array) {
+            if (params[key] instanceof Array
+                && ARRAY_PARAM_NAMES.indexOf(key) === -1
+            ) {
                 params[key] = params[key].join(',');
             }
         }