Compare segments and periods (in API and UI) (#14365)

* Allow row metadata to be datatables in API output.

* Fill out initial DataComparisonFilter.

* fixing some issues

* couple more fixes

* couple more fixes + initial system test

* more fixes

* Finish up segment comparison system test and get to pass.

* Soft limit for number of segments/periods.

* Add supportsComparison method.

* Add UX code for adding/removing/seeing data comparisons + code to forward query parameters in AJAX requests + allow broadcast to handle multi-value query param values.

* Start showing comparison tables in html tables.

* Adding all comparison rows to html table visualization and adding "all visits" segment translation and add currently selected segment to comparisons table.

* Show totals ratio for comparison rows.

* finished poc html table visualization support for comparison

* start working on comparisons support in graph visualizations

* Some UX tweaks to htmltable and add comparisons to bar/pie graphs.

* Getting comparisons to work in evolution graphs.

* Get row evolution to work properly in comparison table.

* Get segmented visitor log to work in comparison tables.

* Fix regression in comparisons in evolution graph.

* Get comparisons to work in actions datatable, fix twig issue that results in 100% cpu usage (when reading dataTable param w/ many rows & comparison tables), get overlay/transitions icons to appear, overlay should work properly.

* Get transitions and overlay to work in comparison rows.

* Fixing some datatable API output issues, fixing tests, support comparisons in subtables by forcing idSubtables of comparisons to be sent in request (makes UI work, but not pracitcal for API).

* Remove typo.

* apply original change

* Allow All Visits default segment to be compared.

* working on disabling currently compared segments.

* Get currently compared segments code to work.

* starting on refactoring datacomparisonfilter

* Most of refactor done.

* Get tests to pass and fix a bunch of datatable metadata consistency issues.

* Modify evolution graph to modify compare parameters and show some sort of accurate comparison line graphs.

* Set xaxis labels correctly in tooltips and make sure series data for comparisons is set correctly.

* more fixes to displaying evolution comparisons where compared date ranges vary in length + make sure normal reports w/ no data display the no data message even when comparing

* Show period type in comparison card.

* Unsanitize compare segments.

* Get correct period count.

* Couple more fixes to evolution graph series labels, but still wonky.

* Include comparison series label in comparison output so evolution graph has an easier time of building series data.

* For multi period vs multi period show correct compareDate/comparePeriod for child tables.

* Redesign period selector comparison section and get to work.

* Allow plugins to disable comparisons for specific pages.

* Start supporting comparison in sparkline visualization.

* Get sparkline points & lengths to work correctly when comparing.

* Fix comparison enabling check.

* Pick series and shade colors.

* Rewriting comparison card to show individual serieses.

* Rewrite comparisons cards to only show segments as cards and individual serieses inside the cards.

* Use comparison colors and shades in evolution graph + fix a couple bugs.

* Tweak series colors and fix a couple regressions to comparison totals calculation.

* Add ratio tooltip suffix explaining comparison percent.

* fix typo

* Forward comparison params in report export.

* tweak series colors again + add tooltip with visitssummary metrics to comparison rows + fix a bug in using array query params in piwik-api + fix bug in formatting of comparison table metrics

* Tooltip fixes, start on sparklines supporting comparison, modify comparison filter to only calculate change metrics against periods since they are time related.

* Sparkline comparison support.

* Tweak line thickness and set metric index properly in jqplot data generator.

* In sparklines comparison, show evolution for compared period, rewrite top tooltip to be better, fix tooltip issues when multiple metrics used in evolution graph, and get comparison to previous period to work.

* Update submodule

* Make things look ok w/ a very long segment name, add numbers to compared datatable row labels, fix pie chart colors + a couple other regressions.

* more bug fixes

* Fix query param retrieval issue.

* Do not throw if no comparison params specified, just do nothing.

* try to fix a couple warnings

* Another query param get fix.

* Do not save comparison parameters.

* pass by reference

* fix JS error

* DO not set compare params if not set in URL for dashboard widgets.

* Fix comparison table styling in dashboard.

* Expand bar graph if there are too many bars when comparing.

* tweak comparison bar graph sizing

* make sure flatten works w/ comparison

* Apply compute processed metrics to comparison tables.

* Hack to get Goals.get to be formatted during comparison.

* Fix ordering of yunits in evolution graph.

* If rows are selected, incorporate into comparison series names.

* Format revenue properly in goals comparison sparklines.

* First working attempt at adding Referrers.get method for use w/ Sparklines visualization.

* get referrers sparklines to work w/ comparison

* Finish using new referrers API method and get referrers sparklines/evolution graph to play well w/ each other in comparison mode.

* Simplify table comparison view if only comparing periods, no segments.

* Take into account visible rows when calculating series metric index.

* Get comparison to work when totals rows are added to tables.

* Show series color in evolution graph tooltip.

* Fix error when loading row evolution/segmented visitor log for compared ranges.

* fix regression in normal subtable loading

* Fix row style

* Forward comparisonIdSubtables parameter if present so it is used when changing limit/offset

* Initialize the row index prefix to the filter offset.

* Do not show period header if only segments compared in table.

* Add UI tests and fix issues so they pass locally.

* quick tweak

* Fix PHP error

* Updating screenshots

* Fixing several bugs and updating expected screenshots.

* Fix comparison tests and clear some TODO.

* Prefix referrers metrics.

* Revert "apply original change"

This reverts commit 8f6ceb0430e5c7306a777498199ad7db21fd7175.

* Show period label if comparing two periods of same type.

* segment sanitization fixes

* More segment fixes.

* Another fix to the tooltip.

* Fix related reports when comparing + make totals tooltip clearer + store segment + pretty title in datatable metadata so it does not have to be looked up every time.

* Allow disabling comparisons for individual uses of visualizations.

* Remove limit on hover for actions tables + fix subtable expansion for normal actions tables.

* Make sure parameters are arrays.

* Stricter check for empty parameters.

* Allow first compared segment to be "removed".

* several more fixes

* Fixing table cell alignment and width and everything else that broke while making changes (hopefully).

* Several fixes, including xss fixes and test fixes and bug fixes for comparisons.

* more table css tweaks

* Correct workings of previous period/year comparison + always convert periods to ranges when comparing in evolution graph.

* Correct workings of previous period/year comparison + always convert periods to ranges when comparing in evolution graph + more css tweaks.

* fix more test regressions

* Forgot to add file

* fix several TODO as well ass get comparison sparklines to have right colors in widgets.

* Use DataTable metadata instead of getting available segments.

* When comparing periods that do not uniformly support unique visitors, do not display unique visitors metric.

* Small refactor and make sure sparklines shows over period w/o using lastN.

* more refactoring and fixes

* some more refactoring

* Move comparison index math to helper methods.

* Use piwikUrl.getSearchParam

* Process comparison tables like normal tables in API.getProcessedReport.

* remove some code redundancy

* use new format date method

* Add first working unit test for comparisons service.

* Finish writing unit test for comparison service.

* refactor comparisons service and fix a couple regressions

* Fix more TODO items and refactoring.

* Fill out more TODO.

* Remove more TODO.

* Fixing some tests.

* another test fix

* FIx some more tests.

* More test fixes and regression fixes.

* Do not add segments to summary rows in actions reports.

* more test fixes

* fix more tests

* more test fixes

* Fixing more tests.

* Fixing more tests + debugging failing one.

* Fix twig loop issue

* Make sure empty compare params are not used in URL.

* Remove cached request array.

* Support comparison rows in multirow evolution popover and LabelFilter.

* Tweak placement of some icons.

* Forward current segment in reporting menu links.

* Fix for split dimension view.

* tweak css

* Add more tests.

* Add year to xlabels in evolution graph (but not xaxis tick).

* applying review feedback

* Apply more PR feedback

* remove debugging code

* tweak event docs

* Fix test.

* fix some test

* fix a test and regression

* updating tes files + fixing test

* Fix regression

* Fix dropdown z-index issue (or workaround really).

* Fixing tests again.

* Update screenshots

* Fix bug and remove some debugging code

* Apply review feedback.

* Make sure ratio tooltips show in widgetized mode.

* Fix some UI tests.

* Fix tests

* Fix a couple more tests.

7 files changed, 1534 insertions, 9 deletions

diff --git a/plugins/API/API.php b/plugins/API/API.php
index cfdbe92985..8d21ab11a8 100644
--- a/plugins/API/API.php
+++ b/plugins/API/API.php
@@ -476,7 +476,7 @@ class API extends \Piwik\Plugin\API
      * @param bool|int $idDimension
      * @return array
      */
-    public function getRowEvolution($idSite, $period, $date, $apiModule, $apiAction, $label = false, $segment = false, $column = false, $language = false, $idGoal = false, $legendAppendMetric = true, $labelUseAbsoluteUrl = true, $idDimension = false)
+    public function getRowEvolution($idSite, $period, $date, $apiModule, $apiAction, $label = false, $segment = false, $column = false, $language = false, $idGoal = false, $legendAppendMetric = true, $labelUseAbsoluteUrl = true, $idDimension = false, $labelSeries = false)
     {
         // check if site exists
         $idSite = (int) $idSite;
@@ -504,7 +504,7 @@ class API extends \Piwik\Plugin\API
 
         $rowEvolution = new RowEvolution();
         return $rowEvolution->getRowEvolution($idSite, $period, $date, $apiModule, $apiAction, $label, $segment, $column,
-            $language, $apiParameters, $legendAppendMetric, $labelUseAbsoluteUrl);
+            $language, $apiParameters, $legendAppendMetric, $labelUseAbsoluteUrl, $labelSeries);
     }
 
     /**
@@ -526,6 +526,10 @@ class API extends \Piwik\Plugin\API
         foreach ($urls as $url) {
             $params = Request::getRequestArrayFromString($url . '&format=php&serialize=0');
 
+            if (!empty($params['method']) && $params['method'] === 'API.getBulkRequest') {
+                continue;
+            }
+
             if (isset($params['urls']) && $params['urls'] == $urls) {
                 // by default 'urls' is added to $params as Request::getRequestArrayFromString adds all $_GET/$_POST
                 // default parameters
@@ -620,6 +624,37 @@ class API extends \Piwik\Plugin\API
         return $values;
     }
 
+    /**
+     * Returns category/subcategory pairs as "CategoryId.SubcategoryId" for whom comparison features should
+     * be disabled.
+     *
+     * @return string[]
+     */
+    public function getPagesComparisonsDisabledFor()
+    {
+        $pages = [];
+
+        /**
+         * If your plugin has pages where you'd like comparison features to be disabled, you can add them
+         * via this event. Add the pages as "CategoryId.SubcategoryId".
+         *
+         * **Example**
+         *
+         * ```
+         * public function getPagesComparisonsDisabledFor(&$pages)
+         * {
+         *     $pages[] = "General_Visitors.MyPlugin_MySubcategory";
+         *     $pages[] = "MyPlugin.myControllerAction"; // if your plugin defines a whole page you want comparison disabled for
+         * }
+         * ```
+         *
+         * @param string[] &$pages
+         */
+        Piwik::postEvent('API.getPagesComparisonsDisabledFor', [&$pages]);
+
+        return $pages;
+    }
+
     private function findSegment($segmentName, $idSite)
     {
         $segmentsMetadata = $this->getSegmentsMetadata($idSite, $_hideImplementationData = false);
diff --git a/plugins/API/Filter/DataComparisonFilter.php b/plugins/API/Filter/DataComparisonFilter.php
new file mode 100644
index 0000000000..78872f3b41
--- /dev/null
+++ b/plugins/API/Filter/DataComparisonFilter.php
@@ -0,0 +1,590 @@
+<?php
+/**
+ * Piwik - free/libre analytics platform
+ *
+ * @link http://piwik.org
+ * @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
+ */
+
+namespace Piwik\Plugins\API\Filter;
+
+use Piwik\API\Request;
+use Piwik\Common;
+use Piwik\Config;
+use Piwik\DataTable;
+use Piwik\DataTable\DataTableInterface;
+use Piwik\DataTable\Simple;
+use Piwik\Http\BadRequestException;
+use Piwik\Metrics;
+use Piwik\Period;
+use Piwik\Period\Factory;
+use Piwik\Piwik;
+use Piwik\Plugin\Report;
+use Piwik\Plugins\API\Filter\DataComparisonFilter\ComparisonRowGenerator;
+use Piwik\Segment;
+use Piwik\Segment\SegmentExpression;
+use Piwik\Site;
+
+/**
+ * Handles the API portion of the data comparison feature.
+ *
+ * If the `compareSegments`/`comparePeriods`/`compareDates` parameters are supplied this class will fetch
+ * the data to compare with and store this data next to each row in the root report.
+ *
+ * Additionally, `..._change` columns will be added that show the percentage change for a column. This is only
+ * done when comparing periods, since segments are subsets of visits, so it doesn't make sense to consider
+ * differences between them as "changes".
+ *
+ * ### Comparing multiple periods
+ *
+ * It is possible to compare multiple periods with other multiple periods. For example, it
+ * is possible to compare period=day, date=2018-02-03,2018-02-13 with period=day, date=2018-03-01,2018-03-15.
+ * When done, this filter will compare the first period in the first set w/ the first period in the second set,
+ * etc. So in the previous example, 2018-02-03 will be compared with 2018-03-01, 2018-02-04 with 2018-03-02, etc.
+ *
+ * ### Metadata
+ *
+ * This filter adds the following metadata to DataTables:
+ *
+ * - 'compareSegments': The list of segments being compared. The first entry will always be the value of the `segment` query param.
+ * - 'comparePeriods': The list of labels of periods being compared. The first entry will always be the value of the
+ *                     `period` query param.
+ * - 'compareDates': The list of dates being compared. The first entry will always be the value of the `date` query param.
+ * - 'comparisonSeries': Prettified labels for every comparison series in order.
+ *
+ * This filter adds the following metadata to rows in the comparison DataTables:
+ *
+ * - 'compareSegment': The segment of the data for the comparison row.
+ * - 'compareSegmentPretty': The prettified label for the segment.
+ * - 'comparePeriod': The period label for the data in the comparison row. This does not have to match a value in the
+ *                    `comparePeriods` query parameter if comparing multiple periods.
+ * - 'compareDate': The date for the period in the data in the comparison row. This does not have to match a value in the
+ *                    `compareDates` query parameter if comparing multiple periods.
+ * - 'comparePeriodPretty': The prettified label for the period.
+ * - 'compareSeriesPretty': Prettified label for the comparison data represented by the row. This will match an entry
+ *                          in the DataTable's `comparisonSeries` metadata.
+ */
+class DataComparisonFilter
+{
+    /**
+     * @var array
+     */
+    private $request;
+
+    /**
+     * @var int
+     */
+    private $segmentCompareLimit;
+
+    /**
+     * @var int
+     */
+    private $periodCompareLimit;
+
+    /**
+     * @var string
+     */
+    private $segmentName;
+
+    /**
+     * @var string[]
+     */
+    private $compareSegments;
+
+    /**
+     * @var string[]
+     */
+    private $compareDates;
+
+    /**
+     * @var string[]
+     */
+    private $comparePeriods;
+
+    /**
+     * @var int[]
+     */
+    private $compareSegmentIndices;
+
+    /**
+     * @var int[]
+     */
+    private $comparePeriodIndices;
+
+    /**
+     * @var bool
+     */
+    private $isRequestMultiplePeriod;
+
+    /**
+     * @var ComparisonRowGenerator
+     */
+    private $comparisonRowGenerator;
+
+    /**
+     * @var array
+     */
+    private $columnMappings;
+
+    public function __construct($request, Report $report = null)
+    {
+        $this->request = $request;
+
+        $generalConfig = Config::getInstance()->General;
+        $this->segmentCompareLimit = (int) $generalConfig['data_comparison_segment_limit'];
+        $this->checkComparisonLimit($this->segmentCompareLimit, 'data_comparison_segment_limit');
+
+        $this->periodCompareLimit = (int) $generalConfig['data_comparison_period_limit'];
+        $this->checkComparisonLimit($this->periodCompareLimit, 'data_comparison_period_limit');
+
+        $this->segmentName = $this->getSegmentNameFromReport($report);
+
+        $this->compareSegments = self::getCompareSegments();
+        if (count($this->compareSegments) > $this->segmentCompareLimit + 1) {
+            throw new BadRequestException(Piwik::translate('General_MaximumNumberOfSegmentsComparedIs', [$this->segmentCompareLimit]));
+        }
+
+        $this->compareDates = self::getCompareDates($request);
+        $this->comparePeriods = self::getComparePeriods($request);
+
+        if (count($this->compareDates) !== count($this->comparePeriods)) {
+            throw new BadRequestException(Piwik::translate('General_CompareDatesParamMustMatchComparePeriods', ['compareDates', 'comparePeriods']));
+        }
+
+        if (count($this->compareDates) > $this->periodCompareLimit + 1) {
+            throw new BadRequestException(Piwik::translate('General_MaximumNumberOfPeriodsComparedIs', [$this->periodCompareLimit]));
+        }
+
+        if (count($this->compareSegments) == 1
+            && count($this->comparePeriods) == 1
+        ) {
+            return;
+        }
+
+        $this->checkMultiplePeriodCompare();
+
+        // map segments/periods to their indexes in the query parameter arrays for comparisonIdSubtable matching
+        $this->compareSegmentIndices = array_flip($this->compareSegments);
+        foreach ($this->comparePeriods as $index => $period) {
+            $date = $this->compareDates[$index];
+            $this->comparePeriodIndices[$period][$date] = $index;
+        }
+
+        $this->columnMappings = $this->getColumnMappings();
+        $this->comparisonRowGenerator = new ComparisonRowGenerator($this->segmentName, $this->isRequestMultiplePeriod(), $this->columnMappings);
+    }
+
+    public static function isCompareParamsPresent($request = null)
+    {
+        return !empty(Common::getRequestVar('compareSegments', [], $type = 'array', $request))
+            || !empty(Common::getRequestVar('comparePeriods', [], $type = 'array', $request))
+            || !empty(Common::getRequestVar('compareDates', [], $type = 'array', $request));
+    }
+
+    /**
+     * @param DataTable\DataTableInterface $table
+     */
+    public function compare(DataTable\DataTableInterface $table)
+    {
+        if (empty($this->compareSegments)
+            && empty($this->comparePeriods)
+        ) {
+            return;
+        }
+
+        $method = Common::getRequestVar('method', $default = null, $type = 'string', $this->request);
+        if ($method == 'Live') {
+            throw new \Exception("Data comparison is not enabled for the Live API.");
+        }
+
+        // optimization, if empty, single table, don't need to make extra queries
+        if ($table->getRowsCount() == 0) {
+            return;
+        }
+
+        $comparisonSeries = [];
+
+        // fetch data first
+        $reportsToCompare = self::getReportsToCompare($this->compareSegments, $this->comparePeriods, $this->compareDates);
+        foreach ($reportsToCompare as $index => $modifiedParams) {
+            $compareMetadata = $this->getMetadataFromModifiedParams($modifiedParams);
+            $comparisonSeries[] = $compareMetadata['compareSeriesPretty'];
+
+            $compareTable = $this->requestReport($method, $modifiedParams);
+            $this->comparisonRowGenerator->compareTables($compareMetadata, $table, $compareTable);
+        }
+
+        // calculate changes (including processed metric changes)
+        // NOTE: it doesn't make to sense to calculate these values for segments, since segments are subsets of all visits, where periods are
+        //       time periods (so things can change from one to another).
+        if (count($this->comparePeriods) > 1) {
+            $this->compareChangePercents($table);
+        }
+
+        // format comparison table metrics
+        $this->formatComparisonTables($table);
+
+        // add comparison parameters as metadata
+        $table->filter(function (DataTable $singleTable) use ($comparisonSeries) {
+            if (isset($this->compareSegments)) {
+                $singleTable->setMetadata('compareSegments', $this->compareSegments);
+            }
+
+            if (isset($this->comparePeriods)) {
+                $singleTable->setMetadata('comparePeriods', $this->comparePeriods);
+            }
+
+            if (isset($this->compareDates)) {
+                $singleTable->setMetadata('compareDates', $this->compareDates);
+            }
+
+            $singleTable->setMetadata('comparisonSeries', $comparisonSeries);
+        });
+    }
+
+    public static function getReportsToCompare($compareSegments, $comparePeriods, $compareDates)
+    {
+        $permutations = [];
+
+        // NOTE: the order of these loops determines the order of the rows in the comparison table. ie,
+        // if we loop over dates then segments, then we'll see comparison rows change segments before changing
+        // periods. this is because this loop determines in what order we fetch report data.
+        foreach ($compareDates as $index => $date) {
+            foreach ($compareSegments as $segment) {
+                $period = $comparePeriods[$index];
+
+                $params = [];
+                $params['segment'] = $segment;
+
+                if (!empty($period)
+                    && !empty($date)
+                ) {
+                    $params['date'] = $date;
+                    $params['period'] = $period;
+                }
+
+                $permutations[] = $params;
+            }
+        }
+
+        return $permutations;
+    }
+
+    /**
+     * @param $paramsToModify
+     * @return DataTable
+     */
+    private function requestReport($method, $paramsToModify)
+    {
+        $params = array_merge(
+            [
+                'filter_limit' => -1,
+                'filter_offset' => 0,
+                'filter_sort_column' => '',
+                'filter_truncate' => -1,
+                'compare' => 0,
+                'totals' => 1,
+                'disable_queued_filters' => 1,
+                'format_metrics' => 0,
+                'label' => '',
+                'flat' => Common::getRequestVar('flat', 0, 'int', $this->request),
+            ],
+            $paramsToModify
+        );
+
+        $params['keep_totals_row'] = Common::getRequestVar('keep_totals_row', 0, 'int', $this->request);
+        $params['keep_totals_row_label'] = Common::getRequestVar('keep_totals_row_label', '', 'string', $this->request);
+
+        if (!isset($params['idSite'])) {
+            $params['idSite'] = Common::getRequestVar('idSite', null, 'string', $this->request);
+        }
+        if (!isset($params['period'])) {
+            $params['period'] = Common::getRequestVar('period', null, 'string', $this->request);
+        }
+        if (!isset($params['date'])) {
+            $params['date'] = Common::getRequestVar('date', null, 'string', $this->request);
+        }
+
+        $idSubtable = Common::getRequestVar('idSubtable', 0, 'int', $this->request);
+        if ($idSubtable > 0) {
+            $comparisonIdSubtables = Common::getRequestVar('comparisonIdSubtables', $default = false, 'json', $this->request);
+            if (empty($comparisonIdSubtables)) {
+                throw new \Exception("Comparing segments/periods with subtables only works when the comparison idSubtables are supplied as well.");
+            }
+
+            $segmentIndex = empty($paramsToModify['segment']) ? 0 : $this->compareSegmentIndices[$paramsToModify['segment']];
+            $periodIndex = empty($paramsToModify['period']) ? 0 : $this->comparePeriodIndices[$paramsToModify['period']][$paramsToModify['date']];
+            $seriesIndex = self::getComparisonSeriesIndex(null, $periodIndex, $segmentIndex, count($this->compareSegments));
+
+            if (!isset($comparisonIdSubtables[$seriesIndex])) {
+                throw new \Exception("Invalid comparisonIdSubtables parameter: no idSubtable found for segment $segmentIndex and period $periodIndex");
+            }
+
+            $comparisonIdSubtable = $comparisonIdSubtables[$seriesIndex];
+            if ($comparisonIdSubtable === -1) { // no subtable in comparison row
+                $table = new DataTable();
+                $table->setMetadata('site', new Site($params['idSite']));
+                $table->setMetadata('period', Period\Factory::build($params['period'], $params['date']));
+                return $table;
+            }
+
+            $params['idSubtable'] = $comparisonIdSubtable;
+        }
+
+        return Request::processRequest($method, $params);
+    }
+
+    private function formatComparisonTables(DataTableInterface $tableOrMap)
+    {
+        $tableOrMap->filter(function (DataTable $table) {
+            $rows = $table->getRows();
+
+            $totalRow = $table->getTotalsRow();
+            if ($totalRow) {
+                $rows[] = $totalRow;
+            }
+
+            foreach ($rows as $row) {
+                /** @var DataTable $comparisonTable */
+                $comparisonTable = $row->getComparisons();
+                if (!empty($comparisonTable)) { // sanity check
+                    $columnMappings = $this->columnMappings;
+                    $comparisonTable->filter(DataTable\Filter\ReplaceColumnNames::class, [$columnMappings]);
+                }
+
+                $subtable = $row->getSubtable();
+                if ($subtable) {
+                    $this->formatComparisonTables($subtable);
+                }
+            }
+        });
+    }
+
+    private function checkComparisonLimit($n, $configName)
+    {
+        if ($n <= 1) {
+            throw new \Exception("The [General] $configName INI config option must be greater than 1.");
+        }
+    }
+
+    private function getMetadataFromModifiedParams($modifiedParams)
+    {
+        $metadata = [];
+
+        $period = isset($modifiedParams['period']) ? $modifiedParams['period'] : reset($this->comparePeriods);
+        $date = isset($modifiedParams['date']) ? $modifiedParams['date'] : reset($this->compareDates);
+        $segment = isset($modifiedParams['segment']) ? $modifiedParams['segment'] : reset($this->compareSegments);
+
+        $metadata['compareSegment'] = $segment;
+
+        $segmentObj = new Segment($segment, []);
+        $metadata['compareSegmentPretty'] = $segmentObj->getStoredSegmentName(false);
+
+        $metadata['comparePeriod'] = $period;
+        $metadata['compareDate'] = $date;
+
+        $prettyPeriod = Factory::build($period, $date)->getLocalizedLongString();
+        $metadata['comparePeriodPretty'] = ucfirst($prettyPeriod);
+
+        $metadata['compareSeriesPretty'] = self::getComparisonSeriesLabelSuffixFromParts(
+            $metadata['comparePeriodPretty'], $metadata['compareSegmentPretty']);
+
+        return $metadata;
+    }
+
+    private static function getComparisonSeriesLabelSuffixFromParts($periodPretty, $segmentPretty)
+    {
+        $comparisonLabels = [
+            $periodPretty,
+            $segmentPretty,
+        ];
+        $comparisonLabels = array_filter($comparisonLabels);
+
+        return '(' . implode(') (', $comparisonLabels) . ')';
+    }
+
+    private function getSegmentNameFromReport(Report $report = null)
+    {
+        if (empty($report)) {
+            return null;
+        }
+
+        $dimension = $report->getDimension();
+        if (empty($dimension)) {
+            return null;
+        }
+
+        $segments = $dimension->getSegments();
+        if (empty($segments)) {
+            return null;
+        }
+
+        /** @var \Piwik\Plugin\Segment $segment */
+        $segment     = reset($segments);
+        $segmentName = $segment->getSegment();
+        return $segmentName;
+    }
+
+    private function checkMultiplePeriodCompare()
+    {
+        if ($this->isRequestMultiplePeriod()) {
+            foreach ($this->comparePeriods as $index => $period) {
+                if (!Period::isMultiplePeriod($this->compareDates[$index], $period)) {
+                    throw new \Exception("Cannot compare: original request is multiple period and cannot be compared with single periods.");
+                }
+            }
+        } else {
+            foreach ($this->comparePeriods as $index => $period) {
+                if (Period::isMultiplePeriod($this->compareDates[$index], $period)) {
+                    throw new \Exception("Cannot compare: original request is single period and cannot be compared with multiple periods.");
+                }
+            }
+        }
+    }
+
+    private function isRequestMultiplePeriod()
+    {
+        if ($this->isRequestMultiplePeriod === null) {
+            $period = Common::getRequestVar('period', $default = null, 'string', $this->request);
+            $date = Common::getRequestVar('date', $default = null, 'string', $this->request);
+
+            $this->isRequestMultiplePeriod = Period::isMultiplePeriod($date, $period);
+        }
+        return $this->isRequestMultiplePeriod;
+    }
+
+    private function compareChangePercents(DataTableInterface $result)
+    {
+        $segmentCount = count($this->compareSegments);
+
+        $result->filter(function (DataTable $table) use ($segmentCount) {
+            $rows = $table->getRows();
+
+            $totalRow = $table->getTotalsRow();
+            if ($totalRow) {
+                $rows[] = $totalRow;
+            }
+
+            foreach ($rows as $row) {
+                $comparisons = $row->getComparisons();
+                if (empty($comparisons)) {
+                    continue;
+                }
+
+                /** @var DataTable\Row[] $rows */
+                $rows = array_values($comparisons->getRows());
+                foreach ($rows as $index => $compareRow) {
+                    if ($index < $segmentCount) {
+                        continue; // do not calculate for first period
+                    }
+
+                    list($periodIndex, $segmentIndex) = self::getIndividualComparisonRowIndices($table, $index, $segmentCount);
+
+                    $otherPeriodRowIndex = $segmentIndex;
+                    $otherPeriodRow = $comparisons[$otherPeriodRowIndex];
+
+                    foreach ($compareRow->getColumns() as $name => $value) {
+                        $valueToCompare = $otherPeriodRow ? $otherPeriodRow->getColumn($name) : 0;
+                        $valueToCompare = $valueToCompare ?: 0;
+
+                        $change = DataTable\Filter\CalculateEvolutionFilter::calculate($value, $valueToCompare, $precision = 1, $appendPercent = false);
+
+                        if ($change >= 0) {
+                            $change = '+' . $change;
+                        }
+                        $change .= '%';
+
+                        $compareRow->addColumn($name . '_change', $change);
+                    }
+                }
+            }
+        });
+    }
+
+    /**
+     * Returns the period and segment indices for a given comparison index.
+     *
+     * @param DataTable|null $table
+     * @param $comparisonRowIndex
+     * @param null $segmentCount
+     * @return array
+     */
+    public static function getIndividualComparisonRowIndices($table, $comparisonRowIndex, $segmentCount = null)
+    {
+        $segmentCount = $segmentCount ?: count($table->getMetadata('compareSegments'));
+        $segmentIndex = $comparisonRowIndex % $segmentCount;
+        $periodIndex = floor($comparisonRowIndex / $segmentCount);
+        return [$periodIndex, $segmentIndex];
+    }
+
+    /**
+     * Returns the series index for a comparison based on the period and segment indices.
+     *
+     * @param DataTable|null $table
+     * @param int $periodIndex
+     * @param int $segmentIndex
+     * @param int|null $segmentCount
+     * @return int
+     */
+    public static function getComparisonSeriesIndex($table, $periodIndex, $segmentIndex, $segmentCount = null)
+    {
+        $segmentCount = $segmentCount ?: count($table->getMetadata('compareSegments'));
+        return $periodIndex * $segmentCount + $segmentIndex;
+    }
+
+    private static function getCompareSegments($request = null)
+    {
+        $segments = Common::getRequestVar('compareSegments', $default = [], $type = 'array', $request);
+        array_unshift($segments, Common::getRequestVar('segment', '', 'string', $request));
+        $segments = Common::unsanitizeInputValues($segments);
+        return $segments;
+    }
+
+    private static function getComparePeriods($request = null)
+    {
+        $periods = Common::getRequestVar('comparePeriods', $default = [], $type = 'array', $request);
+        array_unshift($periods, Common::getRequestVar('period', '', 'string', $request));
+        return array_values($periods);
+    }
+
+    private static function getCompareDates($request = null)
+    {
+        $dates = Common::getRequestVar('compareDates', $default = [], $type = 'array', $request);
+        array_unshift($dates, Common::getRequestVar('date', '', 'string', $request));
+        return array_values($dates);
+    }
+
+    /**
+     * Returns the pretty series label for a specific comparison based on the currently set comparison query parameters.
+     *
+     * @param int $labelSeriesIndex The index of the comparison. Comparison series order is determined by {@see self::getReportsToCompare()}.
+     */
+    public static function getPrettyComparisonLabelFromSeriesIndex($labelSeriesIndex)
+    {
+        $compareSegments = self::getCompareSegments();
+        $comparePeriods = self::getComparePeriods();
+        $compareDates = self::getCompareDates();
+
+        list($periodIndex, $segmentIndex) = self::getIndividualComparisonRowIndices(null, $labelSeriesIndex, count($compareSegments));
+
+        $segmentObj = new Segment($compareSegments[$segmentIndex], []);
+        $prettySegment = $segmentObj->getStoredSegmentName(false);
+
+        $prettyPeriod = Factory::build($comparePeriods[$periodIndex], $compareDates[$periodIndex])->getLocalizedLongString();
+        $prettyPeriod = ucfirst($prettyPeriod);
+
+        return self::getComparisonSeriesLabelSuffixFromParts($prettyPeriod, $prettySegment);
+    }
+
+    private function getColumnMappings()
+    {
+        $allMappings = Metrics::getMappingFromIdToName();
+
+        $mappings = [];
+        foreach ($allMappings as $index => $name) {
+            $mappings[$index] = $name;
+            $mappings[$index . '_change'] = $name . '_change';
+        }
+        return $mappings;
+    }
+}
\ No newline at end of file
diff --git a/plugins/API/Filter/DataComparisonFilter/ComparisonRowGenerator.php b/plugins/API/Filter/DataComparisonFilter/ComparisonRowGenerator.php
new file mode 100644
index 0000000000..eaed08c13b
--- /dev/null
+++ b/plugins/API/Filter/DataComparisonFilter/ComparisonRowGenerator.php
@@ -0,0 +1,239 @@
+<?php
+/**
+ * Piwik - free/libre analytics platform
+ *
+ * @link http://piwik.org
+ * @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
+ */
+
+namespace Piwik\Plugins\API\Filter\DataComparisonFilter;
+
+use Piwik\DataTable;
+use Piwik\DataTable\DataTableInterface;
+use Piwik\DataTable\Simple;
+use Piwik\Metrics;
+use Piwik\Period;
+use Piwik\Segment;
+use Piwik\Segment\SegmentExpression;
+
+class ComparisonRowGenerator
+{
+    /**
+     * @var bool
+     */
+    private $isRequestMultiplePeriod;
+
+    /**
+     * @var string
+     */
+    private $segmentNameForReport;
+
+    /**
+     * @var array
+     */
+    private $columnMappings;
+
+    public function __construct($segmentNameForReport, $isRequestMultiplePeriod, $columnMappings)
+    {
+        $this->segmentNameForReport = $segmentNameForReport;
+        $this->isRequestMultiplePeriod = $isRequestMultiplePeriod;
+        $this->columnMappings = $columnMappings;
+    }
+
+    public function compareTables($compareMetadata, DataTableInterface $tables, DataTableInterface $compareTables = null)
+    {
+        if ($tables instanceof DataTable) {
+            $this->compareTable($compareMetadata, $tables, $compareTables, $compareTables);
+        } else if ($tables instanceof DataTable\Map) {
+            $childTablesArray = array_values($tables->getDataTables());
+            $compareTablesArray = isset($compareTables) ? array_values($compareTables->getDataTables()) : [];
+
+            $isDatePeriod = $tables->getKeyName() == 'date';
+
+            foreach ($childTablesArray as $index => $childTable) {
+                $compareChildTable = isset($compareTablesArray[$index]) ? $compareTablesArray[$index] : null;
+                $this->compareTables($compareMetadata, $childTable, $compareChildTable);
+            }
+
+            // in case one of the compared periods has more periods than the main one, we want to fill the result with empty datatables
+            // so the comparison data is still present. this allows us to see that data in an evolution report.
+            if ($isDatePeriod) {
+                $lastTable = end($childTablesArray);
+
+                /** @var Period $lastPeriod */
+                $lastPeriod = $lastTable->getMetadata('period');
+                $periodType = $lastPeriod->getLabel();
+
+                for ($i = count($childTablesArray); $i < count($compareTablesArray); ++$i) {
+                    $periodChangeCount = $i - count($childTablesArray) + 1;
+                    $newPeriod = Period\Factory::build($periodType, $lastPeriod->getDateStart()->addPeriod($periodChangeCount, $periodType));
+
+                    // create an empty table for the main request
+                    $newTable = new DataTable();
+                    $newTable->setAllTableMetadata($lastTable->getAllTableMetadata());
+                    $newTable->setMetadata('period', $newPeriod);
+
+                    if ($newPeriod->getLabel() === 'week' || $newPeriod->getLabel() === 'range') {
+                        $periodLabel = $newPeriod->getRangeString();
+                    } else {
+                        $periodLabel = $newPeriod->getPrettyString();
+                    }
+
+                    $tables->addTable($newTable, $periodLabel);
+
+                    // compare with the empty table
+                    $compareTable = $compareTablesArray[$i];
+                    $this->compareTables($compareMetadata, $newTable, $compareTable);
+                }
+            }
+        } else {
+            throw new \Exception("Unexpected DataTable type: " . get_class($tables));
+        }
+    }
+
+    private function compareTable($compareMetadata, DataTable $table, DataTable $rootCompareTable = null, DataTable $compareTable = null)
+    {
+        // if there are no rows in the table because the metrics are 0, add one so we can still set comparison values
+        if ($table->getRowsCount() == 0) {
+            $table->addRow(new DataTable\Row());
+        }
+
+        foreach ($table->getRows() as $row) {
+            $label = $row->getColumn('label');
+
+            $compareRow = null;
+            if ($compareTable instanceof Simple) {
+                $compareRow = $compareTable->getFirstRow() ?: null;
+            } else if ($compareTable instanceof DataTable) {
+                $compareRow = $compareTable->getRowFromLabel($label) ?: null;
+            }
+
+            $this->compareRow($table, $compareMetadata, $row, $compareRow, $rootCompareTable);
+        }
+
+        $totalsRow = $table->getTotalsRow();
+        if (!empty($totalsRow)) {
+            $compareRow = $compareTable ? $compareTable->getTotalsRow() : null;
+            $this->compareRow($table, $compareMetadata, $totalsRow, $compareRow, $rootCompareTable);
+        }
+
+        if ($compareTable) {
+            $totals = $compareTable->getMetadata('totals');
+            if (!empty($totals)) {
+                $totals = $this->replaceIndexesInTotals($totals);
+                $comparisonTotalsEntry = array_merge($compareMetadata, [
+                    'totals' => $totals,
+                ]);
+
+                $allTotalsTables = $table->getMetadata('comparisonTotals');
+                $allTotalsTables[] = $comparisonTotalsEntry;
+                $table->setMetadata('comparisonTotals', $allTotalsTables);
+            }
+        }
+    }
+
+    private function compareRow(DataTable $table, $compareMetadata, DataTable\Row $row, DataTable\Row $compareRow = null, DataTable $rootTable = null)
+    {
+        $comparisonDataTable = $row->getComparisons();
+        if (empty($comparisonDataTable)) {
+            $comparisonDataTable = new DataTable();
+            $comparisonDataTable->setMetadata(DataTable::EXTRA_PROCESSED_METRICS_METADATA_NAME,
+                $table->getMetadata(DataTable::EXTRA_PROCESSED_METRICS_METADATA_NAME));
+            $row->setComparisons($comparisonDataTable);
+        }
+
+        $this->addIndividualChildPrettifiedMetadata($compareMetadata, $rootTable);
+
+        $columns = [];
+        if ($compareRow) {
+            foreach ($compareRow as $name => $value) {
+                if (!is_numeric($value)
+                    || $name == 'label'
+                ) {
+                    continue;
+                }
+
+                $columns[$name] = $value;
+            }
+        } else {
+            foreach ($row as $name => $value) {
+                if (!is_numeric($value)
+                    || $name == 'label'
+                ) {
+                    continue;
+                }
+
+                $columns[$name] = 0;
+            }
+        }
+
+        $newRow = new DataTable\Row([
+            DataTable\Row::COLUMNS => $columns,
+            DataTable\Row::METADATA => $compareMetadata,
+        ]);
+
+        // set subtable
+        $newRow->setMetadata('idsubdatatable', -1);
+        if ($compareRow) {
+            $subtableId = $compareRow->getMetadata('idsubdatatable_in_db') ?: $compareRow->getIdSubDataTable();
+            if ($subtableId) {
+                $newRow->setMetadata('idsubdatatable', $subtableId);
+            }
+        }
+
+        // add segment metadatas
+        if ($row->getMetadata('segment')) {
+            $newSegment = $row->getMetadata('segment');
+            if ($newRow->getMetadata('compareSegment')) {
+                $newSegment = Segment::combine($newRow->getMetadata('compareSegment'), SegmentExpression::AND_DELIMITER, $newSegment);
+            }
+            $newRow->setMetadata('segment', $newSegment);
+        } else if ($this->segmentNameForReport
+            && $row->getMetadata('segmentValue') !== false
+        ) {
+            $segmentValue = $row->getMetadata('segmentValue');
+            $newRow->setMetadata('segment', sprintf('%s==%s', $this->segmentNameForReport, urlencode($segmentValue)));
+        }
+
+        $comparisonDataTable->addRow($newRow);
+
+        // recurse on subtable if there
+        $subtable = $row->getSubtable();
+        if ($subtable
+            && $compareRow
+        ) {
+            $this->compareTable($compareMetadata, $subtable, $rootTable, $compareRow->getSubtable());
+        }
+    }
+
+    private function addIndividualChildPrettifiedMetadata(array &$metadata, DataTable $parentTable = null)
+    {
+        if ($parentTable
+            && $this->isRequestMultiplePeriod
+        ) {
+            /** @var Period $period */
+            $period = $parentTable->getMetadata('period');
+            if (empty($period)) {
+                return;
+            }
+
+            $prettyPeriod = $period->getLocalizedLongString();
+            $metadata['comparePeriodPretty'] = ucfirst($prettyPeriod);
+
+            $metadata['comparePeriod'] = $period->getLabel();
+            $metadata['compareDate'] = $period->getDateStart()->toString();
+        }
+    }
+
+    private function replaceIndexesInTotals($totals)
+    {
+        foreach ($totals as $index => $value) {
+            if (isset($this->columnMappings[$index])) {
+                $name = $this->columnMappings[$index];
+                $totals[$name] = $totals[$index];
+                unset($totals[$index]);
+            }
+        }
+        return $totals;
+    }
+}
\ No newline at end of file
diff --git a/plugins/API/ProcessedReport.php b/plugins/API/ProcessedReport.php
index 79e01da316..6e963ec416 100644
--- a/plugins/API/ProcessedReport.php
+++ b/plugins/API/ProcessedReport.php
@@ -600,15 +600,17 @@ class ProcessedReport
      * - extract row metadata to a separate Simple $rowsMetadata
      *
      * @param int $idSite enables monetary value formatting based on site currency
-     * @param Simple $simpleDataTable
+     * @param DataTable $simpleDataTable
      * @param array $metadataColumns
      * @param boolean $hasDimension
      * @param bool $returnRawMetrics If set to true, the original metrics will be returned
      * @param bool|null $formatMetrics
      * @return array DataTable $enhancedDataTable filtered metrics with human readable format & Simple $rowsMetadata
      */
-    private function handleSimpleDataTable($idSite, $simpleDataTable, $metadataColumns, $hasDimension, $returnRawMetrics = false, $formatMetrics = null)
+    private function handleSimpleDataTable($idSite, $simpleDataTable, $metadataColumns, $hasDimension, $returnRawMetrics = false, $formatMetrics = null, $keepMetadata = false)
     {
+        $comparisonColumns = $this->getComparisonColumns($metadataColumns);
+
         // new DataTable to store metadata
         $rowsMetadata = new DataTable();
 
@@ -634,7 +636,11 @@ class ProcessedReport
                 }
             }
 
-            $enhancedRow = new Row();
+            $c = [];
+            if ($keepMetadata) {
+                $c[Row::METADATA] = $row->getMetadata();
+            }
+            $enhancedRow = new Row($c);
             $enhancedDataTable->addRow($enhancedRow);
 
             foreach ($rowMetrics as $columnName => $columnValue) {
@@ -669,11 +675,23 @@ class ProcessedReport
                 }
             }
 
+            /** @var DataTable $comparisons */
+            $comparisons = $row->getComparisons();
+
+            if (!empty($comparisons)
+                && $comparisons->getRowsCount() > 0
+            ) {
+                list($newComparisons, $ignore) = $this->handleSimpleDataTable($idSite, $comparisons, $comparisonColumns, true, $returnRawMetrics, $formatMetrics, $keepMetadata = true);
+                $enhancedRow->setComparisons($newComparisons);
+            }
+
             // If report has a dimension, extract metadata into a distinct DataTable
             if ($hasDimension) {
                 $rowMetadata = $row->getMetadata();
                 $idSubDataTable = $row->getIdSubDataTable();
 
+                unset($rowMetadata[Row::COMPARISONS_METADATA_NAME]);
+
                 // always add a metadata row - even if empty, so the number of rows and metadata are equal and can be matched directly
                 $metadataRow = new Row();
                 $rowsMetadata->addRow($metadataRow);
@@ -830,6 +848,10 @@ class ProcessedReport
             return $value;
         }
 
+        if (strpos($columnName, '_change') !== false) { // comparison change columns are formatted by DataComparisonFilter
+            return $value == '0' ? '+0%' : $value;
+        }
+
         // Display time in human readable
 		if (strpos($columnName, 'time_generation') !== false) {
 			return $formatter->getPrettyTimeFromSeconds($value, true);
@@ -853,4 +875,13 @@ class ProcessedReport
 
         return $value;
     }
+
+    private function getComparisonColumns(array $metadataColumns)
+    {
+        $result = $metadataColumns;
+        foreach ($metadataColumns as $columnName => $columnTranslation) {
+            $result[$columnName . '_change'] = Piwik::translate('General_ChangeInX', lcfirst($columnName));
+        }
+        return $result;
+    }
 }
diff --git a/plugins/API/RowEvolution.php b/plugins/API/RowEvolution.php
index 9025eb3436..24e695c88b 100644
--- a/plugins/API/RowEvolution.php
+++ b/plugins/API/RowEvolution.php
@@ -19,6 +19,7 @@ use Piwik\DataTable\Filter\SafeDecodeLabel;
 use Piwik\DataTable\Row;
 use Piwik\Period;
 use Piwik\Piwik;
+use Piwik\Plugins\API\Filter\DataComparisonFilter;
 use Piwik\Site;
 use Piwik\Url;
 
@@ -36,7 +37,7 @@ class RowEvolution
         'getPageUrl'
     );
 
-    public function getRowEvolution($idSite, $period, $date, $apiModule, $apiAction, $label = false, $segment = false, $column = false, $language = false, $apiParameters = array(), $legendAppendMetric = true, $labelUseAbsoluteUrl = true)
+    public function getRowEvolution($idSite, $period, $date, $apiModule, $apiAction, $label = false, $segment = false, $column = false, $language = false, $apiParameters = array(), $legendAppendMetric = true, $labelUseAbsoluteUrl = true, $labelSeries = '')
     {
         // validation of requested $period & $date
         if ($period == 'range') {
@@ -49,7 +50,7 @@ class RowEvolution
         }
 
         $label = DataTablePostProcessor::unsanitizeLabelParameter($label);
-        $labels = Piwik::getArrayFromApiParameter($label);
+        $labels = Piwik::getArrayFromApiParameter($label, $onlyUnique = empty($labelSeries));
 
         $metadata = $this->getRowEvolutionMetaData($idSite, $period, $date, $apiModule, $apiAction, $language, $apiParameters);
 
@@ -72,7 +73,8 @@ class RowEvolution
                 $labels,
                 $column,
                 $legendAppendMetric,
-                $labelUseAbsoluteUrl
+                $labelUseAbsoluteUrl,
+                $labelSeries
             );
         } else {
             $data = $this->getSingleRowEvolution(
@@ -420,8 +422,13 @@ class RowEvolution
     /** Get row evolution for a multiple labels */
     private function getMultiRowEvolution(DataTable\Map $dataTable, $metadata, $apiModule, $apiAction, $labels, $column,
                                           $legendAppendMetric = true,
-                                          $labelUseAbsoluteUrl = true)
+                                          $labelUseAbsoluteUrl = true,
+                                          $labelSeries = '')
     {
+        $labelSeries = explode(',', $labelSeries);
+        $labelSeries = array_filter($labelSeries, 'strlen');
+        $labelSeries = array_map('intval', $labelSeries);
+
         if (!isset($metadata['metrics'][$column])) {
             // invalid column => use the first one that's available
             $metrics = array_keys($metadata['metrics']);
@@ -455,6 +462,11 @@ class RowEvolution
                 $cleanLabel = $this->cleanOriginalLabel($label);
                 $actualLabels[$labelIdx] = $cleanLabel;
             }
+
+            if (isset($labelSeries[$labelIdx])) {
+                $labelSeriesIndex = $labelSeries[$labelIdx];
+                $actualLabels[$labelIdx] .= ' ' . DataComparisonFilter::getPrettyComparisonLabelFromSeriesIndex($labelSeriesIndex);
+            }
         }
 
         // convert rows to be array($column.'_'.$labelIdx => $value) as opposed to
diff --git a/plugins/API/tests/Integration/Filter/DataComparisonFilter/ComparisonRowGeneratorTest.php b/plugins/API/tests/Integration/Filter/DataComparisonFilter/ComparisonRowGeneratorTest.php
new file mode 100644
index 0000000000..1f9f3f17ec
--- /dev/null
+++ b/plugins/API/tests/Integration/Filter/DataComparisonFilter/ComparisonRowGeneratorTest.php
@@ -0,0 +1,613 @@
+<?php
+/**
+ * Piwik - free/libre analytics platform
+ *
+ * @link http://piwik.org
+ * @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
+ */
+
+namespace Piwik\Plugins\API\tests\Integration\Filter\DataComparisonFilter;
+
+use Piwik\DataTable;
+use Piwik\Period\Factory;
+use Piwik\Plugins\API\Filter\DataComparisonFilter\ComparisonRowGenerator;
+use Piwik\Plugins\SegmentEditor\API;
+use Piwik\Tests\Framework\TestCase\IntegrationTestCase;
+
+class ComparisonRowGeneratorTest extends IntegrationTestCase
+{
+    const TEST_SEGMENT = 'browserCode==ff';
+    const OTHER_SEGMENT = 'operatingSystemCode=WIN';
+
+    protected static function beforeTableDataCached()
+    {
+        parent::beforeTableDataCached();
+
+        API::getInstance()->add('test segment', self::TEST_SEGMENT);
+    }
+
+    public function test_compareTables_shouldCompareTwoDataTablesCorrectly()
+    {
+        $table1 = $this->makeTable([
+            ['label' => 'row1', 'nb_visits' => 5, 'nb_actions' => 10],
+            ['label' => 'row2', 'nb_visits' => 10, 'nb_actions' => 25],
+            ['label' => 'row3', 'nb_visits' => 20],
+            ['label' => 'row4', 'nb_actions' => 30],
+        ]);
+
+        $table2 = $this->makeTable([
+            ['label' => 'row1', 'nb_visits' => 10, 'nb_actions' => 5],
+            ['label' => 'row3', 'somethingelse' => 25],
+        ]);
+
+        $compareMetadata = [
+            'compareSegment' => self::TEST_SEGMENT,
+            'comparePeriod' => 'day',
+            'compareDate' => '2012-03-04',
+        ];
+
+        $comparisonRowGenerator = new ComparisonRowGenerator('reportSegment', false, []);
+        $comparisonRowGenerator->compareTables($compareMetadata, $table1, $table2);
+
+        $xmlContent = $this->toXml($table1);
+
+        $expectedXml = <<<END
+<?xml version="1.0" encoding="utf-8" ?>
+<result>
+	<row>
+		<label>row1</label>
+		<nb_visits>5</nb_visits>
+		<nb_actions>10</nb_actions>
+		<comparisons>
+			<row>
+				<nb_visits>10</nb_visits>
+				<nb_actions>5</nb_actions>
+				<compareSegment>browserCode==ff</compareSegment>
+				<comparePeriod>day</comparePeriod>
+				<compareDate>2012-03-04</compareDate>
+				<idsubdatatable>-1</idsubdatatable>
+			</row>
+		</comparisons>
+	</row>
+	<row>
+		<label>row2</label>
+		<nb_visits>10</nb_visits>
+		<nb_actions>25</nb_actions>
+		<comparisons>
+			<row>
+				<nb_visits>0</nb_visits>
+				<nb_actions>0</nb_actions>
+				<compareSegment>browserCode==ff</compareSegment>
+				<comparePeriod>day</comparePeriod>
+				<compareDate>2012-03-04</compareDate>
+				<idsubdatatable>-1</idsubdatatable>
+			</row>
+		</comparisons>
+	</row>
+	<row>
+		<label>row3</label>
+		<nb_visits>20</nb_visits>
+		<comparisons>
+			<row>
+				<somethingelse>25</somethingelse>
+				<compareSegment>browserCode==ff</compareSegment>
+				<comparePeriod>day</comparePeriod>
+				<compareDate>2012-03-04</compareDate>
+				<idsubdatatable>-1</idsubdatatable>
+			</row>
+		</comparisons>
+	</row>
+	<row>
+		<label>row4</label>
+		<nb_actions>30</nb_actions>
+		<comparisons>
+			<row>
+				<nb_actions>0</nb_actions>
+				<compareSegment>browserCode==ff</compareSegment>
+				<comparePeriod>day</comparePeriod>
+				<compareDate>2012-03-04</compareDate>
+				<idsubdatatable>-1</idsubdatatable>
+			</row>
+		</comparisons>
+	</row>
+</result>
+END;
+        $this->assertEquals($expectedXml, $xmlContent);
+    }
+
+    public function test_compareTables_shouldUseFirstTableRowsForComparisons()
+    {
+        $table1 = $this->makeTable([
+            ['label' => 'row1', 'nb_visits' => 10, 'nb_actions' => 5],
+            ['label' => 'row3', 'somethingelse' => 25],
+        ]);
+
+        $table2 = $this->makeTable([
+            ['label' => 'row1', 'nb_visits' => 5, 'nb_actions' => 10],
+            ['label' => 'row2', 'nb_visits' => 10, 'nb_actions' => 25],
+            ['label' => 'row3', 'nb_visits' => 20],
+            ['label' => 'row4', 'nb_actions' => 30],
+        ]);
+
+        $compareMetadata = [
+            'compareSegment' => self::TEST_SEGMENT,
+            'comparePeriod' => 'day',
+            'compareDate' => '2012-03-04',
+        ];
+
+        $comparisonRowGenerator = new ComparisonRowGenerator('reportSegment', false, []);
+        $comparisonRowGenerator->compareTables($compareMetadata, $table1, $table2);
+
+        $xmlContent = $this->toXml($table1);
+
+        $expectedXml = <<<END
+<?xml version="1.0" encoding="utf-8" ?>
+<result>
+	<row>
+		<label>row1</label>
+		<nb_visits>10</nb_visits>
+		<nb_actions>5</nb_actions>
+		<comparisons>
+			<row>
+				<nb_visits>5</nb_visits>
+				<nb_actions>10</nb_actions>
+				<compareSegment>browserCode==ff</compareSegment>
+				<comparePeriod>day</comparePeriod>
+				<compareDate>2012-03-04</compareDate>
+				<idsubdatatable>-1</idsubdatatable>
+			</row>
+		</comparisons>
+	</row>
+	<row>
+		<label>row3</label>
+		<somethingelse>25</somethingelse>
+		<comparisons>
+			<row>
+				<nb_visits>20</nb_visits>
+				<compareSegment>browserCode==ff</compareSegment>
+				<comparePeriod>day</comparePeriod>
+				<compareDate>2012-03-04</compareDate>
+				<idsubdatatable>-1</idsubdatatable>
+			</row>
+		</comparisons>
+	</row>
+</result>
+END;
+        $this->assertEquals($expectedXml, $xmlContent);
+    }
+
+    public function test_compareTables_shouldCompareTwoDataTableMapsCorrectly()
+    {
+        $tableSet1 = $this->makeTableMap([
+            '2012-01-01' => [
+                ['label' => 'row1', 'nb_visits' => 10, 'nb_actions' => 15],
+                ['label' => 'row2', 'nb_visits' => 15, 'nb_actions' => 15],
+                ['label' => 'row3', 'nb_visits' => 20, 'nb_actions' => 10],
+            ],
+            '2012-02-01' => [
+                ['label' => 'row2', 'nb_visits' => 25, 'nb_actions' => 25],
+            ],
+            '2012-03-01' => [
+                ['label' => 'row3', 'nb_visits' => 20, 'nb_actions' => 10],
+                ['label' => 'row4', 'nb_visits' => 40, 'nb_actions' => 50],
+            ],
+        ]);
+
+        $tableSet2 = $this->makeTableMap([
+            '2012-01-01' => [
+                ['label' => 'row1', 'nb_visits' => 10, 'nb_actions' => 15],
+            ],
+            '2012-02-01' => [
+                ['label' => 'row2', 'nb_visits' => 15, 'nb_actions' => 15],
+                ['label' => 'row3', 'nb_visits' => 20, 'nb_actions' => 10],
+            ],
+            '2012-03-01' => [
+                ['label' => 'row2', 'nb_visits' => 25, 'nb_actions' => 25],
+                ['label' => 'row3', 'nb_visits' => 20, 'nb_actions' => 10],
+                ['label' => 'row4', 'nb_visits' => 40, 'nb_actions' => 50],
+            ],
+        ]);
+
+        $compareMetadata = [
+            'compareSegment' => self::OTHER_SEGMENT,
+            'comparePeriod' => 'month',
+            'compareDate' => '2012-01-01,2012-03-01',
+        ];
+
+        $comparisonRowGenerator = new ComparisonRowGenerator('reportSegment', false, []);
+        $comparisonRowGenerator->compareTables($compareMetadata, $tableSet1, $tableSet2);
+
+        $xmlContent = $this->toXml($tableSet1);
+
+        $expectedXml = <<<END
+<?xml version="1.0" encoding="utf-8" ?>
+<results>
+	<result date="2012-01">
+		<row>
+			<label>row1</label>
+			<nb_visits>10</nb_visits>
+			<nb_actions>15</nb_actions>
+			<comparisons>
+				<row>
+					<nb_visits>10</nb_visits>
+					<nb_actions>15</nb_actions>
+					<compareSegment>operatingSystemCode=WIN</compareSegment>
+					<comparePeriod>month</comparePeriod>
+					<compareDate>2012-01-01,2012-03-01</compareDate>
+					<idsubdatatable>-1</idsubdatatable>
+				</row>
+			</comparisons>
+		</row>
+		<row>
+			<label>row2</label>
+			<nb_visits>15</nb_visits>
+			<nb_actions>15</nb_actions>
+			<comparisons>
+				<row>
+					<nb_visits>0</nb_visits>
+					<nb_actions>0</nb_actions>
+					<compareSegment>operatingSystemCode=WIN</compareSegment>
+					<comparePeriod>month</comparePeriod>
+					<compareDate>2012-01-01,2012-03-01</compareDate>
+					<idsubdatatable>-1</idsubdatatable>
+				</row>
+			</comparisons>
+		</row>
+		<row>
+			<label>row3</label>
+			<nb_visits>20</nb_visits>
+			<nb_actions>10</nb_actions>
+			<comparisons>
+				<row>
+					<nb_visits>0</nb_visits>
+					<nb_actions>0</nb_actions>
+					<compareSegment>operatingSystemCode=WIN</compareSegment>
+					<comparePeriod>month</comparePeriod>
+					<compareDate>2012-01-01,2012-03-01</compareDate>
+					<idsubdatatable>-1</idsubdatatable>
+				</row>
+			</comparisons>
+		</row>
+	</result>
+	<result date="2012-02">
+		<row>
+			<label>row2</label>
+			<nb_visits>25</nb_visits>
+			<nb_actions>25</nb_actions>
+			<comparisons>
+				<row>
+					<nb_visits>15</nb_visits>
+					<nb_actions>15</nb_actions>
+					<compareSegment>operatingSystemCode=WIN</compareSegment>
+					<comparePeriod>month</comparePeriod>
+					<compareDate>2012-01-01,2012-03-01</compareDate>
+					<idsubdatatable>-1</idsubdatatable>
+				</row>
+			</comparisons>
+		</row>
+	</result>
+	<result date="2012-03">
+		<row>
+			<label>row3</label>
+			<nb_visits>20</nb_visits>
+			<nb_actions>10</nb_actions>
+			<comparisons>
+				<row>
+					<nb_visits>20</nb_visits>
+					<nb_actions>10</nb_actions>
+					<compareSegment>operatingSystemCode=WIN</compareSegment>
+					<comparePeriod>month</comparePeriod>
+					<compareDate>2012-01-01,2012-03-01</compareDate>
+					<idsubdatatable>-1</idsubdatatable>
+				</row>
+			</comparisons>
+		</row>
+		<row>
+			<label>row4</label>
+			<nb_visits>40</nb_visits>
+			<nb_actions>50</nb_actions>
+			<comparisons>
+				<row>
+					<nb_visits>40</nb_visits>
+					<nb_actions>50</nb_actions>
+					<compareSegment>operatingSystemCode=WIN</compareSegment>
+					<comparePeriod>month</comparePeriod>
+					<compareDate>2012-01-01,2012-03-01</compareDate>
+					<idsubdatatable>-1</idsubdatatable>
+				</row>
+			</comparisons>
+		</row>
+	</result>
+</results>
+END;
+        $this->assertEquals($expectedXml, $xmlContent);
+    }
+
+    public function test_compareTables_shouldCompareTwoDataTaleMapsOfDifferentLengthsCorrectly_whenFirstIsLonger()
+    {
+        $tableSet1 = $this->makeTableMap([
+            '2012-01-01' => [
+                ['label' => 'row1', 'nb_visits' => 10, 'nb_actions' => 15],
+                ['label' => 'row2', 'nb_visits' => 15, 'nb_actions' => 15],
+                ['label' => 'row3', 'nb_visits' => 20, 'nb_actions' => 10],
+            ],
+            '2012-02-01' => [
+                ['label' => 'row2', 'nb_visits' => 25, 'nb_actions' => 25],
+            ],
+            '2012-03-01' => [
+                ['label' => 'row3', 'nb_visits' => 20, 'nb_actions' => 10],
+                ['label' => 'row4', 'nb_visits' => 40, 'nb_actions' => 50],
+            ],
+        ]);
+
+        $tableSet2 = $this->makeTableMap([
+            '2012-01-01' => [
+                ['label' => 'row1', 'nb_visits' => 10, 'nb_actions' => 15],
+            ],
+            '2012-02-01' => [
+                // empty
+            ],
+        ]);
+
+        $compareMetadata = [
+            'compareSegment' => self::OTHER_SEGMENT,
+            'comparePeriod' => 'month',
+            'compareDate' => '2012-01-01,2012-03-01',
+        ];
+
+        $comparisonRowGenerator = new ComparisonRowGenerator('reportSegment', false, []);
+        $comparisonRowGenerator->compareTables($compareMetadata, $tableSet1, $tableSet2);
+
+        $xmlContent = $this->toXml($tableSet1);
+
+        $expectedXml = <<<END
+<?xml version="1.0" encoding="utf-8" ?>
+<results>
+	<result date="2012-01">
+		<row>
+			<label>row1</label>
+			<nb_visits>10</nb_visits>
+			<nb_actions>15</nb_actions>
+			<comparisons>
+				<row>
+					<nb_visits>10</nb_visits>
+					<nb_actions>15</nb_actions>
+					<compareSegment>operatingSystemCode=WIN</compareSegment>
+					<comparePeriod>month</comparePeriod>
+					<compareDate>2012-01-01,2012-03-01</compareDate>
+					<idsubdatatable>-1</idsubdatatable>
+				</row>
+			</comparisons>
+		</row>
+		<row>
+			<label>row2</label>
+			<nb_visits>15</nb_visits>
+			<nb_actions>15</nb_actions>
+			<comparisons>
+				<row>
+					<nb_visits>0</nb_visits>
+					<nb_actions>0</nb_actions>
+					<compareSegment>operatingSystemCode=WIN</compareSegment>
+					<comparePeriod>month</comparePeriod>
+					<compareDate>2012-01-01,2012-03-01</compareDate>
+					<idsubdatatable>-1</idsubdatatable>
+				</row>
+			</comparisons>
+		</row>
+		<row>
+			<label>row3</label>
+			<nb_visits>20</nb_visits>
+			<nb_actions>10</nb_actions>
+			<comparisons>
+				<row>
+					<nb_visits>0</nb_visits>
+					<nb_actions>0</nb_actions>
+					<compareSegment>operatingSystemCode=WIN</compareSegment>
+					<comparePeriod>month</comparePeriod>
+					<compareDate>2012-01-01,2012-03-01</compareDate>
+					<idsubdatatable>-1</idsubdatatable>
+				</row>
+			</comparisons>
+		</row>
+	</result>
+	<result date="2012-02">
+		<row>
+			<label>row2</label>
+			<nb_visits>25</nb_visits>
+			<nb_actions>25</nb_actions>
+			<comparisons>
+				<row>
+					<nb_visits>0</nb_visits>
+					<nb_actions>0</nb_actions>
+					<compareSegment>operatingSystemCode=WIN</compareSegment>
+					<comparePeriod>month</comparePeriod>
+					<compareDate>2012-01-01,2012-03-01</compareDate>
+					<idsubdatatable>-1</idsubdatatable>
+				</row>
+			</comparisons>
+		</row>
+	</result>
+	<result date="2012-03">
+		<row>
+			<label>row3</label>
+			<nb_visits>20</nb_visits>
+			<nb_actions>10</nb_actions>
+			<comparisons>
+				<row>
+					<nb_visits>0</nb_visits>
+					<nb_actions>0</nb_actions>
+					<compareSegment>operatingSystemCode=WIN</compareSegment>
+					<comparePeriod>month</comparePeriod>
+					<compareDate>2012-01-01,2012-03-01</compareDate>
+					<idsubdatatable>-1</idsubdatatable>
+				</row>
+			</comparisons>
+		</row>
+		<row>
+			<label>row4</label>
+			<nb_visits>40</nb_visits>
+			<nb_actions>50</nb_actions>
+			<comparisons>
+				<row>
+					<nb_visits>0</nb_visits>
+					<nb_actions>0</nb_actions>
+					<compareSegment>operatingSystemCode=WIN</compareSegment>
+					<comparePeriod>month</comparePeriod>
+					<compareDate>2012-01-01,2012-03-01</compareDate>
+					<idsubdatatable>-1</idsubdatatable>
+				</row>
+			</comparisons>
+		</row>
+	</result>
+</results>
+END;
+        $this->assertEquals($expectedXml, $xmlContent);
+    }
+
+    public function test_compareTables_shouldCompareTwoDataTaleMapsOfDifferentLengthsCorrectly_whenFirstIsShorter()
+    {
+        $tableSet1 = $this->makeTableMap([
+            '2012-01-01' => [
+                ['label' => 'row1', 'nb_visits' => 10, 'nb_actions' => 15],
+                ['label' => 'row2', 'nb_visits' => 15, 'nb_actions' => 15],
+                ['label' => 'row3', 'nb_visits' => 20, 'nb_actions' => 10],
+            ],
+            '2012-02-01' => [
+                // empty
+            ],
+        ]);
+
+        $tableSet2 = $this->makeTableMap([
+            '2012-01-01' => [
+                ['label' => 'row1', 'nb_visits' => 10, 'nb_actions' => 15],
+            ],
+            '2012-02-01' => [
+                ['label' => 'row2', 'nb_visits' => 15, 'nb_actions' => 15],
+                ['label' => 'row3', 'nb_visits' => 20, 'nb_actions' => 10],
+            ],
+            '2012-03-01' => [
+                ['label' => 'row2', 'nb_visits' => 25, 'nb_actions' => 25],
+                ['label' => 'row3', 'nb_visits' => 20, 'nb_actions' => 10],
+                ['label' => 'row4', 'nb_visits' => 40, 'nb_actions' => 50],
+            ],
+        ]);
+
+        $compareMetadata = [
+            'compareSegment' => self::OTHER_SEGMENT,
+            'comparePeriod' => 'month',
+            'compareDate' => '2012-01-01,2012-03-01',
+        ];
+
+        $comparisonRowGenerator = new ComparisonRowGenerator('reportSegment', false, []);
+        $comparisonRowGenerator->compareTables($compareMetadata, $tableSet1, $tableSet2);
+
+        $xmlContent = $this->toXml($tableSet1);
+
+        $expectedXml = <<<END
+<?xml version="1.0" encoding="utf-8" ?>
+<results>
+	<result date="2012-01">
+		<row>
+			<label>row1</label>
+			<nb_visits>10</nb_visits>
+			<nb_actions>15</nb_actions>
+			<comparisons>
+				<row>
+					<nb_visits>10</nb_visits>
+					<nb_actions>15</nb_actions>
+					<compareSegment>operatingSystemCode=WIN</compareSegment>
+					<comparePeriod>month</comparePeriod>
+					<compareDate>2012-01-01,2012-03-01</compareDate>
+					<idsubdatatable>-1</idsubdatatable>
+				</row>
+			</comparisons>
+		</row>
+		<row>
+			<label>row2</label>
+			<nb_visits>15</nb_visits>
+			<nb_actions>15</nb_actions>
+			<comparisons>
+				<row>
+					<nb_visits>0</nb_visits>
+					<nb_actions>0</nb_actions>
+					<compareSegment>operatingSystemCode=WIN</compareSegment>
+					<comparePeriod>month</comparePeriod>
+					<compareDate>2012-01-01,2012-03-01</compareDate>
+					<idsubdatatable>-1</idsubdatatable>
+				</row>
+			</comparisons>
+		</row>
+		<row>
+			<label>row3</label>
+			<nb_visits>20</nb_visits>
+			<nb_actions>10</nb_actions>
+			<comparisons>
+				<row>
+					<nb_visits>0</nb_visits>
+					<nb_actions>0</nb_actions>
+					<compareSegment>operatingSystemCode=WIN</compareSegment>
+					<comparePeriod>month</comparePeriod>
+					<compareDate>2012-01-01,2012-03-01</compareDate>
+					<idsubdatatable>-1</idsubdatatable>
+				</row>
+			</comparisons>
+		</row>
+	</result>
+	<result date="2012-02">
+		<row>
+			<comparisons>
+				<row>
+					<compareSegment>operatingSystemCode=WIN</compareSegment>
+					<comparePeriod>month</comparePeriod>
+					<compareDate>2012-01-01,2012-03-01</compareDate>
+					<idsubdatatable>-1</idsubdatatable>
+				</row>
+			</comparisons>
+		</row>
+	</result>
+	<result date="2012-03">
+		<row>
+			<comparisons>
+				<row>
+					<compareSegment>operatingSystemCode=WIN</compareSegment>
+					<comparePeriod>month</comparePeriod>
+					<compareDate>2012-01-01,2012-03-01</compareDate>
+					<idsubdatatable>-1</idsubdatatable>
+				</row>
+			</comparisons>
+		</row>
+	</result>
+</results>
+END;
+        $this->assertEquals($expectedXml, $xmlContent);
+    }
+
+    private function makeTable(array $rows)
+    {
+        $table = new DataTable();
+        $table->addRowsFromSimpleArray($rows);
+        return $table;
+    }
+
+    private function makeTableMap(array $tableRows)
+    {
+        $result = new DataTable\Map();
+        $result->setKeyName('date');
+        foreach ($tableRows as $label => $rows) {
+            $period = Factory::build('month', $label);
+
+            $table = $this->makeTable($rows);
+            $table->setMetadata('period', $period);
+
+            $result->addTable($table, $period->getPrettyString());
+        }
+        return $result;
+    }
+
+    private function toXml(DataTable\DataTableInterface $table)
+    {
+        $renderer = new DataTable\Renderer\Xml();
+        $renderer->setTable($table);
+        return $renderer->render();
+    }
+}
\ No newline at end of file
diff --git a/plugins/API/tests/Unit/XmlRendererTest.php b/plugins/API/tests/Unit/XmlRendererTest.php
index 833d303aac..a026f7778a 100644
--- a/plugins/API/tests/Unit/XmlRendererTest.php
+++ b/plugins/API/tests/Unit/XmlRendererTest.php
@@ -28,6 +28,11 @@ class XmlRendererTest extends \PHPUnit_Framework_TestCase
         DataTable\Manager::getInstance()->deleteAll();
     }
 
+    public function tearDown()
+    {
+        DataTable\Manager::getInstance()->deleteAll();
+    }
+
     public function test_renderSuccess_shouldIncludeMessage()
     {
         $response = $this->builder->renderSuccess('ok');