Switch UI tests from phantomjs to chrome headless (#14421)

* Make sure xss entries is always an array.

* Update expected test files and submodule.

* Regenerate broken omnifixture dump.

* poc conversion to chrome headless for UI tests, single test works locally

* Get single test to pass on travis & local w/ headless chrome.

* Remove old diff viewer generation, and get output to look the same as before when there is a failure.

* Add global timeout & get ViewDataTableTest to pass.

* Convert BarGraph_spec.js.

* Convert EmptySite_spec.

* Update EvolutionGraph spec for chrome headless.

* Convert GoalsTable test & find/replace for common changes.

* Convert MeasurableManager.

* Another find & replace.

* Get Menus/OptOutForm to pass w/ chrome headless

* Convert PeriodSelector, PieGraph & PivotByDimension UI tests.

* undo unrelated debugging change

* Convert QuickAccess UI tests

* Converts ActionsDataTable UI tests

* reset viewport after each testsuite

* Converts RowEvolution UI tests

* Converts Theme UI tests

* Converts SiteSelector UI tests

* Converts CustomVariables UI test

* Converts DBStats UI test

* Ignore empty responses when calling api/controller

* Converts Dashboard UI tests

* Converts Live UI tests

* Converts SimpleUI tests

* Converts Installation UI tests

* Converts ImageGraph UI tests

* Converts Login UI tests

* Converts Marketplace UI tests

* Converts Insights UI tests

* Converts ReportExporting UI tests

* Converts UIIntegration tests

* Updates ViewDataTable UI tests

* Converts CoreUpdater UI tests

* Converts UsersManager UI tests

* Converts Morpheus UI tests

* Converts MultiSites UI tests

* Fix testEnvironment.callApi handling of array parameters

* Converts Overlay UI tests

* Converts PrivacyManager UI tests

* Converts ScheduledReports UI tests

* Converts SegmentEditor UI tests

* compare pixel difference in images

* Converts PeriodSelector UI tests

* allow a pixel difference

* Converts SingleMetricView UI tests

* Converts SitesManager UI tests

* Converts Transitions UI tests

* Converts MeasurableManager UI tests

* Move Mouse out of screen after each test suite

* Converts PieGraph UI tests

* Ensure cursor is not shown in focused input elements

* Converts VisitorMap UI tests

* Converts CustomLogo UI tests

* Converts BarGraph UI tests

* Converts EvolutionGraph UI tests

* Converts IntranetMeasurable UI tests

* Converts TrackingFailures UI tests

* Converts CampaignBuilder UI tests

* Converts TagManagerTeaser UI tests

* fix request url rewrite for changed port

* Converts TwoFactorAuth UI tests

* do not print full responses in logs, but size instead

* improve additional style injection

* use submodule branches

* fix request interception

* update dependencies

* [TEMP] disable dangerous link checks

* [TEMP] limit tests

* Use puppeteer@next and make sure diffs get uploaded properly.

* Use correct fs function.

* Allow timeout to be specified on command line for UI tests and debug console logs.

* timeout option tweak

* Setup diff dir once before running tests not before each suite.

* fix last commit

* Update screenshots and fix some specs.

* Convert some tests that were not converted during merge.

* updating screenshots

* Updating screenshots and fixing some tests.

* more test fixes

* couple more test fixes

* More test fixes and plugin updates.

* wait for jquery

* Fix some screenshots and tests.

* more test fixes

* debug travis test failure

* remove .only

* more test fixes and updated expected files

* another test fixing iteration

* typo

* another quick test change

* more tests updates

* Test fixes and & more debugging.

* more debugging and test fixes

* more fixes and debugging.

* remove .only

* Another round of fixed tests.

* more debugging & fixes

* more test changes

* more debugging and fixes

* Fixing more tests + some bug fixes.

* Couple more fixes.

* Couple more fixes.

* Fixing tracking failures test

* more fixes

* Last couple fixes hopefully.

* couple more test fixes

* more fixes, bug fix in usersmanager, some test updates

* Some more test fixes/changes.

* more and more fixes

* hoping for a green build

* Do not compare against existing files in dashboard_spec.

* more fixes, re-enabling travis

* really re-enable travis

* Install puppeteer on travis-ci.

* more fixes

* try to fix random failues

* remove return

* Convert some login tests I forgot and update UsersManager expected file

* Fix test for primary key update

* Update AddPrimaryKey.php

* More test fixes + implement synchronous comparison threshold (so no resemblejs)

* More wait fixes + reset token detection fix.

* couple more fixes

* Fix login/overlay screenshot issue & magick command running issue.

* identify is the executable on travis (hopefully) + fix some random failures

* Another run.

* Last fix.

* two more tweaks

* typo

8 files changed, 58 insertions, 63 deletions

diff --git a/plugins/Transitions/tests/UI/Transitions_spec.js b/plugins/Transitions/tests/UI/Transitions_spec.js
index 47ad47d429..576cd416c1 100644
--- a/plugins/Transitions/tests/UI/Transitions_spec.js
+++ b/plugins/Transitions/tests/UI/Transitions_spec.js
@@ -14,69 +14,64 @@ describe("Transitions", function () {
         urlBase = 'module=CoreHome&action=index&' + generalParams;
 
 
-    function selectValue(page, field, title)
+    async function selectValue(field, title)
     {
-        page.execCallback(function () {
-            page.webpage.evaluate(function(field) {
-                $(field + ' input.select-dropdown').click()
-            }, field);
-        });
-        page.wait(800);
-        page.execCallback(function () {
-            page.webpage.evaluate(function(field, title) {
-                $(field + ' .dropdown-content.active li:contains("' + title + '"):first').click()
-            }, field, title);
-        });
-    };
-
-    it('should load the transitions popup correctly for the page titles report', function (done) {
-        expect.screenshot('transitions_popup_titles').to.be.captureSelector('.ui-dialog', function (page) {
-            page.load("?" + urlBase + "#?" + generalParams + "&category=General_Actions&subcategory=Actions_SubmenuPageTitles");
-
-            page.mouseMove('div.dataTable tbody tr:eq(2)');
-            page.mouseMove('a.actionTransitions:visible'); // necessary to get popover to display
-            page.click('a.actionTransitions:visible');
-        }, done);
+        await page.webpage.evaluate((field) => {
+            $(field + ' input.select-dropdown').click()
+        }, field);
+        await page.waitFor(500);
+        await page.webpage.evaluate((field, title) => {
+            $(field + ' .dropdown-content.active li:contains("' + title + '"):first').click()
+        }, field, title);
+    }
+
+    it('should load the transitions popup correctly for the page titles report', async function() {
+        await page.goto("?" + urlBase + "#?" + generalParams + "&category=General_Actions&subcategory=Actions_SubmenuPageTitles");
+
+        await (await page.jQuery('div.dataTable tbody tr:eq(2)')).hover();
+        await (await page.jQuery('a.actionTransitions:visible')).hover(); // necessary to get popover to display
+        await (await page.jQuery('a.actionTransitions:visible')).click();
+
+        await page.waitForNetworkIdle();
+
+        expect(await page.screenshotSelector('.ui-dialog')).to.matchImage('transitions_popup_titles');
     });
 
-    it('should load the transitions popup correctly for the page urls report', function (done) {
-        expect.screenshot('transitions_popup_urls').to.be.captureSelector('.ui-dialog', function (page) {
-            page.load("?" + urlBase + "#?" + generalParams + "&category=General_Actions&subcategory=General_Pages&"
+    it('should load the transitions popup correctly for the page urls report', async function() {
+        await page.goto("?" + urlBase + "#?" + generalParams + "&category=General_Actions&subcategory=General_Pages&"
                     + "popover=RowAction$3ATransitions$3Aurl$3Ahttp$3A$2F$2Fpiwik.net$2Fdocs$2Fmanage-websites$2F");
-            page.mouseMove('.Transitions_CurveTextRight');
-        }, done);
+        await page.hover('.Transitions_CurveTextRight');
+
+        expect(await page.screenshotSelector('.ui-dialog')).to.matchImage('transitions_popup_urls');
     });
 
-    it('should show no data message in selector', function (done) {
-        expect.screenshot('transitions_report_no_data_widget').to.be.captureSelector('body', function (page) {
-            page.load("?module=Widgetize&action=iframe&widget=1&moduleToWidgetize=Transitions&actionToWidgetize=getTransitions&idSite=1&period=day&date=today&disableLink=1&widget=1");
-        }, done);
+    it('should show no data message in selector', async function () {
+        await page.goto("?module=Widgetize&action=iframe&widget=1&moduleToWidgetize=Transitions&actionToWidgetize=getTransitions&idSite=1&period=day&date=today&disableLink=1&widget=1");
+        expect(await page.screenshotSelector('body')).to.matchImage('transitions_report_no_data_widget');
     });
 
-    it('should show report in reporting ui with data', function (done) {
-        expect.screenshot('transitions_report_with_data_report').to.be.captureSelector('.pageWrap', function (page) {
-            page.load("?" + urlBase + "#?" + generalParams + "&category=General_Actions&subcategory=Transitions_Transitions");
-            page.wait(1000);
-        }, done);
+    it('should show report in reporting ui with data', async function () {
+        await page.goto("?" + urlBase + "#?" + generalParams + "&category=General_Actions&subcategory=Transitions_Transitions");
+        await page.waitForNetworkIdle();
+        expect(await page.screenshotSelector('.pageWrap')).to.matchImage('transitions_report_with_data_report');
     });
 
-    it('should show report in widget ui in selector', function (done) {
-        expect.screenshot('transitions_report_with_data_widget').to.be.captureSelector('body', function (page) {
-            page.load("?module=Widgetize&action=iframe&widget=1&moduleToWidgetize=Transitions&actionToWidgetize=getTransitions&"+generalParams+"&disableLink=1&widget=1");
-            page.wait(1000);
-        }, done);
+    it('should show report in widget ui in selector', async function () {
+        await page.goto("?module=Widgetize&action=iframe&widget=1&moduleToWidgetize=Transitions&actionToWidgetize=getTransitions&"+generalParams+"&disableLink=1&widget=1");
+        await page.waitForNetworkIdle();
+        expect(await page.screenshotSelector('body')).to.matchImage('transitions_report_with_data_widget');
     });
 
-    it('should be possible to switch report', function (done) {
-        expect.screenshot('transitions_report_switch_url').to.be.captureSelector('body', function (page) {
-            selectValue(page, '[name="actionName"]', 'category/meta');
-        }, done);
+    it('should be possible to switch report', async function () {
+        await selectValue('[name="actionName"]', 'category/meta');
+        await page.waitForNetworkIdle();
+        expect(await page.screenshotSelector('body')).to.matchImage('transitions_report_switch_url');
     });
 
-    it('should be possible to show page titles', function (done) {
-        expect.screenshot('transitions_report_switch_type_title').to.be.captureSelector('body', function (page) {
-            selectValue(page, '[name="actionType"]', 'Title');
-        }, done);
+    it('should be possible to show page titles', async function () {
+        await selectValue('[name="actionType"]', 'Title');
+        await page.waitForNetworkIdle();
+        expect(await page.screenshotSelector('body')).to.matchImage('transitions_report_switch_type_title');
     });
 
 });
\ No newline at end of file
diff --git a/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_popup_titles.png b/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_popup_titles.png
index f7132df281..c14197f2b9 100644
--- a/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_popup_titles.png
+++ b/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_popup_titles.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:968db487718975a719f6c6498e30ff1790f8c2b01288f810e6e3ddcdc7200c44
-size 44104
+oid sha256:53a609b627fb548299f8597204e9fdac5c5c50e280dea9c3f140a4e9fa0da821
+size 46144
diff --git a/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_popup_urls.png b/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_popup_urls.png
index 5de05f048b..2d594036c0 100644
--- a/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_popup_urls.png
+++ b/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_popup_urls.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a51440cf19417600bb16fe15179a1d5712dc5c8f8b51ce3c0698f726c3495a42
-size 68081
+oid sha256:4c9aeacc5370b3d78ef9167ae8258217a0e6596d42a4f7a5dbdc7b6b5c63bd71
+size 79272
diff --git a/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_report_no_data_widget.png b/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_report_no_data_widget.png
index 15b4e54a70..0f20ce33b9 100644
--- a/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_report_no_data_widget.png
+++ b/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_report_no_data_widget.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:976d6bf0bb11d7d4b60faf797e23fe78abd188b2d8956431b5f31c7363a0bb84
-size 26265
+oid sha256:79b079ae0a707f4ee006d568aba6e10a73cf12ca5129167b33ab9fb797736edb
+size 27066
diff --git a/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_report_switch_type_title.png b/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_report_switch_type_title.png
index eb1fd2914f..a1c946e9fa 100644
--- a/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_report_switch_type_title.png
+++ b/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_report_switch_type_title.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bf28366b7ba8fa3e97fc3fffe0980b3bb6e797465bacf88937f1334de623301c
-size 135466
+oid sha256:9c59a9838be72164ab04cc656482524fbde1283ac0145ea9aeca904f827a5a58
+size 182529
diff --git a/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_report_switch_url.png b/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_report_switch_url.png
index 6988ecfbc9..a7baf0cd0e 100644
--- a/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_report_switch_url.png
+++ b/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_report_switch_url.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:48e58ecff0cbcb06568c2f7e61c2fff3afb1001455c2c859f6dd2471fc6cc960
-size 97853
+oid sha256:cd8e2ab9ea0b00bea3a0e3bdf27b3f016434e5e9bc6b42b77ac2db5177ef5ef5
+size 112588
diff --git a/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_report_with_data_report.png b/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_report_with_data_report.png
index 5938af1cac..9f728ea683 100644
--- a/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_report_with_data_report.png
+++ b/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_report_with_data_report.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f6b573e75aa94dca586b082d29f9237df0b78d0a6da9543210c800c498179490
-size 155679
+oid sha256:6739330e6a19a8375e3dd953de98507c809612a2cf22fa6ee4629a0960cfac6a
+size 194568
diff --git a/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_report_with_data_widget.png b/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_report_with_data_widget.png
index d9c4fe2d03..9de366099e 100644
--- a/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_report_with_data_widget.png
+++ b/plugins/Transitions/tests/UI/expected-screenshots/Transitions_transitions_report_with_data_widget.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e86fe7b1000f086744347d8023e76f967d5b1e3835721f7ee398dab18adc7568
-size 151398
+oid sha256:138be5309a8f9c424d8fff33bd1395ed8c3ca9334409afcf53acf9ab4d0c5eed
+size 187612