Switch UI tests from phantomjs to chrome headless (#14421)

* Make sure xss entries is always an array.

* Update expected test files and submodule.

* Regenerate broken omnifixture dump.

* poc conversion to chrome headless for UI tests, single test works locally

* Get single test to pass on travis & local w/ headless chrome.

* Remove old diff viewer generation, and get output to look the same as before when there is a failure.

* Add global timeout & get ViewDataTableTest to pass.

* Convert BarGraph_spec.js.

* Convert EmptySite_spec.

* Update EvolutionGraph spec for chrome headless.

* Convert GoalsTable test & find/replace for common changes.

* Convert MeasurableManager.

* Another find & replace.

* Get Menus/OptOutForm to pass w/ chrome headless

* Convert PeriodSelector, PieGraph & PivotByDimension UI tests.

* undo unrelated debugging change

* Convert QuickAccess UI tests

* Converts ActionsDataTable UI tests

* reset viewport after each testsuite

* Converts RowEvolution UI tests

* Converts Theme UI tests

* Converts SiteSelector UI tests

* Converts CustomVariables UI test

* Converts DBStats UI test

* Ignore empty responses when calling api/controller

* Converts Dashboard UI tests

* Converts Live UI tests

* Converts SimpleUI tests

* Converts Installation UI tests

* Converts ImageGraph UI tests

* Converts Login UI tests

* Converts Marketplace UI tests

* Converts Insights UI tests

* Converts ReportExporting UI tests

* Converts UIIntegration tests

* Updates ViewDataTable UI tests

* Converts CoreUpdater UI tests

* Converts UsersManager UI tests

* Converts Morpheus UI tests

* Converts MultiSites UI tests

* Fix testEnvironment.callApi handling of array parameters

* Converts Overlay UI tests

* Converts PrivacyManager UI tests

* Converts ScheduledReports UI tests

* Converts SegmentEditor UI tests

* compare pixel difference in images

* Converts PeriodSelector UI tests

* allow a pixel difference

* Converts SingleMetricView UI tests

* Converts SitesManager UI tests

* Converts Transitions UI tests

* Converts MeasurableManager UI tests

* Move Mouse out of screen after each test suite

* Converts PieGraph UI tests

* Ensure cursor is not shown in focused input elements

* Converts VisitorMap UI tests

* Converts CustomLogo UI tests

* Converts BarGraph UI tests

* Converts EvolutionGraph UI tests

* Converts IntranetMeasurable UI tests

* Converts TrackingFailures UI tests

* Converts CampaignBuilder UI tests

* Converts TagManagerTeaser UI tests

* fix request url rewrite for changed port

* Converts TwoFactorAuth UI tests

* do not print full responses in logs, but size instead

* improve additional style injection

* use submodule branches

* fix request interception

* update dependencies

* [TEMP] disable dangerous link checks

* [TEMP] limit tests

* Use puppeteer@next and make sure diffs get uploaded properly.

* Use correct fs function.

* Allow timeout to be specified on command line for UI tests and debug console logs.

* timeout option tweak

* Setup diff dir once before running tests not before each suite.

* fix last commit

* Update screenshots and fix some specs.

* Convert some tests that were not converted during merge.

* updating screenshots

* Updating screenshots and fixing some tests.

* more test fixes

* couple more test fixes

* More test fixes and plugin updates.

* wait for jquery

* Fix some screenshots and tests.

* more test fixes

* debug travis test failure

* remove .only

* more test fixes and updated expected files

* another test fixing iteration

* typo

* another quick test change

* more tests updates

* Test fixes and & more debugging.

* more debugging and test fixes

* more fixes and debugging.

* remove .only

* Another round of fixed tests.

* more debugging & fixes

* more test changes

* more debugging and fixes

* Fixing more tests + some bug fixes.

* Couple more fixes.

* Couple more fixes.

* Fixing tracking failures test

* more fixes

* Last couple fixes hopefully.

* couple more test fixes

* more fixes, bug fix in usersmanager, some test updates

* Some more test fixes/changes.

* more and more fixes

* hoping for a green build

* Do not compare against existing files in dashboard_spec.

* more fixes, re-enabling travis

* really re-enable travis

* Install puppeteer on travis-ci.

* more fixes

* try to fix random failues

* remove return

* Convert some login tests I forgot and update UsersManager expected file

* Fix test for primary key update

* Update AddPrimaryKey.php

* More test fixes + implement synchronous comparison threshold (so no resemblejs)

* More wait fixes + reset token detection fix.

* couple more fixes

* Fix login/overlay screenshot issue & magick command running issue.

* identify is the executable on travis (hopefully) + fix some random failures

* Another run.

* Last fix.

* two more tweaks

* typo

9 files changed, 53 insertions, 51 deletions

diff --git a/plugins/SitesManager/tests/UI/SitesManager_spec.js b/plugins/SitesManager/tests/UI/SitesManager_spec.js
index 3e21e12f26..80fa3e548d 100644
--- a/plugins/SitesManager/tests/UI/SitesManager_spec.js
+++ b/plugins/SitesManager/tests/UI/SitesManager_spec.js
@@ -13,74 +13,76 @@ describe("SitesManager", function () {
 
     var url = "?module=SitesManager&action=index&idSite=1&period=day&date=yesterday&showaddsite=false";
 
-    function assertScreenshotEquals(screenshotName, done, test)
+    async function assertScreenshotEquals(screenshotName, test)
     {
-        expect.screenshot(screenshotName).to.be.captureSelector('#content', test, done);
+        await test();
+        await page.waitForNetworkIdle();
+        pageWrap = await page.$('#content');
+        expect(await pageWrap.screenshot()).to.matchImage(screenshotName);
     }
 
-    function loadNextPage(page)
+    async function loadNextPage()
     {
-        page.click('.SitesManager .paging:first .next');
+        await (await page.jQuery('.SitesManager .paging:first .next')).click();
     }
 
-    function loadPreviousPage(page)
+    async function loadPreviousPage()
     {
-        page.click('.SitesManager .paging:first .prev');
+        await (await page.jQuery('.SitesManager .paging:first .prev')).click();
     }
 
-    function searchForText(page, textToAppendToSearchField)
+    async function searchForText(textToAppendToSearchField)
     {
-        page.sendKeys(".SitesManager .search:first input", textToAppendToSearchField);
-        page.click('.SitesManager .search:first img');
-        page.wait(150);
+        await (await page.jQuery('.SitesManager .search:first input')).type(textToAppendToSearchField);
+        await (await page.jQuery('.SitesManager .search:first img')).click();
     }
 
-    it("should load correctly and show page 0", function (done) {
-        assertScreenshotEquals("loaded", done, function (page) {
-            page.load(url);
+    it("should load correctly and show page 0", async function() {
+        await assertScreenshotEquals("loaded", async function () {
+            await page.goto(url);
         });
     });
 
-    it("should show page 1 when clicking next", function (done) {
-        assertScreenshotEquals("page_1", done, function (page) {
-            loadNextPage(page);
+    it("should show page 1 when clicking next", async function() {
+        await assertScreenshotEquals("page_1", async function () {
+            await loadNextPage();
         });
     });
 
-    it("should show page 2 when clicking next", function (done) {
-        assertScreenshotEquals("page_2", done, function (page) {
-            loadNextPage(page);
+    it("should show page 2 when clicking next", async function() {
+        await assertScreenshotEquals("page_2", async function () {
+            await loadNextPage();
         });
     });
 
-    it("should show page 1 when clicking prev", function (done) {
-        assertScreenshotEquals("page_1_again", done, function (page) {
-            loadPreviousPage(page);
+    it("should show page 1 when clicking prev", async function() {
+        await assertScreenshotEquals("page_1_again", async function () {
+            await loadPreviousPage();
         });
     });
 
-    it("should search for websites and reset page to 0", function (done) {
-        assertScreenshotEquals("search", done, function (page) {
-            searchForText(page, 'SiteTes');
+    it("should search for websites and reset page to 0", async function() {
+        await assertScreenshotEquals("search", async function () {
+            await searchForText('SiteTes');
         });
     });
 
-    it("should page within search result to page 1", function (done) {
-        assertScreenshotEquals("search_page_1", done, function (page) {
-            loadNextPage(page);
+    it("should page within search result to page 1", async function() {
+        await assertScreenshotEquals("search_page_1", async function () {
+            await loadNextPage();
         });
     });
 
-    it("should search for websites no result", function (done) {
-        assertScreenshotEquals("search_no_result", done, function (page) {
-            searchForText(page, 'RanDoMSearChTerm');
+    it("should search for websites no result", async function() {
+        await assertScreenshotEquals("search_no_result", async function () {
+            await searchForText('RanDoMSearChTerm');
         });
     });
 
-    it("should load the global settings page", function (done) {
-        assertScreenshotEquals("global_settings", done, function (page) {
-            page.load('?module=SitesManager&action=globalSettings&idSite=1&period=day&date=yesterday&showaddsite=false');
-            page.evaluate(function () {
+    it("should load the global settings page", async function() {
+        await assertScreenshotEquals("global_settings", async function () {
+            await page.goto('?module=SitesManager&action=globalSettings&idSite=1&period=day&date=yesterday&showaddsite=false');
+            await page.evaluate(function () {
                 $('.form-help:contains(UTC time is)').hide();
             });
         });
diff --git a/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_global_settings.png b/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_global_settings.png
index c44d63d14e..e492ffe772 100644
--- a/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_global_settings.png
+++ b/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_global_settings.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:55a38c96258d87bed6de5226cad8bc195eb3154a6b03f3f2fdeb5a8f450213e7
-size 276564
+oid sha256:71b1b2c46a19b1f9c2104765349d1af58bd254573316ed1e586ab8eeae71e09d
+size 270387
diff --git a/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_loaded.png b/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_loaded.png
index 4ebca6449a..f6cbfb6ad1 100644
--- a/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_loaded.png
+++ b/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_loaded.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b3497758d0c7c1284f1ae67f85c30c25af6d5208b73b4b27f98e1d4d811810f6
-size 180718
+oid sha256:8c0229518783993a1f1cf872cbd6cbabd5679e5c19a42728bc6f2b6a3ca9d68d
+size 178782
diff --git a/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_page_1.png b/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_page_1.png
index aad7c74fb2..86ad7a5d48 100644
--- a/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_page_1.png
+++ b/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_page_1.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:36a8a43b3542fdeef8d4c238b9b28192b5fc1130fd979c05f5d7f91daa1a36af
-size 185832
+oid sha256:e28989ceb5e6eab830def15fccb4f82bfeb7d920c7c4c550a2de08ed15db6610
+size 183384
diff --git a/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_page_1_again.png b/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_page_1_again.png
index 4ff88edcfc..f003e9d317 100644
--- a/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_page_1_again.png
+++ b/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_page_1_again.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b83a016f0f4d53242f02d0b077dbbda663780b460ae5d79c8cbe3639bb58c088
-size 185845
+oid sha256:cfd3d1895fb9b37611dbb7d7c05c5b63019db12db9efbce816e8f4d6f800bea2
+size 183386
diff --git a/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_page_2.png b/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_page_2.png
index ed4963a612..a0b2d07cc6 100644
--- a/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_page_2.png
+++ b/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_page_2.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b8805c0ee7c9c33b5ba5856e2fc2e960a365232965880e463af94214240d1afa
-size 187799
+oid sha256:1169bc39ce10023d0f56d870d787398627865aeaac5fb18bad7d06661a5bfa15
+size 186204
diff --git a/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_search.png b/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_search.png
index e6d1804357..22cbce055c 100644
--- a/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_search.png
+++ b/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_search.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e2d1cd3d110f616ec24900389d083ddaa00031769670d3455aa5626fbc6a57f2
-size 178750
+oid sha256:f48635b935f9ccb0bd5cb10888c7bfad375fc703549da4432d61c0a5dbab023e
+size 178029
diff --git a/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_search_no_result.png b/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_search_no_result.png
index b1fdc40221..9972ec6f16 100644
--- a/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_search_no_result.png
+++ b/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_search_no_result.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1279f37d94f1dc39a81160b81f254bc766b1b437698838b60fd4b78af95ae0d2
-size 44948
+oid sha256:749fdac586feb9a7a28006c1528ccd1fded5fe714a22081e68b745a956ae35ae
+size 45739
diff --git a/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_search_page_1.png b/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_search_page_1.png
index 8507dadcf4..51a26d252d 100644
--- a/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_search_page_1.png
+++ b/plugins/SitesManager/tests/UI/expected-screenshots/SitesManager_search_page_1.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1e00aee6453a3740e301c6aaee9b5ad5e2782dfc569d3de8b79813ab12fef397
-size 183881
+oid sha256:302b87783a36fc0c039e2bd0b9bda5fa18f35d2774e2e73c0150d19a329e2fd5
+size 182728