Switch UI tests from phantomjs to chrome headless (#14421)

* Make sure xss entries is always an array.

* Update expected test files and submodule.

* Regenerate broken omnifixture dump.

* poc conversion to chrome headless for UI tests, single test works locally

* Get single test to pass on travis & local w/ headless chrome.

* Remove old diff viewer generation, and get output to look the same as before when there is a failure.

* Add global timeout & get ViewDataTableTest to pass.

* Convert BarGraph_spec.js.

* Convert EmptySite_spec.

* Update EvolutionGraph spec for chrome headless.

* Convert GoalsTable test & find/replace for common changes.

* Convert MeasurableManager.

* Another find & replace.

* Get Menus/OptOutForm to pass w/ chrome headless

* Convert PeriodSelector, PieGraph & PivotByDimension UI tests.

* undo unrelated debugging change

* Convert QuickAccess UI tests

* Converts ActionsDataTable UI tests

* reset viewport after each testsuite

* Converts RowEvolution UI tests

* Converts Theme UI tests

* Converts SiteSelector UI tests

* Converts CustomVariables UI test

* Converts DBStats UI test

* Ignore empty responses when calling api/controller

* Converts Dashboard UI tests

* Converts Live UI tests

* Converts SimpleUI tests

* Converts Installation UI tests

* Converts ImageGraph UI tests

* Converts Login UI tests

* Converts Marketplace UI tests

* Converts Insights UI tests

* Converts ReportExporting UI tests

* Converts UIIntegration tests

* Updates ViewDataTable UI tests

* Converts CoreUpdater UI tests

* Converts UsersManager UI tests

* Converts Morpheus UI tests

* Converts MultiSites UI tests

* Fix testEnvironment.callApi handling of array parameters

* Converts Overlay UI tests

* Converts PrivacyManager UI tests

* Converts ScheduledReports UI tests

* Converts SegmentEditor UI tests

* compare pixel difference in images

* Converts PeriodSelector UI tests

* allow a pixel difference

* Converts SingleMetricView UI tests

* Converts SitesManager UI tests

* Converts Transitions UI tests

* Converts MeasurableManager UI tests

* Move Mouse out of screen after each test suite

* Converts PieGraph UI tests

* Ensure cursor is not shown in focused input elements

* Converts VisitorMap UI tests

* Converts CustomLogo UI tests

* Converts BarGraph UI tests

* Converts EvolutionGraph UI tests

* Converts IntranetMeasurable UI tests

* Converts TrackingFailures UI tests

* Converts CampaignBuilder UI tests

* Converts TagManagerTeaser UI tests

* fix request url rewrite for changed port

* Converts TwoFactorAuth UI tests

* do not print full responses in logs, but size instead

* improve additional style injection

* use submodule branches

* fix request interception

* update dependencies

* [TEMP] disable dangerous link checks

* [TEMP] limit tests

* Use puppeteer@next and make sure diffs get uploaded properly.

* Use correct fs function.

* Allow timeout to be specified on command line for UI tests and debug console logs.

* timeout option tweak

* Setup diff dir once before running tests not before each suite.

* fix last commit

* Update screenshots and fix some specs.

* Convert some tests that were not converted during merge.

* updating screenshots

* Updating screenshots and fixing some tests.

* more test fixes

* couple more test fixes

* More test fixes and plugin updates.

* wait for jquery

* Fix some screenshots and tests.

* more test fixes

* debug travis test failure

* remove .only

* more test fixes and updated expected files

* another test fixing iteration

* typo

* another quick test change

* more tests updates

* Test fixes and & more debugging.

* more debugging and test fixes

* more fixes and debugging.

* remove .only

* Another round of fixed tests.

* more debugging & fixes

* more test changes

* more debugging and fixes

* Fixing more tests + some bug fixes.

* Couple more fixes.

* Couple more fixes.

* Fixing tracking failures test

* more fixes

* Last couple fixes hopefully.

* couple more test fixes

* more fixes, bug fix in usersmanager, some test updates

* Some more test fixes/changes.

* more and more fixes

* hoping for a green build

* Do not compare against existing files in dashboard_spec.

* more fixes, re-enabling travis

* really re-enable travis

* Install puppeteer on travis-ci.

* more fixes

* try to fix random failues

* remove return

* Convert some login tests I forgot and update UsersManager expected file

* Fix test for primary key update

* Update AddPrimaryKey.php

* More test fixes + implement synchronous comparison threshold (so no resemblejs)

* More wait fixes + reset token detection fix.

* couple more fixes

* Fix login/overlay screenshot issue & magick command running issue.

* identify is the executable on travis (hopefully) + fix some random failures

* Another run.

* Last fix.

* two more tweaks

* typo

10 files changed, 95 insertions, 73 deletions

diff --git a/plugins/Live/tests/UI/Live_spec.js b/plugins/Live/tests/UI/Live_spec.js
index 74cb330f3a..1c0d111fc0 100644
--- a/plugins/Live/tests/UI/Live_spec.js
+++ b/plugins/Live/tests/UI/Live_spec.js
@@ -19,80 +19,102 @@ describe("Live", function () {
         }
     });
 
-    it('should show visitor log', function (done) {
-        expect.screenshot('visitor_log').to.be.captureSelector('.reporting-page', function (page) {
-            page.load("?module=CoreHome&action=index&idSite=1&period=year&date=2010-01-03#?idSite=1&period=year&date=2010-01-03&category=General_Visitors&subcategory=Live_VisitorLog");
-            page.wait(4500);
-        }, done);
+    it('should show visitor log', async function() {
+        await page.goto("?module=CoreHome&action=index&idSite=1&period=year&date=2010-01-03#?idSite=1&period=year&date=2010-01-03&category=General_Visitors&subcategory=Live_VisitorLog");
+
+        await page.waitForNetworkIdle();
+
+        var report = await page.$('.reporting-page');
+        expect(await report.screenshot()).to.matchImage('visitor_log');
     });
 
-    it('should expand grouped actions', function (done) {
-        expect.screenshot('visitor_log_expand_actions').to.be.captureSelector('.dataTableVizVisitorLog .card.row:first-child', function (page) {
-            page.click('.dataTableVizVisitorLog .repeat.icon-refresh');
-        }, done);
+    it('should expand grouped actions', async function() {
+        await page.click('.dataTableVizVisitorLog .repeat.icon-refresh');
+        await page.mouse.move(-10, -10);
+
+        var report = await page.$('.dataTableVizVisitorLog .card.row:first-child');
+        expect(await report.screenshot()).to.matchImage('visitor_log_expand_actions');
     });
 
-    it('should show visitor profile', function (done) {
-        expect.screenshot('visitor_profile').to.be.captureSelector('.ui-dialog', function (page) {
-            page.evaluate(function(){
-                $('.card:first-child .visitor-log-visitor-profile-link').click();
-            });
-            page.wait(6000);
-        }, done);
+    it('should show visitor profile', async function() {
+        await page.evaluate(function(){
+            $('.card:first-child .visitor-log-visitor-profile-link').click();
+        });
+
+        await page.waitForSelector('.ui-dialog');
+        await page.waitForNetworkIdle();
+
+        var dialog = await page.$('.ui-dialog');
+        expect(await dialog.screenshot()).to.matchImage('visitor_profile');
     });
 
-    it('should hide all action details', function (done) {
-        expect.screenshot('visitor_profile_actions_hidden').to.be.captureSelector('.ui-dialog', function (page) {
-            page.evaluate(function(){
-                $('.visitor-profile-toggle-actions').click();
-            }, 500);
-        }, done);
+    it('should hide all action details', async function() {
+        await page.evaluate(function(){
+            $('.visitor-profile-toggle-actions').click();
+        });
+
+        await page.mouse.move(0, 0);
+
+        var dialog = await page.$('.ui-dialog');
+        expect(await dialog.screenshot()).to.matchImage('visitor_profile_actions_hidden');
     });
 
-    it('should show visit details', function (done) {
-        expect.screenshot('visitor_profile_visit_details').to.be.captureSelector('.ui-dialog', function (page) {
-            page.evaluate(function(){
-                $('.visitor-profile-visit-title')[0].click();
-            }, 200);
-        }, done);
+    it('should show visit details', async function() {
+        await page.evaluate(function(){
+            $('.visitor-profile-visit-title')[0].click();
+        });
+
+        var dialog = await page.$('.ui-dialog');
+        expect(await dialog.screenshot()).to.matchImage('visitor_profile_visit_details');
     });
 
-    it('should show action details', function (done) {
-        expect.screenshot('visitor_profile_action_details').to.be.captureSelector('.ui-dialog', function (page) {
-            page.click('.visitor-profile-visits li:first-child .visitor-profile-show-actions', 200);
-        }, done);
+    it('should show action details', async function() {
+        await page.click('.visitor-profile-visits li:first-child .visitor-profile-show-actions');
+        await page.waitForNetworkIdle();
+
+        await page.mouse.move(-10, -10);
+
+        var dialog = await page.$('.ui-dialog');
+        expect(await dialog.screenshot()).to.matchImage('visitor_profile_action_details');
     });
 
-    it('should show action tooltip', function (done) {
-        expect.screenshot('visitor_profile_action_tooltip').to.be.captureSelector('.ui-tooltip:visible', function (page) {
-            page.mouseMove('.visitor-profile-visits li:first-child .visitor-profile-actions .action:first-child', 200);
-        }, done);
+    it('should show action tooltip', async function() {
+        var action = await page.jQuery('.visitor-profile-visits li:first-child .visitor-profile-actions .action:first-child');
+        await action.hover();
+        await page.waitForSelector('.ui-tooltip');
+
+        const elem = await page.$('.ui-tooltip');
+        expect(await elem.screenshot()).to.matchImage('visitor_profile_action_tooltip');
     });
 
-    it('should show limited profile message', function (done) {
-        expect.screenshot('visitor_profile_limited').to.be.captureSelector('.ui-dialog', function (page) {
+    it('should show limited profile message', async function () {
+        // Limit number of shown visits to 5
+        testEnvironment.overrideConfig('General', 'live_visitor_profile_max_visits_to_aggregate', 5);
+        testEnvironment.save();
 
-            // Limit number of shown visits to 5
-            testEnvironment.overrideConfig('General', 'live_visitor_profile_max_visits_to_aggregate', 5);
-            testEnvironment.save();
+        await page.goto("?module=CoreHome&action=index&idSite=1&period=year&date=2010-01-03#?idSite=1&period=year&date=2010-01-03&category=General_Visitors&subcategory=Live_VisitorLog");
+        await page.evaluate(function(){
+            $('.card:first-child .visitor-log-visitor-profile-link').click();
+        });
 
-            page.load("?module=CoreHome&action=index&idSite=1&period=year&date=2010-01-03#?idSite=1&period=year&date=2010-01-03&category=General_Visitors&subcategory=Live_VisitorLog");
-            page.evaluate(function(){
-                $('.card:first-child .visitor-log-visitor-profile-link').click();
-            });
-            page.wait(6000);
-        }, done);
+        await page.waitForSelector('.ui-dialog');
+        await page.waitForNetworkIdle();
+        await page.mouse.move(-10, -10);
+
+        var dialog = await page.$('.ui-dialog');
+        expect(await dialog.screenshot()).to.matchImage('visitor_profile_limited');
     });
 
-    it('should show visitor log purge message when purged and no data', function (done) {
-        expect.screenshot('visitor_log_purged').to.be.captureSelector('.reporting-page', function (page) {
+    it('should show visitor log purge message when purged and no data', async function() {
+        testEnvironment.overrideConfig('Deletelogs', 'delete_logs_enable', 1);
+        testEnvironment.overrideConfig('Deletelogs', 'delete_logs_older_than', 4000);
+        testEnvironment.save();
 
-            testEnvironment.overrideConfig('Deletelogs', 'delete_logs_enable', 1);
-            testEnvironment.overrideConfig('Deletelogs', 'delete_logs_older_than', 4000);
-            testEnvironment.save();
+        await page.goto("?module=CoreHome&action=index&idSite=1&period=year&date=2005-01-03#?idSite=1&period=year&date=2005-01-03&category=General_Visitors&subcategory=Live_VisitorLog");
+
+        await page.waitForNetworkIdle();
 
-            page.load("?module=CoreHome&action=index&idSite=1&period=year&date=2005-01-03#?idSite=1&period=year&date=2005-01-03&category=General_Visitors&subcategory=Live_VisitorLog");
-            page.wait(4000);
-        }, done);
+        var report = await page.$('.reporting-page');
+        expect(await report.screenshot()).to.matchImage('visitor_log_purged');
     });
 });
diff --git a/plugins/Live/tests/UI/expected-screenshots/Live_visitor_log.png b/plugins/Live/tests/UI/expected-screenshots/Live_visitor_log.png
index de66647b67..5fcc726b2d 100644
--- a/plugins/Live/tests/UI/expected-screenshots/Live_visitor_log.png
+++ b/plugins/Live/tests/UI/expected-screenshots/Live_visitor_log.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:aad3b53871687402544b5c57156ac0267d8a877d144b6f8bfb77e10d90a1d226
-size 406036
+oid sha256:403d52d525566412b7f6556d9ea109f883fd0fdd79f1a36b0a97ab610f403d81
+size 382454
diff --git a/plugins/Live/tests/UI/expected-screenshots/Live_visitor_log_expand_actions.png b/plugins/Live/tests/UI/expected-screenshots/Live_visitor_log_expand_actions.png
index 96dae07601..7670bd0320 100644
--- a/plugins/Live/tests/UI/expected-screenshots/Live_visitor_log_expand_actions.png
+++ b/plugins/Live/tests/UI/expected-screenshots/Live_visitor_log_expand_actions.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:913895364cb38d6276a91451f738b69895b94edcb9a7d4237c0386866e475a56
-size 52381
+oid sha256:61e520ac19eaf7e2ad0a3c000d4a9f2b422337031ec76c7895f3b92d84157e87
+size 51024
diff --git a/plugins/Live/tests/UI/expected-screenshots/Live_visitor_log_purged.png b/plugins/Live/tests/UI/expected-screenshots/Live_visitor_log_purged.png
index 3d2495240f..9a58bbeeb5 100644
--- a/plugins/Live/tests/UI/expected-screenshots/Live_visitor_log_purged.png
+++ b/plugins/Live/tests/UI/expected-screenshots/Live_visitor_log_purged.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3842f59a2d202904c85ce2fd66a8ba22792ca44c34eaa3f1beebabba78a1694d
-size 22152
+oid sha256:accc0fc97ad435098d85eb5d75caa76cc1077d75ee25ccf81e66c9e585ccb89f
+size 21444
diff --git a/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile.png b/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile.png
index c15564a9e4..01108007f1 100644
--- a/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile.png
+++ b/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d50f6e9c3331332b65d96dfe2825811a80876682f7ad7ad0e1edadb072f63f30
-size 438117
+oid sha256:239c3cff34bdc37ef6e1a0a8f85ee537d548b2b0fcf255b25ce90f436c17caa8
+size 417219
diff --git a/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile_action_details.png b/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile_action_details.png
index e8415d3643..c420414212 100644
--- a/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile_action_details.png
+++ b/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile_action_details.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6a8da7d0caf9d3a23e0cc673c73ad080475302c71ac3eb4d1343420621d1ffeb
-size 300987
+oid sha256:3baf4e26d25cb2cc08ba78a0c6d199ffadc88aff819af2882abbe7706824b85f
+size 286558
diff --git a/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile_action_tooltip.png b/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile_action_tooltip.png
index 76d57c1187..9ede4f978f 100644
--- a/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile_action_tooltip.png
+++ b/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile_action_tooltip.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9b844e25eea64abd30aff3db3dd560080f744d47853a0ae4360bca661f0893b6
-size 11399
+oid sha256:317e2481c429a9f264a813a237c5d0f69c80db3962cf0107490ca22e380f80a8
+size 11663
diff --git a/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile_actions_hidden.png b/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile_actions_hidden.png
index 546c643871..4fc4cfc765 100644
--- a/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile_actions_hidden.png
+++ b/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile_actions_hidden.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0b4a4235de0ad5281424618dc3019e64a8a5bf4621a1345161ab149396612cac
-size 266595
+oid sha256:13493d125dc03edd1e3b698e9b86056f5e38ab4744aa117cf3d350a470d79b48
+size 254843
diff --git a/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile_limited.png b/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile_limited.png
index 6ffe864232..690d51b874 100644
--- a/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile_limited.png
+++ b/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile_limited.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9d9ea756ae1ebb15f947222c6baca7a64417662e50f6a9b98cdd8fe4fe463472
-size 322583
+oid sha256:407bbf4891ff749ec90af841adf6c26848848dcc2146d016c3f9c3a4c7d8abad
+size 309267
diff --git a/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile_visit_details.png b/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile_visit_details.png
index 79771b0caa..c1880e2da4 100644
--- a/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile_visit_details.png
+++ b/plugins/Live/tests/UI/expected-screenshots/Live_visitor_profile_visit_details.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:cfa721a104ec02f180f83db1d412e31b6420e27a10083cf66f7b5c0d5b6405b5
-size 277071
+oid sha256:8ff64c409f8124f250da602fb6a4cfe2d7068245a0b70820b8ea7909b52ed1c4
+size 264543