refs #1700 basic performance analytics: handle server generation time for each page and page title

CORE

 * formatting sub-second times
 * getColumn() method on data table array (in order to behave the same as the regular data table class)
 * data tables can store in their meta data, which columns are empty (this is used in order to dynamically hide the new "generation time" column)
 * ViewDataTable and Api.getProcessedReport act according to the empty column meta data


SCHEMA

 * new column custom_float_1 in log_link_visit_action
 * new version to apply the change


TRACKER

 * Piwik_Tracker::setGenerationTime
 * tracking parameter "generation_time_me"
 * value is stored in new custom_float_1 column
 * the log importer can handle a group "generation_time_micro" which can be used in a custom log format. _micro is used because apache can log the time in microseconds but piwik processes milliseconds.
 * note: extension of JS tracker still missing


ACTIONS PLUGIN

 * for pages and page titles, add new columns sum_time_generation and nb_hits_with_time_generation to the blob archives
 * if they are set, compute avg_time_generation on the API level. if not, remove the columns and mark them as empty in the data table meta data.
 * show new column "avg. generation time" in the pages and page titles reports


plus TESTS for everything

1 files changed, 7 insertions, 0 deletions

diff --git a/misc/log-analytics/import_logs.py b/misc/log-analytics/import_logs.py
index 330e540b2b..785c6c69dc 100755
--- a/misc/log-analytics/import_logs.py
+++ b/misc/log-analytics/import_logs.py
@@ -1153,6 +1153,8 @@ class Recorder(object):
                 urllib.quote(args['url'], ''),
                 ("/From = %s" % urllib.quote(args['urlref'], '') if args['urlref'] != ''  else '')
             )
+        if hit.generation_time_milli > 0:
+            args['generation_time_ms'] = hit.generation_time_milli
         return args
 
     def _record_hits(self, hits):
@@ -1435,6 +1437,11 @@ class Parser(object):
             except (ValueError, IndexError):
                 # Some lines or formats don't have a length (e.g. 304 redirects, IIS logs)
                 hit.length = 0
+                
+            try:
+			    hit.generation_time_milli = int(match.group('generation_time_micro')) / 1000
+            except IndexError:
+                hit.generation_time_milli = 0
 
             if config.options.log_hostname:
                 hit.host = config.options.log_hostname