Added piwik release timeline to the TODO file + organized tasks

Changed config class so that it uses 2 config files now (one default and one user specific where you can setup new values for the settings)

10 files changed, 185 insertions, 49 deletions

diff --git a/README b/README
index 7a24ed256f..9c1e1026a0 100644
--- a/README
+++ b/README
@@ -1,15 +1,20 @@
 	Piwik - piwik.org
 	-----------------
+	
+Mission statement
+« To create, as a community, the leading international open source web analytics application, 
+   providing access to all functionality through open components and open APIs, 
+   while focusing on the user experience. »
 
 Install 
 =======
 1 - copy paste the javascript code found in log.htm into your website
 2 - enable pdo_mysql in your php.ini
-2 - launch index.php, you should see some text (without any PHP warning/error message)	
-3 - you can log as super user login = root, password = nintendo (change them in the config/config.ini.php file)
-4 - go to ?module=SitesManager and add a website
-5 - go to misc/generateVisits.php to generate some visits
-6 - go to your piwik/ directory to see the statistics 
+3 - launch index.php, you should see some text (without any PHP warning/error message)	
+4 - you can log as super user login = root, password = nintendo (change them in the config/config.ini.php file)
+5 - go to ?module=SitesManager and add a website
+6 - go to misc/generateVisits.php to generate some visits
+7 - go to your piwik/ directory to see the statistics 
 
 Requirements
 ============
@@ -18,15 +23,8 @@ Requirements
 - php PDO module enabled
 - OS / server independant
 
-
 Some modules
 ============
-Login
-?module=Login
-
-Logout
-?module=Logout
-
 User manager admin UI 
 ?module=UsersManager
 
@@ -42,6 +40,12 @@ or yesterday visits information in JSON
 You can find the full listing of the APIs provided by the plugins on
 ?module=API&action=listAllAPI
 
+Login
+?module=Login
+
+Logout
+?module=Logout
+
 
 Some other stuff
 ================ 
@@ -54,6 +58,9 @@ To generate the API documentation
 # ./misc/generateDoc.sh
 then check the documentation at piwik/documentation/ in your browser
 
+Want to help? The /TODO file lists all the known bugs and the features to implement.
+Feel free to have a look and send us patches. 
+
 Contact
 =======
 Any remark, question, suggestion or critic, 
diff --git a/TODO b/TODO
index 8f27734a90..de36c40a24 100644
--- a/TODO
+++ b/TODO
@@ -1,22 +1,23 @@
-FEATURES
-========
-- TRANSLATIONS
-	- All strings in the application
-	- JS strings to be translated
-	- Generic exception system
+SMALL FEATURES
+==============
 - when a method is called and doesn't make any access check, 
   we can throw a NoticeException and display the message 
   in debug mode in the API returned value in the field "notice" or "debug"
 - in the piwik.php process, we could do without all the information in the cookie except for the idvisitor
   we could select the information last_action_time, last_id_action, etc. assuming we have the idvisitor in the cookie
   this would allow to save the logs later by big bulk
-- Purge log_ when archiving is done
 - Provide Config file feature for plugins (works as the translation system)
 - CHANGE setColumnsToDisplay TO setColumnsIdsToDisplay
-- Limit number of rows saved in the datatable during the archiving process
+- in the config.ini.php, it is not possible to set a configuration settings without setting the whole category
+  we should have some sort of merging that would make possible to set only a sub configuration variable from the global.ini.php into the config.ini.php
+  this way, during an upgrade, we could keep only the config/config.ini.php and overwrite the other config files with the update piwik version
+  and we would keep the changes made to some configuration variables
 
 BUGS
 ====
+- The numbers in Visits summary seem wrong when compared to phpmyvisites, but the nb of actiosn under the action table for example is correct
+  http://ns21996.ovh.net/piwik/?module=Home&action=index&idSite=1&period=day&date=yesterday
+- Purge log_ when archiving is done
 - archiving is creating some wrong lines in the archive_numeric tables for future dates
 - hide the generator + phpdocumentor script from being triggered by the release
 - monthly unique visitors count is WRONG
@@ -38,8 +39,8 @@ BUGS
 - is_numeric is not valid. Replace by is_int OR is_float
 - images.google.X is not counted as a search engine. needs to read the prev variable and then the q variable.
 
-User Interface
-==============
+User Interface bugs
+===================
 - searching on countries should search on the final label and not the iso code
 - define the default output type on a per table basis
 - design a generic system to add _blocks_ in _pages_ (different from the _views_ associated to the controllers)
@@ -47,8 +48,8 @@ User Interface
 - add the columns containing the % values based on the nb_visits or nb_visitors
 - when a given output is chosen (Tag cloud) hide the icon under the datatable
 
-Browsers bugs
-==============
+Browsers specific bugs
+======================
 Opera 9.23 windows
 	- actions expanding / collapse doesnt work. always expand. multiple clicks load the content multiple times
 	- the columns width are changing when sorting the columns, annoying effect
@@ -77,8 +78,80 @@ When commenting
 
 RELEASE PROCESS
 ===============
-Delete from the release
+Delete from the SVN files export:
 - documentation/
 - libs/phpDocumentor
 - tmp
 
+
+
+           ------------------
+				Timeline
+           ------------------
+           
+
+Todo before alpha public release
+================================
+# Core
+   - Installation process so that installing Piwik only requires to upload the directory
+   - Interface architecture: widgets style page building, menu editing by plugins
+   - Fixed all known critical/important bugs
+   - Testing piwik on a > 10,000 visits per day website. Check the memory required by the archiving
+   
+# Plugins
+   - Openads integration working
+
+# Misc
+   - Launch a blog for the Piwik development and public announce
+   - Clean all the google docs piwik related and save them in the TRAC wiki + task tracker
+
+Deadline: 5 november 2007
+
+Todo before beta public release
+===============================
+# Core
+  - easy way to access historical data (eg. the last 30 days)
+  - building RSS feeds of every information
+  - Limit number of rows saved in the datatable during the archiving process
+  - Testing the piwik on some huge websites (> 50,000 visits per day) and make sure it works: Db optimization, php memory profiling
+  - Theming system so that we can create different look n feel for the piwik interface
+
+# Plugins
+  - Automatic Outlink tracking + download tracking plugin
+  - Automatically delete/keep onlyq  some parameters from the URLs before processing it during the visitor logging process
+    for example the url example.com/page/index.php?userid=8571498752487&module=homepage could be example.com/page/index.php?module=homepage after removing the userid parameter
+  - Exclude the webmaster based on a cookie. The cookie is then checked in piwik.php and the visitor is not counted in the stats
+  - Exclude the webmaster based on IP / IP range. The ip is then checked in piwik.php and the visitor is not counted in the stats
+  - Precise geolocalisation using Maxmind database (.dat file OR apache module OR php module)
+  - Last N visitors information 
+
+# Documentation
+  - All core classes phpdoc documented + documentation reviewed by 1 other developer
+  - Write simple plugins that would be useful and at the same time would help as "How to write Piwik Plugins" tutorials
+  - Document the database design and the concepts
+  - Document the Piwik architecture: API concept, Plugins concept, Logging engine, Archiving process, Reporting process, MVC model, DataTable concept
+
+Deadline: 20 december 2007
+
+Todo before stable release
+==========================
+# Core
+  - Translations framework ready (all strings translatable) + translated in french and english
+    To translate: all strings in templates, all strings in JS variables, design a generic system for the php exceptions to handle multilingual error messages
+  - Update process that makes it easy to update piwik
+  - versioning in APIs to make sure we don't break the plugins/widgets when updating the APIs
+  - complete profiling of the source and fix the easy to fix problems
+  
+# Plugins
+  - Select a logo to be displayed when calling the piwik.php 
+  	currently a 1*1 transparent Gif is displayed but we could have some nice logos
+  - Import phpmyvisites v2.x statistics into piwik easily (we have to reach the 15,000 users of phpmyvisites)
+  - Goals setup and tracking. We define a goal by a URL to reach (or a part of a url), goal conversion check, goal information (referers, countries, keywords, etc.)
+  - PDF export
+  - Email sending containing statistics inline or attached file
+
+# Documentation
+  - End user documentation that covers the main point. Not complete but it would make it easy to get started with Piwik for a non technical user.
+ 
+
+Deadline = Q1 2008
\ No newline at end of file
diff --git a/config/config.ini.php b/config/global.ini.php
index 78ffaa7846..3e70007366 100755
--- a/config/config.ini.php
+++ b/config/global.ini.php
@@ -9,7 +9,7 @@ password		= nintendo
 dbname			= piwiktrunk
 adapter			= PDO_MYSQL ; PDO_MYSQL or MYSQLI
 tables_prefix	= piwik_
-profiler 		= true
+profiler 		= false
 
 [database_tests : database]
 dbname			= piwiktests
@@ -35,20 +35,29 @@ enabled[] 		= VisitTime
 enabled[] 		= VisitorInterest
 enabled[] 		= ExamplePlugin
 
-[Debug]
-always_archive_data = false
-
 [Plugins_LogStats]
 enabled[] 		= Provider
 
+[Debug]
+; if set to true, the archiving process will always be triggered, even if the archive has already been computed
+; this is useful when making changes to the archiving code so we can 
+always_archive_data = false
+
 
 [General]
 ; Time in seconds after which an archive will be computed again. 
 ; This setting is used only for today's statistics.
-time_before_archive_considered_outdated = 15
+time_before_archive_considered_outdated = 30
 
+; character used to automatically create categories in the "Action" "Downloads" reports
+; for example a URL like "example.com/blog/development/first-post" will create 
+; the page first-post in the subcategory development which belongs to the blog category
 action_category_delimiter = /
 
+; default sorting order used by all datatables (desc or asc)
+dataTable_default_sort_order = desc
+
+; default number of elements in the datatable
 dataTable_default_limit = 10
 
 
@@ -133,14 +142,23 @@ log				= tmp/logs/
 
 
 [smarty]
+; the list of directories in which to look for templates
 template_dir[]	= plugins
 template_dir[]	= themes/default
+template_dir[]	= themes
+
+; smarty provided plugins
+plugins_dir[] 	= libs/Smarty/plugins
+; smarty plugins provided by piwik 
+plugins_dir[]	= modules/SmartyPlugins
+
+; where to store the compiled smarty templates
 compile_dir		= tmp/templates_c
+
 config_dir		= tmp/configs
 cache_dir		= tmp/cache
 
-plugins_dir[] 	= libs/Smarty/plugins
-plugins_dir[]	= modules/SmartyPlugins
-        
+; error reporting inside Smarty
 error_reporting = E_ALL|E_NOTICE
-debugging		= TRUE
\ No newline at end of file
+; should be set to false in a piwik release
+debugging		= true
\ No newline at end of file
diff --git a/misc/Installation specification.txt b/misc/Installation specification.txt
index 0fae6f9017..31945aae7a 100644
--- a/misc/Installation specification.txt
+++ b/misc/Installation specification.txt
@@ -4,3 +4,7 @@ Installation composed of different steps
 Plugin can add / delete steps from the installation
 The installation uses a different set of base templates
 But can use the output of some plugins
+
+
+Installation is triggered
+- when config file not found
\ No newline at end of file
diff --git a/misc/generateVisitsData/Referers.php b/misc/generateVisitsData/Referers.php
index ad6f85dc1e..0f03dc32aa 100644
--- a/misc/generateVisitsData/Referers.php
+++ b/misc/generateVisitsData/Referers.php
@@ -1254,7 +1254,6 @@ $referers = array(
 "http://www.google.fr/search?q=test site web&hl=fr&lr=lang_fr&start=10&sa=N",
 "http://www.google.com/search?hl=fr&q=related:perso.orange.fr/hattonbros/bateau.htm",
 "http://www.mon-evenement.com/phpmv2/index.php?&error_login=1",
-"http://www.web-analytique.com/les-interviews/interview--matthieu-aubry-createur-de-phpmyvisites.html",
 "http://search.msn.fr/results.aspx?q=logiciel libre serveur&FORM=MSNH",
 "http://www.google.fr/search?client=firefox-a&rls=org.mozilla%3Afr%3Aofficial&channel=s&hl=fr&q=phpmyvisit&meta=&btnG=Recherche Google",
 "http://www.google.fr/search?hl=fr&client=firefox-a&channel=s&rls=org.mozilla:fr:official&hs=h4f&sa=X&oi=spell&resnum=0&ct=result&cd=1&q=analyse d%27audience de site&spell=1",
@@ -1967,7 +1966,6 @@ $referers = array(
 "http://www.joomlafrance.org/8/32.html",
 "http://www.stif-idf.fr/phpmyvisites/login.php",
 "http://www.philippejoret.com/",
-"http://www.matthias-kuehl.de/",
 "http://www.psn3.com/Pin,sylvestre/fiche.html",
 "http://meshlab.sourceforge.net/phpmv2/index.php?site=1&date=2007-03-21&mod=view_referers&period=3",
 "http://stats.1foteam.com/",
@@ -2096,7 +2094,6 @@ $referers = array(
 "http://www.swalif.net/softs/showthread.php?t=143689",
 "http://www.jakpsatweb.cz/pocitadla.html",
 "http://www.educlasse.ch/",
-"http://www.google.com/search?hl=en&client=firefox-a&rls=org.mozilla%3Afr%3Aofficial&hs=RSg&q=matthieu aubry&btnG=Search",
 "http://www.skunk.powa.fr/shadow/portal.php",
 "http://www.phpscripts-fr.net/scripts/derniers.php",
 "http://www.phpmyvisites.net/prerequis.html",
@@ -5005,7 +5002,6 @@ $referers = array(
 "http://www.parisrandovelo.com/index.php?page=promo/biosc1.html",
 "http://www.phpmyvisites.us/phpmv2/index.php?lang=en-utf-8.php&mod=clickheat.view_clickheat&site=1&adminsite=1&date=2007-07-25&period=1&action=",
 "http://www.google.fr/search?client=firefox-a&rls=org.mozilla%3Afr%3Aofficial&channel=s&hl=fr&q=phpmyvisites&meta=&btnG=Recherche Google",
-"http://www.web-analytique.com/actualites/2006/08/12/interview-matthieu-aubry-createur-de-phpmyvisites/",
 "http://www.rouen.fr/quartier/conseilsdequartier/agenda/19062007/reunionduconseildequartiergrammont",
 "http://www.afmd.asso.fr/afmd_actualites.php",
 "http://www.mycodes.net/soft/6536.htm",
@@ -6515,7 +6511,6 @@ $referers = array(
 "http://ane.voillemont-deiber.qc.ca/welcome/index.php",
 "http://www.phpmyvisites.us/",
 "http://sourceforge.net/projects/phpmyvisites",
-"http://www.google.fr/search?source=ig&hl=fr&q=Matthieu Aubry&btnG=Recherche Google&meta=",
 "http://www.google.fr/search?hl=fr&q=phpMyVisites perdu mot de passe administrateur&meta=lr%3Dlang_fr",
 "http://www.journaldufreenaute.fr/03/08/2007/amelioration-des-stats-pour-les-pages-persos.html",
 "http://www.journaldufreenaute.fr/03/08/2007/amelioration-des-stats-pour-les-pages-persos.html",
diff --git a/modules/API/Request.php b/modules/API/Request.php
index 7f45ba91ff..1380139c9d 100644
--- a/modules/API/Request.php
+++ b/modules/API/Request.php
@@ -288,7 +288,7 @@ class Piwik_API_Request
 						),
 			'Sort' => array(
 								'filter_sort_column' 		=> array('string', Piwik_Archive::INDEX_NB_VISITS),
-								'filter_sort_order' 		=> array('string', 'desc'),
+								'filter_sort_order' 		=> array('string', Zend_Registry::get('config')->General->dataTable_default_sort_order),
 						),
 			'Limit' => array(
 								'filter_offset' 			=> array('integer', '0'),
diff --git a/modules/Config.php b/modules/Config.php
index a228312c78..c48f32be29 100755
--- a/modules/Config.php
+++ b/modules/Config.php
@@ -5,15 +5,25 @@
  */
 require_once "Zend/Config/Ini.php";
 require_once "Zend/Registry.php";
-class Piwik_Config extends Zend_Config_Ini
+class Piwik_Config
 {
-	function __construct($pathIniFile = null)
+	protected $urlToPiwikHelpMissingValueInConfigurationFile = 
+		'http://piwik.svn.sourceforge.net/viewvc/piwik/trunk/config/global.ini.php';
+		
+	protected $defaultConfig 				= null;
+	protected $userConfig 					= null;
+	protected $pathIniFileUserConfig 		= null;
+	protected $pathIniFileDefaultConfig 	= null;
+	
+	function __construct($pathIniFileUserConfig = null)
 	{
-		if(is_null($pathIniFile))
+		if(is_null($pathIniFileUserConfig))
 		{	
-			$pathIniFile = PIWIK_INCLUDE_PATH . '/config/config.ini.php';
+			$this->pathIniFileUserConfig = PIWIK_INCLUDE_PATH . '/config/config.ini.php';
+			$this->pathIniFileDefaultConfig = PIWIK_INCLUDE_PATH . '/config/global.ini.php';
 		}
-		parent::__construct($pathIniFile, null, true);
+		$this->userConfig = new Zend_Config_Ini($this->pathIniFileUserConfig, null, true);
+		$this->defaultConfig = new Zend_Config_Ini($this->pathIniFileDefaultConfig, null, true);
 		
 		Zend_Registry::set('config', $this);
 		
@@ -31,5 +41,27 @@ class Piwik_Config extends Zend_Config_Ini
 	{		
 		Zend_Registry::set('tablesPrefix', $this->database->tables_prefix);
 	}
+	
+	public function __set($name, $value)
+	{
+		$this->userConfig->$name = $value;
+	}
+	
+	public function __get($name)
+    {
+        if(null !== ($valueInUserConfig = $this->userConfig->$name))
+        {
+//        	return $valueInUserConfig;
+        }
+        if(null !== ($valueInDefaultConfig = $this->defaultConfig->$name))
+        {
+        	return $valueInDefaultConfig;
+        }
+        
+        throw new Exception("The configuration parameter $name couldn't be found in your configuration file.
+						<br>Try to replace your default configuration file ({$this->pathIniFileDefaultConfig}) with 
+					the <a href='".$this->urlToPiwikHelpMissingValueInConfigurationFile."'>default piwik configuration file</a> ");
+    }
+	
 }
 
diff --git a/modules/DataTable/Renderer/CSV.php b/modules/DataTable/Renderer/CSV.php
index d2e52a0bb0..eb7608b17d 100644
--- a/modules/DataTable/Renderer/CSV.php
+++ b/modules/DataTable/Renderer/CSV.php
@@ -15,7 +15,7 @@
   * The record delimiter string is the carriage return and line feed (<cr><lf>).
   * The text qualifier string is a quotation mark (").
   * If the text contains an embedded delimiter string or qualifier string, the text qualifier is placed around the text, and the embedded qualifier strings are doubled.
-* Formatting and layout are ignored.
+  * Formatting and layout are ignored.
   */
 require_once "DataTable/Renderer/PHP.php";
 class Piwik_DataTable_Renderer_CSV extends Piwik_DataTable_Renderer
diff --git a/modules/ErrorHandler.php b/modules/ErrorHandler.php
index d0dcb81452..db402eb902 100755
--- a/modules/ErrorHandler.php
+++ b/modules/ErrorHandler.php
@@ -10,8 +10,14 @@ function Piwik_ErrorHandler($errno, $errstr, $errfile, $errline)
     debug_print_backtrace();
     $backtrace = ob_get_contents();
     ob_end_clean();
-    Zend_Registry::get('logger_error')->log($errno, $errstr, $errfile, $errline, $backtrace);
     
+    try {
+    	Zend_Registry::get('logger_error')->log($errno, $errstr, $errfile, $errline, $backtrace);
+    }catch(Exception $e){
+    	// in case the error occurs before the logger creation, we simply display it
+    	print("<pre>$errstr \nin '$errfile' at the line $errline\n\n$backtrace\n</pre>");
+    	exit;
+    }
 	switch($errno)
 	{
 		case E_ERROR:
diff --git a/modules/ExceptionHandler.php b/modules/ExceptionHandler.php
index 5d17925841..dc3b057916 100644
--- a/modules/ExceptionHandler.php
+++ b/modules/ExceptionHandler.php
@@ -10,12 +10,13 @@ function Piwik_ExceptionHandler(Exception $exception)
 	} catch(Exception $e) {
 		print("<br> <b>Exception</b>: '". $exception->getMessage()."'<br>");
 		
-		print("Backtrace:<br><pre>");
+		print("<br><small>Backtrace:<br><pre>");
 		print($exception->getTraceAsString());
 		print("</pre>");
-		print("<br> -------------------------- <br>
+		print("-------------------------- <br>
 			This exception occured and also raised this exception: ");
 		print("'" . $e->getMessage()."'");
+		print("</small>");
 	}
 }