2009-03-22 Miguel de Icaza <miguel@novell.com>

	Initial support to provide code completion facilities to consumers
	of the evaluator API.
	
	* cs-tokenizer.cs (CompleteOnEOF): this new property is used to
	support the completion engine.   When we reach the end of the
	input stream instead of returning EOF, when this flag is true the
	tokenizer instead produces:

		One GENERATE_COMPLETION token: this token then must be
		handled in the grammar at every point where the user
		would likely request a completion.

		As many COMPLETE_COMPLETION tokens as necessary.   These
		tokens are generated to assist the parser in unwinding and
		producing a valid parse tree.    

	The parser rules do not have to be perfect, the parser needs to be
	augmented with judicious use of GENERATE_COMPLETION tokens to
	improve the areas where we can provide completion and the parser
	needs to add support for COMPLETE_COMPLETION tokens in productions
	to make them work.

	It is common to not have enough support for COMPLETE_COMPLETION
	under certain rules and that even if we generated the
	GENERATE_COMPLETION token that the resulting tree will be invalid
	due to the missing rules that support COMPLETE_COMPLETION.

	The final EOF token is produced by having the parser notify the
	tokenizer when it reaches the root production that the next token
	should be EOF.

	* support.cs (CompletionResult): New Exception.   This exception
	is thrown to return the completion results when one of the special
	completion expressions is reached.

	This exception is thrown by the completing ExpressionStatements
	classes that live in complete.cs

	* complete.cs (CompletingExpression): a new base class for
	completing expressions.   This derives from the
	ExpressionStatement class and not from Expression as it allows
	completion to happen not only where expressions are expected in
	the grammar, but also where statements are expected.

	(CompletionSimpleName): A new class used to provide completions
	for SimpleNames.     This currently only resolves to local
	variables from the evaluator context (GetVars call).

	(CompletionMemberAccess): Implements support for completing member
	access patterns. 

	* cs-parser.jay: Add support for completion in a few places. 

	* eval.cs (GetCompletions): New public API for the evaluator that
	returns a list of possible completions given the input.   The
	return value is an array of completions 

	* anonymous.cs (Compatible): If the exception thrown from the
	resolved expression is a CompletionResult exception let that one
	through instead of printing a diagnostic error in the try/catch. 
	


svn path=/trunk/mcs/; revision=132033

1 files changed, 76 insertions, 0 deletions

diff --git a/mcs/docs/compiler.txt b/mcs/docs/compiler.txt
index f6be665cf95..6971c437236 100755
--- a/mcs/docs/compiler.txt
+++ b/mcs/docs/compiler.txt
@@ -956,7 +956,83 @@
 
 * Code Completion
 
+	Support for code completion is available to allow the compiler
+	to provide a list of possible completions at any given point
+	int he parsing process.   This is used for Tab-completion in
+	an interactive shell or visual aids in GUI shells for possible
+	method completions. 
+
+	This method is available as part of the Evaluator API where a
+	special method GetCompletions returns a list of possible
+	completions given a partial input.
+
+	The parser and tokenizer work together so that the tokenizer
+	upon reaching the end of the input generates the following
+	tokens: GENERATE_COMPLETION followed by as many
+	COMPLETE_COMPLETION token and finally the EOF token.
+
+	GENERATE_COMPLETION needs to be handled in every production
+	where the user is likely to press the TAB key in the shell (or
+	in the future the GUI, or an explicit request in an IDE).
+	COMPLETE_COMPLETION must be handled throughout the grammar to
+	provide a way of completing the parsed expression.  See below
+	for details.
+
+	For the member access case, I have added productions that
+	mirror the non-completing productions, for example:
+
+	  primary_expression DOT IDENTIFIER GENERATE_COMPLETION 
+	  {
+	        LocatedToken lt = (LocatedToken) $3;
+	        $$ = new CompletionMemberAccess ((Expression) $1, lt.Value, lt.Location);
+	  }
 	
+	This mirrors:
+
+	  primary_expression DOT IDENTIFIER opt_type_argument_list
+	  {
+	        LocatedToken lt = (LocatedToken) $3;
+	        $$ = new MemberAccess ((Expression) $1, lt.Value, (TypeArguments) $4, lt.Location);
+	  }
+	
+	The CompletionMemberAccess is a new kind of
+	Mono.CSharp.Expression that does the actual lookup.  It
+	internally mimics some of the MemberAccess code but has been
+	tuned for this particular use.
+
+	After this initial token is processed GENERATE_COMPLETION the
+	tokenizer will emit COMPLETE_COMPLETION tokens.  This is done
+	to help the parser basically produce a valid result from the
+	partial input it received.  For example it is able to produce
+	a valid AST from "(x" even if no parenthesis has been closed.
+	This is achieved by sprinkling the grammar with productions
+	that can cope with this "winding down" token, for example this
+	is what parenthesized_expression looks like now:
+
+	  parenthesized_expression
+	          : OPEN_PARENS expression CLOSE_PARENS
+	            {
+	                  $$ = new ParenthesizedExpression ((Expression) $2);
+	            }
+	          //
+	          // New production
+	          //
+	          | OPEN_PARENS expression COMPLETE_COMPLETION
+	            {
+	                  $$ = new ParenthesizedExpression ((Expression) $2);
+	            }
+	          ;
+	  
+	  Once we have wrapped up everything we generate the last EOF token.
+
+	When the AST is complete we actually trigger the regular semantic
+	analysis process.   The DoResolve method of each node in our abstract
+	syntax tree will compute the result and communicate the possible
+	completions by throwing an exception of type CompletionResult.
+
+	So for example if the user type "T" and the completion is
+	"ToString" we return "oString".
+
 * Miscellaneous
 
 ** Error Processing.