Welcome to mirror list, hosted at ThFree Co, Russian Federation.

github.com/ianj-als/pcl.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorIan Johnson <ian.johnson@appliedlanguage.com>2013-11-13 18:53:07 +0400
committerIan Johnson <ian.johnson@appliedlanguage.com>2013-11-13 18:53:07 +0400
commit5425bdcc53176bfa89cf50eb7d7be2bd4c416a5c (patch)
tree0e51310b511dca992da82d66ea6d238baba50a68
parent4a00214cc8703307df99bb4081459729cfa581ab (diff)
Re-factored variable and function name generation. And started documenting the imperative PCL syntax.
-rw-r--r--documentation/chapters/adapter/adapter.tex114
-rw-r--r--documentation/chapters/adapter/diagrams/README1
-rw-r--r--documentation/chapters/adapter/diagrams/any_identifier.pngbin0 -> 2148 bytes
-rw-r--r--documentation/chapters/adapter/diagrams/command.pngbin0 -> 13796 bytes
-rw-r--r--documentation/chapters/adapter/diagrams/component.pngbin0 -> 9664 bytes
-rw-r--r--documentation/chapters/adapter/diagrams/condition_expression.pngbin0 -> 20943 bytes
-rw-r--r--documentation/chapters/adapter/diagrams/configuration.pngbin0 -> 5129 bytes
-rw-r--r--documentation/chapters/adapter/diagrams/configuration_identifier.pngbin0 -> 1900 bytes
-rw-r--r--documentation/chapters/adapter/diagrams/function_call.pngbin0 -> 7584 bytes
-rw-r--r--documentation/chapters/adapter/diagrams/function_name.pngbin0 -> 874 bytes
-rw-r--r--documentation/chapters/adapter/diagrams/identifier.pngbin0 -> 9514 bytes
-rw-r--r--documentation/chapters/adapter/diagrams/imports.pngbin0 -> 4281 bytes
-rw-r--r--documentation/chapters/adapter/diagrams/input_signal.pngbin0 -> 933 bytes
-rw-r--r--documentation/chapters/adapter/diagrams/literal.pngbin0 -> 23652 bytes
-rw-r--r--documentation/chapters/adapter/diagrams/output_signal.pngbin0 -> 933 bytes
-rw-r--r--documentation/chapters/adapter/diagrams/pcl-do-grammar.ebnf44
-rw-r--r--documentation/chapters/adapter/diagrams/pcl_module.pngbin0 -> 933 bytes
-rw-r--r--documentation/chapters/adapter/diagrams/pcl_module_alias.pngbin0 -> 874 bytes
-rw-r--r--documentation/chapters/adapter/diagrams/port_definition.pngbin0 -> 3931 bytes
-rw-r--r--documentation/chapters/adapter/diagrams/qualified_identifier.pngbin0 -> 12838 bytes
-rw-r--r--documentation/chapters/adapter/diagrams/return.pngbin0 -> 4971 bytes
-rw-r--r--documentation/chapters/adapter/diagrams/return_output.pngbin0 -> 6226 bytes
-rw-r--r--documentation/chapters/adapter/diagrams/signal_name.pngbin0 -> 933 bytes
-rw-r--r--documentation/chapters/adapter/diagrams/variable.pngbin0 -> 933 bytes
-rw-r--r--documentation/chapters/compiler/compiler.tex3
-rw-r--r--documentation/pcl-manual.tex3
-rw-r--r--examples/let_binding/let_binding.pcl3
-rw-r--r--examples/parallel_sleep/sleep.pcl9
-rw-r--r--src/pclc/visitors/do_executor_visitor.py118
-rw-r--r--src/pclc/visitors/executor_visitor.py29
-rw-r--r--src/pclc/visitors/first_pass_resolver_visitor.py14
-rw-r--r--src/pclc/visitors/pcl_executor_visitor.py72
-rw-r--r--src/pclc/visitors/scoped_name_generator.py57
33 files changed, 343 insertions, 124 deletions
diff --git a/documentation/chapters/adapter/adapter.tex b/documentation/chapters/adapter/adapter.tex
index 68b6696..7a01e4a 100644
--- a/documentation/chapters/adapter/adapter.tex
+++ b/documentation/chapters/adapter/adapter.tex
@@ -1,14 +1,113 @@
\chapter{Adapting to PCL}
-Today PCL gives you a convenient way of composing pre-existing PCL component together in order to build packages of computation. To build PCL components from your existing programs a Python file is required which wraps your code. The Python contains six functions that inform PCLc about the nature of the component defined:
+Adapting pre-existing executables to PCL can be achieved in two ways, they are:
\begin{itemize}
-\item Component's name,
-\item Input and output port specifications,
-\item Configuration and pre-processing configuration, and
-\item The component's computation.
+\item \textbf{Imperative PCL}: The PCL language supports an imperative style grammar which allows component authors to use runtime libraries to run external executables. This portion of the PCL grammar is not \emph{Turing complete} since there are no looping constructs. This feature of PCL exists to quickly initialise pre-requisites of an external executable, run the executable, post-process a result, and return the result. If it turns out your component needs a more complex pre- and post-processing, or a component cannot be, or need not be an external exectuable, the second approach can be used.
+\item \textbf{Python Wrapper}: A Python file, containing, six functions can be written that informs PCLc about the nature fo the component. Properties defined are: component's name, input and output port specifications, configuration and pre-processing configuration, and the component's computation. Since the computation of the component is described using Python, an arbitrarily complex component can be written.
\end{itemize}
Care must be taken when adapting your existing work to PCL pipelines. Threading issues and batch or on-line processing must be considered as the dynamics of your final pipeline may depend on it. Also, any state that may need to accumulate over the lifetime of a PCL component must be handled by the adaptor for your programs.
+\section{Imperative PCL}
+The imperative PCL language is a free-form language which allows the programmer to use arbitrary white-space to format your component definitions. Comments are a single line and should start with the \texttt{\#} and can appear at any point in a PCL file.
+\begin{figure}[h!]
+ \centering
+ \includegraphics[scale=\DiagramScale,angle=90]{chapters/adapter/diagrams/component}
+ \caption{Imperative PCL file syntax.}
+ \label{fig:imperative-pcl-top-level}
+\end{figure}
+The top level syntax of a PCL file is shown in Figure \ref{fig:imperative-pcl-top-level}, and consists of the following sections:
+\begin{itemize}
+\item \textbf{Imports}: Imports can be optionally specified. The imports here are PCL runtime libraries. These are written in Python and are modules which contain functions. Imports must specify an alias and this is use to call the functions, e.g., \texttt{list.insert(...)}.
+\item \textbf{Component}: This starts the component definition and provides the name. The component's name must be the same as the filename. E.g., a component in \texttt{fred.pcl} must be called \texttt{fred}.
+\item \textbf{Inputs}: Defines the inputs of the component. This information is used to verify that the outputs of a previous component is compatible with another. Only \emph{one} input port can be defined.
+\item \textbf{Outputs}: Defines the outputs of the component. This information is used to verify that the inputs of a subsequent component is compatible with another. Only \emph{one} output port can be defined.
+\item \textbf{Configuration}: Optional configuration for the component. This is static data that shall be used to construct imported components used in this component.
+\item \textbf{Commands}: This portion is the component definition. It is a list of commands which are executed in order from top to bottom.
+\end{itemize}
+
+An example imperative PCL file can be seen in Figure \ref{fig:imperative-pcl-example}. This example can be found in the \texttt{parallel\_sleep} example in the PCL Git repository.
+\begin{figure}[h!]
+ \begin{verbatim}
+import pcl.system.process as process
+import pcl.util.list as list
+
+component sleep
+ input sleep_time
+ output complete
+ configuration sleep_command
+ do
+ cmd <- list.cons(@sleep_command, sleep_time)
+ process.callAndCheck(cmd)
+
+ return complete <- True
+ \end{verbatim}
+ \caption{Example imperative PCL file.}
+ \label{fig:imperative-pcl-example}
+\end{figure}
+
+\subsection{Imports}
+\begin{figure}[h!]
+ \centering
+ \includegraphics[scale=\DiagramScale]{chapters/adapter/diagrams/imports}
+ \caption{\texttt{imports} : Importing PCL runtime functions.}
+ \label{fig:imperative-pcl-imports}
+\end{figure}
+Imperative PCL files can use runtime library functions to import functionality. These runtime libraries are Python files which contain functions. The names of functions are the name of the function in PCL. Figure \ref{fig:imperative-pcl-imports} shows the syntax for importing. The environment variable \texttt{PCL\_IMPORT\_PATH} is a colon separated list of directories from which a search shall take place for the PCL runtime libraries. If this environment variable is not set then the current working directory is used as a starting point for the component search.
+
+Each imported component must specify an alias. This is the name by which this component shall be referred to in this PCL file. E.g., \texttt{import pcl.util.list as list} shall import a PCL runtime library called \texttt{list} from the package \texttt{pcl.util} and shall be refereed to as, i.e. has the alias, \texttt{list}.
+
+Figure \ref{fig:imperative-pcl-import-non-terminals} shows how the non-terminals expand in the import syntax.
+\begin{figure}[h!]
+ \centering
+ \begin{subfigure}[b]{0.4\textwidth}
+ \includegraphics[scale=\DiagramScale]{chapters/adapter/diagrams/pcl_module}
+ \caption{\texttt{pcl\_module} expansion}
+ \end{subfigure}
+ ~
+ \begin{subfigure}[b]{0.4\textwidth}
+ \includegraphics[scale=\DiagramScale]{chapters/adapter/diagrams/pcl_module_alias}
+ \caption{\texttt{pcl\_module\_alias} expansion}
+ \end{subfigure}
+ \caption{\texttt{pcl\_module} \& \texttt{pcl\_module\_alias} : Imperative import PCL non-terminals.}
+ \label{fig:imperative-pcl-import-non-terminals}
+\end{figure}
+
+\subsection{Port Definition}
+\begin{figure}[h!]
+ \centering
+ \includegraphics[scale=\DiagramScale]{chapters/adapter/diagrams/port_definition}
+ \caption{\texttt{port\_definition} : Imperative PCL port definition.}
+ \label{fig:imperative-pcl-port-def}
+\end{figure}
+A port definition informs the PCL compiler about the nature of a component's input or an output. Components defined with imperative PCL can only have one input and one output port. Figure \ref{fig:pcl-port-def} shows the syntax for this grammatical construct.
+
+Again, ports carry one or more \emph{signals}. A signal is a piece of data that flows through ports and has a unique name to that port, and can be fully qualified. The signal names, for a port, are declared in a port definition. Signal names are read-only.
+
+\subsection{Configuration}
+\begin{figure}[h!]
+ \centering
+ \includegraphics[scale=\DiagramScale]{chapters/adapter/diagrams/configuration}
+ \caption{\texttt{configuration} : Imperative PCL configuration.}
+ \label{fig:imperative-pcl-config}
+\end{figure}
+A component's configuration is static and read-only data. Configuration data is named using identifiers, which can be fully qualified. Figure \ref{fig:imperative-pcl-config} shows the configuration syntax. Configuration identifiers may be used at any point where a variable, or input signal name can be used, e.g., and \emph{if} command, or function call. In imperative PCL zero or more configuration identifiers can be declared.
+
+\subsection{Commands}
+The command syntax is shown in Figure \ref{fig:imperative-pcl-command}. Each command yields a value which can, optionally, be assign to a write-once ``variable''.
+\begin{figure}[h!]
+ \centering
+ \includegraphics[scale=0.45,angle=90]{chapters/adapter/diagrams/command}
+ \caption{\texttt{command} : Imperative PCL commmands.}
+ \label{fig:imperative-pcl-command}
+\end{figure}
+
+\subsubsection{Function Calls}
+
+
+\subsubsection{Let Bindings}
+
+\subsubsection{If Commands}
+
\section{Python Wrapper}
The Python wrappers for your programs can inhabit the same hierarchical package structure as your PCL hierarchy. This is because the PCL hierarchy mirrors the Python one\footnote{This is the reason why \texttt{\_\_init\_\_.py} files must be manually placed in directories in your PCL heirarchy which have no PCL files.}.
@@ -90,7 +189,4 @@ def initialise(config):
\caption{\texttt{sleep.py}: An example Python wrapper for PCL.}
\label{fig:python-wrapper}
\end{figure}
-This wrapper if used in the \texttt{parallel\_sleep} example PCL which can be found in \texttt{examples/parallel\_sleep} directory of your Git clone.
-
-\section{Future Work}
-It is envisaged that the PCL syntax will be extended in order that these ``bottom level'' PCL components can be defined in PCL. This will no longer require that these components be defined in Python wrappers. However, this is for v2!
+This wrapper is the Python implementation of the imperative PCL \texttt{sleep} component shown in Figure \ref{fig:imperative-pcl-example}.
diff --git a/documentation/chapters/adapter/diagrams/README b/documentation/chapters/adapter/diagrams/README
new file mode 100644
index 0000000..71538f6
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/README
@@ -0,0 +1 @@
+Railroad diagrams made with this web-site: http://railroad.my28msec.com/rr/ui
diff --git a/documentation/chapters/adapter/diagrams/any_identifier.png b/documentation/chapters/adapter/diagrams/any_identifier.png
new file mode 100644
index 0000000..c5e0cc0
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/any_identifier.png
Binary files differ
diff --git a/documentation/chapters/adapter/diagrams/command.png b/documentation/chapters/adapter/diagrams/command.png
new file mode 100644
index 0000000..b64d812
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/command.png
Binary files differ
diff --git a/documentation/chapters/adapter/diagrams/component.png b/documentation/chapters/adapter/diagrams/component.png
new file mode 100644
index 0000000..0efefb3
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/component.png
Binary files differ
diff --git a/documentation/chapters/adapter/diagrams/condition_expression.png b/documentation/chapters/adapter/diagrams/condition_expression.png
new file mode 100644
index 0000000..ebc4c5c
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/condition_expression.png
Binary files differ
diff --git a/documentation/chapters/adapter/diagrams/configuration.png b/documentation/chapters/adapter/diagrams/configuration.png
new file mode 100644
index 0000000..88295da
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/configuration.png
Binary files differ
diff --git a/documentation/chapters/adapter/diagrams/configuration_identifier.png b/documentation/chapters/adapter/diagrams/configuration_identifier.png
new file mode 100644
index 0000000..2d12b1b
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/configuration_identifier.png
Binary files differ
diff --git a/documentation/chapters/adapter/diagrams/function_call.png b/documentation/chapters/adapter/diagrams/function_call.png
new file mode 100644
index 0000000..9111e93
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/function_call.png
Binary files differ
diff --git a/documentation/chapters/adapter/diagrams/function_name.png b/documentation/chapters/adapter/diagrams/function_name.png
new file mode 100644
index 0000000..ae239aa
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/function_name.png
Binary files differ
diff --git a/documentation/chapters/adapter/diagrams/identifier.png b/documentation/chapters/adapter/diagrams/identifier.png
new file mode 100644
index 0000000..ac4af9c
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/identifier.png
Binary files differ
diff --git a/documentation/chapters/adapter/diagrams/imports.png b/documentation/chapters/adapter/diagrams/imports.png
new file mode 100644
index 0000000..f0058dc
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/imports.png
Binary files differ
diff --git a/documentation/chapters/adapter/diagrams/input_signal.png b/documentation/chapters/adapter/diagrams/input_signal.png
new file mode 100644
index 0000000..57d7b6e
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/input_signal.png
Binary files differ
diff --git a/documentation/chapters/adapter/diagrams/literal.png b/documentation/chapters/adapter/diagrams/literal.png
new file mode 100644
index 0000000..12de31c
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/literal.png
Binary files differ
diff --git a/documentation/chapters/adapter/diagrams/output_signal.png b/documentation/chapters/adapter/diagrams/output_signal.png
new file mode 100644
index 0000000..f0ea025
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/output_signal.png
Binary files differ
diff --git a/documentation/chapters/adapter/diagrams/pcl-do-grammar.ebnf b/documentation/chapters/adapter/diagrams/pcl-do-grammar.ebnf
new file mode 100644
index 0000000..b2afe63
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/pcl-do-grammar.ebnf
@@ -0,0 +1,44 @@
+component ::= imports? 'component' identifier ('input' | 'inputs') port_definition ('output' | 'outputs') port_definition configuration? 'do' (command)* return_output
+
+
+imports ::= (('import' pcl_module 'as' pcl_module_alias) ('import' pcl_module 'as' pcl_module_alias)*)
+
+pcl_module ::= any_identifier
+
+pcl_module_alias ::= identifier
+
+
+port_definition ::= ((identifier | qualified_identifier) (',' (identifier | qualified_identifier))*)
+
+
+configuration ::= 'configuration' ((identifier | qualified_identifier) (',' (identifier | qualified_identifier))*)
+
+
+command ::= ((variable '<-')? function_call) | (variable '<-')? 'let' ((variable '<-' function_call) (variable '<-' function_call)*) 'in' function_call | (variable '<-')? 'if' condition_expression 'then' ((command)*)? return 'else' ((command)*)? return 'endif'
+
+function_call ::= pcl_module_alias '.' function_name '(' (((input_signal | variable | configuration_identifier) (',' (input_signal | variable | configuration_identifier))*))? ')'
+
+return_output ::= 'return' ((output_signal '<-' (input_signal | variable | literal)) (',' (output_signal '<-' (input_signal | variable | literal)))*)
+
+return ::= 'return' ('()' | input_signal | variable | configuration_identifier)
+
+function_name ::= identifier
+
+variable ::= any_identifier
+
+input_signal ::= any_identifier
+
+output_signal ::= any_identifier
+
+
+condition_expression ::= condition_expression ('or' | 'and' | 'xor' | '==' | '!=' | '>' | '<' | '>=' | '<=') condition_expression | (input_signal | variable | configuration_identifier | literal) | '(' condition_expression ')'
+
+any_identifier ::= identifier | qualified_identifier
+
+configuration_identifier ::= '@' any_identifier
+
+qualified_identifier ::= [a-zA-Z_][a-zA-Z0-9_]*('.'[a-zA-Z_][a-zA-Z0-9_]*)+
+
+identifier ::= [a-zA-Z_][a-zA-Z0-9_]*
+
+literal ::= [-]?[0-9]+('.'[0-9]+([eE][-+]?[0-9]+)?)? | '"' ('\'.|[^"])* '"' | [Tt][Rr][Uu][Ee] | [Ff][Aa][Ll][Ss][Ee]
diff --git a/documentation/chapters/adapter/diagrams/pcl_module.png b/documentation/chapters/adapter/diagrams/pcl_module.png
new file mode 100644
index 0000000..f0ea025
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/pcl_module.png
Binary files differ
diff --git a/documentation/chapters/adapter/diagrams/pcl_module_alias.png b/documentation/chapters/adapter/diagrams/pcl_module_alias.png
new file mode 100644
index 0000000..a387db9
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/pcl_module_alias.png
Binary files differ
diff --git a/documentation/chapters/adapter/diagrams/port_definition.png b/documentation/chapters/adapter/diagrams/port_definition.png
new file mode 100644
index 0000000..fa9d78b
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/port_definition.png
Binary files differ
diff --git a/documentation/chapters/adapter/diagrams/qualified_identifier.png b/documentation/chapters/adapter/diagrams/qualified_identifier.png
new file mode 100644
index 0000000..c89dc81
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/qualified_identifier.png
Binary files differ
diff --git a/documentation/chapters/adapter/diagrams/return.png b/documentation/chapters/adapter/diagrams/return.png
new file mode 100644
index 0000000..0a8e3bc
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/return.png
Binary files differ
diff --git a/documentation/chapters/adapter/diagrams/return_output.png b/documentation/chapters/adapter/diagrams/return_output.png
new file mode 100644
index 0000000..ce78e57
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/return_output.png
Binary files differ
diff --git a/documentation/chapters/adapter/diagrams/signal_name.png b/documentation/chapters/adapter/diagrams/signal_name.png
new file mode 100644
index 0000000..f0ea025
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/signal_name.png
Binary files differ
diff --git a/documentation/chapters/adapter/diagrams/variable.png b/documentation/chapters/adapter/diagrams/variable.png
new file mode 100644
index 0000000..f0ea025
--- /dev/null
+++ b/documentation/chapters/adapter/diagrams/variable.png
Binary files differ
diff --git a/documentation/chapters/compiler/compiler.tex b/documentation/chapters/compiler/compiler.tex
index 3de4329..9ebba6d 100644
--- a/documentation/chapters/compiler/compiler.tex
+++ b/documentation/chapters/compiler/compiler.tex
@@ -1,4 +1,3 @@
-\newcommand{\DiagramScale}{0.6}
\chapter{PCL Compiler}
PLCc is the PCL compiler. It is located in the \texttt{src/pclc} directory of the Git clone and your path should be set according. This chapter introduces the PCL syntax using \emph{railroad diagrams}. Railroad diagrams illustrate valid PCL and are read from left to right, following the lines as a train would. The symbols in the yellow ovals are to be typed \emph{as is}. The symbols in the tan rectangles references another railroad diagram. The referenced diagram should be used to expand the rectangle in more valid PCL. Hexagons contain \emph{character classes} and specify a range of characters that will be accepted.
@@ -94,7 +93,7 @@ component rainbow
\caption{\texttt{configuration} : Component configuration.}
\label{fig:pcl-config}
\end{figure}
-A component's configuration is static data that is primarily used for constructing import components. Configuration data is named using identifiers, which can be fully qualified. Figure \ref{fig:pcl-config} shows the configuration syntax. Configuration identifiers may also be used in \emph{if} components see Section \ref{sec:if} for details. A PCL can declare zero or more configuration identifiers.
+A component's configuration is static data that is primarily used for constructing import components. Configuration data is named using identifiers, which can be fully qualified. Figure \ref{fig:pcl-config} shows the configuration syntax. Configuration identifiers may also be used in \emph{if} components see Section \ref{sec:if} for details. A PCL component can declare zero or more configuration identifiers.
Figure \ref{fig:pcl-config-example} shows an example of configuration being declared in a PCL file. Here the \texttt{parallel\_sleep} component shall be constructed using two configuration values, namely \texttt{sleep\_command} and \texttt{sleep\_time}.
\begin{figure}[h!]
diff --git a/documentation/pcl-manual.tex b/documentation/pcl-manual.tex
index c4810a1..a74f1db 100644
--- a/documentation/pcl-manual.tex
+++ b/documentation/pcl-manual.tex
@@ -42,6 +42,8 @@
\usepackage{natbib}
\usepackage{booktabs}
\usepackage{graphicx}
+\usepackage{caption}
+\usepackage{subcaption}
\graphicspath{{expt/}}
\usepackage{setspace}
@@ -52,6 +54,7 @@
%%% Macro definitions for Commonly used symbols
\newcommand{\ReleaseVersion}{1.0.3-beta}
+\newcommand{\DiagramScale}{0.6}
\begin{document}
\title{\huge{Pipeline Creation Language (PCL)}\\
diff --git a/examples/let_binding/let_binding.pcl b/examples/let_binding/let_binding.pcl
index 7e68e74..4f1fac2 100644
--- a/examples/let_binding/let_binding.pcl
+++ b/examples/let_binding/let_binding.pcl
@@ -7,7 +7,8 @@ component let_binding
output filename.new
configuration working.directory
do
- # Create the pathname of the re-encoded corpus
+ basename <- list.cons()
+ # Create the new pathname
result <- let
basename <- path.basename(filename)
pieces <- path.splitext(basename)
diff --git a/examples/parallel_sleep/sleep.pcl b/examples/parallel_sleep/sleep.pcl
index 038b108..4ff87b2 100644
--- a/examples/parallel_sleep/sleep.pcl
+++ b/examples/parallel_sleep/sleep.pcl
@@ -18,16 +18,13 @@
#
import pcl.system.process as process
import pcl.util.list as list
-import pcl.util.string as string
component sleep
input sleep_time
output complete
configuration sleep_command
do
- cmd.list <- list.cons()
- list.append(cmd.list, @sleep_command)
- list.append(cmd.list, sleep_time)
- cmd.line <- string.join(cmd.list, " ")
- process.callAndCheck(cmd.line)
+ cmd <- list.cons(@sleep_command, sleep_time)
+ process.callAndCheck(cmd)
+
return complete <- True
diff --git a/src/pclc/visitors/do_executor_visitor.py b/src/pclc/visitors/do_executor_visitor.py
index a198fc2..86ede37 100644
--- a/src/pclc/visitors/do_executor_visitor.py
+++ b/src/pclc/visitors/do_executor_visitor.py
@@ -38,6 +38,7 @@ from parser.conditional_expressions import ConditionalExpression, \
UnaryConditionalExpression, \
TerminalConditionalExpression
from parser.expressions import StateIdentifier, Identifier, Literal
+from scoped_name_generator import ScopedNameGenerator
class IntermediateRepresentation(object):
@@ -96,15 +97,14 @@ class IntermediateRepresentation(object):
if isinstance(node.object, Command):
self.bindings.append(node)
- __TEMP_FUNC_FMT = "____func_%d"
- def __init__(self):
+ def __init__(self, variable_name_generator, function_name_generator):
self.__root = list()
self.__current_node = None
self.__current_if = None
self.__current_let = None
- self.__func_table = dict()
- self.__func_no = 0
+ self.__var_name_generator = variable_name_generator
+ self.__func_name_generator = function_name_generator
def __add_node(self, node, update_current_node = True):
if node.parent is None:
@@ -152,7 +152,7 @@ class IntermediateRepresentation(object):
generate_func_args = lambda args, scope: ", ".join([executor_visitor._generate_terminal(a, scope) for a in args])
generate_func_call = lambda f, scope: "____%s(%s)" % (f.name, generate_func_args(f.arguments, scope))
- top_function_name = self.__get_function_name(executor_visitor._module.definition.identifier)
+ top_function_name = self.__func_name_generator.get_name(executor_visitor._module.definition.identifier)
code = [("def %s(a, s):" % top_function_name, "+")]
for node in self.__root:
code.extend(self.__generate_code(node,
@@ -173,23 +173,28 @@ class IntermediateRepresentation(object):
scope = function['scope']
if is_instrumented:
- code.append(("____instr_command_begin('%s', %d, '%s', a, s)" % (function.filename, function.lineno, function), ""))
-
- tmp_var = executor_visitor._get_temp_var(function)
- code.append(("%s = %s" % (tmp_var,
- generate_function_call(function, scope)), ""))
- code.append(("return %s" % tmp_var, ""))
+ code.append(("____instr_command_begin('%s', %d, '%s', a, s)" % (function.filename, function.lineno, function),
+ ""))
+
+ tmp_var = self.__var_name_generator.get_name(function, scope)
+ code.append(("%s = %s" % (tmp_var, generate_function_call(function, scope)),
+ ""))
+ code.append(("return %s" % tmp_var,
+ ""))
elif isinstance(node, IntermediateRepresentation.IRCommandNode):
# Command action code generation
command = node.object
scope = command['scope']
- code.append(("def %s(a, s):" % self.__get_function_name(command), "+"))
+ code.append(("def %s(a, s):" % self.__func_name_generator.get_name(command),
+ "+"))
if command.identifier:
- code.append(("%s = %s" % (executor_visitor._get_temp_var(command.identifier), \
- generate_function_call(command.function, scope)), ""))
+ code.append(("%s = %s" % (self.__var_name_generator.get_name(command.identifier, scope), \
+ generate_function_call(command.function, scope)),
+ ""))
else:
- code.append((generate_function_call(command.function, scope), ""))
+ code.append((generate_function_call(command.function, scope),
+ ""))
for child in node.children:
more_code = self.__generate_code(child,
@@ -200,20 +205,25 @@ class IntermediateRepresentation(object):
code.append((None, "-"))
if is_instrumented:
- value_var = executor_visitor._get_temp_var(command)
- code.append(("____instr_command_begin('%s', %d, '%s', a, s)" % (command.filename, command.lineno, command), ""))
- code.append(("%s = %s(a, s)" % (value_var,
- self.__lookup_function_name(command)), ""))
- code.append(("return %s" % value_var, ""))
+ value_var = self.__var_name_generator.get_name(command, scope)
+ code.append(("____instr_command_begin('%s', %d, '%s', a, s)" % (command.filename, command.lineno, command),
+ ""))
+ code.append(("%s = %s(a, s)" % (value_var, self.__func_name_generator.lookup_name(command)),
+ ""))
+ code.append(("return %s" % value_var,
+ ""))
else:
- code.append(("return %s(a, s)" % self.__lookup_function_name(command), ""))
+ code.append(("return %s(a, s)" % self.__func_name_generator.lookup_name(command),
+ ""))
elif isinstance(node, IntermediateRepresentation.IRIfNode):
# If command action code generation
if_command = node.object
scope = if_command['scope']
- code.append(("def %s(a, s):" % self.__get_function_name(if_command), "+"))
- code.append(("if %s:" % executor_visitor._generate_condition(if_command.condition, scope), "+"))
+ code.append(("def %s(a, s):" % self.__func_name_generator.get_name(if_command),
+ "+"))
+ code.append(("if %s:" % executor_visitor._generate_condition(if_command.condition, scope),
+ "+"))
for then_node in node.then_block:
more_code = self.__generate_code(then_node,
@@ -233,27 +243,34 @@ class IntermediateRepresentation(object):
code.extend([(None, "-"), (None, "-")])
if if_command.identifier:
- code.append(("%s = %s(a, s)" % (executor_visitor._get_temp_var(if_command.identifier), \
- self.__lookup_function_name(if_command)), ""))
+ code.append(("%s = %s(a, s)" % (self.__var_name_generator.get_name(if_command.identifier, scope), \
+ self.__func_name_generator.lookup_name(if_command)),
+ ""))
else:
- code.append(("%s(a, s)" % self.__lookup_function_name(if_command), ""))
+ code.append(("%s(a, s)" % self.__func_name_generator.lookup_name(if_command),
+ ""))
elif isinstance(node, IntermediateRepresentation.IRReturnNode):
return_command = node.object
scope = return_command['scope']
if return_command.value:
- code.append(("return %s" % executor_visitor._generate_terminal(return_command.value, scope), ""))
+ code.append(("return %s" % executor_visitor._generate_terminal(return_command.value, scope),
+ ""))
elif len(return_command.mappings) == 0:
- code.append(("return None", ""))
+ code.append(("return None",
+ ""))
else:
code.append(("return {%s}" % ", ".join(["'%s' : %s" % \
(m.to, executor_visitor._generate_terminal(m.from_, scope)) \
- for m in return_command.mappings]), ""))
+ for m in return_command.mappings]),
+ ""))
elif isinstance(node, IntermediateRepresentation.IRLetNode):
# Let command
let_command = node.object
+ scope = let_command['scope']
- code.append(("def %s(a, s):" % self.__get_function_name(let_command), "+"))
+ code.append(("def %s(a, s):" % self.__func_name_generator.get_name(let_command),
+ "+"))
for binding in node.bindings:
more_code = self.__generate_code(binding,
@@ -264,22 +281,15 @@ class IntermediateRepresentation(object):
code.append((None, "-"))
if let_command.identifier:
- code.append(("%s = %s(a, s)" % (executor_visitor._get_temp_var(let_command.identifier), \
- self.__lookup_function_name(let_command)), ""))
+ code.append(("%s = %s(a, s)" % (self.__var_name_generator.get_name(let_command.identifier, scope), \
+ self.__func_name_generator.lookup_name(let_command)),
+ ""))
else:
- code.append(("%s(a, s)" % self.__lookup_function_name(let_command), ""))
+ code.append(("%s(a, s)" % self.__func_name_generator.lookup_name(let_command),
+ ""))
return code
- def __get_function_name(self, function):
- func_name = IntermediateRepresentation.__TEMP_FUNC_FMT % self.__func_no
- self.__func_table[function] = func_name
- self.__func_no += 1
- return func_name
-
- def __lookup_function_name(self, function):
- return self.__func_table[function]
-
@multimethodclass
class DoExecutorVisitor(ExecutorVisitor):
@@ -287,11 +297,18 @@ class DoExecutorVisitor(ExecutorVisitor):
"def ____instr_command_begin(filename, lineno, cmd_type, a, s):\n" \
" print >> sys.stderr, '%s: %s: Component %s begining %s, at line %d (%s), with input %s and state %s' % (datetime.datetime.now().strftime('%x %X.%f'), threading.current_thread().name, get_name(), cmd_type, lineno, filename, a, {skey : s[skey] for skey in filter(lambda k: k != '____prev_', s.keys())})\n"
+ __VAR_NAME_PREFIX = "____tmp"
+ __FUNC_NAME_PREFIX = "____func"
+
def __init__(self, filename_root, is_instrumented):
- ExecutorVisitor.__init__(self, filename_root, "", is_instrumented)
- self.__ir = IntermediateRepresentation()
- self.__func_no = 0
- self.__func_table = dict()
+ var_name_generator = ScopedNameGenerator(DoExecutorVisitor.__VAR_NAME_PREFIX)
+ ExecutorVisitor.__init__(self,
+ filename_root,
+ var_name_generator,
+ "",
+ is_instrumented)
+ self.__func_name_generator = ScopedNameGenerator(DoExecutorVisitor.__FUNC_NAME_PREFIX)
+ self.__ir = IntermediateRepresentation(var_name_generator, self.__func_name_generator)
if self._is_instrumented:
self._write_line(DoExecutorVisitor.__INSTRUMENTATION_FUNCTION)
self._write_line()
@@ -312,19 +329,16 @@ class DoExecutorVisitor(ExecutorVisitor):
self._write_line()
self._write_line()
self._write_function("get_name",
- [("return '%s'" % \
- component.identifier,
+ [("return '%s'" % component.identifier,
"")])
# The get inputs function
self._write_function("get_inputs",
- "return %s" % \
- type_formatting_fn(component.inputs))
+ "return %s" % type_formatting_fn(component.inputs))
# The get outputs function
self._write_function("get_outputs",
- "return %s" % \
- type_formatting_fn(component.outputs))
+ "return %s" % type_formatting_fn(component.outputs))
# The get configuration function
self._write_function("get_configuration",
diff --git a/src/pclc/visitors/executor_visitor.py b/src/pclc/visitors/executor_visitor.py
index c217b3d..69f8b15 100644
--- a/src/pclc/visitors/executor_visitor.py
+++ b/src/pclc/visitors/executor_visitor.py
@@ -32,6 +32,8 @@ from parser.conditional_expressions import ConditionalExpression, \
UnaryConditionalExpression, \
TerminalConditionalExpression
from parser.expressions import StateIdentifier, Identifier, Literal
+from scoped_name_generator import ScopedNameGenerator
+
class ExecutorVisitor(object):
__INDENTATION = " "
@@ -41,21 +43,27 @@ class ExecutorVisitor(object):
"# This file was automatically generated by PCLc on\n" \
"# %(datetime)s.\n" \
"#\n"
- __TEMP_VAR = "____tmp_%d"
- def __init__(self, filename_root, imports = "", is_instrumented = False):
+ def __init__(self, filename_root, variable_name_generator, imports = "", is_instrumented = False):
# Check that the __init__.py file exists in the
# working directory
if os.path.isfile('__init__.py') is False:
open('__init__.py', 'w').close()
self._object_file = open('%s.py' % filename_root, 'w')
self._indent_level = 0
- self._tmp_var_no = 0
- self._var_table = dict()
+
+ # Variable generation
+ self._variable_generator = variable_name_generator
+
+ # Write header to object file
header_args = {'datetime' : \
datetime.datetime.now().strftime("%A %d %B %Y at %H:%M:%S")}
self._write_line((ExecutorVisitor.__HEADER + imports) % header_args)
+
+ # Instrumented object code?
self._is_instrumented = is_instrumented
+
+ # Python conditional operators
self.__conditional_operators = {AndConditionalExpression : 'and',
OrConditionalExpression : 'or',
XorConditionalExpression : '^',
@@ -102,20 +110,11 @@ class ExecutorVisitor(object):
self._write_lines(body_lines)
self._write_line()
- def _get_temp_var(self, expr):
- temp_var = ExecutorVisitor.__TEMP_VAR % self._tmp_var_no
- self._var_table[expr] = temp_var
- self._tmp_var_no += 1
- return temp_var
-
- def _lookup_var(self, expr):
- return self._var_table[expr]
-
- def _generate_terminal(self, terminal, scope):
+ def _generate_terminal(self, terminal, scope = None):
if isinstance(terminal, StateIdentifier):
return "s['%s']" % terminal.identifier
elif scope is not None and isinstance(terminal, Identifier) and terminal in scope:
- return self._lookup_var(terminal)
+ return self._variable_generator.lookup_name(terminal, scope)
elif isinstance(terminal, Identifier):
return "a['%s']" % terminal
elif isinstance(terminal, Literal):
diff --git a/src/pclc/visitors/first_pass_resolver_visitor.py b/src/pclc/visitors/first_pass_resolver_visitor.py
index 6d26f1b..da97321 100644
--- a/src/pclc/visitors/first_pass_resolver_visitor.py
+++ b/src/pclc/visitors/first_pass_resolver_visitor.py
@@ -813,13 +813,13 @@ class FirstPassResolverVisitor(ResolverVisitor):
# The imports
import_symbol_dict = self._module.resolution_symbols['imports']
- # This error is a little pointless.
- #if not import_symbol_dict.has_key(package_alias):
- # self._add_errors("ERROR: %(filename)s at line %(lineno)d, unknown function package alias %(alias)s",
- # [function],
- # lambda f: {'filename' : f.filename,
- # 'lineno' : f.lineno,
- # 'alias' : package_alias})
+ # Check the package alias has been imported
+ if not import_symbol_dict.has_key(Identifier(None, -1, package_alias)):
+ self._add_errors("ERROR: %(filename)s at line %(lineno)d, unknown function package alias %(alias)s",
+ [function],
+ lambda f: {'filename' : f.filename,
+ 'lineno' : f.lineno,
+ 'alias' : package_alias})
# The key for the imports is an Identifier, so create an Identifier
# from the string package alias derived from the function call.
diff --git a/src/pclc/visitors/pcl_executor_visitor.py b/src/pclc/visitors/pcl_executor_visitor.py
index 0f07c4c..06e9332 100644
--- a/src/pclc/visitors/pcl_executor_visitor.py
+++ b/src/pclc/visitors/pcl_executor_visitor.py
@@ -57,6 +57,7 @@ from parser.mappings import Mapping, \
LiteralMapping
from parser.module import Module
from executor_visitor import ExecutorVisitor
+from scoped_name_generator import ScopedNameGenerator
@multimethodclass
@@ -73,8 +74,15 @@ class PCLExecutorVisitor(ExecutorVisitor):
"def ____instr_component_construction(component_decl_id, component_id, component_config, invoked_component, decl_line_no):\n" \
" print >> sys.stderr, '%s: %s: Component %s is constructing %s (id = %s) with configuration %s (%s instance declared at line %d)' % (datetime.datetime.now().strftime('%x %X.%f'), threading.current_thread().name, get_name(), component_decl_id, component_id, component_config, invoked_component, decl_line_no)\n"
+ __COMP_NAME_PREFIX = "____comp"
+
def __init__(self, filename_root, is_instrumented = False):
- ExecutorVisitor.__init__(self, filename_root, PCLExecutorVisitor.__IMPORTS, is_instrumented)
+ self.__comp_name_generator = ScopedNameGenerator(PCLExecutorVisitor.__COMP_NAME_PREFIX)
+ ExecutorVisitor.__init__(self,
+ filename_root,
+ self.__comp_name_generator,
+ PCLExecutorVisitor.__IMPORTS,
+ is_instrumented)
if self._is_instrumented:
self._write_line(PCLExecutorVisitor.__INSTRUMENTATION_FUNCTIONS)
self._write_line()
@@ -85,9 +93,7 @@ class PCLExecutorVisitor(ExecutorVisitor):
@multimethod(Import)
def visit(self, an_import):
- self._write_line("import %s as ____%s" % \
- (an_import.module_name, \
- an_import.alias))
+ self._write_line("import %s as ____%s" % (an_import.module_name, an_import.alias))
@multimethod(Component)
def visit(self, component):
@@ -195,10 +201,11 @@ class PCLExecutorVisitor(ExecutorVisitor):
initialise_fn = [t for triple in decl_zipper for t in triple]
# Store variables in variable table
for decl in self._module.resolution_symbols['components']:
- self._var_table[IdentifierExpression(None,
- 0,
- decl.identifier,
- Expression(None, 0))] = str(decl.identifier)
+ self._variable_generator.register_name(IdentifierExpression(None,
+ 0,
+ decl.identifier,
+ Expression(None, 0)),
+ str(decl.identifier))
self._write_function("initialise",
[(smt, "") for smt in initialise_fn],
["config"])
@@ -209,55 +216,55 @@ class PCLExecutorVisitor(ExecutorVisitor):
@multimethod(object)
def visit(self, nowt):
- for expr in self._var_table.keys():
+ for expr in self._variable_generator.iter_names():
if expr.parent == None:
self._write_line()
- self._write_line("return %s" % self._lookup_var(expr))
+ self._write_line("return %s" % self._variable_generator.lookup_name(expr))
break
self._object_file.close()
@multimethod(UnaryExpression)
def visit(self, unary_expr):
- var_name = self._var_table.pop(unary_expr.expression)
- self._var_table[unary_expr] = var_name
+ var_name = self._variable_generator.remove_name(unary_expr.expression)
+ self._variable_generator.register_name(unary_expr, var_name)
@multimethod(CompositionExpression)
def visit(self, comp_expr):
self._write_line("%s = %s >> %s" % \
- (self._get_temp_var(comp_expr),
- self._lookup_var(comp_expr.left),
- self._lookup_var(comp_expr.right)))
+ (self._variable_generator.get_name(comp_expr),
+ self._variable_generator.lookup_name(comp_expr.left),
+ self._variable_generator.lookup_name(comp_expr.right)))
@multimethod(ParallelWithTupleExpression)
def visit(self, para_tuple_expr):
self._write_line("%s = %s ** %s" % \
- (self._get_temp_var(para_tuple_expr),
- self._lookup_var(para_tuple_expr.left),
- self._lookup_var(para_tuple_expr.right)))
+ (self._variable_generator.get_name(para_tuple_expr),
+ self._variable_generator.lookup_name(para_tuple_expr.left),
+ self._variable_generator.lookup_name(para_tuple_expr.right)))
@multimethod(ParallelWithScalarExpression)
def visit(self, para_scalar_expr):
self._write_line("%s = %s & %s" % \
- (self._get_temp_var(para_scalar_expr),
- self._lookup_var(para_scalar_expr.left),
- self._lookup_var(para_scalar_expr.right)))
+ (self._variable_generator.get_name(para_scalar_expr),
+ self._variable_generator.lookup_name(para_scalar_expr.left),
+ self._variable_generator.lookup_name(para_scalar_expr.right)))
@multimethod(FirstExpression)
def visit(self, first_expr):
self._write_line("%s = %s.first()" % \
- (self._get_temp_var(first_expr),
- self._lookup_var(first_expr.expression)))
+ (self._variable_generator.get_name(first_expr),
+ self._variable_generator.lookup_name(first_expr.expression)))
@multimethod(SecondExpression)
def visit(self, second_expr):
self._write_line("%s = %s.second()" % \
- (self._get_temp_var(second_expr),
- self._lookup_var(second_expr.expression)))
+ (self._variable_generator.get_name(second_expr),
+ self._variable_generator.lookup_name(second_expr.expression)))
@multimethod(SplitExpression)
def visit(self, split_expr):
self._write_line("%s = cons_split_wire()" % \
- self._get_temp_var(split_expr))
+ self._variable_generator.get_name(split_expr))
@multimethod(MergeExpression)
def visit(self, merge_expr):
@@ -272,7 +279,7 @@ class PCLExecutorVisitor(ExecutorVisitor):
for m in merge_expr.literal_mapping]
mapping = ", ".join(top_mappings + bottom_mappings + literal_mappings)
self._write_line("%s = cons_unsplit_wire(lambda t, b: {%s})" % \
- (self._get_temp_var(merge_expr),
+ (self._get_var(merge_expr),
mapping))
@staticmethod
@@ -285,22 +292,23 @@ class PCLExecutorVisitor(ExecutorVisitor):
@multimethod(WireExpression)
def visit(self, wire_expr):
- self._write_line("%s = %s" % (self._get_temp_var(wire_expr), PCLExecutorVisitor.__build_wire_expr(wire_expr.mapping)))
+ self._write_line("%s = %s" % (self._variable_generator.get_name(wire_expr),
+ PCLExecutorVisitor.__build_wire_expr(wire_expr.mapping)))
@multimethod(WireTupleExpression)
def visit(self, wire_tuple_expr):
self._write_line("%s = %s ** %s" % \
- (self._get_temp_var(wire_tuple_expr),
+ (self._variable_generator.get_name(wire_tuple_expr),
PCLExecutorVisitor.__build_wire_expr(wire_tuple_expr.top_mapping),
PCLExecutorVisitor.__build_wire_expr(wire_tuple_expr.bottom_mapping)))
@multimethod(IfExpression)
def visit(self, if_expr):
self._write_line("%s = cons_if_component(lambda a, s: %s, %s, %s)" % \
- (self._get_temp_var(if_expr),
+ (self._variable_generator.get_name(if_expr),
self._generate_condition(if_expr.condition),
- self._lookup_var(if_expr.then),
- self._lookup_var(if_expr.else_)))
+ self._variable_generator.lookup_name(if_expr.then),
+ self._variable_generator.lookup_name(if_expr.else_)))
@multimethod(AndConditionalExpression)
def visit(self, and_cond_expr):
diff --git a/src/pclc/visitors/scoped_name_generator.py b/src/pclc/visitors/scoped_name_generator.py
new file mode 100644
index 0000000..35223f1
--- /dev/null
+++ b/src/pclc/visitors/scoped_name_generator.py
@@ -0,0 +1,57 @@
+#
+# Copyright Capita Translation and Interpreting 2013
+#
+# This file is part of Pipeline Creation Language (PCL).
+#
+# Pipeline Creation Language (PCL) is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Pipeline Creation Language (PCL) is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Pipeline Creation Language (PCL). If not, see <http://www.gnu.org/licenses/>.
+#
+class ScopedNameGenerator(object):
+ def __init__(self, name_prefix):
+ self.__prefix = name_prefix
+ self.__table = dict()
+ self.__number = 0
+ # A default scope
+ self.__table[None] = dict()
+
+ def get_name(self, thing, scope = None):
+ name = "%s_%d" % (self.__prefix, self.__number)
+ if scope in self.__table:
+ scoped_table = self.__table[scope]
+ else:
+ scoped_table = dict()
+ self.__table[scope] = scoped_table
+
+ scoped_table[thing] = name
+ self.__number += 1
+ return name
+
+ def register_name(self, thing, name, scope = None):
+ if scope in self.__table:
+ scoped_table = self.__table[scope]
+ else:
+ scoped_table = dict()
+ self.__table[scope] = scoped_table
+
+ scoped_table[thing] = name
+
+ def lookup_name(self, thing, scope = None):
+ scoped_table = self.__table[scope]
+ return scoped_table[thing]
+
+ def iter_names(self, scope = None):
+ return self.__table[scope].iterkeys()
+
+ def remove_name(self, thing, scope = None):
+ scoped_table = self.__table[scope]
+ return scoped_table.pop(thing)