Welcome to mirror list, hosted at ThFree Co, Russian Federation.

github.com/ianj-als/pypeline.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorIan Johnson <ian.johnson@appliedlanguage.com>2012-09-10 16:57:01 +0400
committerIan Johnson <ian.johnson@appliedlanguage.com>2012-09-10 16:57:01 +0400
commit06179fa27eb1690eb5889029fe3ee45dda4c4589 (patch)
tree2af20f66933aeacbacea4fda06d10e333ad260a0
parentd6a1a9b3fbeb7528b5e9fd7c0c5db4e41d59bbcc (diff)
Batch sub-process components output is now a function of value and state.
-rw-r--r--README.md8
-rw-r--r--src/pypeline/helpers/helpers.py12
2 files changed, 10 insertions, 10 deletions
diff --git a/README.md b/README.md
index 6bd9c6f..f4bfd3b 100644
--- a/README.md
+++ b/README.md
@@ -114,16 +114,16 @@ The state mutator function shall take one argument, the state object, and return
#### Constructing a Batch Subprocess Pipeline Component
helpers.cons_batch_subprocess_component(process_pipe,
- input_feed_function,
+ input_generator_function,
output_function,
state_mutator = None)
-Construct a pipeline component whose computation requires many values to be sent to the sub-process. An input feed function is required that shall provide the values for the computation. This function shall be a generator that takes two arguments: the value, and the state. This function shall yield objects, that once "stringyfied", shall be sent, as one line, to the `stdin` of the sub-process. The `stdout` is ignored.
+Construct a pipeline component whose computation requires many values to be sent to the sub-process. An input generator function is required that shall provide the values for the computation. This function shall be a generator that takes two arguments: the value, and the state. This function shall yield objects, that once "stringyfied", shall be sent, as one line, to the `stdin` of the sub-process. The `stdout` of the sub-process is ignored.
-The output function generates the value that shall be passed to the subsequent pipeline component.
+The output function generates the value that shall be passed to the subsequent pipeline component. This function shall take two arguments: the input value to the components, and the state object.
input_feed_function :: a -> s -> b
- output_function :: s -> c
+ output_function :: a -> s -> c
The state mutator function shall take one argument, the state object, and return a mutated state of object if desired. If no state mutator function is specified the stat flows through the component unchanged.
diff --git a/src/pypeline/helpers/helpers.py b/src/pypeline/helpers/helpers.py
index 2fadad0..1d059d1 100644
--- a/src/pypeline/helpers/helpers.py
+++ b/src/pypeline/helpers/helpers.py
@@ -66,16 +66,16 @@ def cons_subprocess_component(process_pipe,
def cons_batch_subprocess_component(process_pipe,
- input_feed_function,
+ input_generator_function,
output_function,
state_mutator = None):
- """Construct a pipeline component using a Popen object. Batch subprocesses shall accept a single line on stdin. An input feed function shall be provided that yields objects, that once "stringyfied", are presented to the subprocess' stdin. This function takes tow arguments: the value and the state objects. It is the responsibility of the feed function implementer to yield an EOF if necessary. The returned object shall be a Kleisli arrow representing this pipeline component."""
+ """Construct a pipeline component using a Popen object. Batch subprocesses shall accept a single line on stdin. An input generator function shall be provided that yields objects, that once "stringyfied", are presented to the subprocess' stdin. This function takes tow arguments: the value and the state objects. It is the responsibility of the feed function implementer to yield an EOF if necessary. The returned object shall be a Kleisli arrow representing this pipeline component."""
if not isinstance(process_pipe, subprocess.Popen):
raise ValueError("Must be a Popen process")
- if input_feed_function is None:
+ if input_generator_function is None or output_function is None:
raise ValueError("Subprocess components must specify both " +
- "input and output forming functions")
+ "input generator and output functions")
#
# This bind function handles the 'process'
@@ -88,14 +88,14 @@ def cons_batch_subprocess_component(process_pipe,
# and feed it to the underlying subprocess.
# This function shall return a value, that when stringyfied and
# injected into stdin, the subprocess will understand
- for transformed_a in input_feed_function(a, s):
+ for transformed_a in input_generator_function(a, s):
# Communicate with the subprocess
if transformed_a is not None:
print >> process_pipe.stdin, str(transformed_a).strip()
process_pipe.stdin.flush()
# Get the new a
- new_a = output_function(s)
+ new_a = output_function(a, s)
# Mutate the state
next_s = state_mutator(s) if state_mutator else s