Add test communicating marian-decoder from Python subprocessline-by-line

5 files changed, 75 insertions, 0 deletions

diff --git a/tests/decoder/inputs/setup.sh b/tests/decoder/inputs/setup.sh
new file mode 100644
index 0000000..59913a8
--- /dev/null
+++ b/tests/decoder/inputs/setup.sh
@@ -0,0 +1,2 @@
+test -f $MRT_MODELS/wmt16_systems/en-de/model.npz || exit 1
+
diff --git a/tests/decoder/inputs/subprocess.expected b/tests/decoder/inputs/subprocess.expected
new file mode 100644
index 0000000..b463ecd
--- /dev/null
+++ b/tests/decoder/inputs/subprocess.expected
@@ -0,0 +1,21 @@
+Command: ['/mnt/zisa0/romang/marian/marian-dev/build/marian-decoder', '-c', '/fs/zisa0/romang/marian/mrt/models/wmt16_systems/marian.en-de.yml', '--log', 'subprocess.log']
+Sending:   India and Japan prime ministers meet in Tokyo
+Receiving: Indien und Japan Ministerpräsidenten treffen sich in Tokio
+Sending:   high on the agenda are plans for greater nuclear co-operation .
+Receiving: auf der Tagesordnung stehen die Pläne für eine verstärkte nukleare Zusammenarbeit .
+Sending:   India is also reportedly hoping for a deal on defence collaboration between the two nations .
+Receiving: auch Indien hofft angeblich auf eine Einigung zur Verteidigungs@@ kooperation beider Nationen .
+Sending:   he is due to appear in Kar@@ ra@@ tha Magistr@@ ates Court on September 23 .
+Receiving: er soll am 23. September in Kar@@ ra@@ tha Magistr@@ ats erscheinen .
+Sending:   the motorcycle was seized and im@@ po@@ unded for three months .
+Receiving: das Motorrad wurde für drei Monate beschlagnahmt und im@@ matri@@ k@@ uliert .
+Sending:   George Web@@ ster accused of N@@ air@@ n and Pit@@ loch@@ ry hotel rap@@ es
+Receiving: George Web@@ ster warf N@@ air@@ n und Pit@@ loch@@ ry Hotel@@ vergewal@@ tigung vor
+Sending:   a man is to stand trial accused of rap@@ ing women at two hotels .
+Receiving: ein Mann soll vor Gericht stehen , der Frauen in zwei Hotels vergewaltigt hat .
+Sending:   George Web@@ ster , 28 , faced the charges during a hearing at the High Court in Glasgow .
+Receiving: George Web@@ ster , 28 , stellte sich bei einer Anhörung vor dem Ober@@ landes@@ gericht in Glasgow die Vorwürfe .
+Sending:   judge Lady Ra@@ e set a trial date for November 17 at the High Court in Edinburgh .
+Receiving: Rich@@ terin Lady Ra@@ e hat einen Prozes@@ stag für den 17. November beim Ober@@ landes@@ gericht Edinburgh festgelegt .
+Sending:   re@@ connecting With the Very American Ideal That Labor Rights Are Human Rights
+Receiving: Wieder@@ ansch@@ aff@@ ung mit der Very American Ideal , die Arbeits@@ rechte sind Menschenrechte
diff --git a/tests/decoder/inputs/subprocess_script.py b/tests/decoder/inputs/subprocess_script.py
new file mode 100644
index 0000000..c17e0c4
--- /dev/null
+++ b/tests/decoder/inputs/subprocess_script.py
@@ -0,0 +1,29 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+import sys
+from subprocess import Popen, PIPE
+
+
+marian_path = sys.argv[1]
+config_file = sys.argv[2]
+text_file = sys.argv[3]
+
+
+cmd = ["{}/marian-decoder".format(marian_path), "-c", config_file, "--log", "subprocess.log"]
+sys.stdout.write("Command: {}\n".format(cmd))
+p = Popen(cmd, stdin=PIPE, stdout=PIPE, stderr=PIPE)
+
+
+with open(text_file) as f:
+    text = f.readlines()[:10]
+
+for sent in text:
+    sys.stdout.write("Sending:   {}".format(sent))
+    p.stdin.write(sent)
+    p.stdin.flush()
+    out = p.stdout.readline()
+    sys.stdout.write("Receiving: {}".format(out))
+
+p.stdin.close()
+p.terminate()
diff --git a/tests/decoder/inputs/test_subprocess_line_by_line.sh b/tests/decoder/inputs/test_subprocess_line_by_line.sh
new file mode 100644
index 0000000..28c5fe1
--- /dev/null
+++ b/tests/decoder/inputs/test_subprocess_line_by_line.sh
@@ -0,0 +1,11 @@
+#!/bin/bash
+
+# Exit on error
+set -e
+
+# Test code goes here
+timeout 10 python subprocess_script.py $MRT_MARIAN $MRT_MODELS/wmt16_systems/marian.en-de.yml text.in > subprocess.out
+$MRT_TOOLS/diff.sh subprocess.out subprocess.expected > subprocess.diff
+
+# Exit with success code
+exit 0
diff --git a/tests/decoder/inputs/text.in b/tests/decoder/inputs/text.in
new file mode 100644
index 0000000..8d21d17
--- /dev/null
+++ b/tests/decoder/inputs/text.in
@@ -0,0 +1,12 @@
+India and Japan prime ministers meet in Tokyo
+high on the agenda are plans for greater nuclear co-operation .
+India is also reportedly hoping for a deal on defence collaboration between the two nations .
+he is due to appear in Kar@@ ra@@ tha Magistr@@ ates Court on September 23 .
+the motorcycle was seized and im@@ po@@ unded for three months .
+George Web@@ ster accused of N@@ air@@ n and Pit@@ loch@@ ry hotel rap@@ es
+a man is to stand trial accused of rap@@ ing women at two hotels .
+George Web@@ ster , 28 , faced the charges during a hearing at the High Court in Glasgow .
+judge Lady Ra@@ e set a trial date for November 17 at the High Court in Edinburgh .
+re@@ connecting With the Very American Ideal That Labor Rights Are Human Rights
+the police used tear gas .
+the demonstrators demanded Shar@@ if 's resignation , which he refused .