Welcome to mirror list, hosted at ThFree Co, Russian Federation.

github.com/mapsme/omim.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
path: root/tools
diff options
context:
space:
mode:
authorAlex Zolotarev <deathbaba@gmail.com>2012-07-02 06:56:28 +0400
committerAlex Zolotarev <alex@maps.me>2015-09-23 01:40:38 +0300
commit3000488be4e75fe8223a21de3a8f148fb821d9b0 (patch)
tree16344ece6040fbf96df261f2e3a5b0a0a35b0847 /tools
parent6cc193db450881d8f4503c636dc1aeb43d10016a (diff)
Added tool to generate/update string resources
Diffstat (limited to 'tools')
-rw-r--r--tools/twine/Gemfile2
-rw-r--r--tools/twine/LICENSE30
-rw-r--r--tools/twine/README.md151
-rw-r--r--tools/twine/Rakefile8
-rwxr-xr-xtools/twine/bin/twine8
-rw-r--r--tools/twine/lib/twine.rb11
-rw-r--r--tools/twine/lib/twine/cli.rb183
-rw-r--r--tools/twine/lib/twine/encoding.rb20
-rw-r--r--tools/twine/lib/twine/formatters.rb10
-rw-r--r--tools/twine/lib/twine/formatters/abstract.rb75
-rw-r--r--tools/twine/lib/twine/formatters/android.rb215
-rw-r--r--tools/twine/lib/twine/formatters/apple.rb122
-rw-r--r--tools/twine/lib/twine/formatters/jquery.rb96
-rw-r--r--tools/twine/lib/twine/runner.rb301
-rw-r--r--tools/twine/lib/twine/stringsfile.rb201
-rw-r--r--tools/twine/lib/twine/version.rb3
-rw-r--r--tools/twine/test/fixtures/en-1.json12
-rw-r--r--tools/twine/test/fixtures/en-1.strings10
-rw-r--r--tools/twine/test/fixtures/fr-1.xml9
-rw-r--r--tools/twine/test/fixtures/strings-1.txt16
-rw-r--r--tools/twine/test/fixtures/test-output-1.txt11
-rw-r--r--tools/twine/test/fixtures/test-output-2.txt11
-rw-r--r--tools/twine/test/fixtures/test-output-3.txt17
-rw-r--r--tools/twine/test/fixtures/test-output-4.txt20
-rw-r--r--tools/twine/test/fixtures/test-output-5.txt11
-rw-r--r--tools/twine/test/twine_test.rb58
-rwxr-xr-xtools/twine/twine3
-rw-r--r--tools/twine/twine.gemspec30
28 files changed, 1644 insertions, 0 deletions
diff --git a/tools/twine/Gemfile b/tools/twine/Gemfile
new file mode 100644
index 0000000000..e45e65f871
--- /dev/null
+++ b/tools/twine/Gemfile
@@ -0,0 +1,2 @@
+source :rubygems
+gemspec
diff --git a/tools/twine/LICENSE b/tools/twine/LICENSE
new file mode 100644
index 0000000000..00585cc828
--- /dev/null
+++ b/tools/twine/LICENSE
@@ -0,0 +1,30 @@
+Software License Agreement (BSD License)
+
+Copyright (c) 2012, Mobiata, LLC
+All rights reserved.
+
+Redistribution and use of this software in source and binary forms, with or
+without modification, are permitted provided that the following conditions are
+met:
+
+* Redistributions of source code must retain the above copyright notice, this
+ list of conditions and the following disclaimer.
+
+* Redistributions in binary form must reproduce the above copyright notice,
+ this list of conditions and the following disclaimer in the documentation
+ and/or other materials provided with the distribution.
+
+* Neither the name of the organization nor the names of its contributors may be
+ used to endorse or promote products derived from this software without
+ specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
diff --git a/tools/twine/README.md b/tools/twine/README.md
new file mode 100644
index 0000000000..fc0027c1d4
--- /dev/null
+++ b/tools/twine/README.md
@@ -0,0 +1,151 @@
+# Twine
+
+Twine is a command line tool for managing your strings and their translations. These strings are all stored in a master text file and then Twine uses this file to import and export strings in a variety of file types, including iOS and Mac OS X `.strings` files as well as Android `.xml` files. This allows individuals and companies to easily share strings across multiple projects, as well as export strings in any format the user wants.
+
+## Install
+
+### As a Gem
+
+Twine is most easily installed as a Gem.
+
+ $ gem install twine
+
+### From Source
+
+You can also run Twine directly from source. However, it requires [rubyzip][rubyzip] in order to create and read standard zip files.
+
+ $ gem install rubyzip
+ $ git clone git://github.com/mobiata/twine.git
+ $ cd twine
+ $ ./twine --help
+
+Make sure you run the `twine` executable at the root of the project as it properly sets up your Ruby library path. The `bin/twine` executable does not.
+
+## String File Format
+
+Twine stores all of its strings in a single file. The format of this file is a slight variant of the [Git][git] config file format, which itself is based on the old [Windows INI file][INI] format. The entire file is broken up into sections, which are created by placing the section name between two pairs of square brackets. Sections are optional, but they are a recommended way of breaking your strings into smaller, more manageable chunks.
+
+Each grouping section contains N string definitions. These string definitions start with the string key placed within a single pair of square brackets. This string definition then contains a number of key-value pairs, including a comment, a comma-separated list of tags (which are used by Twine to select a subset of strings), and all of the translations.
+
+### Tags
+
+Tags are used by Twine as a way to only work with a subset of your strings at any given point in time. Each string can be assigned zero or more tags which are separated by commas. Tags are optional, though highly recommended. You can get a list of all strings currently missing tags by executing the `generate-report` command.
+
+### Whitespace
+
+Whitepace in this file is mostly ignored. If you absolutely need to put spaces at the beginning or end of your translated string, you can wrap the entire string in a pair of `` ` `` characters. If your actual string needs to start *and* end with a grave accent, you can wrap it in another pair of `` ` `` characters. See the example, below.
+
+### Example
+
+ [[General]]
+ [yes]
+ en = Yes
+ es = Sí
+ fr = Oui
+ ja = はい
+ [no]
+ en = No
+ fr = Non
+ ja = いいえ
+
+ [[Errors]]
+ [path_not_found_error]
+ en = The file '%@' could not be found.
+ tags = app1,app6
+ comment = An error describing when a path on the filesystem could not be found.
+ [network_unavailable_error]
+ en = The network is currently unavailable.
+ tags = app1
+ comment = An error describing when the device can not connect to the internet.
+
+ [[Escaping Example]]
+ [list_item_separator]
+ en = `, `
+ tags = mytag
+ comment = A string that should be placed between multiple items in a list. For example: Red, Green, Blue
+ [grave_accent_quoted_string]
+ en = ``%@``
+ tags = myothertag
+ comment = This string will evaluate to `%@`.
+
+## Usage
+
+ Usage: twine COMMAND STRINGS_FILE [INPUT_OR_OUTPUT_PATH] [--lang LANG1,LANG2...] [--tags TAG1,TAG2,TAG3...] [--format FORMAT]
+
+### Commands
+
+#### `generate-string-file`
+
+This command creates an Apple or Android strings file from the master strings data file.
+
+ $ twine generate-string-file /path/to/strings.txt values-ja.xml --tags common,app1
+ $ twine generate-string-file /path/to/strings.txt Localizable.strings --lang ja --tags mytag
+ $ twine generate-string-file /path/to/strings.txt all-english.strings --lang en
+
+#### `generate-all-string-files`
+
+This command is a convenient way to call `generate-string-file` multiple times. It uses standard Mac OS X, iOS, and Android conventions to figure out exactly which files to create given a parent directory. For example, if you point it to a parent directory containing `en.lproj`, `fr.lproj`, and `ja.lproj` subdirectories, Twine will create a `Localizable.strings` file of the appropriate language in each of them. This is often the command you will want to execute during the build phase of your project.
+
+ $ twine generate-all-string-files /path/to/strings.txt /path/to/project/locales/directory --tags common,app1
+
+#### `consume-string-file`
+
+This command slurps all of the strings from a `.strings` or `.xml` file and incorporates the translated text into the master strings data file. This is a simple way to incorporate any changes made to a single file by one of your translators. It will only identify strings that already exist in the master data file.
+
+ $ twine consume-string-file /path/to/strings.txt fr.strings
+ $ twine consume-string-file /path/to/strings.txt Localizable.strings --lang ja
+ $ twine consume-string-file /path/to/strings.txt es.xml
+
+#### `consume-all-string-files`
+
+This command reads in a folder containing many `.strings` or `.xml` files. These files should be in a standard folder hierarchy so that twine knows the language of each file. When combined with the `--developer-language`, `--consume-comments`, and `--consume-all` flags, this command is a great way to create your initial strings data file from an existing iOS or Android project. Just make sure that you create a blank strings.txt file, first!
+
+ $ twine consume-all-string-files strings.txt Resources/Locales --developer-language en --consume-all --consume-comments
+
+#### `generate-loc-drop`
+
+This command is a convenient way to generate a zip file containing files created by the `generate-string-file` command. It is often used for creating a single zip containing a large number of strings in all languages which you can then hand off to your translation team.
+
+ $ twine generate-loc-drop /path/to/strings.txt LocDrop1.zip
+ $ twine generate-loc-drop /path/to/strings.txt LocDrop2.zip --lang en,fr,ja,ko --tags common,app1
+
+#### `consume-loc-drop`
+
+This command is a convenient way of taking a zip file and executing the `consume-string-file` command on each file within the archive. It is most often used to incorporate all of the changes made by the translation team after they have completed work on a localization drop.
+
+ $ twine consume-loc-drop /path/to/strings.txt LocDrop2.zip
+
+#### `generate-report`
+
+This command gives you useful information about your strings. It will tell you how many strings you have, how many have been translated into each language, and whether your master strings data file has any duplicate string keys.
+
+ $ twine generate-report /path/to/strings.txt
+
+## Creating Your First strings.txt File
+
+The easiest way to create your first strings.txt file is to run the `consume-all-string-files` command. The one caveat is to first create a blank strings.txt file to use as your starting point. Then, just point the `consume-all-string-files` command at a directory in your project containing all of your iOS, OS X, or Android strings files.
+
+ $ touch strings.txt
+ $ twine consume-all-string-files strings.txt Resources/Locales --developer-language en --consume-all --consume-comments
+
+## Twine and Your Build Process
+
+It is easy to incorporate Twine right into your iOS and OS X app build processes.
+
+1. In your project folder, create all of the `.lproj` directories that you need. It does not really matter where they are. We tend to put them in `Resources/Locales/`.
+2. Run the `generate-all-string-files` command to create all of the string files you need in these directories. For example,
+
+ $ twine generate-all-string-files strings.txt Resources/Locales/ --tags tag1,tag2
+
+ Make sure you point Twine at your strings data file, the directory that contains all of your `.lproj` directories, and the tags that describe the strings you want to use for this project.
+3. Drag the `Resources/Locales/` directory to the Xcode project navigator so that Xcode knows to include all of these strings files in your build.
+4. In Xcode, navigate to the "Build Phases" tab of your target.
+5. Click on the "Add Build Phase" button and select "Add Run Script".
+6. Drag the new "Run Script" build phase up so that it runs earlier in the build process. It doesn't really matter where, as long as it happens before the resources are copied to your bundle.
+7. Edit your script to run the exact same command you ran in step (2) above.
+
+Now, whenever you build your application, Xcode will automatically invoke Twine to make sure that your `.strings` files are up-to-date.
+
+[rubyzip]: http://rubygems.org/gems/rubyzip
+[git]: http://git-scm.org/
+[INI]: http://en.wikipedia.org/wiki/INI_file
diff --git a/tools/twine/Rakefile b/tools/twine/Rakefile
new file mode 100644
index 0000000000..db49633b40
--- /dev/null
+++ b/tools/twine/Rakefile
@@ -0,0 +1,8 @@
+require 'rake'
+require 'rake/testtask'
+
+Rake::TestTask.new do |t|
+ t.test_files = %w(test/twine_test.rb)
+end
+
+task :default => :test
diff --git a/tools/twine/bin/twine b/tools/twine/bin/twine
new file mode 100755
index 0000000000..690f51f28f
--- /dev/null
+++ b/tools/twine/bin/twine
@@ -0,0 +1,8 @@
+#!/usr/bin/env ruby
+require 'twine'
+begin
+ Twine::Runner.run(ARGV)
+rescue Twine::Error => e
+ STDERR.puts e.message
+ exit
+end
diff --git a/tools/twine/lib/twine.rb b/tools/twine/lib/twine.rb
new file mode 100644
index 0000000000..71d3bdce7f
--- /dev/null
+++ b/tools/twine/lib/twine.rb
@@ -0,0 +1,11 @@
+module Twine
+ class Error < StandardError
+ end
+
+ require 'twine/cli'
+ require 'twine/encoding'
+ require 'twine/formatters'
+ require 'twine/runner'
+ require 'twine/stringsfile'
+ require 'twine/version'
+end
diff --git a/tools/twine/lib/twine/cli.rb b/tools/twine/lib/twine/cli.rb
new file mode 100644
index 0000000000..d83b85da2b
--- /dev/null
+++ b/tools/twine/lib/twine/cli.rb
@@ -0,0 +1,183 @@
+require 'optparse'
+
+module Twine
+ class CLI
+ def initialize(args, options)
+ @options = options
+ @args = args
+ end
+
+ def self.parse_args(args, options)
+ new(args, options).parse_args
+ end
+
+ def parse_args
+ parser = OptionParser.new do |opts|
+ opts.banner = 'Usage: twine COMMAND STRINGS_FILE [INPUT_OR_OUTPUT_PATH] [--lang LANG1,LANG2...] [--tags TAG1,TAG2,TAG3...] [--format FORMAT]'
+ opts.separator ''
+ opts.separator 'The purpose of this script is to convert back and forth between multiple data formats, allowing us to treat our strings (and translations) as data stored in a text file. We can then use the data file to create drops for the localization team, consume similar drops returned by the localization team, generate reports on the strings, as well as create formatted string files to ship with your products. Twine currently supports iOS, OS X, Android, and jquery-localize string files.'
+ opts.separator ''
+ opts.separator 'Commands:'
+ opts.separator ''
+ opts.separator 'generate-string-file -- Generates a string file in a certain LANGUAGE given a particular FORMAT. This script will attempt to guess both the language and the format given the filename and extension. For example, "ko.xml" will generate a Korean language file for Android.'
+ opts.separator ''
+ opts.separator 'generate-all-string-files -- Generates all the string files necessary for a given project. The parent directory to all of the locale-specific directories in your project should be specified as the INPUT_OR_OUTPUT_PATH. This command will most often be executed by your build script so that each build always contains the most recent strings.'
+ opts.separator ''
+ opts.separator 'consume-string-file -- Slurps all of the strings from a translated strings file into the specified STRINGS_FILE. If you have some files returned to you by your translators you can use this command to incorporate all of their changes. This script will attempt to guess both the language and the format given the filename and extension. For example, "ja.strings" will assume that the file is a Japanese iOS strings file.'
+ opts.separator ''
+ opts.separator 'consume-all-string-files -- Slurps all of the strings from a directory into the specified STRINGS_FILE. If you have some files returned to you by your translators you can use this command to incorporate all of their changes. This script will attempt to guess both the language and the format given the filename and extension. For example, "ja.strings" will assume that the file is a Japanese iOS strings file.'
+ opts.separator ''
+ opts.separator 'generate-loc-drop -- Generates a zip archive of strings files in any format. The purpose of this command is to create a very simple archive that can be handed off to a translation team. The translation team can unzip the archive, translate all of the strings in the archived files, zip everything back up, and then hand that final archive back to be consumed by the consume-loc-drop command. This command assumes that --include-untranslated has been specified on the command line.'
+ opts.separator ''
+ opts.separator 'consume-loc-drop -- Consumes an archive of translated files. This archive should be in the same format as the one created by the generate-loc-drop command.'
+ opts.separator ''
+ opts.separator 'generate-report -- Generates a report containing data about your strings. For example, it will tell you if you have any duplicate strings or if any of your strings are missing tags. In addition, it will tell you how many strings you have and how many of those strings have been translated into each language.'
+ opts.separator ''
+ opts.separator 'General Options:'
+ opts.separator ''
+ opts.on('-l', '--lang LANGUAGES', Array, 'The language code(s) to use for the specified action.') do |langs|
+ @options[:languages] = langs
+ end
+ opts.on('-t', '--tags TAGS', Array, 'The tag(s) to use for the specified action. Only strings with that tag will be processed. Do not specify any tags to match all strings in the strings data file.') do |tags|
+ @options[:tags] = tags
+ end
+ opts.on('-u', '--untagged', 'If you have specified tags using the --tags flag, then only those tags will be selected. If you also want to select all strings that are untagged, then you can specify this option to do so.') do |u|
+ @options[:untagged] = true
+ end
+ formats = []
+ Formatters::FORMATTERS.each do |formatter|
+ formats << formatter::FORMAT_NAME
+ end
+ opts.on('-f', '--format FORMAT', "The file format to read or write (#{formats.join(', ')}). Additional formatters can be placed in the formats/ directory.") do |format|
+ lformat = format.downcase
+ if !formats.include?(lformat)
+ STDERR.puts "Invalid format: #{format}"
+ end
+ @options[:format] = lformat
+ end
+ opts.on('-a', '--consume-all', 'Normally, when consuming a string file, Twine will ignore any string keys that do not exist in your master file.') do |a|
+ @options[:consume_all] = true
+ end
+ opts.on('-s', '--include-untranslated', 'This flag will cause any Android string files that are generated to include strings that have not yet been translated for the current language.') do |s|
+ @options[:include_untranslated] = true
+ end
+ opts.on('-o', '--output-file OUTPUT_FILE', 'Write the new strings database to this file instead of replacing the original file. This flag is only useful when running the consume-string-file or consume-loc-drop commands.') do |o|
+ @options[:output_path] = o
+ end
+ opts.on('-d', '--developer-language LANG', 'When writing the strings data file, set the specified language as the "developer language". In practice, this just means that this language will appear first in the strings data file.') do |d|
+ @options[:developer_language] = d
+ end
+ opts.on('-c', '--consume-comments', 'Normally, when consuming a string file, Twine will ignore all comments in the file. With this flag set, any comments encountered will be read and parsed into the strings data file. This is especially useful when creating your first strings data file from an existing project.') do |c|
+ @options[:consume_comments] = true
+ end
+ opts.on('-e', '--encoding ENCODING', 'Twine defaults to encoding all output files in UTF-8. This flag will tell Twine to use an alternate encoding for these files. For example, you could use this to write Apple .strings files in UTF-16. This flag currently only works with Apple .strings files and is currently only supported in Ruby 1.9.3 or greater.') do |e|
+ if !"".respond_to?(:encode)
+ raise Twine::Error.new "The --encoding flag is only supported on Ruby 1.9.3 or greater."
+ end
+ @options[:output_encoding] = e
+ end
+ opts.on('-h', '--help', 'Show this message.') do |h|
+ puts opts.help
+ exit
+ end
+ opts.on('--version', 'Print the version number and exit.') do |x|
+ puts "Twine version #{Twine::VERSION}"
+ exit
+ end
+ opts.separator ''
+ opts.separator 'Examples:'
+ opts.separator ''
+ opts.separator '> twine generate-string-file strings.txt ko.xml --tags FT'
+ opts.separator '> twine generate-all-string-files strings.txt Resources/Locales/ --tags FT,FB'
+ opts.separator '> twine consume-string-file strings.txt ja.strings'
+ opts.separator '> twine consume-all-string-files strings.txt Resources/Locales/ --developer-language en'
+ opts.separator '> twine generate-loc-drop strings.txt LocDrop5.zip --tags FT,FB --format android --lang de,en,en-GB,ja,ko'
+ opts.separator '> twine consume-loc-drop strings.txt LocDrop5.zip'
+ opts.separator '> twine generate-report strings.txt'
+ end
+ parser.parse! @args
+
+ if @args.length == 0
+ puts parser.help
+ exit
+ end
+
+ @options[:command] = @args[0]
+
+ if !VALID_COMMANDS.include? @options[:command]
+ raise Twine::Error.new "Invalid command: #{@options[:command]}"
+ end
+
+ if @args.length == 1
+ raise Twine::Error.new 'You must specify your strings file.'
+ end
+
+ @options[:strings_file] = @args[1]
+
+ case @options[:command]
+ when 'generate-string-file'
+ if @args.length == 3
+ @options[:output_path] = @args[2]
+ elsif @args.length > 3
+ raise Twine::Error.new "Unknown argument: #{@args[3]}"
+ else
+ raise Twine::Error.new 'Not enough arguments.'
+ end
+ if @options[:languages] and @options[:languages].length > 1
+ raise Twine::Error.new 'Please only specify a single language for the generate-string-file command.'
+ end
+ when 'generate-all-string-files'
+ if ARGV.length == 3
+ @options[:output_path] = @args[2]
+ elsif @args.length > 3
+ raise Twine::Error.new "Unknown argument: #{@args[3]}"
+ else
+ raise Twine::Error.new 'Not enough arguments.'
+ end
+ when 'consume-string-file'
+ if @args.length == 3
+ @options[:input_path] = @args[2]
+ elsif @args.length > 3
+ raise Twine::Error.new "Unknown argument: #{@args[3]}"
+ else
+ raise Twine::Error.new 'Not enough arguments.'
+ end
+ if @options[:languages] and @options[:languages].length > 1
+ raise Twine::Error.new 'Please only specify a single language for the consume-string-file command.'
+ end
+ when 'consume-all-string-files'
+ if @args.length == 3
+ @options[:input_path] = @args[2]
+ elsif @args.length > 3
+ raise Twine::Error.new "Unknown argument: #{@args[3]}"
+ else
+ raise Twine::Error.new 'Not enough arguments.'
+ end
+ when 'generate-loc-drop'
+ @options[:include_untranslated] = true
+ if @args.length == 3
+ @options[:output_path] = @args[2]
+ elsif @args.length > 3
+ raise Twine::Error.new "Unknown argument: #{@args[3]}"
+ else
+ raise Twine::Error.new 'Not enough arguments.'
+ end
+ if !@options[:format]
+ raise Twine::Error.new 'You must specify a format.'
+ end
+ when 'consume-loc-drop'
+ if @args.length == 3
+ @options[:input_path] = @args[2]
+ elsif @args.length > 3
+ raise Twine::Error.new "Unknown argument: #{@args[3]}"
+ else
+ raise Twine::Error.new 'Not enough arguments.'
+ end
+ when 'generate-report'
+ if @args.length > 2
+ raise Twine::Error.new "Unknown argument: #{@args[2]}"
+ end
+ end
+ end
+ end
+end
diff --git a/tools/twine/lib/twine/encoding.rb b/tools/twine/lib/twine/encoding.rb
new file mode 100644
index 0000000000..d268a36802
--- /dev/null
+++ b/tools/twine/lib/twine/encoding.rb
@@ -0,0 +1,20 @@
+module Twine
+ module Encoding
+ def self.encoding_for_path path
+ File.open(path, 'rb') do |f|
+ begin
+ a = f.readbyte
+ b = f.readbyte
+ if (a == 0xfe && b == 0xff)
+ return 'UTF-16BE'
+ elsif (a == 0xff && b == 0xfe)
+ return 'UTF-16LE'
+ end
+ rescue EOFError
+ end
+ end
+
+ 'UTF-8'
+ end
+ end
+end
diff --git a/tools/twine/lib/twine/formatters.rb b/tools/twine/lib/twine/formatters.rb
new file mode 100644
index 0000000000..c2358d7926
--- /dev/null
+++ b/tools/twine/lib/twine/formatters.rb
@@ -0,0 +1,10 @@
+require 'twine/formatters/abstract'
+require 'twine/formatters/android'
+require 'twine/formatters/apple'
+require 'twine/formatters/jquery'
+
+module Twine
+ module Formatters
+ FORMATTERS = [Formatters::Apple, Formatters::Android, Formatters::JQuery]
+ end
+end
diff --git a/tools/twine/lib/twine/formatters/abstract.rb b/tools/twine/lib/twine/formatters/abstract.rb
new file mode 100644
index 0000000000..8cd6c4591f
--- /dev/null
+++ b/tools/twine/lib/twine/formatters/abstract.rb
@@ -0,0 +1,75 @@
+module Twine
+ module Formatters
+ class Abstract
+ attr_accessor :strings
+ attr_accessor :options
+
+ def self.can_handle_directory?(path)
+ return false
+ end
+
+ def initialize(strings, options)
+ @strings = strings
+ @options = options
+ end
+
+ def set_translation_for_key(key, lang, value)
+ if @strings.strings_map.include?(key)
+ @strings.strings_map[key].translations[lang] = value
+ elsif @options[:consume_all]
+ STDERR.puts "Adding new string '#{key}' to strings data file."
+ arr = @strings.sections.select { |s| s.name == 'Uncategorized' }
+ current_section = arr ? arr[0] : nil
+ if !current_section
+ current_section = StringsSection.new('Uncategorized')
+ @strings.sections.insert(0, current_section)
+ end
+ current_row = StringsRow.new(key)
+ current_section.rows << current_row
+ @strings.strings_map[key] = current_row
+ @strings.strings_map[key].translations[lang] = value
+ else
+ STDERR.puts "Warning: '#{key}' not found in strings data file."
+ end
+ if !@strings.language_codes.include?(lang)
+ @strings.add_language_code(lang)
+ end
+ end
+
+ def set_comment_for_key(key, comment)
+ if @strings.strings_map.include?(key)
+ @strings.strings_map[key].comment = comment
+ end
+ end
+
+ def default_file_name
+ raise NotImplementedError.new("You must implement default_file_name in your formatter class.")
+ end
+
+ def determine_language_given_path(path)
+ raise NotImplementedError.new("You must implement determine_language_given_path in your formatter class.")
+ end
+
+ def read_file(path, lang)
+ raise NotImplementedError.new("You must implement read_file in your formatter class.")
+ end
+
+ def write_file(path, lang)
+ raise NotImplementedError.new("You must implement write_file in your formatter class.")
+ end
+
+ def write_all_files(path)
+ if !File.directory?(path)
+ raise Twine::Error.new("Directory does not exist: #{path}")
+ end
+
+ Dir.foreach(path) do |item|
+ lang = determine_language_given_path(item)
+ if lang
+ write_file(File.join(path, item, default_file_name), lang)
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/tools/twine/lib/twine/formatters/android.rb b/tools/twine/lib/twine/formatters/android.rb
new file mode 100644
index 0000000000..a79f5545ef
--- /dev/null
+++ b/tools/twine/lib/twine/formatters/android.rb
@@ -0,0 +1,215 @@
+# encoding: utf-8
+
+require 'rexml/document'
+
+module Twine
+ module Formatters
+ class Android < Abstract
+ FORMAT_NAME = 'android'
+ EXTENSION = '.xml'
+ DEFAULT_FILE_NAME = 'strings.xml'
+ LANG_CODES = Hash[
+ 'zh' => 'zh-Hans',
+ 'zh-rCN' => 'zh-Hans',
+ 'zh-rHK' => 'zh-Hant',
+ 'en-rGB' => 'en-UK',
+ 'in' => 'id',
+ 'nb' => 'no'
+ # TODO: spanish
+ ]
+ DEFAULT_LANG_CODES = Hash[
+ 'zh-TW' => 'zh-Hant' # if we don't have a zh-TW translation, try zh-Hant before en
+ ]
+
+ def self.can_handle_directory?(path)
+ Dir.entries(path).any? { |item| /^values.*$/.match(item) }
+ end
+
+ def default_file_name
+ return DEFAULT_FILE_NAME
+ end
+
+ def determine_language_given_path(path)
+ path_arr = path.split(File::SEPARATOR)
+ path_arr.each do |segment|
+ if segment == 'values'
+ return @strings.language_codes[0]
+ else
+ match = /^values-(.*)$/.match(segment)
+ if match
+ lang = match[1]
+ lang = LANG_CODES.fetch(lang, lang)
+ lang.sub!('-r', '-')
+ return lang
+ end
+ end
+ end
+
+ return
+ end
+
+ def read_file(path, lang)
+ File.open(path, 'r:UTF-8') do |f|
+ current_section = nil
+ doc = REXML::Document.new(f)
+ doc.elements.each('resources/string') do |ele|
+ key = ele.attributes["name"]
+ value = ele.text || ''
+ value.gsub!('\\\'', '\'')
+ value.gsub!('\\"', '"')
+ value.gsub!(/\n/, '')
+ value.gsub!('&lt;', '<')
+ value.gsub!('&amp;', '&')
+ value = iosify_substitutions(value)
+ set_translation_for_key(key, lang, value)
+ end
+ end
+ end
+
+ def write_file(path, lang)
+ default_lang = nil
+ if DEFAULT_LANG_CODES.has_key?(lang)
+ default_lang = DEFAULT_LANG_CODES[lang]
+ end
+ File.open(path, 'w:UTF-8') do |f|
+ f.puts "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<!-- Android Strings File -->\n<!-- Generated by Twine #{Twine::VERSION} -->\n<!-- Language: #{lang} -->"
+ f.write '<resources>'
+ @strings.sections.each do |section|
+ printed_section = false
+ section.rows.each do |row|
+ if row.matches_tags?(@options[:tags], @options[:untagged])
+ if !printed_section
+ f.puts ''
+ if section.name && section.name.length > 0
+ section_name = section.name.gsub('--', '—')
+ f.puts "\t<!-- #{section_name} -->"
+ end
+ printed_section = true
+ end
+
+ key = row.key
+
+ value = row.translated_string_for_lang(lang, default_lang)
+ if !value && @options[:include_untranslated]
+ value = row.translated_string_for_lang(@strings.language_codes[0])
+ end
+
+ if value # if values is nil, there was no appropriate translation, so let Android handle the defaulting
+ value = String.new(value) # use a copy to prevent modifying the original
+
+ # Android enforces the following rules on the values
+ # 1) apostrophes and quotes must be escaped with a backslash
+ value.gsub!('\'', '\\\\\'')
+ value.gsub!('"', '\\\\"')
+ # 2) ampersand and less-than must be in XML-escaped form
+ value.gsub!('&', '&amp;')
+ value.gsub!('<', '&lt;')
+ # 3) fix substitutions (e.g. %s/%@)
+ value = androidify_substitutions(value)
+
+ comment = row.comment
+ if comment
+ comment = comment.gsub('--', '—')
+ end
+
+ if comment && comment.length > 0
+ f.puts "\t<!-- #{comment} -->\n"
+ end
+ f.puts "\t<string name=\"#{key}\">#{value}</string>"
+ end
+ end
+ end
+ end
+
+ f.puts '</resources>'
+ end
+ end
+
+ def iosify_substitutions(str)
+ # 1) use "@" instead of "s" for substituting strings
+ str.gsub!(/%([0-9\$]*)s/, '%\1@')
+
+ # 2) if substitutions are numbered, see if we can remove the numbering safely
+ expectedSub = 1
+ startFound = false
+ foundSub = 0
+ str.each_char do |c|
+ if startFound
+ if c == "%"
+ # this is a literal %, keep moving
+ startFound = false
+ elsif c.match(/\d/)
+ foundSub *= 10
+ foundSub += Integer(c)
+ elsif c == "$"
+ if expectedSub == foundSub
+ # okay to keep going
+ startFound = false
+ expectedSub += 1
+ else
+ # the numbering appears to be important (or non-existent), leave it alone
+ return str
+ end
+ end
+ elsif c == "%"
+ startFound = true
+ foundSub = 0
+ end
+ end
+
+ # if we got this far, then the numbering (if any) is in order left-to-right and safe to remove
+ if expectedSub > 1
+ str.gsub!(/%\d+\$(.)/, '%\1')
+ end
+
+ return str
+ end
+
+ def androidify_substitutions(str)
+ # 1) use "s" instead of "@" for substituting strings
+ str.gsub!(/%([0-9\$]*)@/, '%\1s')
+
+ # 2) if there is more than one substitution in a string, make sure they are numbered
+ substituteCount = 0
+ startFound = false
+ str.each_char do |c|
+ if startFound
+ if c == "%"
+ # ignore as this is a literal %
+ elsif c.match(/\d/)
+ # leave the string alone if it already has numbered substitutions
+ return str
+ else
+ substituteCount += 1
+ end
+ startFound = false
+ elsif c == "%"
+ startFound = true
+ end
+ end
+
+ if substituteCount > 1
+ currentSub = 1
+ startFound = false
+ newstr = ""
+ str.each_char do |c|
+ if startFound
+ if !(c == "%")
+ newstr = newstr + "#{currentSub}$"
+ currentSub += 1
+ end
+ startFound = false
+ elsif c == "%"
+ startFound = true
+ end
+ newstr = newstr + c
+ end
+ return newstr
+ else
+ return str
+ end
+ end
+
+ end
+ end
+end
diff --git a/tools/twine/lib/twine/formatters/apple.rb b/tools/twine/lib/twine/formatters/apple.rb
new file mode 100644
index 0000000000..70cb10db78
--- /dev/null
+++ b/tools/twine/lib/twine/formatters/apple.rb
@@ -0,0 +1,122 @@
+module Twine
+ module Formatters
+ class Apple < Abstract
+ FORMAT_NAME = 'apple'
+ EXTENSION = '.strings'
+ DEFAULT_FILE_NAME = 'Localizable.strings'
+
+ def self.can_handle_directory?(path)
+ Dir.entries(path).any? { |item| /^.+\.lproj$/.match(item) }
+ end
+
+ def default_file_name
+ return DEFAULT_FILE_NAME
+ end
+
+ def determine_language_given_path(path)
+ path_arr = path.split(File::SEPARATOR)
+ path_arr.each do |segment|
+ match = /^(.+)\.lproj$/.match(segment)
+ if match
+ return match[1]
+ end
+ end
+
+ return
+ end
+
+ def read_file(path, lang)
+ encoding = Twine::Encoding.encoding_for_path(path)
+ sep = nil
+ if !encoding.respond_to?(:encode)
+ # This code is not necessary in 1.9.3 and does not work as it did in 1.8.7.
+ if encoding.end_with? 'LE'
+ sep = "\x0a\x00"
+ elsif encoding.end_with? 'BE'
+ sep = "\x00\x0a"
+ else
+ sep = "\n"
+ end
+ end
+
+ if encoding.index('UTF-16')
+ mode = "rb:#{encoding}"
+ else
+ mode = "r:#{encoding}"
+ end
+
+ File.open(path, mode) do |f|
+ last_comment = nil
+ while line = (sep) ? f.gets(sep) : f.gets
+ if encoding.index('UTF-16')
+ if line.respond_to? :encode!
+ line.encode!('UTF-8')
+ else
+ require 'iconv'
+ line = Iconv.iconv('UTF-8', encoding, line).join
+ end
+ end
+ match = /"((?:[^"\\]|\\.)+)"\s*=\s*"((?:[^"\\]|\\.)*)"/.match(line)
+ if match
+ key = match[1]
+ key.gsub!('\\"', '"')
+ value = match[2]
+ value.gsub!('\\"', '"')
+ set_translation_for_key(key, lang, value)
+ if last_comment
+ set_comment_for_key(key, last_comment)
+ end
+ end
+ if @options[:consume_comments]
+ match = /\/\* (.*) \*\//.match(line)
+ if match
+ last_comment = match[1]
+ else
+ last_comment = nil
+ end
+ end
+ end
+ end
+ end
+
+ def write_file(path, lang)
+ default_lang = @strings.language_codes[0]
+ encoding = @options[:output_encoding] || 'UTF-8'
+ File.open(path, "w:#{encoding}") do |f|
+ f.puts "/**\n * Apple Strings File\n * Generated by Twine #{Twine::VERSION}\n * Language: #{lang}\n */"
+ @strings.sections.each do |section|
+ printed_section = false
+ section.rows.each do |row|
+ if row.matches_tags?(@options[:tags], @options[:untagged])
+ f.puts ''
+ if !printed_section
+ if section.name && section.name.length > 0
+ f.print "/********** #{section.name} **********/\n\n"
+ end
+ printed_section = true
+ end
+
+ key = row.key
+ key = key.gsub('"', '\\\\"')
+
+ value = row.translated_string_for_lang(lang, default_lang)
+ value = value.gsub('"', '\\\\"')
+
+ comment = row.comment
+ if comment
+ comment = comment.gsub('*/', '* /')
+ end
+
+ if comment && comment.length > 0
+ f.print "/* #{comment} */\n"
+ end
+
+ f.print "\"#{key}\" = \"#{value}\";\n"
+ end
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/tools/twine/lib/twine/formatters/jquery.rb b/tools/twine/lib/twine/formatters/jquery.rb
new file mode 100644
index 0000000000..14f12f4f97
--- /dev/null
+++ b/tools/twine/lib/twine/formatters/jquery.rb
@@ -0,0 +1,96 @@
+module Twine
+ module Formatters
+ class JQuery < Abstract
+ FORMAT_NAME = 'jquery'
+ EXTENSION = '.json'
+ DEFAULT_FILE_NAME = 'localize.json'
+
+ def self.can_handle_directory?(path)
+ Dir.entries(path).any? { |item| /^.+\.json$/.match(item) }
+ end
+
+ def default_file_name
+ return DEFAULT_FILE_NAME
+ end
+
+ def determine_language_given_path(path)
+ path_arr = path.split(File::SEPARATOR)
+ path_arr.each do |segment|
+ match = /^((.+)-)?([^-]+)\.json$/.match(segment)
+ if match
+ return match[3]
+ end
+ end
+
+ return
+ end
+
+ def read_file(path, lang)
+ begin
+ require "json"
+ rescue LoadError
+ raise Twine::Error.new "You must run 'gem install json' in order to read or write jquery-localize files."
+ end
+
+ open(path) do |io|
+ json = JSON.load(io)
+ json.each do |key, value|
+ set_translation_for_key(key, lang, value)
+ end
+ end
+ end
+
+ def write_file(path, lang)
+ begin
+ require "json"
+ rescue LoadError
+ raise Twine::Error.new "You must run 'gem install json' in order to read or write jquery-localize files."
+ end
+
+ default_lang = @strings.language_codes[0]
+ encoding = @options[:output_encoding] || 'UTF-8'
+ File.open(path, "w:#{encoding}") do |f|
+ f.puts "/**\n * JQuery Language File\n * Generated by Twine\n * Language: #{lang}\n */"
+ f.puts "{"
+
+ @strings.sections.each_with_index do |section, si|
+ printed_section = false
+ section.rows.each_with_index do |row, ri|
+ if row.matches_tags?(@options[:tags], @options[:untagged])
+ if !printed_section
+ f.puts ''
+ if section.name && section.name.length > 0
+ f.puts "/* #{section.name} */"
+ end
+ printed_section = true
+ end
+
+ key = row.key
+ key = key.gsub('"', '\\\\"')
+
+ value = row.translated_string_for_lang(lang, default_lang)
+ value = value.gsub('"', '\\\\"')
+
+ comment = row.comment
+ if comment
+ comment = comment.gsub('*/', '* /')
+ end
+
+ f.print "\"#{key}\":\"#{value}\","
+
+ if comment && comment.length > 0
+ f.print " /* #{comment} */\n"
+ else
+ f.print "\n"
+ end
+ end
+ end
+ end
+ f.seek(-2, IO::SEEK_CUR)
+ f.puts "\n}"
+
+ end
+ end
+ end
+ end
+end
diff --git a/tools/twine/lib/twine/runner.rb b/tools/twine/lib/twine/runner.rb
new file mode 100644
index 0000000000..7e6b89e377
--- /dev/null
+++ b/tools/twine/lib/twine/runner.rb
@@ -0,0 +1,301 @@
+require 'tmpdir'
+
+module Twine
+ VALID_COMMANDS = ['generate-string-file', 'generate-all-string-files', 'consume-string-file', 'consume-all-string-files', 'generate-loc-drop', 'consume-loc-drop', 'generate-report']
+
+ class Runner
+ def initialize(args)
+ @options = {}
+ @args = args
+ end
+
+ def self.run(args)
+ new(args).run
+ end
+
+ def run
+ # Parse all CLI arguments.
+ CLI::parse_args(@args, @options)
+ read_strings_data
+ execute_command
+ end
+
+ def read_strings_data
+ @strings = StringsFile.new
+ @strings.read @options[:strings_file]
+ end
+
+ def write_strings_data(path)
+ if @options[:developer_language]
+ @strings.set_developer_language_code(@options[:developer_language])
+ end
+ @strings.write(path)
+ end
+
+ def execute_command
+ case @options[:command]
+ when 'generate-string-file'
+ generate_string_file
+ when 'generate-all-string-files'
+ generate_all_string_files
+ when 'consume-string-file'
+ consume_string_file
+ when 'consume-all-string-files'
+ consume_all_string_files
+ when 'generate-loc-drop'
+ generate_loc_drop
+ when 'consume-loc-drop'
+ consume_loc_drop
+ when 'generate-report'
+ generate_report
+ end
+ end
+
+ def generate_string_file
+ lang = nil
+ if @options[:languages]
+ lang = @options[:languages][0]
+ end
+
+ read_write_string_file(@options[:output_path], false, lang)
+ end
+
+ def generate_all_string_files
+ if !File.directory?(@options[:output_path])
+ raise Twine::Error.new("Directory does not exist: #{@options[:output_path]}")
+ end
+
+ format = @options[:format]
+ if !format
+ format = determine_format_given_directory(@options[:output_path])
+ end
+ if !format
+ raise Twine::Error.new "Could not determine format given the contents of #{@options[:output_path]}"
+ end
+
+ formatter = formatter_for_format(format)
+
+ formatter.write_all_files(@options[:output_path])
+ end
+
+ def consume_string_file
+ lang = nil
+ if @options[:languages]
+ lang = @options[:languages][0]
+ end
+
+ read_write_string_file(@options[:input_path], true, lang)
+ output_path = @options[:output_path] || @options[:strings_file]
+ write_strings_data(output_path)
+ end
+
+ def consume_all_string_files
+ if !File.directory?(@options[:input_path])
+ raise Twine::Error.new("Directory does not exist: #{@options[:output_path]}")
+ end
+
+ Dir.glob(File.join(@options[:input_path], "**/*")) do |item|
+ if File.file?(item)
+ begin
+ read_write_string_file(item, true, nil)
+ rescue Twine::Error => e
+ STDERR.puts "#{e.message}"
+ end
+ end
+ end
+
+ output_path = @options[:output_path] || @options[:strings_file]
+ write_strings_data(output_path)
+ end
+
+ def read_write_string_file(path, is_read, lang)
+ if is_read && !File.file?(path)
+ raise Twine::Error.new("File does not exist: #{path}")
+ end
+
+ format = @options[:format]
+ if !format
+ format = determine_format_given_path(path)
+ end
+ if !format
+ raise Twine::Error.new "Unable to determine format of #{path}"
+ end
+
+ formatter = formatter_for_format(format)
+
+ if !lang
+ lang = determine_language_given_path(path)
+ end
+ if !lang
+ lang = formatter.determine_language_given_path(path)
+ end
+ if !lang
+ raise Twine::Error.new "Unable to determine language for #{path}"
+ end
+
+ if !@strings.language_codes.include? lang
+ @strings.language_codes << lang
+ end
+
+ if is_read
+ formatter.read_file(path, lang)
+ else
+ formatter.write_file(path, lang)
+ end
+ end
+
+ def generate_loc_drop
+ begin
+ require 'zip/zip'
+ rescue LoadError
+ raise Twine::Error.new "You must run 'gem install rubyzip' in order to create or consume localization drops."
+ end
+
+ if File.file?(@options[:output_path])
+ File.delete(@options[:output_path])
+ end
+
+ Dir.mktmpdir do |dir|
+ Zip::ZipFile.open(@options[:output_path], Zip::ZipFile::CREATE) do |zipfile|
+ zipfile.mkdir('Locales')
+
+ formatter = formatter_for_format(@options[:format])
+ @strings.language_codes.each do |lang|
+ if @options[:languages] == nil || @options[:languages].length == 0 || @options[:languages].include?(lang)
+ file_name = lang + formatter.class::EXTENSION
+ real_path = File.join(dir, file_name)
+ zip_path = File.join('Locales', file_name)
+ formatter.write_file(real_path, lang)
+ zipfile.add(zip_path, real_path)
+ end
+ end
+ end
+ end
+ end
+
+ def consume_loc_drop
+ if !File.file?(@options[:input_path])
+ raise Twine::Error.new("File does not exist: #{@options[:input_path]}")
+ end
+
+ begin
+ require 'zip/zip'
+ rescue LoadError
+ raise Twine::Error.new "You must run 'gem install rubyzip' in order to create or consume localization drops."
+ end
+
+ Dir.mktmpdir do |dir|
+ Zip::ZipFile.open(@options[:input_path]) do |zipfile|
+ zipfile.each do |entry|
+ if !entry.name.end_with?'/' and !File.basename(entry.name).start_with?'.'
+ real_path = File.join(dir, entry.name)
+ FileUtils.mkdir_p(File.dirname(real_path))
+ zipfile.extract(entry.name, real_path)
+ begin
+ read_write_string_file(real_path, true, nil)
+ rescue Twine::Error => e
+ STDERR.puts "#{e.message}"
+ end
+ end
+ end
+ end
+ end
+
+ output_path = @options[:output_path] || @options[:strings_file]
+ write_strings_data(output_path)
+ end
+
+ def generate_report
+ total_strings = 0
+ strings_per_lang = {}
+ all_keys = Set.new
+ duplicate_keys = Set.new
+ keys_without_tags = Set.new
+ @strings.language_codes.each do |code|
+ strings_per_lang[code] = 0
+ end
+
+ @strings.sections.each do |section|
+ section.rows.each do |row|
+ total_strings += 1
+
+ if all_keys.include? row.key
+ duplicate_keys.add(row.key)
+ else
+ all_keys.add(row.key)
+ end
+
+ row.translations.each_key do |code|
+ strings_per_lang[code] += 1
+ end
+
+ if row.tags == nil || row.tags.length == 0
+ keys_without_tags.add(row.key)
+ end
+ end
+ end
+
+ # Print the report.
+ puts "Total number of strings = #{total_strings}"
+ @strings.language_codes.each do |code|
+ puts "#{code}: #{strings_per_lang[code]}"
+ end
+
+ if duplicate_keys.length > 0
+ puts "\nDuplicate string keys:"
+ duplicate_keys.each do |key|
+ puts key
+ end
+ end
+
+ if keys_without_tags.length == total_strings
+ puts "\nNone of your strings have tags."
+ elsif keys_without_tags.length > 0
+ puts "\nStrings without tags:"
+ keys_without_tags.each do |key|
+ puts key
+ end
+ end
+ end
+
+ def determine_language_given_path(path)
+ code = File.basename(path, File.extname(path))
+ if !@strings.language_codes.include? code
+ code = nil
+ end
+
+ code
+ end
+
+ def determine_format_given_path(path)
+ ext = File.extname(path)
+ Formatters::FORMATTERS.each do |formatter|
+ if formatter::EXTENSION == ext
+ return formatter::FORMAT_NAME
+ end
+ end
+
+ return
+ end
+
+ def determine_format_given_directory(directory)
+ Formatters::FORMATTERS.each do |formatter|
+ if formatter.can_handle_directory?(directory)
+ return formatter::FORMAT_NAME
+ end
+ end
+
+ return
+ end
+
+ def formatter_for_format(format)
+ Formatters::FORMATTERS.each do |formatter|
+ if formatter::FORMAT_NAME == format
+ return formatter.new(@strings, @options)
+ end
+ end
+
+ return
+ end
+ end
+end
diff --git a/tools/twine/lib/twine/stringsfile.rb b/tools/twine/lib/twine/stringsfile.rb
new file mode 100644
index 0000000000..f9316858b8
--- /dev/null
+++ b/tools/twine/lib/twine/stringsfile.rb
@@ -0,0 +1,201 @@
+module Twine
+ class StringsSection
+ attr_reader :name
+ attr_reader :rows
+
+ def initialize(name)
+ @name = name
+ @rows = []
+ end
+ end
+
+ class StringsRow
+ attr_reader :key
+ attr_accessor :comment
+ attr_accessor :tags
+ attr_reader :translations
+
+ def initialize(key)
+ @key = key
+ @comment = nil
+ @tags = nil
+ @translations = {}
+ end
+
+ def matches_tags?(tags, include_untagged)
+ if tags == nil || tags.length == 0
+ # The user did not specify any tags. Everything passes.
+ return true
+ elsif @tags == nil || @tags.length == 0
+ # This row has no tags.
+ return (include_untagged) ? true : false
+ else
+ tags.each do |tag|
+ if @tags.include? tag
+ return true
+ end
+ end
+ end
+
+ return false
+ end
+
+ def translated_string_for_lang(lang, default_lang=nil)
+ if @translations[lang]
+ return @translations[lang]
+ elsif default_lang.respond_to?("each")
+ default_lang.each do |def_lang|
+ if @translations[def_lang]
+ return @translations[def_lang]
+ end
+ end
+ return nil
+ else
+ return @translations[default_lang]
+ end
+ end
+ end
+
+ class StringsFile
+ attr_reader :sections
+ attr_reader :strings_map
+ attr_reader :language_codes
+
+ def initialize
+ @sections = []
+ @strings_map = {}
+ @language_codes = []
+ end
+
+ def read(path)
+ if !File.file?(path)
+ raise Twine::Error.new("File does not exist: #{path}")
+ end
+
+ File.open(path, 'r:UTF-8') do |f|
+ line_num = 0
+ current_section = nil
+ current_row = nil
+ while line = f.gets
+ parsed = false
+ line.strip!
+ line_num += 1
+
+ if line.length == 0
+ next
+ end
+
+ if line.length > 4 && line[0, 2] == '[['
+ match = /^\[\[(.+)\]\]$/.match(line)
+ if match
+ current_section = StringsSection.new(match[1].strip)
+ @sections << current_section
+ parsed = true
+ end
+ elsif line.length > 2 && line[0, 1] == '['
+ match = /^\[(.+)\]$/.match(line)
+ if match
+ current_row = StringsRow.new(match[1].strip)
+ @strings_map[current_row.key] = current_row
+ if !current_section
+ current_section = StringsSection.new('')
+ @sections << current_section
+ end
+ current_section.rows << current_row
+ parsed = true
+ end
+ else
+ match = /^([^=]+)=(.*)$/.match(line)
+ if match
+ key = match[1].strip
+ value = match[2].strip
+ if value[0,1] == '`' && value[-1,1] == '`'
+ value = value[1..-2]
+ end
+
+ case key
+ when "comment"
+ current_row.comment = value
+ when 'tags'
+ current_row.tags = value.split(',')
+ else
+ if !@language_codes.include? key
+ add_language_code(key)
+ end
+ current_row.translations[key] = value
+ end
+ parsed = true
+ end
+ end
+
+ if !parsed
+ raise Twine::Error.new("Unable to parse line #{line_num} of #{path}: #{line}")
+ end
+ end
+ end
+ end
+
+ def write(path)
+ dev_lang = @language_codes[0]
+
+ File.open(path, 'w:UTF-8') do |f|
+ @sections.each do |section|
+ if f.pos > 0
+ f.puts ''
+ end
+
+ f.puts "[[#{section.name}]]"
+
+ section.rows.each do |row|
+ f.puts "\t[#{row.key}]"
+ value = row.translations[dev_lang]
+ if !value
+ puts "Warning: #{row.key} does not exist in developer language '#{dev_lang}'"
+ else
+ if value[0,1] == ' ' || value[-1,1] == ' ' || (value[0,1] == '`' && value[-1,1] == '`')
+ value = '`' + value + '`'
+ end
+ f.puts "\t\t#{dev_lang} = #{value}"
+ end
+
+ if row.tags && row.tags.length > 0
+ tag_str = row.tags.join(',')
+ f.puts "\t\ttags = #{tag_str}"
+ end
+ if row.comment && row.comment.length > 0
+ f.puts "\t\tcomment = #{row.comment}"
+ end
+ @language_codes[1..-1].each do |lang|
+ value = row.translations[lang]
+ if value && value != row.translations[dev_lang]
+ if value[0,1] == ' ' || value[-1,1] == ' ' || (value[0,1] == '`' && value[-1,1] == '`')
+ value = '`' + value + '`'
+ end
+ f.puts "\t\t#{lang} = #{value}"
+ end
+ end
+ end
+ end
+ end
+ end
+
+ def add_language_code(code)
+ if @language_codes.length == 0
+ @language_codes << code
+ elsif !@language_codes.include?(code)
+ dev_lang = @language_codes[0]
+ @language_codes << code
+ @language_codes.delete(dev_lang)
+ @language_codes.sort!
+ @language_codes.insert(0, dev_lang)
+ end
+ end
+
+ def set_developer_language_code(code)
+ if @language_codes.include?(code)
+ @language_codes.delete(code)
+ end
+ @language_codes.insert(0, code)
+ end
+ end
+end
diff --git a/tools/twine/lib/twine/version.rb b/tools/twine/lib/twine/version.rb
new file mode 100644
index 0000000000..199cb1a640
--- /dev/null
+++ b/tools/twine/lib/twine/version.rb
@@ -0,0 +1,3 @@
+module Twine
+ VERSION = '0.2.2'
+end
diff --git a/tools/twine/test/fixtures/en-1.json b/tools/twine/test/fixtures/en-1.json
new file mode 100644
index 0000000000..26c23f4c2c
--- /dev/null
+++ b/tools/twine/test/fixtures/en-1.json
@@ -0,0 +1,12 @@
+/**
+ * JQuery Language File
+ * Generated by Twine
+ * Language: en
+ */
+{
+
+/* My Strings */
+"key1":"key1-english",
+"key3":"key3-english",
+"key5":"A new string"
+}
diff --git a/tools/twine/test/fixtures/en-1.strings b/tools/twine/test/fixtures/en-1.strings
new file mode 100644
index 0000000000..35cb6ca7e1
--- /dev/null
+++ b/tools/twine/test/fixtures/en-1.strings
@@ -0,0 +1,10 @@
+/**
+ * iOS Strings File
+ * Generated by Twine
+ * Language: en
+ */
+
+/* My Strings */
+"key1" = "key1-english";
+"key3" = "key3-english";
+"key5" = "A new string";
diff --git a/tools/twine/test/fixtures/fr-1.xml b/tools/twine/test/fixtures/fr-1.xml
new file mode 100644
index 0000000000..78cd8f34d4
--- /dev/null
+++ b/tools/twine/test/fixtures/fr-1.xml
@@ -0,0 +1,9 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!-- Android Strings File -->
+<!-- Generated by Twine -->
+<!-- Language: fr -->
+<resources>
+ <string name="key1">key1-french</string>
+ <string name="key2">key2-french</string>
+ <string name="key3">key3-french</string>
+</resources>
diff --git a/tools/twine/test/fixtures/strings-1.txt b/tools/twine/test/fixtures/strings-1.txt
new file mode 100644
index 0000000000..ec5da6065c
--- /dev/null
+++ b/tools/twine/test/fixtures/strings-1.txt
@@ -0,0 +1,16 @@
+[[My Strings]]
+ [key1]
+ en = key1-english
+ tags = tag1
+ es = key1-spanish
+ fr = key1-french
+ [key2]
+ en = key2-english
+ tags = tag2
+ fr = key2-french
+ [key3]
+ en = key3-english
+ tags = tag1,tag2
+ es = key3-spanish
+ [key4]
+ en = key4-english
diff --git a/tools/twine/test/fixtures/test-output-1.txt b/tools/twine/test/fixtures/test-output-1.txt
new file mode 100644
index 0000000000..bc051c3973
--- /dev/null
+++ b/tools/twine/test/fixtures/test-output-1.txt
@@ -0,0 +1,11 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!-- Android Strings File -->
+<!-- Generated by Twine <%= Twine::VERSION %> -->
+<!-- Language: fr -->
+<resources>
+ <!-- My Strings -->
+ <string name="key1">key1-french</string>
+ <string name="key2">key2-french</string>
+ <string name="key3">key3-english</string>
+ <string name="key4">key4-english</string>
+</resources>
diff --git a/tools/twine/test/fixtures/test-output-2.txt b/tools/twine/test/fixtures/test-output-2.txt
new file mode 100644
index 0000000000..c530a59b52
--- /dev/null
+++ b/tools/twine/test/fixtures/test-output-2.txt
@@ -0,0 +1,11 @@
+/**
+ * Apple Strings File
+ * Generated by Twine <%= Twine::VERSION %>
+ * Language: en
+ */
+
+/********** My Strings **********/
+
+"key1" = "key1-english";
+
+"key3" = "key3-english";
diff --git a/tools/twine/test/fixtures/test-output-3.txt b/tools/twine/test/fixtures/test-output-3.txt
new file mode 100644
index 0000000000..1f26afe869
--- /dev/null
+++ b/tools/twine/test/fixtures/test-output-3.txt
@@ -0,0 +1,17 @@
+[[My Strings]]
+ [key1]
+ en = key1-english
+ tags = tag1
+ es = key1-spanish
+ fr = key1-french
+ [key2]
+ en = key2-english
+ tags = tag2
+ fr = key2-french
+ [key3]
+ en = key3-english
+ tags = tag1,tag2
+ es = key3-spanish
+ fr = key3-french
+ [key4]
+ en = key4-english
diff --git a/tools/twine/test/fixtures/test-output-4.txt b/tools/twine/test/fixtures/test-output-4.txt
new file mode 100644
index 0000000000..98f81f295b
--- /dev/null
+++ b/tools/twine/test/fixtures/test-output-4.txt
@@ -0,0 +1,20 @@
+[[Uncategorized]]
+ [key5]
+ en = A new string
+
+[[My Strings]]
+ [key1]
+ en = key1-english
+ tags = tag1
+ es = key1-spanish
+ fr = key1-french
+ [key2]
+ en = key2-english
+ tags = tag2
+ fr = key2-french
+ [key3]
+ en = key3-english
+ tags = tag1,tag2
+ es = key3-spanish
+ [key4]
+ en = key4-english
diff --git a/tools/twine/test/fixtures/test-output-5.txt b/tools/twine/test/fixtures/test-output-5.txt
new file mode 100644
index 0000000000..02aa52d773
--- /dev/null
+++ b/tools/twine/test/fixtures/test-output-5.txt
@@ -0,0 +1,11 @@
+/**
+ * JQuery Language File
+ * Generated by Twine
+ * Language: en
+ */
+{
+
+/* My Strings */
+"key1":"key1-english",
+"key3":"key3-english"
+}
diff --git a/tools/twine/test/twine_test.rb b/tools/twine/test/twine_test.rb
new file mode 100644
index 0000000000..f4bced703e
--- /dev/null
+++ b/tools/twine/test/twine_test.rb
@@ -0,0 +1,58 @@
+require 'ERB'
+require 'rubygems'
+require 'test/unit'
+require 'twine'
+
+class TwineTest < Test::Unit::TestCase
+ def test_generate_string_file_1
+ Dir.mktmpdir do |dir|
+ output_path = File.join(dir, 'fr.xml')
+ Twine::Runner.run(%W(generate-string-file test/fixtures/strings-1.txt #{output_path} --include-untranslated))
+ assert_equal(ERB.new(File.read('test/fixtures/test-output-1.txt')).result, File.read(output_path))
+ end
+ end
+
+ def test_generate_string_file_2
+ Dir.mktmpdir do |dir|
+ output_path = File.join(dir, 'en.strings')
+ Twine::Runner.run(%W(generate-string-file test/fixtures/strings-1.txt #{output_path} -t tag1))
+ assert_equal(ERB.new(File.read('test/fixtures/test-output-2.txt')).result, File.read(output_path))
+ end
+ end
+
+ def test_generate_string_file_3
+ Dir.mktmpdir do |dir|
+ output_path = File.join(dir, 'en.json')
+ Twine::Runner.run(%W(generate-string-file test/fixtures/strings-1.txt #{output_path} -t tag1))
+ assert_equal(ERB.new(File.read('test/fixtures/test-output-5.txt')).result, File.read(output_path))
+ end
+ end
+
+ def test_consume_string_file_1
+ Dir.mktmpdir do |dir|
+ output_path = File.join(dir, 'strings.txt')
+ Twine::Runner.run(%W(consume-string-file test/fixtures/strings-1.txt test/fixtures/fr-1.xml -o #{output_path} -l fr))
+ assert_equal(File.read('test/fixtures/test-output-3.txt'), File.read(output_path))
+ end
+ end
+
+ def test_consume_string_file_2
+ Dir.mktmpdir do |dir|
+ output_path = File.join(dir, 'strings.txt')
+ Twine::Runner.run(%W(consume-string-file test/fixtures/strings-1.txt test/fixtures/en-1.strings -o #{output_path} -l en -a))
+ assert_equal(File.read('test/fixtures/test-output-4.txt'), File.read(output_path))
+ end
+ end
+
+ def test_consume_string_file_3
+ Dir.mktmpdir do |dir|
+ output_path = File.join(dir, 'strings.txt')
+ Twine::Runner.run(%W(consume-string-file test/fixtures/strings-1.txt test/fixtures/en-1.json -o #{output_path} -l en -a))
+ assert_equal(File.read('test/fixtures/test-output-4.txt'), File.read(output_path))
+ end
+ end
+
+ def test_generate_report_1
+ Twine::Runner.run(%w(generate-report test/fixtures/strings-1.txt))
+ end
+end
diff --git a/tools/twine/twine b/tools/twine/twine
new file mode 100755
index 0000000000..fc9aa6eecf
--- /dev/null
+++ b/tools/twine/twine
@@ -0,0 +1,3 @@
+#!/bin/sh
+BASEDIR=$(dirname $0)
+ruby -rubygems -I $BASEDIR/lib $BASEDIR/bin/twine $@
diff --git a/tools/twine/twine.gemspec b/tools/twine/twine.gemspec
new file mode 100644
index 0000000000..d26db2a47a
--- /dev/null
+++ b/tools/twine/twine.gemspec
@@ -0,0 +1,30 @@
+$LOAD_PATH.unshift 'lib'
+require 'twine/version'
+
+Gem::Specification.new do |s|
+ s.name = "twine"
+ s.version = Twine::VERSION
+ s.date = Time.now.strftime('%Y-%m-%d')
+ s.summary = "Manage strings and their translations for your iOS and Android projects."
+ s.homepage = "https://github.com/mobiata/twine"
+ s.email = "twine@mobiata.com"
+ s.authors = [ "Sebastian Celis" ]
+ s.has_rdoc = false
+
+ s.files = %w( Gemfile README.md LICENSE )
+ s.files += Dir.glob("lib/**/*")
+ s.files += Dir.glob("bin/**/*")
+ s.files += Dir.glob("test/**/*")
+ s.test_file = 'test/twine_test.rb'
+
+ s.required_ruby_version = ">= 1.8.7"
+ s.add_runtime_dependency('rubyzip', "~> 0.9.5")
+ s.add_development_dependency('rake', "~> 0.9.2")
+
+ s.executables = %w( twine )
+ s.description = <<desc
+ Twine is a command line tool for managing your strings and their translations.
+
+ It is geared toward Mac OS X, iOS, and Android developers.
+desc
+end