Age | Commit message (Collapse) | Author |
|
|
|
|
|
|
|
|
|
Adds `--early-stopping-on first|all|any` allowing to decide if early stopping should take into account only first, all, or any validation metrics.
Feature request: https://github.com/marian-nmt/marian-dev/issues/850
Regression tests: https://github.com/marian-nmt/marian-regression-tests/pull/79
|
|
|
|
|
|
|
|
vs-ssh.visualstudio.com:v3/machinetranslation/Marian/marian-dev
|
|
|
|
|
|
Support for RMSNorm as drop-in replace for LayerNorm from _Biao Zhang; Rico Sennrich (2019). Root Mean Square Layer Normalization_. Enabled in Transformer model via `--transformer-postprocess dar` instead of `dan`.
|
|
|
|
bias addition kernel for CUDA < 11 (#778)
Co-authored-by: Marcin Junczys-Dowmunt <marcinjd@microsoft.com>
|
|
This reverts commit 096c48e51cd2e61bb275345d7cca99cbfd6bc5c7.
|
|
vocabulary
This PR extends the --allow-special feature to default vocabulary items as well. If the default vocabulary is provided with symbols ostensibly generated from the SentencePiece Byte Fallback mechanism, we suppress the control characters from that list.
|
|
This PR adds cleaner suppression of unwanted output words. We identified a situation where SPM with byte-fallback can generate random bytes with output-sampling.
That is particularly harmful when that random bytes happens to be a newline symbol. Here we suppress newline in output unless explicitly wanted.
|
|
Fixes #842 #843 #844
|
|
|
|
Display decoder time statistics if requested
|
|
There seems to be no benefit from TC_MALLOC any more, hence removing.
|
|
|
|
* fix model loading on architectures where size_t is 32bit
* Update the changelog
Co-authored-by: Roman Grundkiewicz <rgrundkiewicz@gmail.com>
|
|
* unit tests for binary file operations
* adjust changelog
* Set file_ in TemporaryFile for MSVC
Co-authored-by: Roman Grundkiewicz <rgrundkiewicz@gmail.com>
|
|
|
|
This PR updates Windows build using Visual Studio CMake compilation with Ninja. It does not affect standard VS compilation or Windows builds on Azure/GitHub CI.
List of changes:
- Fixed syntax in the script installing dependencies via vcpkg.
- Removed installation of Protobuf (already included as a submodule) and Boost 1.72 (the previous solution no longer works with new vcpkg).
- Disabled compilation of marian-server in the default setting due to Boost issues.
- Disabled compilation of NCCL in the default setting due to an error (see comment in the code).
- Updated vs/README.
|
|
|
|
Adds support for Microsoft-internal binary shortlist format.
|
|
* Fix missing float template specialisation for elem::Plus
* Update CHANGELOG.md
|
|
If --train-sets or --valid-sets are not provided, the fallback to the
hard-coded paths does not occur. This commit requires that these
entities have a non-empty value.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* Fix omp variable names
|
|
* Update simple-websocket-server submodule
* Update VERSION
|
|
|
|
* Doxygen structure for expression graph operators
* Document arithmetic expression operations
* Document comparison expression operations
* Document exp/log and trig operations
* Add missing implementation for cos/tan
* Document expression manipulation operations
* Document misc math operations
* Overview of operators
* Document activation functions
* Document element-wise min/max
* Document debugging/checkpoint operators
* Document topk/argmin/argmax operations
* Document index-based operations
* Document reduction operations
* Document lambda expression operators
* Document product operations
* Document softmax, cross-entropy, unlikelihood operations
* Document dropout operations
* Document scalar product and weighted average operations
* Document layer normalization, highway and pooling operations
* Document shift expression operator
* Extra details on rules for adding specializations to .inc files
* Add SinNodeOp example for specialization documentation
* Additional details in tensor operator documentation
* Remove brief command from doxygen comments
* Prefer @ style doxygen functions to \
* Document n-ary function macros
* Enable .cu and .inc files in documentation
* Add a comment about ONNX mapping
* Remove empty lines in doxygen
* Update CHANGELOG
Co-authored-by: Roman Grundkiewicz <rgrundkiewicz@gmail.com>
|
|
* add API docs for expression_graph.h
* change API docs to doxygen-readable format
* add API docs for node_initializers
* update doxygen configure file
* add hyperlinks and remove layers section from graph documentation
* fixing typos and links on graph doc
|
|
This adds support for boost 1.75
|
|
|
|
|
|
gradient exchanges
* Refactors MPI-related code
* Adds node-local updates with occasional inter-node updates
* decouples batch-reading across nodes
|