diff options
| author | Martin Junczys-Dowmunt <Marcin.JunczysDowmunt@microsoft.com> | 2020-05-22 02:20:52 +0300 |
|---|---|---|
| committer | Martin Junczys-Dowmunt <Marcin.JunczysDowmunt@microsoft.com> | 2020-05-22 02:20:52 +0300 |
| commit | 5a439245b1aa0145c9e2c0fb15de178b70b30c59 (patch) | |
| tree | b9f98b510948f1b7ccd3f2b0418d491b69b7ce52 /CMakeLists.txt | |
| parent | 77a420740c79782d251bfe43a10279e3cfb10781 (diff) | |
Merged PR 12923: LSH indexing to replace short list
This PR adds the FAISS LSH index to the Marian as a CPU-side optimization in the output layer of transformer models via KNN search.
* It allows to replace the potentially harmful short-list with a ML free approximation of the final matrix multiply. With increasing number of K neighbors and the size of the chosen hash in bits the approximation becomes more accurate, but also slower. For model dimensions of 512, the sweet spot seems to be around k=100-150 and nbits=1024-1536
* Enable during CPU-side decoding via `--output-approx-knn k nbits`
* Add a `lambda` node that allows to create custom new nodes, most useful for CPU use right now.
Diffstat (limited to 'CMakeLists.txt')
| -rw-r--r-- | CMakeLists.txt | 3 |
1 files changed, 3 insertions, 0 deletions
diff --git a/CMakeLists.txt b/CMakeLists.txt index 8e7b643c..9c83579a 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -138,6 +138,9 @@ else(MSVC) set(INTRINSICS "-msse4.1") endif() + add_definitions(-DFINTEGER=long) + set(EXT_LIBS ${EXT_LIBS} faiss) + if(USE_FBGEMM) set(EXT_LIBS ${EXT_LIBS} fbgemm dl) add_definitions(-DUSE_FBGEMM=1) |