Welcome to mirror list, hosted at ThFree Co, Russian Federation.

github.com/bitextor/bicleaner-ai.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2022-09-30Update Hub client to 0.10ZJaume
2022-09-23Add test suite (#21)Jaume Zaragoza
* Basic test for full model training * Extend full train test * Add train lite test * Ensure reproducibility of frequence noise * Unit test for noise generation * Add Tokenizer class test * Remove old test corpus file * Add classifier tests Download files on pytest setup to the test dir to avoid downloading it every time. Test normal, calibrated and raw modes. * Download models only in classifier test * Delete args object to avoid interference between tests
2022-09-15Set SentencePiece seedZJaume
2022-09-15Flatten full model directory (again)ZJaume
2022-09-15Fix transformer trainingZJaume
Ignore synthetic noise tag when loading data. Don't return tuples in datagen for transformer Fix TokenAndPositionEmbeddings call
2022-09-05Look for model and vocab subdirectories when loadingZJaume
2022-09-05Fix typo when writing version to metadataZJaume
2022-08-26Remove unused import in __init__.pyZJaume
2022-08-24Restore classifier layer loading for old modelsZJaume
2022-08-23Write current package version to metadataZJaume
2022-08-23Move package version to setup.cfgZJaume
2022-08-19Don't load Hardrules objects if disabledZJaume
2022-08-17typoMarta Bañón
2022-08-17typoMarta Bañón
2022-08-09Restore retrocompatibility with older modelsZJaume
2022-08-09Fix loading local modelsZJaume
2022-08-09Speed improvements using padding longest and no max_lengthZJaume
2022-07-27Be more informative when model not foundZJaume
2022-07-27Remove unset variableZJaume
2022-07-27Download models from HFHub if possibleZJaume
2022-07-27Introduce model namesZJaume
2022-07-27Remove vocab and model file config values, use only dirZJaume
2022-07-27Flatten the full model directoryZJaume
2022-07-27Overwrite XLMRConfig classZJaume
This makes the classes more compatible with HF API and to be able to load them later more easily.
2022-07-27Update versionZJaume
2022-07-25Redirect predict progbar to stder in debug modeZJaume
2022-07-11Fix imports in classifier and trainZJaume
2022-07-11Introduce BICLEANER_AI_THREADS to control number of threadsZJaume
`--processes` option is now deprecated in favor of the environment variable. The TensorFlow threads is now set prior to initialization to avoid errors.
2022-07-05Redirect all Keras progbars to stderrZJaume
Seems that Keras developers won't accept writing progbars to stderr, see [here](https://github.com/keras-team/keras/pull/12019)
2022-07-05Set inter/intra_op parallelism to 0 by defaultZJaume
There's no need of setting it to max cpu, TF will set an optimal value
2022-07-05Change XLMR subclass name to pass HF signature checkZJaume
2022-07-05Metrics rename `update_state`ZJaume
2022-06-16Force no verbosity in predict by defaultZJaume
2022-03-09Add Keras backend alias to custom_objs when loading lite modelsZJaume
2022-03-02Use empty string for user_defined symbols as defaultZJaume
None was being included in the vocab as a piece
2022-03-01Explicit input shape when calibrating with TF 2.8ZJaume
2022-02-17Headers supportCristian García Romero
2022-02-17Update to Hardrules 2.0ZJaume
2021-12-15Encode sentences during batching and unify Generator classZJaume
Huge memory savings due to vectorized and padded arrays don't stay in memory and are processed when needed. Also, speed penalty is negligible because workers process batches in parallel.
2021-11-23Hide Transformers and Tensorflow logging messages on executable scriptsZJaume
2021-11-15Merge branch 'dev'ZJaume
2021-11-10Avoid write metadata file too earlyZJaume
2021-11-09Avoid generating empty sentences in synthetic noiseZJaume
2021-11-08Remove useless logging info messageZJaume
2021-11-05Update HF Transformers, no longer needed single GPU for predictionZJaume
2021-11-05Restore starting capital letter in frequency noiseZJaume
2021-11-05Fix writing metadata when no lm is trainedZJaume
2021-11-04Add save valid and use 'validation' naming instead of 'developmentZJaume
2021-11-04Add MCC as validation metric in XLMR, use default names for metricsZJaume
2021-07-07Load weights instead of full model when loading fails due to bad marshal ↵ZJaume
data (model saved with different Python version)