AWS Machine Learning Blog

MAY 8, 2023

With kernel auto-tuning, the engine selects the best algorithm for the target GPU, maximizing hardware utilization. Overall, TensorRT’s combination of techniques results in faster inference and lower latency compared to other inference engines. These functions are used during the inference step.

ML 88

ML BERT Deep Learning Auto-complete 88

Towards AI

JULY 19, 2023

github.com Their core repos consist of SparseML: a toolkit that includes APIs, CLIs, scripts and libraries that apply optimization algorithms such as pruning and quantization to any neural network. DeepSparse: a CPU inference engine for sparse models. Follow their code on GitHub. SparseZoo: a model repo for sparse models.

NLP 96

NLP Neural Network Natural Language Processing Computer Vision 96

Towards AI

JULY 21, 2023

github.com Their core repos consist of SparseML: a toolkit that includes APIs, CLIs, scripts and libraries that apply optimization algorithms such as pruning and quantization to any neural network. DeepSparse: a CPU inference engine for sparse models. Follow their code on GitHub. SparseZoo: a model repo for sparse models.

NLP 52

NLP Neural Network Natural Language Processing Computer Vision 52