Sebastian Ruder

JANUARY 6, 2020

This post gathers ten ML and NLP research directions that I found exciting and impactful in 2019. 2019 ) and other variants. 2019 ), MoCo ( He et al., 2019 ), MoCo ( He et al., In speech, representations learned with a multi-layer CNN ( Schneider et al., 2019 ) or bidirectional CPC ( Kawakami et al.,

NLP 52

NLP ML Neural Network BERT 52

NLP People

JANUARY 2, 2019

So, what’s new in the world of machine translation and what can we expect in 2019? Nonetheless, it was interesting to see Google AI present a systematic comparison of BPE and character-based ways of handling out-of-vocabulary words from deep learning perspective. The post was initially published at the TAUS Blog.

BERT 52

BERT Natural Language Processing Computational Linguistics NLP 52

The Stanford AI Lab Blog

FEBRUARY 24, 2022

This series of blog posts will discuss our theoretical work which seeks to analyze recent empirical methods which use unlabeled data. In this first post, we’ll analyze self-training , which is a very impactful algorithmic paradigm for semi-supervised learning and domain adaptation. 2019 , Berthelot et al., 2019 , Xie et al.,

Deep Learning 40

Deep Learning Algorithm Explainability 40

AWS Machine Learning Blog

FEBRUARY 29, 2024

Large-scale deep learning has recently produced revolutionary advances in a vast array of fields. is a startup dedicated to the mission of democratizing artificial intelligence technologies through algorithmic and software innovations that fundamentally change the economics of deep learning. Founded in 2021, ThirdAI Corp.

Neural Network 96

Neural Network Deep Learning Machine Learning Algorithm 96

Dlabs.ai

MARCH 17, 2021

And her research expertise spans AI, machine learning , deep learning , computer vision , and cognitive neuroscience. To find out more about deep learning, check this Dlabs.AI He’s a Principal Scientist at Google DeepMind and Team Lead of the Deep Learning group.

Deep Learning 92

Deep Learning Artificial Intelligence Artificial Intelligence Neural Network 92

Flipboard

JULY 10, 2023

Figure 5: Architecture of Convolutional Autoencoder for Image Segmentation (source: Bandyopadhyay, “Autoencoders in Deep Learning: Tutorial & Use Cases [2023],” V7Labs , 2023 ). VAEs can generate new samples from the learned latent distribution, making them ideal for image generation and style transfer tasks.

Computer Vision 110

Computer Vision Deep Learning Neural Network Machine Learning 110

Heartbeat

JUNE 14, 2023

Label Smoothing Equation [5] In their 2019 paper “ When does label smoothing help? ”, Hinton et al. [5] logits of the final layer), which helps reduce overconfidence and subsequently reduces the network’s ECE. ” Advances in neural information processing systems 32 (2019). [6] Measuring Calibration in Deep Learning.

Neural Network 59

Neural Network Computer Vision Deep Learning Explainability 59

NVIDIA

DECEMBER 4, 2023

Indeed, NVIDIA GPUs have won every round of MLPerf training and inference tests since the benchmark was released in 2019. A RedHat software engineering team put it succinctly in a blog : “GPUs have become the foundation of artificial intelligence.” The NVIDIA AI platform includes hundreds of software libraries and apps.

Neural Network 135

Neural Network AI AI LLM 135

Viso.ai

DECEMBER 21, 2023

This blog will introduce you to the computer vision visionaries behind these achievements. In recognition by LinkedIn, Kozyrkov received the #1 Top Voice in Data Science and Analytics in 2019. His contributions to ML, deep learning , computer vision, and NLP underscore his influence in the rapidly evolving AI landscape.

Convolutional Neural Networks 52

Convolutional Neural Networks Neural Network Computer Vision Deep Learning 52

Heartbeat

DECEMBER 8, 2023

He focused on generative AI trained on large language models, The strength of the deep learning era of artificial intelligence has lead to something of a renaissance in corporate R&D in information technology, according to Yann LeCun, chief AI. Hinton is viewed as a leading figure in the deep learning community.

Convolutional Neural Networks 52

Convolutional Neural Networks Neural Network Deep Learning LLM 52

AWS Machine Learning Blog

APRIL 19, 2023

The DJL is a deep learning framework built from the ground up to support users of Java and JVM languages like Scala, Kotlin, and Clojure. With the DJL, integrating this deep learning is simple. When we did our research online, the Deep Java Library showed up on the top. The architecture of DJL is engine agnostic.

ML 72

ML Deep Learning Python Auto-classification 72

AWS Machine Learning Blog

OCTOBER 5, 2023

He focuses on developing scalable machine learning algorithms. His research interests are in the area of natural language processing, explainable deep learning on tabular data, and robust analysis of non-parametric space-time clustering. an AI start-up, and worked as the CEO and Chief Scientist in 2019–2021.

Machine Learning 91

Machine Learning Big Data Deep Learning Natural Language Processing 91

AWS Machine Learning Blog

DECEMBER 12, 2023

In this post, we’ll summarize training procedure of GPT NeoX on AWS Trainium , a purpose-built machine learning (ML) accelerator optimized for deep learning training. In this post, we showed cost-efficient training of LLMs on AWS deep learning hardware. Ben Snyder is an applied scientist with AWS Deep Learning.

Deep Learning 91

Deep Learning Large Language Models Machine Learning Big Data 91

Heartbeat

DECEMBER 14, 2023

LeCun received the 2018 Turing Award (often referred to as the "Nobel Prize of Computing"), together with Yoshua Bengio and Geoffrey Hinton, for their work on deep learning. Hinton is viewed as a leading figure in the deep learning community. > Finished chain. ") > Entering new AgentExecutor chain.

Convolutional Neural Networks 52

Convolutional Neural Networks Neural Network Deep Learning Computer Scientist 52

Pickl AI

APRIL 8, 2024

This blog explores 13 major AI blunders, highlighting issues like algorithmic bias, lack of transparency, and job displacement. This blog delves into 13 of the biggest recent AI failure examples that have occurred in recent years, highlighting the challenges and ethical considerations that accompany this rapidly evolving technology.

Artificial Intelligence 52

Artificial Intelligence Artificial Intelligence Algorithm AI 52

Towards AI

JULY 20, 2023

in ICLR 2019, that leverage DQN to solve web navigation problems. For better readability, you may want… Read the full blog for free on Medium. Specifically, we discuss an algorithm, named QWeb proposed by Gur et al. Join thousands of data leaders on the AI newsletter.

Algorithm 40

Algorithm OpenAI AI AI 40

Heartbeat

OCTOBER 10, 2023

It was developed by Facebook AI Research and released in 2019. Interpretability: Like many other deep learning models, RoBERTa is frequently referred to as a “black box.” Overfitting: RoBERTa, like any deep learning model, is prone to overfitting. It is a state-of-the-art model for a variety of NLP tasks.

BERT 52

BERT NLP Machine Learning Natural Language Processing 52

Defined.ai blog

MARCH 8, 2023

For readers wondering what a “parameter” refers to, here’s an illustrative, if overly simplified example: if a deep learning algorithm were a cake recipe—e.g. BioMedLM’s foundational model was 2019’s GPT-2 , which—while still powerful—is less capable than the current generation of OpenAI’s GPT-3 and the forthcoming GPT-4.

Generative AI 52

Generative AI Large Language Models LLM AI 52

Unite.AI

AUGUST 8, 2023

These advanced AI deep learning models have seamlessly integrated into various applications, from Google's search engine enhancements with BERT to GitHub’s Copilot, which harnesses the capability of Large Language Models (LLMs) to convert simple code snippets into fully functional source codes. How Are LLMs Used?

Generative AI 243

Generative AI ChatGPT Neural Network Convolutional Neural Networks 243

Heartbeat

MARCH 15, 2023

An open-source machine learning model called BERT was developed by Google in 2018 for NLP, but this model had some limitations, and due to this, a modified BERT model called RoBERTa (Robustly Optimized BERT Pre-Training Approach) was developed by the team at Facebook in the year 2019. What is RoBERTa?

BERT 52

BERT NLP Deep Learning Neural Network 52

Heartbeat

JANUARY 23, 2023

2019) set out to determine how well an NLP model could determine whether or not a given sentence adhered to linguistic norms. Editor’s Note: Heartbeat is a contributor-driven online publication and community dedicated to providing premier educational resources for data science, machine learning, and deep learning practitioners.

Natural Language Processing 52

Natural Language Processing NLP Deep Learning Machine Learning 52

Flipboard

JULY 31, 2023

As such, the quality, diversity, and volume of data you feed into your machine learning model can significantly impact the model’s ability to make accurate predictions. Object detection works by using machine learning or deep learning models that learn from many examples of images with objects and their labels.

Machine Learning 104

Machine Learning Computer Vision Deep Learning Convolutional Neural Networks 104

Towards AI

JULY 21, 2023

declassified Blast from the past: Check out this old (2017) blog post from Google introducing transformer models. It leverages an interface across tasks that are grounded on a single knowledge source: the 2019/08/01 Wikipedia snapshot containing 5.9M But to what extent? You may be surprised.” It’s fairly straightforward to execute.

NLP 86

NLP Natural Language Processing Machine Learning ML 86

Google Research AI blog

FEBRUARY 23, 2023

Commensurate with our mission to demonstrate these societal benefits , Google Research’s programs in applied machine learning (ML) have helped place Alphabet among the top five most impactful corporate research institutions in the health and life sciences publications on the Nature Impact Index in every year from 2019 through 2022.

Convolutional Neural Networks 104

Convolutional Neural Networks ML Neural Network Deep Learning 104

Heartbeat

SEPTEMBER 13, 2023

For example, explainability is crucial if a healthcare professional uses a deep learning model for medical diagnoses. Captum allows users to explain both deep learning and traditional machine learning models. Why We Will Never Open Deep Learning's Black Box || Towards Data Science Brent, M.,

Explainability 52

Explainability Machine Learning Deep Learning Explainable AI 52

Mlearning.ai

FEBRUARY 19, 2023

It combines techniques from computational linguistics, probabilistic modeling, deep learning to make computers intelligent enough to grasp the context and the intent of the language. GPT-3 is a successor to the earlier GPT-2 (released in Feb 2019) and GPT-1 (released in June 2018) models . Hope you like it!

Auto-classification 52

Auto-classification Natural Language Processing Auto-complete NLP 52

AWS Machine Learning Blog

APRIL 6, 2023

For example, in the 2019 WAPE value, we trained our model using sales data between 2011–2018 and predicted sales values for the next 12 months (2019 sale). We evaluated the WAPE for all BLs in the auto end market for 2019, 2020, and 2021. In 2019 and 2020, our model achieved less than 0.1

Machine Learning 75

Machine Learning ML Categorization Data Scientist 75

AWS Machine Learning Blog

APRIL 29, 2024

Netflix open sourced the framework in 2019 with integrations to AWS services like AWS Batch , AWS Step Functions (see Unbundling Data Science Workflows with Metaflow and AWS Step Functions ), Kubernetes , and throughput-optimized Amazon Simple Storage Service (Amazon S3), so you can build your own Netflix-scale ML/AI environment in your AWS account.

ML 93

ML Python Data Scientist Deep Learning 93

AssemblyAI

AUGUST 22, 2023

That is, training a language model like GPT-1 in this self-supervised way allows the model to learn general abstractions , and these abstractions allow the model to become better at learning downstream tasks. The same pretrained model can be finetuned for different downstream tasks One year later in 2019, the GPT-2 paper was released.

LLM 242

LLM ChatGPT AI Modeling Chatbots 242

Mlearning.ai

APRIL 11, 2023

Overall , Machine learning’s main objective is to find a tradeoff between the model’s ability to fit the training data and its ability to generalize to new data. In this blog we will talk a bit about the bias-variance tradeoff and drop on double descent phenomenon. This is the so-called bias-variance tradeoff.

Neural Network 52

Neural Network Machine Learning Deep Learning Algorithm 52

Heartbeat

JUNE 6, 2023

Prerequisites To follow along with this tutorial, you will need the following: Basic knowledge of Python and deep learning. We will construct a graph based on the citation links between the papers and use GCNs to classify the papers. Some familiarity with PyTorch and Comet, as these are the tools we will use to implement the GCN.

NLP 59

NLP Convolutional Neural Networks Neural Network Natural Language Processing 59

Heartbeat

MAY 31, 2023

Deep learning-based compression techniques have emerged as a viable alternative to traditional methods. One of these methods is the Variational Autoencoder (VAE) , a generative model that learns a lower-dimensional latent space representation of the image. PyTorch and Comet ML: We will use these to implement the VAE.

Deep Learning 52

Deep Learning ML Machine Learning Neural Network 52

AWS Machine Learning Blog

MAY 10, 2023

In 2019, we made a significant leap forward with the launch of SageMaker Studio, an IDE for ML built on top of JupyterLab that enables you to build, train, tune, debug, deploy, and monitor models from a single application. The distribution is versioned using SemVer and will be released on a regular basis moving forward.

ML 84

ML Generative AI Data Scientist Data Science 84

Heartbeat

DECEMBER 7, 2023

I decided to write a series of blogs on current topics: the elements of data governance that I have been thinking about, reading, and following for a while. Data mesh as a concept was first introduced by Zhamak Dehghani in 2019 and is shaped by four fundamental principles. Even though the titles are new, the ideas are not really new.

Metadata 52

Metadata Data Platform Deep Learning Data Quality 52

NLP People

JULY 24, 2019

The 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019) is starting this week in Florence, Italy. This is especially relevant for the advanced, complex algorithms of the Deep Learning family. Neural Networks are the workhorse of Deep Learning (cf.

NLP 52

NLP Neural Network Deep Learning Natural Language Processing 52

Heartbeat

DECEMBER 28, 2023

Towards Federated Learning at Scale: System Design. Advances and Open Problems in Federated Learning. References: Bonawitz, K., arXiv preprint arXiv:1902.01046. Satyanarayanan, M. The Emergence of Edge Computing. Computer Magazine, 50 (1), 30–39. Kairouz, P., arXiv preprint arXiv:1912.04977. Sandler, M.,

AI 52

AI AI Artificial Intelligence Artificial Intelligence 52

Mlearning.ai

FEBRUARY 6, 2023

2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics. [7] Attention is not not Explanation (2019). The 2019 Conference on Empirical Methods in Natural Language Processing. [8] 2022) Shtetl-Optimized: The Blog of Scott Aaronson. [11] Attention is not Explanation. Serrano, N.

Explainable AI 52

Explainable AI Explainability ChatGPT Chatbots 52