Sebastian Ruder

FEBRUARY 23, 2023

This post gives a brief overview of modularity in deep learning. For modular fine-tuning for NLP, check out our EMNLP 2022 tutorial. Fuelled by scaling laws, state-of-the-art models in machine learning have been growing larger and larger. We give an in-depth overview of modularity in our survey on Modular Deep Learning.

Deep Learning 52

Deep Learning NLP Computer Vision Metadata 52

NLP People

JUNE 5, 2018

Are you looking to study or work in the field of NLP? For this series, NLP People will be taking a closer look at the NLP education & development landscape in different parts of the world, including the best sites for job-seekers and where you can go for the leading NLP-related education programs on offer.

NLP 52

NLP Data Mining Deep Learning Natural Language Processing 52

Sebastian Ruder

JANUARY 24, 2022

2021) 2021 saw many exciting advances in machine learning (ML) and natural language processing (NLP). In computer vision, supervised pre-trained models such as Vision Transformer [2] have been scaled up [3] and self-supervised pre-trained models have started to match their performance [4]. Why is it important?  

NLP 52

NLP ML BERT Computational Linguistics 52

Topbots

JUNE 15, 2023

Language Disparity in Natural Language Processing This digital divide in natural language processing (NLP) is an active area of research. 70% of research papers published in a computational linguistics conference only evaluated English.[ Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifold.

Natural Language Processing 59

Natural Language Processing Computational Linguistics NLP ChatGPT 59

Sebastian Ruder

NOVEMBER 14, 2022

At the same time, a wave of NLP startups has started to put this technology to practical use. I will be focusing on topics related to natural language processing (NLP) and African languages as these are the domains I am most familiar with. This post is partially based on a keynote I gave at the Deep Learning Indaba 2022.

Natural Language Processing 52

Natural Language Processing NLP Computational Linguistics AI 52

Topbots

JANUARY 30, 2023

Making the transition from classical language generation to recognising and responding to specific communicative intents is an important step to achieve better acceptance of user-facing NLP systems, especially in Conversational AI. Association for Computational Linguistics. [2] Association for Computational Linguistics. [4]

Large Language Models 52

Large Language Models LLM Computational Linguistics Natural Language Processing 52

The Stanford AI Lab Blog

JANUARY 21, 2022

Deep learning has enabled improvements in the capabilities of robots on a range of problems such as grasping 1 and locomotion 2 in recent years. Indeed, this recipe of massive, diverse datasets combined with scalable offline learning algorithms (e.g. Deep contextualized word representations. Neumann, M., Gardner, M.,

Robotics 40

Robotics Computational Linguistics Computer Vision BERT 40