BAIR

MARCH 11, 2024

These are comments in HTML. The above header text is needed to format the title, authors, etc. The "example_post" is an example representative image (not GIF) that we use for each post for tweeting (see below as well) and for the emails to subscribers. Please provide this image (and any other images and GIFs) in the blog to the BAIR Blog editors directly.

Robotics 100

Robotics Natural Language Processing Machine Learning Computer Vision 100

BAIR

MARCH 21, 2024

These are comments in HTML. The above header text is needed to format the title, authors, etc. The "example_post" is an example representative image (not GIF) that we use for each post for tweeting (see below as well) and for the emails to subscribers. Please provide this image (and any other images and GIFs) in the blog to the BAIR Blog editors directly.

Computer Vision 100

Computer Vision 100

BAIR

MARCH 11, 2024

These are comments in HTML. The above header text is needed to format the title, authors, etc. The "example_post" is an example representative image (not GIF) that we use for each post for tweeting (see below as well) and for the emails to subscribers. Please provide this image (and any other images and GIFs) in the blog to the BAIR Blog editors directly.

Robotics 344

Robotics Natural Language Processing Machine Learning Computer Vision 344

BAIR

FEBRUARY 17, 2024

These are comments in HTML. The above header text is needed to format the title, authors, etc. The "example_post" is an example representative image (not GIF) that we use for each post for tweeting (see below as well) and for the emails to subscribers. Please provide this image (and any other images and GIFs) in the blog to the BAIR Blog editors directly.

LLM 364

LLM Neural Network AI AI 364

BAIR

NOVEMBER 14, 2023

These are comments in HTML. The above header text is needed to format the title, authors, etc. The "example_post" is an example representative image (not GIF) that we use for each post for tweeting (see below as well) and for the emails to subscribers. Please provide this image (and any other images and GIFs) in the blog to the BAIR Blog editors directly.

Large Language Models 239

Large Language Models ChatGPT AI AI 239

BAIR

NOVEMBER 14, 2023

Asymmetric Certified Robustness via Feature-Convex Neural Networks TLDR : We propose the asymmetric certified robustness problem, which requires certified robustness for only one class and reflects real-world adversarial scenarios. This focused setting allows us to introduce feature-convex classifiers, which produce closed-form and deterministic certified radii on the order of milliseconds.

Neural Network 162

Neural Network Machine Learning Deep Learning 162

BAIR

OCTOBER 17, 2023

Goal Representations for Instruction Following page width. --> A longstanding goal of the field of robot learning has been to create generalist agents that can perform tasks for humans. Natural language has the potential to be an easy-to-use interface for humans to specify arbitrary tasks, but it is difficult to train robots to follow language instructions.

Robotics 238

Robotics 238

BAIR

OCTOBER 16, 2023

Rethinking the Role of PPO in RLHF TL;DR : In RLHF, there’s tension between the reward learning phase, which uses human preference in the form of comparisons, and the RL fine-tuning phase, which optimizes a single, non-comparative reward. What if we performed RL in a comparative way? Figure 1: This diagram illustrates the difference between reinforcement learning from absolute feedback and relative feedback.

Algorithm 225

Algorithm Large Language Models LLM OpenAI 225

BAIR

JULY 14, 2023

Training Diffusion Models with Reinforcement Learning replay Diffusion models have recently emerged as the de facto standard for generating complex, high-dimensional outputs. You may know them for their ability to produce stunning AI art and hyper-realistic synthetic images , but they have also found success in other applications such as drug design and continuous control.

Algorithm 239

Algorithm Robotics Neural Network Chatbots 239

BAIR

JULY 9, 2023

Figure 1: stepwise behavior in self-supervised learning. When training common SSL algorithms, we find that the loss descends in a stepwise fashion (top left) and the learned embeddings iteratively increase in dimensionality (bottom left). Direct visualization of embeddings (right; top three PCA directions shown) confirms that embeddings are initially collapsed to a point, which then expands to a 1D manifold, a 2D manifold, and beyond concurrently with steps in the loss.

Neural Network 197

Neural Network Deep Learning Algorithm 197

BAIR

JUNE 29, 2023

Figure 1: CoarsenConf architecture. (II) Equivariant MLPs are applied to learn the mean and log variance of both the posterior and prior distributions. (III) The posterior (training) or prior (inference) is sampled and fed into the Channel Selection module, where an attention layer is used to learn the optimal pathway from CG to FG structure. (IV) Given the FG latent vector and the RDKit approximation, the decoder $p_theta(X |mathcal{R}, z)$ learns to recover the low-energy FG structure through

Deep Learning 264

Deep Learning Algorithm 264

BAIR

MAY 23, 2023

TL;DR : Text Prompt -> LLM -> Intermediate Representation (such as an image layout) -> Stable Diffusion -> Image. Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, despite their impressive capabilities, diffusion models, such as Stable Diffusion , often struggle to accurately follow the prompts when spatial or common sense reasoning is required.

Large Language Models 363

Large Language Models LLM 363

BAIR

APRIL 6, 2023

Figure 1: “Interactive Fleet Learning” (IFL) refers to robot fleets in industry and academia that fall back on human teleoperators when necessary and continually learn from them over time. In the last few years we have seen an exciting development in robotics and artificial intelligence: large fleets of robots have left the lab and entered the real world.

Robotics 246

Robotics Algorithm Continuous Learning Artificial Intelligence 246

BAIR

APRIL 3, 2023

These are comments in HTML. The above header text is needed to format the title, authors, etc. The "example_post" is an example representative image (not GIF) that we use for each post for tweeting (see below as well) and for the emails to subscribers. Please provide this image (and any other images and GIFs) in the blog to the BAIR Blog editors directly.

Large Language Models 363

Large Language Models ChatGPT OpenAI Chatbots 363

BAIR

APRIL 20, 2022

A demonstration of the RvS policy we learn with just supervised learning and a depth-two MLP. It uses no TD learning, advantage reweighting, or Transformers! Offline reinforcement learning (RL) is conventionally approached using value-based methods based on temporal difference (TD) learning. However, many recent algorithms reframe RL as a supervised learning problem.

Algorithm 130

Algorithm Robotics Automation 130

BAIR

APRIL 25, 2022

Figure 1: Summary of our recommendations for when a practitioner should BC and various imitation learning style methods, and when they should use offline RL approaches. Offline reinforcement learning allows learning policies from previously collected data, which has profound implications for applying RL in domains where running trial-and-error learning is impractical or dangerous, such as safety-critical settings like autonomous driving or medical treatment planning.

Robotics 130

Robotics Algorithm Explainability 130

BAIR

APRIL 29, 2022

Deep reinforcement learning (DRL) is transitioning from a research field focused on game playing to a technology with real-world applications. Notable examples include DeepMind’s work on controlling a nuclear reactor or on improving Youtube video compression , or Tesla attempting to use a method inspired by MuZero for autonomous vehicle behavior planning.

Robotics 147

Robotics Machine Learning Computer Vision ML 147

BAIR

MAY 3, 2022

Figure 1: In real-world applications, we think there exist a human-machine loop where humans and machines are mutually augmenting each other. We call it Artificial Augmented Intelligence. How do we build and evaluate an AI system for real-world applications? In most AI research, the evaluation of AI methods involves a training-validation-testing process.

AI Modeling 147

AI Modeling Automation AI Developer AI Development 147

BAIR

MAY 20, 2022

We recently published the Berkeley Crossword Solver (BCS), the current state of the art for solving American-style crossword puzzles. The BCS combines neural question answering and probabilistic inference to achieve near-perfect performance on most American-style crossword puzzles, like the one shown below: Figure 1: Example American-style crossword puzzle An earlier version of the BCS, in conjunction with Dr.Fill, was the first computer program to outscore all human competitors in the world’s t

147

147

BAIR

JUNE 30, 2022

FIGS (Fast Interpretable Greedy-tree Sums): A method for building interpretable models by simultaneously growing an ensemble of decision trees in competition with one another. Recent machine-learning advances have led to increasingly complex predictive models, often at the cost of interpretability. We often need interpretability, particularly in high-stakes applications such as in clinical decision-making; interpretable models help with all kinds of things, such as identifying errors, leveraging

Machine Learning 145

Machine Learning Algorithm 145

BAIR

JULY 10, 2022

In cooperative multi-agent reinforcement learning (MARL), due to its on-policy nature, policy gradient (PG) methods are typically believed to be less sample efficient than value decomposition (VD) methods, which are off-policy. However, some recent empirical studies demonstrate that with proper input representation and hyper-parameter tuning, multi-agent PG can achieve surprisingly strong performance compared to off-policy VD methods.

Algorithm 147

Algorithm 147

BAIR

AUGUST 29, 2022

Deep neural networks have enabled technological wonders ranging from voice recognition to machine transition to protein engineering, but their design and application is nonetheless notoriously unprincipled. The development of tools and methods to guide this process is one of the grand challenges of deep learning theory. In Reverse Engineering the Neural Tangent Kernel , we propose a paradigm for bringing some principle to the art of architecture design using recent theoretical breakthroughs: fir

Neural Network 147

Neural Network Deep Learning 147

BAIR

SEPTEMBER 19, 2022

To regulate the distribution shift experience by learning-based controllers, we seek a mechanism for constraining the agent to regions of high data density throughout its trajectory (left). Here, we present an approach which achieves this goal by combining features of density models (middle) and Lyapunov functions (right). In order to make use of machine learning and reinforcement learning in controlling real world systems, we must design algorithms which not only achieve good performance, but a

Machine Learning 113

Machine Learning Neural Network Algorithm Robotics 113

BAIR

JANUARY 20, 2023

Reinforcement learning provides a conceptual framework for autonomous agents to learn from experience, analogously to how one might train a pet with treats. But practical applications of reinforcement learning are often far from natural: instead of using RL to learn through trial and error by actually attempting the desired task, typical RL applications use a separate (usually simulated) training phase.

Robotics 275

Robotics Continuous Learning Algorithm 275