Remove high-level-hopes-for-ai-alignment
article thumbnail

Survey of 2,778 AI authors: six parts in pictures

AI Impacts

The 2023 Expert Survey on Progress in AI is out , this time with 2778 participants from six top AI venues (up from about 700 and two in the 2022 ESPAI ), making it probably the biggest ever survey of AI researchers. What makes AI progress go? Here is the preprint. Here is the preprint. Has it sped up?

article thumbnail

High-level hopes for AI alignment

Cold Takes

In previous pieces, I argued that there's a real and large risk of AI systems' aiming to defeat all of humanity combined - and succeeding. I then argued that countermeasures could be challenging, due to some key difficulties of AI safety research. Perhaps the eight-year-old is a mind-reader, or even a young Professor X.)

AI 84
professionals

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Training Diffusion Models with Reinforcement Learning

BAIR

Training Diffusion Models with Reinforcement Learning replay Diffusion models have recently emerged as the de facto standard for generating complex, high-dimensional outputs. To do this, we finetune Stable Diffusion on a variety of objectives, including image compressibility, human-perceived aesthetic quality, and prompt-image alignment.

Algorithm 239
article thumbnail

How RLHF Preference Model Tuning Works (And How Things May Go Wrong)

AssemblyAI

The exploding popularity of conversational AI tools has also raised serious concerns about AI safety. Much of current AI research aims to design LLMs that seek helpful, truthful, and harmless behavior. Despite this, RLHF remains the industry's go-to solution for achieving alignment in LLMs.

LLM 238
article thumbnail

Charles Fisher, Ph.D., CEO & Founder of Unlearn – Interview Series

Unite.AI

is the CEO and Founder of Unlearn , a platform harnessing AI to tackle some of the biggest bottlenecks in clinical development: long trial timelines, high costs, and uncertain outcomes. Their novel AI models analyze vast quantities of patient-level data to forecast patients’ health outcomes. Charles Fisher, Ph.D.,

article thumbnail

Microsoft Researchers Propose A Novel Text Diffusion Model (TREC) that Mitigates the Degradation with Reinforced Conditioning and the Misalignment by Time-Aware Variance Scaling

Marktechpost

This challenge was particularly pronounced in applications requiring high versatility, such as dynamic content creation for websites or personalized dialogue systems, where the context and style could shift rapidly. TR E C employs Time-Aware Variance Scaling, an innovative approach to align the training and sampling processes more closely.

article thumbnail

Pioneering AI Supervision and Reliability: Insights from David Rein and Julian Michael’s Research

NYU Center for Data Science

Tackling the complex and evolving challenges of AI supervision, CDS Junior Research Scientist David Rein and CDS Research Scientist Julian Michael , both members of the NYU Alignment Research Group , are pioneering new methodologies. AI systems are advancing really fast.

AI 81