Remove must-do-error-analysis
article thumbnail

Common Flaws in NLP Evaluation Experiments

Ehud Reiter

We discovered early on in the project that none of the papers we considered replicating had sufficient information for replicability and that only 13% of authors were willing and able to provide the missing information ( paper ) ( blog ). Once researchers may decide this is not worth doing.

NLP 259
article thumbnail

LLMs and Data-to-text

Ehud Reiter

In January I wrote a blog on Can ChatGPT do Data-to-Text? Generally speaking, LLMs do very well on such tasks *if* they are evaluated on a “leaderboard” basis, that is average performance on a test set as measured by automatic or simple human evaluation. which was based on some simple experiments I ran on ChatGPT.

LLM 177
professionals

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

How to achieve Kubernetes observability: Principles and best practices

IBM Journey to AI blog

In this blog, we discuss how Kubernetes observability works, and how organizations can use it to optimize cloud-native IT architectures. Logs Logs include discrete events recorded every time something occurs in the system, such as status or error messages, or transaction details. How does observability work?

article thumbnail

Cyber recovery vs. disaster recovery: What’s the difference? 

IBM Journey to AI blog

While DR can include plans that help deal with cyber threats, it primarily targets a much wider range including natural disasters, human error, massive outages and more. How do cyber recovery and disaster recovery work? Without it, no one will know what to do in the event of a disaster.

article thumbnail

The tsunami of sustainability disclosures facing American multinationals: Is your company prepared?

IBM Journey to AI blog

Globally, there has been an uptick of landmark regulations forcing companies to address sustainability issues like climate change, and to disclose the work they are doing to address these issues. Today, sustainability information may be rife with human error, mostly driven by the complexity of data calculations (e.g.,

ESG 202
article thumbnail

Streamlining supply chain management: Strategies for the future

IBM Journey to AI blog

On top of disruption, companies with global supply chains must also deal with different regulatory environments, cultural norms and market conditions. These technologies can significantly reduce manual labor, minimize errors and speed up processes, leading to increased efficiency and cost savings.

article thumbnail

How to improve your finance operation’s efficiency with generative AI

IBM Journey to AI blog

What is generative AI, what are foundation models, and why do they matter? We must transform from manual processes (that require meticulous analysis, critical thinking and effective communication skills) to AI-powered processes that streamline and improve operational efficiency.