ODSC - Open Data Science

MAY 31, 2023

The combination of data streaming and machine learning (ML) enables you to build one scalable, reliable, but also simple infrastructure for all machine learning tasks using the Apache Kafka ecosystem. I had previously discussed example use cases and architectures that leverage Apache Kafka and machine learning.

Machine Learning 131

Machine Learning Data Ingestion Neural Network Data Science 131

AWS Machine Learning Blog

DECEMBER 19, 2023

Artificial intelligence (AI) and machine learning (ML) are becoming an integral part of systems and processes, enabling decisions in real time, thereby driving top and bottom-line improvements across organizations. However, putting an ML model into production at scale is challenging and requires a set of best practices.

Machine Learning 97

Machine Learning ML Engineer DevOps ML 97

AWS Machine Learning Blog

MAY 16, 2024

For this reason, we built the MLOps architecture to manage the created models and provide real-time services. Solution architecture The following diagram illustrates the solution architecture for serving Neural Collaborative Filtering (NCF) algorithm-based recommendation models as MLOps.

Deep Learning 88

Deep Learning Auto-complete Algorithm ML 88

AWS Machine Learning Blog

APRIL 4, 2024

In this post, we describe how Nielsen Sports modernized a system running thousands of different machine learning (ML) models in production by using Amazon SageMaker multi-model endpoints (MMEs) and reduced operational and financial cost by 75%. It took months to get models with new model architecture to production.

Machine Learning 71

Machine Learning ML Artificial Intelligence Artificial Intelligence 71

IBM Journey to AI blog

AUGUST 4, 2023

Data democratization instead refers to the simplification of all processes related to data, from storage architecture to data management to data security. Architecture for data democratization Data democratization requires a move away from traditional “data at rest” architecture, which is meant for storing static data.

Machine Learning 149

Machine Learning AI AI Automation 149

AWS Machine Learning Blog

NOVEMBER 22, 2023

The AWS Well-Architected Framework provides a systematic way for organizations to learn operational and architectural best practices for designing and operating reliable, secure, efficient, cost-effective, and sustainable workloads in the cloud. This post focuses on the Performance Efficiency pillar of the IDP workload.

IDP 88

IDP ML Machine Learning Automation 88

AWS Machine Learning Blog

MAY 13, 2024

In part 1 of this blog series, we discussed how a large language model (LLM) available on Amazon SageMaker JumpStart can be fine-tuned for the task of radiology report impression generation. Prompt engineering helps to effectively design and improve prompts to get better results on different tasks with LLMs.

Generative AI 99

Generative AI Prompt Engineer Prompt Engineering LLM 99

AWS Machine Learning Blog

FEBRUARY 1, 2024

Data pipelines In cases where you need to provide contextual data to the foundation model using the RAG pattern, you need a data pipeline that can ingest the source data, convert it to embedding vectors, and store the embedding vectors in a vector database. The calling application should invoke the pipeline asynchronously.

Generative AI 105

Generative AI Prompt Engineer Prompt Engineering AI 105

AWS Machine Learning Blog

OCTOBER 2, 2023

This is Part 3 of our series where we design and implement an MLOps pipeline for visual quality inspection at the edge. In this post, we focus on how to automate the edge deployment part of the end-to-end MLOps pipeline. In Part 2 , we showed how to automate the labeling and model training parts of the pipeline.

Automation 84

Automation DevOps ML Machine Learning 84

AWS Machine Learning Blog

FEBRUARY 24, 2023

AWS recently released Amazon SageMaker geospatial capabilities to provide you with satellite imagery and geospatial state-of-the-art machine learning (ML) models, reducing barriers for these types of use cases. For more information, refer to Preview: Use Amazon SageMaker to Build, Train, and Deploy ML Models Using Geospatial Data.

Computer Vision 82

Computer Vision Metadata Machine Learning ML 82

AWS Machine Learning Blog

DECEMBER 6, 2023

However, the popular RAG design pattern with semantic search can’t answer all types of questions that are possible on documents. In this post, we show how to design an intelligent document assistant capable of answering analytical and multi-step reasoning questions in three parts.

Metadata 95

Metadata LLM NLP Conversational AI 95

The MLOps Blog

SEPTEMBER 7, 2023

How to best adapt SOLID principles, how to make use of effective software patterns, how to give the most appropriate names to functions and classes, how to organize code modules, etc. This situation is not different in the ML world. Data Scientists and ML Engineers typically write lots and lots of code.

Software Engineer 52

Software Engineer Machine Learning ETL ML Engineer 52

AWS Machine Learning Blog

AUGUST 31, 2023

As Artificial Intelligence (AI) and Machine Learning (ML) technologies have become mainstream, many enterprises have been successful in building critical business applications powered by ML models at scale in production. They provide a fact sheet of the model that is important for model governance.

ML 85

ML Data Scientist Machine Learning Automation 85

The MLOps Blog

JUNE 27, 2023

Alignment to other tools in the organization’s tech stack Consider how well the MLOps tool integrates with your existing tools and workflows, such as data sources, data engineering platforms, code repositories, CI/CD pipelines, monitoring systems, etc. For example, neptune.ai and Pandas or Apache Spark DataFrames.

Machine Learning 134

Machine Learning Metadata Data Scientist Data Quality 134

Google Research AI blog

MAY 23, 2023

However, with machine learning (ML), we have an opportunity to automate and streamline the code review process, e.g., by proposing code changes based on a comment’s text. As of today, code-change authors at Google address a substantial amount of reviewer comments by applying an ML-suggested edit. 3-way-merge UX in IDE.

ML 140

ML Software Engineer Automation Software Development 140

AWS Machine Learning Blog

MAY 30, 2024

The following diagram illustrates the solution workflow and architecture. The solution adopts microservice design principles, with loosely coupled components that can be deployed together to serve the video analysis and policy evaluation workflow, or independently to integrate into existing pipelines.

Generative AI 106

Generative AI Metadata AI AI 106

AWS Machine Learning Blog

MARCH 1, 2023

Statistical methods and machine learning (ML) methods are actively developed and adopted to maximize the LTV. In this post, we share how Kakao Games and the Amazon Machine Learning Solutions Lab teamed up to build a scalable and reliable LTV prediction solution by using AWS data and ML services such as AWS Glue and Amazon SageMaker.

Automation 84

Automation ETL Data Drift ML 84

AWS Machine Learning Blog

MAY 23, 2023

This is a guest blog post co-written with Vik Pant and Kyle Bassett from PwC. With organizations increasingly investing in machine learning (ML), ML adoption has become an integral part of business transformation strategies. Many organizations start their journey into the world of ML with a model-centric viewpoint.

Machine Learning 71

Machine Learning Neural Network ML Data Scientist 71

AWS Machine Learning Blog

NOVEMBER 22, 2023

By using the Well-Architected Framework and the IDP Well-Architected Custom Lens, you will learn about operational and architectural best practices for designing and operating reliable, secure, efficient, cost-effective, and sustainable workloads in the cloud. This means you’ll need to select your management pattern accordingly.

IDP 83

IDP Data Ingestion Automation Natural Language Processing 83

AWS Machine Learning Blog

MAY 7, 2024

Hugging Face is an open-source machine learning (ML) platform that provides tools and resources for the development of AI projects. They are designed for real-time, interactive, and low-latency workloads and provide auto scaling to manage load fluctuations.

Automation 98

Automation Auto-complete DevOps UX Design 98

AWS Machine Learning Blog

DECEMBER 14, 2023

To ensure the effective operation of this job recommendation engine, it is crucial to implement a large-scale data processing pipeline responsible for extracting and refining features from Talent.com’s aggregated job listings. This can significantly shorten the time needed to deploy the Machine Learning (ML) pipeline to production.

ETL 77

ETL Data Scientist Machine Learning Deep Learning 77

Google Research AI blog

FEBRUARY 2, 2023

For research, it has not only reduced language model latency for users , designed computer architectures , accelerated hardware , assisted protein discovery , and enhanced robotics , but also provided a reliable backend interface for users to search for neural architectures and evolve reinforcement learning algorithms.

Algorithm 95

Algorithm Machine Learning Software Engineer Python 95

AWS Machine Learning Blog

MAY 9, 2024

The rapid advancements in artificial intelligence and machine learning (AI/ML) have made these technologies a transformative force across industries. An effective approach that addresses a wide range of observed issues is the establishment of an AI/ML center of excellence (CoE). What is an AI/ML CoE?

ML 99

ML Generative AI AI AI 99

AWS Machine Learning Blog

NOVEMBER 29, 2023

In this post, we show you how to integrate Amazon SageMaker Clarify LLM evaluation with Amazon SageMaker Pipelines to enable LLM evaluation at scale. Deployment of Amazon SageMaker Pipelines relies on repository interactions and CI/CD pipeline activation.

LLM 78

LLM Data Scientist Algorithm Generative AI 78

AWS Machine Learning Blog

JANUARY 9, 2023

Machine learning (ML) applications are complex to deploy and often require the ability to hyper-scale, and have ultra-low latency requirements and stringent cost budgets. Deploying ML models at scale with optimized cost and compute efficiencies can be a daunting and cumbersome task. Design patterns for building ML applications.

ML 72

ML Auto-complete Auto-classification Deep Learning 72

AWS Machine Learning Blog

JULY 31, 2023

Drug discovery is the research component of this pipeline that generates candidate drugs with the highest likelihood of being effective with the least harm to patients. Precise 3D structures of target proteins can enable drug design with higher specificity and lower likelihood of cross-interactions with other proteins.

Algorithm 75

Algorithm ML Machine Learning Neural Network 75

AWS Machine Learning Blog

APRIL 5, 2024

SageMaker geospatial capabilities make it straightforward for data scientists and machine learning (ML) engineers to build, train, and deploy models using geospatial data. It helps in understanding how proposed infrastructure and land use changes can impact UHI patterns and identifies the key factors influencing UHI.

ML 97

ML Data Scientist Python Machine Learning 97

AWS Machine Learning Blog

NOVEMBER 9, 2023

Building out a machine learning operations (MLOps) platform in the rapidly evolving landscape of artificial intelligence (AI) and machine learning (ML) for organizations is essential for seamlessly bridging the gap between data science experimentation and deployment while meeting the requirements around model performance, security, and compliance.

Data Drift 94

Data Drift Auto-complete ML Automation 94

AWS Machine Learning Blog

AUGUST 31, 2023

As Artificial Intelligence (AI) and Machine Learning (ML) technologies have become mainstream, many enterprises have been successful in building critical business applications powered by ML models at scale in production. They provide a fact sheet of the model that is important for model governance.

ML 52

ML Data Scientist Machine Learning Automation 52

Marktechpost

MAY 10, 2024

No-code is a software design system that implements software without writing a single line of code. In addition to preparing your ML models for action, you can also test them. Use the data dialog to modify your dataset without additional code, then distribute or showcase your ML models across your organization.

AI Tools 97

AI Tools Machine Learning Artificial Intelligence Artificial Intelligence 97

ODSC - Open Data Science

FEBRUARY 17, 2023

ChatGPT: The Google Killer, Distributed Training with PyTorch and Azure ML, and Many Models Batch Training Distributed Training with PyTorch and Azure ML Continue reading to learn the simplest way to do distributed training with PyTorch and Azure ML. Revolution of AI: Will ChatGPT Replace Search Engines in 2023?

ML 52

ML ChatGPT Data Science Machine Learning 52

AWS Machine Learning Blog

MAY 1, 2024

Llama 2 is an auto-regressive language model that uses an optimized transformer architecture and is intended for commercial and research use in English. In this post, we use the Neuron Distributed library to continuously pre-train a Llama 2 model using tensor and pipeline parallelism using SageMaker training jobs.

Auto-complete 84

Auto-complete ML Deep Learning Generative AI 84

TensorFlow

JANUARY 18, 2023

Posted by Chansung Park, Sayak Paul (ML and Cloud GDEs) TensorFlow Extended ( TFX ) is a flexible framework allowing Machine Learning (ML) practitioners to iterate on production-grade ML workflows faster with reliability and resiliency. The pipeline also uses a custom TFX component that is integrated with Hugging Face ?

Data Ingestion 52

Data Ingestion ML DevOps Automation 52

John Snow Labs

SEPTEMBER 11, 2023

This blog article delves into the exciting synergy between the T5 model and Spark NLP, an open-source library built on Apache Spark, which enables seamless integration of cutting-edge NLP capabilities into your projects. If you are not familiar with annotation concept, you can review it and other general concepts in Spark on this page.

NLP 52

NLP Natural Language Processing Python Machine Learning 52

AWS Machine Learning Blog

APRIL 19, 2023

Businesses are increasingly using machine learning (ML) to make near-real-time decisions, such as placing an ad, assigning a driver, recommending a product, or even dynamically pricing products and services. As a result, some enterprises have spent millions of dollars inventing their own proprietary infrastructure for feature management.

ML 75

ML Data Scientist Big Data Machine Learning 75

AWS Machine Learning Blog

SEPTEMBER 28, 2023

In the world of data-driven decision-making, time series forecasting is key in enabling businesses to use historical data patterns to anticipate future outcomes. The following diagram illustrates the architecture of the training pipeline. The following diagram illustrates the inference pipeline.

Machine Learning 84

Machine Learning Automation ML Natural Language Processing 84

IBM Journey to AI blog

OCTOBER 20, 2023

Artificial intelligence platforms enable individuals to create, evaluate, implement and update machine learning (ML) and deep learning models in a more scalable way. This unified experience optimizes the process of developing and deploying ML models by streamlining workflows for increased efficiency.

Machine Learning 182

Machine Learning Automation AI AI 182