AWS Machine Learning Blog

MAY 30, 2023

In this series of posts, we share lessons learned about optimizing costs in Amazon SageMaker. In Part 1 , we showed how to get started using AWS Cost Explorer to identify cost optimization opportunities in SageMaker. SageMaker manages creating the instance and related resources.

ML 77

ML Deep Learning Neural Network Machine Learning 77

AWS Machine Learning Blog

OCTOBER 2, 2023

Solution overview The sample use case used for this series is a visual quality inspection solution that can detect defects on metal tags, which could be deployed as part of a manufacturing process. SageMaker Ground Truth provides out-of-the-box templates for many different labeling task types, including drawing bounding boxes.

Automation 87

Automation DevOps Machine Learning Metadata 87

Flipboard

NOVEMBER 21, 2023

In this post, we bring MATLAB’s machine learning capabilities into Amazon SageMaker , which has several significant benefits: Compute resources : Using the high-performance computing environment offered by SageMaker can speed up machine learning training. We start by training a classifier model on our desktop with MATLAB.

Machine Learning 116

Machine Learning Auto-classification Python Algorithm 116

AWS Machine Learning Blog

APRIL 11, 2023

Amazon SageMaker Studio can help you build, train, debug, deploy, and monitor your models and manage your machine learning (ML) workflows. Amazon SageMaker Pipelines enables you to build a secure, scalable, and flexible MLOps platform within Studio. In our example, we create a SageMaker pipeline running a single processing step.

Data Scientist 71

Data Scientist ML Python ML Engineer 71

AWS Machine Learning Blog

APRIL 24, 2024

The solution uses Amazon SageMaker JumpStart as the core service for model deployment, fine-tuning, and reinforcement learning. In the pipeline, we used Amazon OpenSearch Service for the vector database, and deployed a fine-tuned Mistral-7B-Instruct model on Amazon SageMaker. Answer the score inside of tags.

LLM 103

LLM AI AI Generative AI 103

AWS Machine Learning Blog

FEBRUARY 29, 2024

Amazon SageMaker Studio offers a comprehensive set of capabilities for machine learning (ML) practitioners and data scientists. Deutsche Bahn has been at the forefront in adopting AI, using SageMaker Studio as a key AI platform. AI’s growing influence in large organizations brings crucial challenges in managing AI platforms.

Data Scientist 92

Data Scientist DevOps AI AI 92

AWS Machine Learning Blog

SEPTEMBER 26, 2023

This post shows you how to build and host an ML application with custom containers on Amazon SageMaker. Amazon SageMaker offers built-in algorithms and pre-built SageMaker docker images for model deployment. But, if these don’t fit your needs, you can bring your own containers (BYOC) for hosting on Amazon SageMaker.

ML 96

ML Python Categorization Algorithm 96

AWS Machine Learning Blog

NOVEMBER 14, 2023

In this post, we walk through best practices for managing LoRA fine-tuned models on Amazon SageMaker to address this emerging question. Working with FMs on SageMaker Model Registry In this post, we walk through an end-to-end example of fine-tuning the Llama2 large language model (LLM) using the QLoRA method.

LLM 84

LLM ML Natural Language Processing Machine Learning 84

AWS Machine Learning Blog

DECEMBER 13, 2023

Amazon SageMaker MLOps is a suite of features that includes Amazon SageMaker Projects (CI/CD), Amazon SageMaker Pipelines and Amazon SageMaker Model Registry. SageMaker Pipelines allows for straightforward creation and management of ML workflows, while also offering storage and reuse capabilities for workflow steps.

ML 94

ML Automation Metadata Software Development 94

AWS Machine Learning Blog

NOVEMBER 30, 2023

We are excited to share that Amazon SageMaker HyperPod is now generally available to enable training foundation models with thousands of accelerators up to 40% faster by providing a highly resilient training environment while eliminating the undifferentiated heavy lifting involved in operating large-scale training clusters.

Auto-complete 106

Auto-complete Machine Learning Generative AI Software Engineer 106

AWS Machine Learning Blog

MAY 16, 2024

In this post, we share how LotteON improved their recommendation service using Amazon SageMaker and machine learning operations (MLOps). The main AWS services used are SageMaker, Amazon EMR , AWS CodeBuild , Amazon Simple Storage Service (Amazon S3), Amazon EventBridge , AWS Lambda , and Amazon API Gateway.

Deep Learning 88

Deep Learning Auto-complete Algorithm ML 88

AWS Machine Learning Blog

NOVEMBER 29, 2023

Amazon SageMaker Studio provides a fully managed solution for data scientists to interactively build, train, and deploy machine learning (ML) models. Amazon SageMaker notebook jobs allow data scientists to run their notebooks on demand or on a schedule with a few clicks in SageMaker Studio.

Data Drift 78

Data Drift BERT Data Scientist ML 78

AWS Machine Learning Blog

MAY 30, 2023

In this post, we share lessons learned and walk you through the various ways to analyze your SageMaker usage and identify opportunities for cost optimization. It allows you to filter and group by values such as AWS service, usage type, cost allocation tags, EC2 instance type, and more.

ML 63

ML Machine Learning Computer Vision Categorization 63

AWS Machine Learning Blog

NOVEMBER 30, 2023

Amazon SageMaker is a fully managed service that enables developers and data scientists to quickly and easily build, train, and deploy machine learning (ML) models at scale. SageMaker makes it easy to deploy models into production directly through API calls to the service. SageMaker provides a variety of options to deploy models.

ML 97

ML Auto-complete Python LLM 97

AWS Machine Learning Blog

DECEMBER 12, 2023

This post shows you how you can create a web UI, which we call Chat Studio, to start a conversation and interact with foundation models available in Amazon SageMaker JumpStart such as Llama 2, Stable Diffusion, and other models available on Amazon SageMaker. We cover the following steps: Deploy SageMaker foundation models.

LLM 93

LLM Large Language Models Generative AI Neural Network 93

AWS Machine Learning Blog

FEBRUARY 29, 2024

In this post, we present a framework for automating the creation of a directed acyclic graph (DAG) for Amazon SageMaker Pipelines based on simple configuration files. It then dynamically creates a SageMaker Pipelines DAG based on the steps declared in the configuration files and the interactions and dependencies among steps.

Automation 86

Automation Python Machine Learning ML 86

AWS Machine Learning Blog

AUGUST 21, 2023

Amazon SageMaker Data Wrangler reduces the time it takes to collect and prepare data for machine learning (ML) from weeks to minutes. SageMaker Data Wrangler supports fine-grained data access control with Lake Formation and Amazon Athena connections. The EMR cluster should be created with encryption in transit.

Data Scientist 80

Data Scientist Machine Learning ML Big Data 80

AWS Machine Learning Blog

FEBRUARY 13, 2024

In this post, we discuss how BigBasket used Amazon SageMaker to train their computer vision model for Fast-Moving Consumer Goods (FMCG) product identification, which helped them reduce training time by approximately 50% and save costs by 20%. Use SageMaker and Amazon FSx for Lustre for efficient data augmentation.

Computer Vision 94

Computer Vision Convolutional Neural Networks AI AI 94

AWS Machine Learning Blog

APRIL 4, 2024

In this post, we describe how Nielsen Sports modernized a system running thousands of different machine learning (ML) models in production by using Amazon SageMaker multi-model endpoints (MMEs) and reduced operational and financial cost by 75%. The following figure shows an example of our tagging system.

Machine Learning 71

Machine Learning ML Artificial Intelligence Artificial Intelligence 71

AWS Machine Learning Blog

DECEMBER 14, 2023

Amazon SageMaker Studio offers a broad set of fully managed integrated development environments (IDEs) for machine learning (ML) development, including JupyterLab, Code Editor based on Code-OSS (Visual Studio Code Open Source), and RStudio. Both JupyterLab and Code Editor can be launched using a flexible workspace called Spaces.

Generative AI 92

Generative AI AI Tools ML AI 92

AWS Machine Learning Blog

MARCH 15, 2023

The term legacy code refers to code that was developed to be manually run on a local desktop, and is not built with cloud-ready SDKs such as the AWS SDK for Python (Boto3) or Amazon SageMaker Python SDK. The best practice for migration is to refactor these legacy codes using the Amazon SageMaker API or the SageMaker Python SDK.

Machine Learning 117

Machine Learning Data Scientist ML Python 117

Flipboard

MARCH 22, 2023

In this post, we show how to configure a new OAuth-based authentication feature for using Snowflake in Amazon SageMaker Data Wrangler. Finally, we show how to export the data flow and train a model using SageMaker Autopilot. Access to Amazon SageMaker , an instance of Amazon SageMaker Studio , and a user for Studio.

IDP 100

IDP Data Scientist Categorization Data Quality 100

AWS Machine Learning Blog

APRIL 25, 2023

Amazon SageMaker Model Training helps data scientists run fully managed large-scale training jobs on AWS’s compute infrastructure. SageMaker Training also helps data scientists with advanced tools such as Amazon SageMaker Debugger and Profiler to debug and analyze their large-scale training jobs.

Machine Learning 76

Machine Learning Data Scientist Python ML 76

AWS Machine Learning Blog

JULY 25, 2023

Amazon SageMaker JumpStart is an ML hub offering algorithms, models, and ML solutions. With SageMaker JumpStart, ML practitioners can choose from a growing list of best performing open source FMs. For the complete list, check out Getting started with Amazon SageMaker JumpStart.

Generative AI 76

Generative AI Python AI AI 76

Marktechpost

JANUARY 22, 2024

After human annotation is complete, a machine-learning model automatically examines the tagged pictures to generate the same annotations. Keylabs Keylabs enables users to annotate images with captions, tags, and other information, such as bounding boxes, important points, and semantic segmentation. Makingsense.ai

Computer Vision 85

Computer Vision Machine Learning ML Deep Learning 85

AWS Machine Learning Blog

MAY 30, 2023

In this series of posts, we share lessons learned about optimizing costs in Amazon SageMaker. In Part 1 , we showed how to get started using AWS Cost Explorer to identify cost optimization opportunities in SageMaker. You can also apply additional filters such as account number, EC2 instance type, cost allocation tag, Region, and more.

Auto-complete 66

Auto-complete ML Machine Learning Computer Vision 66

AWS Machine Learning Blog

NOVEMBER 22, 2023

AWS provides tools such as Amazon CloudWatch , tags , and AWS CloudTrail to make it straightforward to accurately identify the cost and usage of workloads, make sure you utilize resources to measure return on investment (ROI), and enable workload owners to optimize their resources and reduce costs.

IDP 77

IDP Auto-classification Machine Learning Auto-complete 77

AWS Machine Learning Blog

AUGUST 29, 2023

In this post, we describe how to create an MLOps workflow for batch inference that automates job scheduling, model monitoring, retraining, and registration, as well as error handling and notification by using Amazon SageMaker , Amazon EventBridge , AWS Lambda , Amazon Simple Notification Service (Amazon SNS), HashiCorp Terraform, and GitLab CI/CD.

Data Scientist 70

Data Scientist Data Quality Python ML 70

AWS Machine Learning Blog

AUGUST 4, 2023

This is the second post in a series discussing the integration of Salesforce Data Cloud and Amazon SageMaker. Use the newly launched capability of the Amazon SageMaker Data Wrangler connector for Salesforce Data Cloud to prepare the data in SageMaker without copying the data from Salesforce Data Cloud.

ML 75

ML Categorization AI AI 75

AWS Machine Learning Blog

MAY 10, 2023

Amazon SageMaker Canvas is a visual interface that enables business analysts to generate accurate ML predictions on their own, without requiring any ML experience or having to write a single line of code. Access to Amazon SageMaker , an instance of Amazon SageMaker Studio , and a user for Studio. Choose Next: Tags.

ML 72

ML Data Analysis Machine Learning Metadata 72

AWS Machine Learning Blog

APRIL 22, 2024

Amazon SageMaker HyperPod is purpose-built to accelerate foundation model (FM) training, removing the undifferentiated heavy lifting involved in managing and optimizing a large training compute cluster. With SageMaker HyperPod, you can train FMs for weeks and months without disruption. Add any optional tags, then choose Next.

Software Engineer 81

Software Engineer Software Development Machine Learning ML 81

AWS Machine Learning Blog

MAY 25, 2023

To understand the overall structure of a RAG-based approach, refer to Question answering using Retrieval Augmented Generation with foundation models in Amazon SageMaker JumpStart. We then demonstrate how to use LangChain for tying everything together: Interfacing with LLMs hosted on Amazon SageMaker.

LLM 74

LLM Data Ingestion Python ML 74

AWS Machine Learning Blog

JULY 3, 2023

In computer vision (CV), adding tags to identify objects of interest or bounding boxes to locate the objects is called labeling. Amazon SageMaker Ground Truth – Alternatively, you can manage your own data labeling workflows and workforce to label data. For a full list of features, refer to Amazon SageMaker Data Labeling Features.

Deep Learning 78

Deep Learning Auto-complete Machine Learning ML 78

AWS Machine Learning Blog

MARCH 10, 2023

Amazon SageMaker Data Wrangler reduces the time it takes to aggregate and prepare data for machine learning (ML) from weeks to minutes in Amazon SageMaker Studio. Solution overview With SageMaker Studio setups, data professionals can quickly identify and connect to existing EMR clusters. Choose Open studio.

ML 73

ML Big Data Machine Learning Data Quality 73

Mlearning.ai

FEBRUARY 6, 2023

Use SageMaker Processing Jobs to easily run inference on your large Dataset with Hugging Face’s Transformer Models Photo by Alex Kulikov on Unsplash This blog will give you a complete walk through of running a distributed batch inference on large data in production. We’ll be using Amazon Sagemaker, a fully managed machine learning service.

ML 52

ML Python Machine Learning Data Scientist 52

AWS Machine Learning Blog

MARCH 15, 2024

In addition to Amazon Bedrock, you can use other AWS services like Amazon SageMaker JumpStart and Amazon Lex to create fully automated and easily adaptable generative AI order processing agents. Furthermore, these libraries are required when calling the Amazon Bedrock API from Amazon SageMaker Studio. awscli>=1.29.57

Generative AI 113

Generative AI Prompt Engineer Prompt Engineering LLM 113

AWS Machine Learning Blog

JANUARY 17, 2023

Amazon SageMaker JumpStart is the Machine Learning (ML) hub of SageMaker providing pre-trained, publicly available models for a wide range of problem types to help you get started with machine learning. Amazon SageMaker JumpStart provides one-click, end-to-end solutions for many common ML use cases. JumpStart solution templates.

Categorization 68

Categorization BERT Machine Learning Neural Network 68