AI/ML and analytics services: what SageMaker AI, Lex and the other AI services do, and Athena, Kinesis, Glue and Quick Sight

Why this matters

Almost every organisation now asks two questions of its data: what can a machine learn from it, and what is it telling us right now? AWS answers both with a long list of services whose names rarely say what they do. "Polly" and "Textract" give no hint that one reads text aloud and the other reads scanned forms. A manager approving a project, or a salesperson describing what a customer could build, needs to match each name to its job without hesitation.

The good news is that this area rewards a simple mental map more than deep technical knowledge. Nobody at this level is expected to build a model or write an analytics pipeline. What matters is recognising which service does which task, and spotting the near-miss: the service that sounds right but works in the opposite direction, or on the wrong kind of data.

This lesson builds that map in two halves: first the artificial intelligence and machine learning services, then the analytics services that collect, prepare, query and present data.

The lesson

AWS AI/ML services

Some terms first. Artificial intelligence (AI) is the broad goal of computers performing tasks that normally need human judgement, such as understanding speech or recognising a face. Machine learning (ML) is the main way of getting there: instead of writing rules by hand, a program learns patterns from example data. The result of that learning is a model, which can then make predictions about new data. Generative AI is a kind of ML that produces new content, such as text, code or images, in response to a request.

AWS offers these capabilities at two levels, and telling them apart is the single most useful idea in this half of the lesson.

• The ML platform. Amazon SageMaker AI is for teams who build, train and deploy their own models, with their own data, algorithms and frameworks. It is aimed at teams with ML expertise and their own training data.
• Pre-trained AI services. Each of these does one job with a model AWS has already trained. A developer sends data through an API (a defined way for one program to call another) and gets a result back. No ML expertise or model training is needed.

On top of these sits Amazon Q, a ready-made generative AI assistant, covered below.

Exam framing: if the scenario says the company has no ML expertise, or wants to add a capability quickly without training anything, the answer is a pre-trained AI service. If it says data scientists want to choose their own algorithms and train custom models, the answer is SageMaker AI.

AWS analytics services

Analytics means turning raw data into answers. It helps to picture a pipeline with four stages, because each analytics service belongs mainly to one of them.

1. Collect. Data arrives, sometimes in batches and sometimes as a continuous stream. Amazon Kinesis handles streams.
2. Store and prepare. Data is usually kept in Amazon S3, often as a data lake: one central store of raw data in many formats. AWS Glue discovers its structure and transforms it. Amazon EMR processes very large data sets with open-source frameworks.
3. Query and analyse. Amazon Athena runs SQL on files where they sit. Amazon Redshift is a data warehouse. Amazon OpenSearch Service searches and analyses text and logs.
4. Visualise. Amazon Quick Sight turns results into dashboards that people can read.

SQL (Structured Query Language) is the standard language for asking questions of tabular data, and it appears throughout this half of the lesson.

What each AI/ML service does: SageMaker AI, Lex and the rest

Amazon SageMaker AI is a fully managed service for the whole ML life cycle: preparing data, building a model in notebooks, training it on managed compute, and deploying it to make predictions. AWS manages the infrastructure; the customer brings the expertise, the data and the choice of algorithm and framework.

The pre-trained AI services are easiest to learn in pairs, because each pair holds a common trap.

Speech in both directions

• Amazon Transcribe converts speech into text (automatic speech recognition). Recorded calls, meetings and video subtitles are typical uses.
• Amazon Polly converts text into lifelike speech, in a choice of voices and languages. Reading articles aloud or speaking replies in an app are typical uses.

A voice interface that listens and replies needs both. Confusing their directions is the most common mistake in this area.

Conversation and translation

• Amazon Lex builds conversational interfaces, or chatbots, that work with voice and text. It works out what the user intends, such as "check my order", and carries out the request. Lex is a building kit: the team designs what the bot can do. It is also used inside Amazon Connect for voice menus, as lesson 21 shows.
• Amazon Translate converts text from one language to another using neural machine translation.

Understanding text

• Amazon Comprehend is natural language processing (NLP): it reads text and finds its meaning. It detects sentiment (positive, negative, neutral or mixed), key phrases, entities such as people, places and organisations, and the language a text is written in. It does not translate; it analyses.

Images and documents

• Amazon Rekognition analyses images and video. It detects objects, scenes, faces, text in images and unsafe or inappropriate content.
• Amazon Textract extracts printed and handwritten text from scanned documents, and also understands their structure: form fields as key-value pairs, and tables. Rekognition can spot words in a photo, but only Textract interprets a form.

Generative AI assistance

• Amazon Q is the AWS generative AI assistant. For business users, Amazon Q Business answers questions in plain language using the company's own connected documents and systems. For developers, Amazon Q Developer works inside popular IDEs (integrated development environments, the programs developers write code in) and the AWS console to suggest and explain code and to help upgrade and transform applications.

The difference between Lex and Q matters. Lex is a toolkit for a team to build its own bot, intent by intent. Q is a finished assistant that a company connects to its data and uses.

The analytics services: Athena, Kinesis, Glue, Quick Sight and others

Amazon Athena is a serverless, interactive query service. Analysts write standard SQL against data stored in Amazon S3, and Athena reads the files in place: nothing is loaded into a database, and there are no servers to manage. It is charged by the amount of data each query scans, so storing data in compressed, columnar formats lowers the cost. Athena is the answer when logs or exports already sit in S3 and someone wants to query them now.

Amazon Kinesis collects, processes and analyses streaming data in real time. A stream is a continuous flow of small records, such as clicks, game events, sensor readings or application logs. Kinesis Data Streams captures that flow so applications can act on it within seconds, for example to update a leaderboard or flag a suspicious payment. Kinesis works on data in motion; Athena works on data at rest.

AWS Glue is a serverless data integration service. Its crawlers examine data sources and discover their structure, called the schema. The AWS Glue Data Catalog records those schemas centrally, so other services know what the data looks like. Glue jobs then run ETL (extract, transform and load): pulling data out of sources, cleaning and reshaping it, and loading it where it will be analysed. Glue and Athena are often used together: Glue catalogues files in S3, and Athena uses that catalogue to query them.

Amazon Quick Sight is the business intelligence (BI) service. It builds interactive dashboards and visualisations that users explore and share in a web browser, with no code. Other services supply the data; Quick Sight presents it to managers.

Three more analytics services are in scope:

• Amazon Redshift is the AWS data warehouse: a store built for complex analytical SQL queries across very large volumes of structured, historical data. It is the answer for company-wide reporting over petabytes of sales history. Contrast it with the databases in lesson 16, such as Amazon RDS and Amazon DynamoDB, which handle the day-to-day transactions of running applications.
• Amazon EMR runs open-source big data frameworks, including Apache Hadoop and Apache Spark, on managed clusters whose size and instance types the customer chooses. "Hadoop" or "Spark" in a scenario points to EMR.
• Amazon OpenSearch Service is a managed service for full-text search, such as searching a product catalogue, and for analysing logs and operational data in near real time.

What the exam is really asking

Most questions here describe a task and offer four services with similar-sounding names. Match the verb and the data type:

• Build and train your own model: SageMaker AI. Use a ready-made capability: a pre-trained AI service.
• Speech to text: Transcribe. Text to speech: Polly. Chatbot: Lex. Language to language: Translate.
• Meaning of text: Comprehend. Pictures and video: Rekognition. Forms and scanned documents: Textract.
• A ready-made assistant over company data, or coding help in an IDE: Amazon Q.
• SQL on files in S3, serverless: Athena. Live streams: Kinesis. Catalogue and ETL: Glue. Dashboards: Quick Sight.
• Data warehouse: Redshift. Hadoop or Spark: EMR. Search and log analytics: OpenSearch Service.

When two answers both seem plausible, ask which one actually performs the step the question describes. Kinesis can carry logs, but OpenSearch Service searches them. Athena can feed a dashboard, but Quick Sight draws it.