Building Real-time AI Applications with Cognitive Services and Stream Analytics
2 h 15 m
Lab Overview
You are the leader of a group of climate scientists who are concerned about the dwindling polar-bear population in the Arctic. As such, your team has placed hundreds of motion-activated cameras at strategic locations throughout the region. Rather than manually examine each photograph to determine whether it contains a polar bear, you have been challenged to devise an automated system that processes data from these cameras in real time and displays an alert on a map when a polar bear is photographed. You need a solution that incorporates real-time stream processing to analyze raw data for potential sightings, and one that incorporates artificial intelligence (AI) and machine learning to determine with a high degree of accuracy whether a photo contains a polar bear. And you need it fast, because climate change won’t wait.

In this lab, you will build such a system using Microsoft Azure and Microsoft Cognitive Services. Specifically, you will use an Azure IoT hub to ingest streaming data from simulated cameras, Azure Storage to store photographs, Azure Stream Analytics to process real-time data streams, Azure Functions to process output from Stream Analytics, Microsoft’s Custom Vision Service to analyze photographs for polar pears, Microsoft Power BI to build a dashboard for visualizing results, and Azure SQL Database as a data source for Power BI.
In this exercise, you will start the Real Time Lab Environment for the lab. This lab will provision a virtual machine with all of the software necessary to complete the hands-on lab. You will also have access to a Microsoft Azure subscription to complete the tasks. Do not create more resources in Azure then the lab requires or your lab may be terminated.
In this exercise, you will create a storage account and an IoT hub and write a Node.js app that connects them to a simulated camera array. Then you will test one of the virtual cameras by having it upload a photograph to blob storage and transmit an event containing the blob URL and other information to the IoT hub.
Azure Stream Analytics is a cloud-based service for ingesting high-velocity data streaming from devices, sensors, applications, Web sites, and other data sources and analyzing that data in real time. It supports a SQL-like query language that works over dynamic data streams and makes analyzing constantly changing data no more difficult than performing queries on static data stored in traditional databases. With Azure Stream Analytics, you can set up jobs that analyze incoming data for anomalies or information of interest and record the results, present notifications on dashboards, or even fire off alerts to mobile devices. And all of it can be done at low cost and with a minimum of effort.

Scenarios for the application of real-time data analytics are many and include fraud detection, identity-theft protection, optimizing the allocation of resources (think of an Uber-like transportation service that sends drivers to areas of increasing demand before that demand peaks), click-stream analysis on Web sites, and countless others. Having the ability to process data as it comes in rather than waiting until after it has been aggregated offers a competitive advantage to businesses that are agile enough to make adjustments on the fly.

In this lab, the second of four in a series, you will create an Azure Stream Analytics job and connect it to the IoT hub you created in the previous exercise. Then you will write an Azure Function to receive the output, and use the two of them together to analyze data streaming in from a simulated camera array.

Microsoft Cognitive Services is a suite of services and APIs backed by machine learning that enables developers to incorporate intelligent features such as facial recognition in photos and videos, sentiment analysis in text, and language understanding into their applications. Microsoft’s Custom Vision Service is among the newest members of the Cognitive Services suite. Its purpose is to create image-classification models that “learn” from labeled images you provide. Want to know if a photo contains a picture of a flower? Train the Custom Vision Service with a collection of flower images, and it can tell you whether the next image includes a flower — or even what type of flower it is.

The Custom Vision Service exposes two APIs: the Custom Vision Training API and the Custom Vision Prediction API. You can build, train, and test image-classification models using the Custom Vision Service portal, or you can build, train, and test them using the Custom Vision Training API. Once a model is trained, you can use the Custom Vision Prediction API to build apps that utilize it. Both are REST APIs that can be called from a variety of programming languages.

In this exercise, you will create a Custom Vision Service model and train it to differentiate between various types of Arctic wildlife. Then you will connect it to the Stream Analytics job you created in the previous lab.

The goal of business intelligence is being able to quickly identify and respond to ever-changing trends in business and industry. Whether you’re a data analyst delivering reports and analytics to your organization or a stakeholder or decision-maker in need of critical insights, Power BI can organize and unify all of your organization’s data to provide a clear view of the world around you.

Microsoft Power BI was created to address the data explosion in commercial and academic organizations, the need to analyze that data, and the need for rich, interactive visuals to represent the data and reveal key insights. It contains a suite of tools that assist in data analysis, from data discovery and collection to data transformation, aggregation, sharing, and collaboration. Moreover, it allows you to create rich visualizations and package them in interactive dashboards.

In this exercise, you will connect Microsoft Power BI to the Azure SQL database you created in the previous exercise to capture information emanating from the virtual camera array you deployed in the Arctic. Then you will use Power BI to build a report that shows in near real-time where polar bears are being spotted.
Real-Time Lab
Not Registered?
Create Account
Already Registered?
What are Labs?

Labs are where you can get hands on experience from what you have learned from lectures. You get to work in real time in virtual machines at your pace.