From the course: Apache Spark Deep Learning Essential Training

Unlock the full course today

Join today to access over 22,500 courses taught by industry experts or purchase this course individually.

Using pretrained models

Using pretrained models - Spark DataFrames Tutorial

From the course: Apache Spark Deep Learning Essential Training

Start my 1-month free trial

Using pretrained models

“

- [Instructor] Deep learning pipelines support running pretrained models in a distributed manner with Spark. You can do this with both batch and streaming data processing. The ImageNet data set has 1,000 different categories of objects. The different categories are here in the synset file. I open this with Wordpad. Now each of these rows corresponds to one of the categories or classes in the ImageNet data set. The first part is the ID and the second part is one or more words to describe the class. We can write code using well-known models that have won competitions classifying the 1,000 different objects. These models are ResNet 50 and Inception version three amongst others. You can then input any image that belongs to one of these 1,000 categories and the deep image predictor method will predict which of these objects have been loaded. This prediction, of course, is done in parallel with all the benefits that come with Spark. Let's head over to the databricks notebook. I'll select…

Contents