From the course: Deep Learning Foundations: Natural Language Processing with TensorFlow
Join today to access over 22,600 courses taught by industry experts or purchase this course individually.
Predicting the next word - TensorFlow Tutorial
From the course: Deep Learning Foundations: Natural Language Processing with TensorFlow
Predicting the next word
- So let's look at how we are going to implement all of these five steps in order to generate new text. So let's quickly import all of the required packages. So we are using the same set of packages same functions and classes. So if you run the cell let's make sure that we're using TensorFlow 2.x. Then the first step is to create a corpus. So here, I've taken this very long string from the book Harry Potter. So this is a text from there. Raw text, long string. And the first step is to tokenize it. So we have to instantiate the tokenizer and then we are required to create a corpus. So corpus is basically a list of sentences. So first of all, let's can word it into lowercase so that's not a problem. And then what we'll do is we will split this entire string on new line characters. So that'll give us sentences split by new lines. So let's give it a run and you see that we have a list of so many sentences now. So this is…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.