Dataproc can help you quickly create clusters. In this video, explore how you can work with Dataproc for managed Apache Spark.
- [Instructor] To see these services, … if you click on the menu in the GCP console, … and you scroll down to the big data section, … you'll see we have Dataproc, Dataflow, Data Fusion, … which we covered previously, and Composer. … So Dataproc as mentioned is managed Hadoop Spark clusters. … What I've done for the sake of time … is I've set up a cluster. … So we can just look at the interface … and get a basic understanding of how you'd interact with it. … The objects here are clusters, jobs, and workflows. … So here's my cluster. … I set up a very small cluster just for demonstration. … In production, I've had as many as 1,000 … or even 10,000 worker nodes. … The idea with these types of services is … that they are massively distributable for huge workloads. … Again, I've been using this in genomics, … but previously, I've used it in finance, … and I've used it in social media sites … where you have just tremendous amounts of data … flowing through the pipelines. … Notice there's an associated cloud storage staging bucket, …
- Enterprise concerns
- Enterprise scenarios
- Setting up your organization’s account
- Managing billing
- Enterprise compute services
- Enterprise storage and database services
- Enterprise data pipelines
- GCP developer and DevOps tools
Skill Level Intermediate
Work with cloud services1m 13s
1. GCP for the Enterprise
2. Enterprise Setup and Security
3. Enterprise Compute
4. Enterprise Storage and Database
5. Enterprise Data Pipelines
6. Dev and DevOps Tools
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.