Google Composer is a data workflow orchestration service. In this video, explore how you can work with Composer for managed Apache Airflow.
- [Instructor] The next tool is Cloud Composer. … This is managed Apache Airflow, … and again is often used in combination … for complex data pipelines with Dataflow and Dataproc. … The idea here is that you get visibility … into your complex pipelines … that are often hosted in Beam … and have plugins such as Dataproc. … So it takes about 15, 20 minutes to set this up. … So I just created a default instance, … and then you get visibility into the Airflow web server. … So I didn't bother to run any jobs, … because the amount of time that it would take. … So if you wanted to understand what type of visibility, … you could just quickly go over to the documentation here, … and you could see that you have on the UI a DAG view. … So again, the directed acyclic graph … from your complex processing tasks, … and you can see, here, this is a pretty good screenshot, … that shows for each task, what is the run of the task? … What is the DAG run? … And then links out to logs and information about it. …
- Enterprise concerns
- Enterprise scenarios
- Setting up your organization’s account
- Managing billing
- Enterprise compute services
- Enterprise storage and database services
- Enterprise data pipelines
- GCP developer and DevOps tools
Skill Level Intermediate
Work with cloud services1m 13s
1. GCP for the Enterprise
2. Enterprise Setup and Security
3. Enterprise Compute
4. Enterprise Storage and Database
5. Enterprise Data Pipelines
6. Dev and DevOps Tools
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.