In this post , we are going to see how to trigger cloud composer-2 DAG from Cloud Functions-2. To start with , first enable the cloud composer API in the Google cloud console.
Next, we will select cloud composer-2 to provision the environment .
Note: CloudComposer-2 has the advantage of Autoscaling (GKE Auto-Pilot) over cloud composer-1. Detailed comparison is available in the Google cloud documentation site below.
Cloud Composer versioning overview | Google Cloud
Cloud Composer 1 | Cloud Composer 2 This page describes Cloud Composer versioning and lists differences between major…
Before provisioning , it is recommended to create a custom network for cloud composer. I have created a node subnet, services subnet and pod subnet. Post creation , these subnet ranges can be selected in the Network configuration of the cloud composer.
It is also to be noted that instead of using the default compute service account, it is recommended to create a custom service account for security purposes. Since this is only a demo, I have used the default compute service account and granted the role to that service account.( If you still get any permission issues, follow the error instructions and provide the required privileges for the listed service account)
I have chosen a small environment size for demo purposes.