--

Yes, you are right, we can use the storage transfer service to transfer b/w GCS buckets.

We can also filter the filenames using regex patterns and execute the transfer at a particular time window or make it event driven.

But the scenario we are trying to achieve here is event driven batch load , I think you cannot make STS intelligent enough to "WAIT" until the arrival of final file and then transfer the data as a batch to the destination GCS bucket.

--

--

Rajathithan Rajasekar
Rajathithan Rajasekar

Written by Rajathithan Rajasekar

I like to write code in Python . Interested in cloud , dataAnalysis, computerVision, ML and deepLearning. https://rajathithanrajasekar.medium.com/membership

No responses yet