Quick way to learn the basics of Apache Beam Programming

Rajathithan Rajasekar
7 min readOct 28, 2023
Photo by Jean Gerber on Unsplash

Apache Beam Katas is a set of interactive beam coding exercises which helps to learn the Apache Beam concepts and Programming Model, Currently the coding exercises are available in Go, Kotlin, Java and Python. The link is given below,

To train in python , download the pycharm community edition and browse the integrated Stepik courses from the menu, Search for “Beam Katas” and once the course is loaded on the IDE, you’re good to go.

This blog post is to quickly go through the coding exercises currently available in this interactive course.

The program output cannot be viewed inside the course, it just verifies and lets you know whether the program works or not. In order to view the output , use the below file “log_elements.py ”in your local IDE of choice.

Introduction:

  1. Create a simple element that takes a hardcoded input element “Hello Beam”.

2. ParDo is a Beam transform for generic parallel processing. The ParDo processing paradigm is similar to the “Map” phase of a Map/Shuffle/Reduce-style algorithm: a ParDo transform considers each element in the input PCollection, performs some processing function on that element, and emits zero, one, or multiple…

--

--

Rajathithan Rajasekar

I like to write code in Python . Interested in cloud , dataAnalysis, computerVision, ML and deepLearning. https://rajathithanrajasekar.medium.com/membership