Get started with Apache Spark Streaming Pulsar

Prerequisites

You need the following:

• Git
• Python >3.7
• pip

Script python

Unlike other python's jobs, for pyspark job you do not need to build the project. You will directly use the python's code on Graal because spark-submit need .py file as argument.

Use

The example script takes as input parameter the broker and admin endpoint of Pulsar, the name of the topic to read from and your token to have the authorization to read and write on the topic.

For that reason, you will need to specify in Graal the following parameters "--pulsar_broker", "--pulsar_admin", "--pulsar_token", "--pulsar_topic"