Get started with Apache Spark Streaming Pulsar
Prerequisites
You need the following:
- Git
- Python >3.7
- pip
Script python
Unlike other python's jobs, for pyspark job you do not need to build the project. You will directly use the python's code on Graal because spark-submit need .py file as argument.
Use
The example script takes as input parameter the broker and admin endpoint of Pulsar, the name of the topic to read from and your token to have the authorization to read and write on the topic.
For that reason, you will need to specify in Graal the following parameters "--pulsar_broker", "--pulsar_admin", "--pulsar_token", "--pulsar_topic"