Graal Platform Documentation

Graal Platform Documentation

  • Docs
  • Help

›Overview

Overview

  • What is Graal Platform?
  • Why use our platform?
  • How Graal Platform works?
  • Concepts
  • Jobs & workflows
  • Security

Quickstart

  • Quickstart

Tutorials

  • Get started with Python
  • Get started with Dask
  • Get started with XGBoost
  • Get started with Apache Spark and Maven
  • Get started with Apache PySpark
  • Get started with Apache Beam and Gradle
  • Use the API
  • Using the command line tool (graalctl)
  • Using secrets
  • Migration from Databricks
  • Get started with Tensorflow
  • Get started with Pytorch
  • Get started with Mxnet
  • Setting up the Hadoop bridge
  • Get started with Apache Flink and Maven
  • Get started with Dbt
  • Get started with Pulsar
  • Get started with Apache Spark Streaming Pulsar
  • Get started with Debezium
  • Get started with the SDK

How-to guides

  • Using Graal Platform with Azure Data Factory
  • Publishing your artefacts with Azure DevOps
  • Using Graal Platform with Apache Airflow
  • Publishing your artefacts with Jenkins
  • Spark
  • Network, VPN, gateway and firewall
  • Logs
  • Pricing

Security

  • Overview
  • Comply with requirements
  • Infrastructures under Graal Systems
  • Responsibilities

Troubleshoot & debug

  • Troubleshooting
  • Common issues
  • Debug jobs

Jobs & workflows

Jobs

Supported jobs are:

  • Apache Spark
  • Bash
  • Ray
  • Python
  • PyTorch
  • Dask
  • Tensorflow (Distributed and Federated)
  • Scikit-learn
  • Spring (Batch and Cloud Data Flow)
  • Apache Hadoop
  • Apache Pulsar Function
  • Distributed XGBoost
  • SAS (soon)
  • R (soon)

Resources

Specify resources for memory and cpu inside the job options:

"resources": {
    "requests": {
        "cpu": "3",
        "memory": "8G"
    },
    "limits": {
        "cpu": "3",
        "memory": "16G"
    }
}

Special characters inside parameters

Parameters are directly used inside the jobs. You have to escape special characters. Example:

"parameters": ["--path", "\"spècia!;ch@\""],

Use \" inside the JSON for this. If you want to escape from a CLI like PowerShell, use single quote for string with special characters

Schedule

Run once by letting an empty schedule or force a schedule of type once

"schedule": {
    "type": "once"
}

For cron job, specify a cron expression, for instance:

  • Every day at 1h: 0 1 * * *
  • Every 3 months at 0h: 0 0 * Jan,Apr,Jul,Oct *

inside a schedule type cron like this:

"schedule": {
    "type": "cron",
    "timezone": "Europe/Paris",
    "cron_expression": "0 1 * * *"
}
← ConceptsSecurity →
  • Resources
  • Special characters inside parameters
  • Schedule
Graal Platform Documentation
Overview
What is Graal Platform?
Quickstart
Apache SparkApache FlinkApache BeamPythonTensorflowDaskDistributed XGBoost
Links
HomeConsoleCopyrights
Copyright © 2023 Graal Systems