Unlocking the Power of Dagster: A Step-by-Step Guide to Using the GraphQL API for Programmatic Job Triggers

Dagster exposes graphQL API to programmatically connect with dagster and do wide variety of operations. If you are coming from REST API world, there could be initial hiccups. This blog should come handy in such situation.

GraphQL schema

In local this is available here — https://localhost:3000/graphql

In dagster cloud, this is available for each branch & prod like this :

• For branch : https://yourOrg.dagster.cloud/your_branch_hash/graphql
• For prod : https://yourOrg.dagster.cloud/prod/graphql

We should be able to reach graphQL browser like below:

Here we can read different API(s) that are available and different params that can be passed.

Postman for dagster

Postman supports GraphQL out of the box.

First step is generating a user token from dagster cloud. Once we have it,

• Method is POST
• URL is graphQL endpoint like (for branch): https://yourOrg.dagster.cloud/055961b61babaeaa0b5e5dc24dbec108f81ddf41/graphql
• In headers, `Dagster-Cloud-Api-Token` should hold the generated user token

Getting list of repositories

Let’s try to get list of repositories

• In body., select GraphQL option.,
• In query, fill following :

query RepositoriesQuery {
  repositoriesOrError {
    ... on RepositoryConnection {
      nodes {
        name
        location {
          name
        }
      }
    }
  }
}

• no parameters required

We should get response like this :

{
    "data": {
        "repositoriesOrError": {
            "nodes": [
                {
                    "name": "__repository__",
                    "location": {
                        "name": "data-pipeline-1"
                    }
                }
            ]
        }
    }
}

Run a dagster job from GraphQL

Request body :

mutation LaunchRunMutation(
  $repositoryLocationName: String!
  $repositoryName: String!
  $jobName: String!
  $runConfigData: RunConfigData!
) {
  launchRun(
    executionParams: {
      selector: {
        repositoryLocationName: $repositoryLocationName
        repositoryName: $repositoryName
        jobName: $jobName
      }
      runConfigData: $runConfigData
    }
  ) {
    __typename
    ... on LaunchRunSuccess {
      run {
        runId
      }
    }
    ... on RunConfigValidationInvalid {
      errors {
        message
        reason
      }
    }
    ... on PythonError {
      message
    }
  }
}

• the above mutution is generic & ready to use. All the variables are given values in the parameter section:

{
  "repositoryLocationName": "data-pipeline-1", 
  "repositoryName": "__repository__",
  "jobName": "data_sync_job",
  "runConfigData": "{}"
}

Jobs with partition

Let’s say a job has a monthly partition. This input need to be passed to dagster. However, partition data is not a runConfigData but a tag information.

mutation LaunchRunMutation(
  $repositoryLocationName: String!
  $repositoryName: String!
  $jobName: String!
  $runConfigData: RunConfigData!
  $tags:[ExecutionTag!]
) {
  launchRun(
    executionParams: {
      selector: {
        repositoryLocationName: $repositoryLocationName
        repositoryName: $repositoryName
        jobName: $jobName
      }
      runConfigData: $runConfigData
      executionMetadata: {
        tags: $tags
      }      
    }
  ) {
    __typename
    ... on LaunchRunSuccess {
      run {
        runId
      }
    }
    ... on RunConfigValidationInvalid {
      errors {
        message
        reason
      }
    }
    ... on PythonError {
      message
    }
  }
}

• The above request body adds tags information and passed as executionMetadata

{
  "repositoryLocationName": "data-pipeline-1", 
  "repositoryName": "__repository__",
  "jobName": "data_sync_job",
  "runConfigData": "{}",
  "tags": {
    "key": "dagster/partition",
    "value": "2024-10-01"
  }
}

• the above parameter contains dagster/partition as one of the tag’s key with partition value. Now we should be able to run a job which has partition