Creating a Pipeline

The starting point of your data journey.

What are Pipelines?

Pipelines are the starting point of your data journey. A pipeline creates the shell that will include a group of models. When building pipelines you may organize the models any way you like but there is one defining factor that all the models in the pipeline will abide by: the run frequency. This will determine how frequently you want to run your pipeline, and thus the models within your pipeline.

Your first Pipeline

To build your first pipeline select the add Pipeline button in the top right corner of your screen. Initially you won't have any pipelines created, so your pipelines page will be empty.

Upon selection a dialogue box will appear for you to fill in the requirements.

Name

Select a name for your pipeline. There are a few rules to creating a name:

• No more than 32 characters long
• Use lowercase letters
• Use numbers but not as the first character
• Include underscores (ie. snake_case)

Once created this cannot be changed. If you are concerned about naming conventions, don't worry, you can always archive the pipeline (more on this later) and move your models to the new pipeline.

Description

Write a description about your pipeline. You can always come back and edit this.

Run Frequency

This is how often you want your pipeline to run. Current frequencies options are by hour, day, week, or month. Once set you cannot change this. But don't worry, you can always archive the pipeline (more on this later) and move your models to the new pipeline.

Click submit. Congratulations you have your first pipeline and are ready to start building models!

Name

Select a name for your pipeline. There are a few rules to creating a name:

• No more than 32 characters long
• Use lowercase letters
• Use numbers but not as the first character
• Include underscores (ie. snake_case)

Once created this cannot be changed. If you are concerned about naming conventions, don't worry, you can always archive the pipeline (more on this later) and move your models to the new pipeline.

Description

Write a description about your pipeline. You can always come back and edit this.

Run Frequency

This is how often you want your pipeline to run. Current frequencies options are by hour, day, week, or month. Once set you cannot change this. But don't worry, you can always archive the pipeline (more on this later) and move your models to the new pipeline.

Click submit. Congratulations you have your first pipeline and are ready to start building models!