We are excited to announce the release of version 2023.02 of our DataOps platform!
We are constantly improving Saagie by incorporating extensive community feedback to simplify data engineering/science workflows and deliver innovative, first-class tools for DataOps.
This version is a major version providing new rich functionalities and focusing on Observability and Smart Conditions of Pipeline. Here are the notable new features and improvements since the previous release:
- A new set of resource observability modules have been added for clusters and pipelines.
- Pipeline functionality has been enhanced to include more advanced orchestration logic, such as conditions on environment variables and job status.
- Saagie now supports Google Cloud Platfom.
Observability and advanced automation in pipelines
Cluster and Pipeline Resource Monitoring
New resource monitoring pages have been added to monitor resource consumption for clusters and pipelines.
At the cluster level, you can access the Operations module to see an overview of your cluster. This page displays the number of projects, jobs, pipelines, and apps created on each platform, as well as resource capacity metrics for CPU, and RAM for each node in the platform.
In the Overview page of pipelines, you can access graphs displaying runtime and resource consumption metrics (CPU and RAM) for each instance.
This added focus on resource monitoring in Saagie will allow data engineers and platform administrators to have a complementary view of clusters and pipelines to track performance and better optimize resource usage on their platforms.
Smart Conditions in Pipelines
You can now create new type of conditions to build more relevant pipelines:
- Conditions based on environment variables
- Conditions based on job status
These new conditions will allow you to implement advanced intelligence in your pipelines. In particular, by completing Environment variables modification already available since 2023.01, the conditions on Environment variables expand the capacity to execute complex branches in Pipeline regarding the combination of variable values.
For more information, see About Conditions in Pipelines.
Google Cloud Platform (GCP) Support
Saagie DataOps Platform is now available Google Cloud Platform GCP.
This new version of the Saagie installer is now compatible with Google Kubernetes Engine (GKE).
Saagie’s Technology Repository Updates
The following technologies have been added to the official Saagie technology repository:
New embedded job technologies:
- DBT 1.3, 1.4
- Python 3.11
- R 4.1, R 4.2
- Rstudio R 4.1 et R 4.2
TIP: Do not forget to update your Saagie repositories to keep your technologies up to date.
We value your opinion
As always, we value your feedback on the solution we build. Stay tuned for new features, updates and improvements. Share your feedback on this announcement. You can also test the platform for free with our free trial!
Thank you for your feedback!
