Domo Tricks: Dataflow Archiving

by bvkonline

(Originally posted October 10, 2022 on Linkedin)

Everyone’s done it. You’ve built out some incredible dataflow that will revolutionize the way your organization operates and 100x the revenue. Fast forward a year, and you’re somehow the only card viewer and have millions of rows hanging out in your Domo instance not being used.

Data Governance and regular clean-ups are an important part of keeping your Domo instance user friendly and compliant. However, there’s sometimes value in keeping the business/dataflow logic and not the extra unused data.

Screenshot of dataflow named "Production | Amazing 100x Revenue Idea"​ with 8 million rows last used 1 year ago

If you’ve labored over the dataflow and business logic but the dataset isn’t being used and you’re sick of the millions of unused rows, consider implementing a standard dataflow archiving process in your instance as part of your governance procedures:

  • Rename the dataflow/datasets and add an “Archive” description, or whatever makes sense based on your organization’s naming conventions (i.e., “Archive | Amazing 100x Revenue Idea”). I’d also add a description to the dataflow and dataset that explains that it has been configured to output zero rows.
  • If your organization utilizes tags, add an #Archive tag.
  • Remove any schedules so the dataflow stops running on a schedule or when the input datasets update.
  • Configure the dataflow’s outputs to return zero rows, which can easily be done in both MySQL and Magic dataflows:

MySQL

In the final output(s), add some nonsense logic that will never be true in your where clause. I’m impartial to “where 1 = 0:”

SQL statement "Select * from profit??? where 1 = 0"​

Magic ETL

The core logic is the same in Magic ETL as it is in MySQL dataflows. In order to output zero rows in your Magic ETL datasets, add a filter tile right before your output dataset(s) and populate logic in your filter formula that will never equate to true:

Filter tile that says "excel = best enterprise data tool"​

Run the dataflow one last time, and it will return datasets with zero rows. You still have the business and dataflow logic if the project gets revisited, but it’s not taking up valuable rows in your Domo instance.

What are other hashtag#DomoGovernance tips and tricks you have?

Leave a comment