Be able to execute a scheduled export to email/Slack/S3 by sending a (API?) request to Sigma
What is the use case?
We have numerous jobs that involve listening for an event (e.g., end-of-day processing). These events can occur after an unknown time but need to occur before a SQL-based report is generated and emailed/Slacked/stored in S3.
How often would this feature be used?
A lot of daily reconciliations could be performed by Sigma instead of being maintained in Python services.
What is the impact of this feature on your organization?
High - we could replace heavy, engineering-related Python jobs with lighter touch Sigma reports
Sigma offers condition based alerts/reports: https://help.sigmacomputing.com/hc/en-us/articles/4408962397331-Conditionally-Scheduled-Reports
If you could add a timestamp or something for your end-of-day processing, you could send the alert when the timestamp = today or something like that.
Does that sound like it’d solve for what you need?
Hi @data_katrina ,
We use the conditionally scheduled reports heavily, but in these cases, the end-of-day processing is an example that is external to our databases. We do not currently have a timestamp in the database that corresponds to the completion of the processing. In addition, our processing could fall outside of a set time window, so even if we used the approach you suggest, there could be times when the reports would not run.
Ah! That makes sense.
I haven’t dug into Sigma’s APIs much, but would this help? Sigma API Documentation
Either way, it sounds like an interesting use case, and I would love to hear if you find a solution!
We looked into the Sigma API docs a little, but I did not see an endpoint that would perform the export. It might take an enhancement to the API. However, I am hopeful we’ll arrive at a solution!
We have found a solution internally by building a callable service that will take in the parameters/filters and call the API, export the results, and ultimately save them to an S3 location. As such, we can close this request.