Q: So if Sigma loads in 10K rows, what if I want to do something like a sum for the entire dataset? The calculation in the browser, does it give the result only for that 10k or the entire dataset?
A: If the query is for all the rows, the query will submit to the warehouse rather than using the cache. The Browser Cache will be utilized only if the question being asked can be fully encompassed by the data that is already present in the cache.
Q: You mentioned that Sigma caches recent queries made to Snowflake. Does that apply to those of us who are not using Snowflake (e.g. Databricks or Postgres)?
A: Postgres and Databricks do not utilize a results cache like Snowflake unfortunately, but we do still have all the same browser cache optimizations. Implementing a results cache (catered to each warehouse’s internal features) is something in development! We will definitely communicate the product’s new news in the Sigma Community.
Q: Do you recommend doing the joins in a Dataset or in the workbook directly for maximum efficiency?
A: In short, doing the join in either place is equally efficient. (You could use Materialization to periodically “save” the results of a join/etc for you, side-stepping the join computational cost each time you’re-query in the workbook). @Sierra recommends doing the join in a Dataset to reduce duplicated work between teammates. Keep an eye out for our upcoming updated Dataset interface that will make the distinction seamless!
Q: Can you show us an example of the step-by-step process to create a flat table from a join?
A: Sandeep answered from 27:50 to 29:25 in the recording.
Q: It’d be really helpful to understand which of these steps apply to all warehouses, not just Snowflake. Is there clarification somewhere on which features are available to all connectors, or what’s not available for non-Snowflake users?
A: Sigma docs note that the results cache is only for some warehouses. Check out the doc listing various supported data warehouses: https://help.sigmacomputing.com/hc/en-us/articles/360037430573
Q: For a previous heavy Excel shop and now heavy Sigma Computing shop, would you recommend using less pivot tables? What are the top 3 or 4 alternatives that are less compute intensive (especially on Snowflake) to pivot tables?
A: “I’m in a similar position, and I’ve had a lot of success with making everything with groupings rather than pivots, and then pivot as my very last step in order to make something a little easier to read at a glance for an end-user.” from Diana Levy. Also, Sandeep answered live from 33:10 to 35:40 in the recording.
Q: Will we have a function to allow for the overall value to show the aggregate over the whole time filtered for instead of the final date within a timeline?
A: Sandeep answered from 43:00 to 44:25 in the recording.
Q: Are Sigma Alpha Queries subject to the same or different query timeout limits as we would experience with Snowflake queries?
A: Sandeep answered from 35:50 to 38:10 in the recording.
Q: If I have many different visuals that refer to the same Sigma flat table in a workbook, will Sigma generate just one query and reuse the result to populate the related visuals or will it generate one different query for each visual?
A: Sandeep answered from 38:45 to 39:49 in the recording.
Q: Where can I see, alpha query log?
A: Sandeep answered from 40:01 to 40:25 in the recording.
Q: Does Sigma have any plans for column-level lineage? If I have a large dataset that multiple people are using, I might want to hide/remove unused columns, but I want to be sure that these columns are not used downstream.
A: Yes, that product request is on our radar. Your use case has been added to our engineering ticket for the product team to understand your needs. Thank you for the feedback!
Q: Do you recommend lookups over joins when it is possible?
I ask because I have experienced bad performance in a table when I make a join, then I realize it would be simpler to do a lookup instead of the join and the run times improve.
A: The answer is it depends! In general, joins tend to perform better than lookups, but if the lookup/join is very simple, it could be either no difference or a very little difference. It also depends on what you’re doing in the workbook later. For example, if you have 12 other child elements, a join will most likely be more efficient. In general, Lookups are more flexible prototyping than Joins as production.
