Snowflake - Use Different Warehouse per Dataset/Workbook

Product Requests
1
  • Describe Feature
    I would like a way to change which Snowflake Warehouse that sigma uses based on the dataset or workbook being accessed.
  • What is the use case?
    We have many different datasources, some in the billions of rows and others only a few million. Currently we have to set single Warehouse for Sigma to use in Snowflake, which has to be big enough for the large sources but overpowered for the smaller. It would be great if each Dataset could be assigned a different Warehouse, so that we do not consume a large number of credits for small datasets.
  • How often would this feature be used?
    Every time a new dataset and workbook is created.
  • What is the impact of this feature on your organization?
    This would enable a huge increase in cost-efficiency of compute credit usage, as we can more easily right-size the warehouse based on the data size.
2

Thoughts on this product request:

Can’t you already do this, by creating 2 separate Snowflake connections. One connected to your large warehouse and the second connected to your small warehouse. Depending on how your data is organized (small tables on db A and large tables on db B OR if all tables are on 1 db) you can select the desired WH for each dataset by selected the connection desired when creating the dataset. I think this does exactly what you’re looking for. Let me know your thoughts!

image
image622×696 46.3 KB

3

Hi Blake,

Thanks for the response. Yes that is a good idea to tackle this issue, particularly the second option. We will look into potentially implementing it for our needs in the short term. I still think it would be helpful to have this functionality in a more formal way in the product, as both options still feel a little hacky for a product that advertises itself as purpose-built for the cloud data warehouse. But if it works I can’t argue with the results haha.

Thanks,
Rob

1 Like
4

Hi @rhughes,

Have you looked into Snowflake’s Query Acceleration Service (QAS)?
https://docs.snowflake.com/en/user-guide/query-acceleration-service.html

It allows you to have a smaller warehouse that scales up should the query need extra resources. You can start with a SMALL warehouse and set QAS to 8x, so Snowflake would grow the warehouse out to the equivalent of a XLARGE warehouse for those queries that need it.

Regards,
David

1 Like