Introduction to Snowflake Data Pipelines
One of the following Snowflake features or a third-party data integration tool (not shown) loads data continuously into a staging table: Snowpipe Snowpipe continuously loads micro-batches of data from an external stage location (Amazon S3, Google Cloud Storage, or Microsoft Azure) into a staging table.
Snowflake’s Time Travel feature does not apply to stored procedures
✅ Looking up old versions of stored procedures must be done outside Snowflake. ✅ Consider keeping a master copy of each stored procedure’s CREATE PROCEDURE command in a source code control system. ✅ If a source code control system is not available, you can partly
Snowflake Organizations
Use SNOWFLAKE ORGANIZATIONS to simplify account management and billing, Database Replication and Failover/Failback, Snowflake Secure Data Sharing, and other account administration tasks. View, create, and manage all of your accounts across different regions and cloud platforms. Snowflake documentation – introduction A central view of all
Snowflake’s Secure Data Sharing
Snowflake’s Secure Data Sharing is transformative to the world of data. What other platform, environment, or vendor can touch this? The World Is Going to Run On Snowflake! Gone are the days of old-school legacy slow, expensive, cumbersome, repetitive, and limiting transferring data. Snowflake enables
Improving performance with Snowflake Clustering
Here is an example of performance improvements we’ve obtained using Snowflake Clustering. The before join between a 9.4 TB, 35.4 billion row table with a 1.5 TB, 1.8 billion row table ran in 1:14:57 and costs $11.99 in compute credits. After clustering both tables, the
Snowflake Role Hierarchies
Take advantage of SNOWFLAKE role hierarchies to align access to database objects with business functions in your organization. ✅ In a role hierarchy, roles are granted to other roles to form an inheritance relationship. ✅ Permissions granted to roles at a lower level are inherited
Data Masking – Snowflake and Data Vault
By Julie Heckman, Director of Data Engineering Whenever an organization embarks on a journey into data warehousing, one of the first questions to arise is “how will we secure the data?” Securing data is a broad concept but I want to focus on confidential data
My View from Snowflake Summit
By David Teplow, Managing Director of Snowflake Services It was an exciting time to be at Snowflake Summit (June 13-16, 2022). It was the first non-virtual Summit in three years and in that time Summit attendance, with nearly 9,000 in-person attendees, grew by more than
Continuous Integration and Deployment Are Data Engineering’s Cruise Control
Julie Heckman, Principal Data Engineer Curious about Continuous Integration and Continuous Deployment in Data Engineering? As the names imply, CI/CD is a nonstop method for taking new code through development, testing and production environments in order to advance the good and press pause on the