Last week I was lucky enough to attend the Snowflake Data Cloud World Tour. I say lucky because the conference was pumping, and the 600+ delegates who crammed into the 5 Star Pullman Hotel in Auckland’s CBD, were itching with excitement. The vendor hall, its low ceiling and lack of natural light, was deafening with chatter as vendors did their best to sell, and the delegates did their best to grab some marketing swag or fill up on a breakfast of fresh fruit and pastries. I had barely walked through the main doors when I bumped into people I knew, and pretty soon we were talking about Snowflake.
I must confess; despite being aware of Snowflake’s reputation, I had never used it in a professional setting. For me, this conference was a fact-finding mission, and I discovered that Snowflake enjoys immense popularity with a dedicated following. If you’re relatively new to the world of Snowflake, it can best be described as an autoscaling cloud data warehouse. To make it more appealing, consider adding data analytics, dashboarding, a data marketplace, and the absence of infrastructure management. These features collectively explain why this conference was one of the year’s hottest tech events (with the exception of the Locus NZ FME User Conference, of course!).
From an FME perspective, Safe Software has developed numerous readers and writers to facilitate data movement to and from the Snowflake platform. Notably, spatial data writers simplify the uploading of spatial data into Snowflake, a feature not supported by other integration platforms. FME Flow Hosted, previously known as FME Cloud, shares a cloud-based, elastic nature with Snowflake, establishing a strong technological partnership between the two platforms.
To understand a little more about that synergy between platforms, have a read through Automating Google Analytics Reporting with the help of FME and Snowflake and HIS Markit Centralises 1.5 Billion Rows of Data in 5 hours.
Back to the event… we were made to feel very welcome in the plenary session, first by the nerve tingling The Haka Experience, who’s powhiri reverberated around the auditorium, and then by successive speakers who each spoke with great enthusiasm about the end-user experience.
After a well-earned refreshment break and with a Snowflake t-shirt and badge under my arm, I started chatting to the vendors to see what they offered. Data Engineers and Location Technologists, though sharing some common terminology, often interpreted words differently. This language barrier was an intriguing challenge, and I’ll save the details for a future blog post.
Returning to the technical breakout sessions, I was captivated by presentations from organizations like Mitre10, Sky, Kiwibank, and Spark. While each session had its unique focus, they all grappled with a common challenge — managing and surfacing vast amounts of data. The topic that dominated these discussions was AI’s role in making data more accessible. Large Language Models, in particular, enable users to query data in plain English, such as asking for sales figures or the best-performing month in 2023.
With my Location Technology hat on I couldn’t help but notice that all the questions lacked a bit of spatial awareness. Questions like “How many customers living more than 5km from a store spend over $1000?” or “Which store has the longest supply chain route?” were notably absent. These should ideally be answerable without the need for a GIS specialist, indicating a language barrier between Data Engineers and Location Technologists.
After lunch (amazing by the way… vegetarians were well catered for, and the numerous petit fours kept the most sweet toothed delegates happy!) – the afternoon passed with more insightful discussions on Snowflake usage and architecture. A session on Streamlit caught my attention, as it addresses the challenge of presenting data to others effectively. Streamlit enables users to quickly and easily build web applications for sharing and visualizing data, much like how platforms like ArcGIS Online or Mapbox simplify complex data sharing in the Location Tech space.
As the day drew to a close, I eagerly anticipated a well-deserved beer and reflected on the event. It had been a highlight, and I look forward to attending next year, possibly as a vendor rather than just a delegate. FME has much to offer Data Engineers, and the combination of FME and Snowflake empowers organizations to harness their spatial information effectively.
More Resources
Working with Snowflake and want to understand more about FME? Let Locus help you discover and maximise the full potential of your organisation’s data. Get in touch.
