Insights from the road: Shifting to a higher gear with StreamSets

It has been a literal and figurative breath of fresh air to get back on the road the last couple of months, meeting with employees, partners, and customers worldwide. While it’s been an adjustment from working from home and riding my favorite local bike trails regularly, there is nothing like face-to-face conversations. Wherever I go, I make it a priority to meet with customers.

Yes, I want to hear their feedback on our products and company. But I also want to learn what keeps them up at night as technology leaders and about the BHAGs (big hairy audacious goals) they have for themselves, their teams, and their companies.

Three weeks ago, I was in the San Francisco Bay Area, where I spent time getting to know the team at our most recent acquisition, StreamSets (more on them later). While in the Bay Area, I also met with executives at two of our customers, one at a top software company and another at a major hardware manufacturer. Sitting with them, I heard many of the same themes that I’ve been hearing from other customers. Several seismic shifts in the technology landscape present great opportunities for digital transformation, but also pose significant challenges to achieving technology leaders’ ultimate visions.

Shifting into gear

The first and most obvious shift is the cloud. The advantages of the cloud — speed, ease of deployment, ease of management– have been proven over and over by cloud computing platforms like AWS, cloud apps like Expensify, and cloud data platforms like Snowflake. But the early days of cloud are over, along with the fantasy that companies could go all-in on one cloud platform and everything would then be simple, cheap, and in one place. Enterprises need to integrate hybrid and multi-cloud environments, or they risk losing visibility and control as their applications and data are scattered across multiple environments. And very few CIOs want to be left at the mercy of being locked into a single cloud stack.

The second shift is self-service. Software AG focuses on integration, which used to be something specialists in IT took care of. But more and more project teams in the business are being empowered to do more on their own, whether it’s spinning up a new application, mapping out a business process, or accessing data. Democratizing certain aspects of IT helps businesses innovate at today’s speed of business, but there’s a risk. Some of today’s self-service tools sacrifice the power features and sophisticated capabilities that large enterprises need to actually run and connect their businesses at scale.

The third shift is the convergence of applications and data. The major applications in companies such as the ERP and CRM systems were the systems of record, and data was extracted from them for reporting and analytics. Today, apps have been fragmented and decomposed, while data analytics platforms such as AWS S3 and Snowflake are becoming the new systems of record– the only place with authoritative data on customers, products, etc. Data needs to flow continuously across applications and platforms, as scattered as they may be across legacy data centers and multiple clouds. As it does, it’s easy to lose your ability to see and govern that data. This application-data convergence zone is powerful. It’s also a bit of the wild, wild west.

Accelerating with StreamSets

I am taking these themes to heart, and I am thrilled to be able to help leaders handle these shifts with the exciting new offering from StreamSets, the company we acquired in April. StreamSets is bringing modern, enterprise-grade data integration capabilities to Software AG, and this week it has released its new Transformer for Snowflake engine.

You can read more about it here, or if you want to dive in, try it here.

For me, this release is significant because it embraces these three seismic shifts.

• Snowflake has become a dominant player in cloud data and analytics platforms. With Transformer for Snowflake, Snowflake users can accelerate their shift to the cloud while still being able to integrate all data across their hybrid environments to Snowflake.
• Transformer for Snowflake is built on Snowpark, which puts powerful transformation capabilities that used to require expert Spark developers into the hands of the typical Snowflake user– enabling self-service without sacrificing power.
• The Snowflake Data Cloud is rapidly becoming a new system of record. Together, StreamSets and webMethods will provide an iPaaS addressing both application and data integration, allowing consolidated, synthesized data to flow across the connected enterprise.

I’m back home now, so it’s time for me to hit the bike trail with my sons (they are navigating their own seismic shifts but that’s a whole other story). I’ll be traveling again soon, where I look forward to meeting more customers and seeing how new offerings like StreamSets Transformer for Snowflake help customers navigate these big changes. It’s time for all of us to shift into a higher gear!