Fast-growing Software as a service (SaaS) and other cloud-based solutions providers in Singapore, and across the region, have developed data apps for a variety of business uses across functions and industries. These companies are receiving huge volumes of valuable data that promises to unlock significant insights for their customers, if managed properly. From marketing apps that provide customer insights, to Internet of Things (IoT) apps that handle device feedback, and data analytics apps that process both historical and near real-time data, the demand for data apps for analysis is growing exponentially.
Various data apps that promise to help companies take advantage of their data in real-time to improve business outcomes are emerging. However, many businesses using these applications struggle to extract and analyze these growing volumes of data efficiently. This is due to challenges that developers often face when building, designing, and supporting these applications including developing a 360-degree view of the customer data, handling IoT device data in near real-time, combining historical and current data for analysis, bringing data together for machine learning (ML) models, and embedding analytics in data-intensive applications.
A key obstacle in overcoming these challenges is app developers’ reliance on legacy architectures that only enable limited scalability, concurrency, and performance. To address these challenges, software companies can turn to cloud data platforms to build and manage their data stack. By adopting a modern, cloud-based data architecture, developers have an opportunity to deliver differentiated and defensible value to customers who need powerful features and real-time insights to run their businesses better.
Why does architecture matter?
While apps have been modernized, the infrastructure powering these apps still runs on a traditional architecture that was built on the assumption that small clusters of machines with predictable amounts and types of structured data would be created largely by internal sources. Not surprisingly, these companies struggle with large volumes of data, as well as schema-less and semi-structured formats from external sources, such as application logs, web applications, mobile devices, social media, sensor data, and IoT data. This legacy architecture, created long before the emergence of the cloud and IaaS and PaaS services, was not built to run massive SaaS applications with semi-structured data. Additionally, traditional data warehouses cannot scale to match data capacity or demand easily, which creates constraints on data availability. Adopting generic architectural plumbing and tools might be a quick and low-cost fix, but such strategy can cause technical challenges down the road that can lead to lower output and a disappointing customer experience.
As a result, investing in new architecture is critical to delivering on customer expectations of data apps. Some key capabilities to look for in a modern data platform for data applications are:
- Decoupled resources which allow apps to scale computing resources independently, and in a linear fashion for each job. It also enables multiple queries to be run against the same data without conflict.
- Elasticity to allow software companies to grow or shrink dynamically and adapt to load changes.
- Support for various data types to provide a holistic view of the data.
- Developer tooling and automation to enable developers to “plug in” services with APIs, applying a building blocks approach, rather than rebuilding with each addition to the app.
- Strong security baked into the design to enable fast development, while protecting against security threats.
To ensure that data apps deliver on their customers’ expectations, software companies need to align technology decisions with long-term product needs, keep evolving customer needs in mind and design with growth and flexibility in mind. App developers need a central repository to provide the workload isolation, instant and near-infinite elasticity, unlimited concurrency, and ability to natively ingest semi-structured data.
Additionally, many data app developers adopt generic low-cost tools that allow for quick development without upfront investment, as well as using traditional data platforms. When developers do not fully consider what is needed from their data stack to deliver powerful data analytics apps, problems can arise down the line. These problems include data storage and computing strains on the system, difficulties supporting semi-structured data, frequent maintenance and upgrades, and a lack of employee resources to configure the platform to their requirements. That will eventually lead to a full re-architecture of the data platform to address these issues, which can leave customers frustrated by latency issues and incomplete data analysis.
In order to develop powerful, modern data apps, software companies need to invest in a modern, cloud-built data platform. Consider the data architecture before technical issues arise to enjoy a lower total cost of ownership from the beginning, remove the restraints of traditional data platforms and deliver fast, differentiated customer experiences.
The author is Geoff Soon, Managing Director, South Asia, Snowflake. Snowflake equips organizations with a single, integrated platform that offers the data warehouse built for the cloud.
If you want to be part of MARKETECH Experts Group, please click here.
