If you had predicted 15 years ago that databases and data platforms would today be one of the hottest areas in technology, you probably would have been dismissed by a lot of people. At that time the database wars seemed to be over. The giants—Oracle, IBM, and Microsoft, had won. The focus of the technology world was consumed by Internet companies–some with a business model but many without one, as a technology companies drove a seemingly unstoppable rise in the stock market into what would prove to be its last few months.
However, such a prediction would have demonstrated an insight that is clear in hindsight: applications created for and driven by the internet would be a key driver in a step function change in the sources, types, and quantities of data being generated.
That brings us to today, when we find ourselves in the midst of a sea change in how companies use data that is remaking the software and business landscape. What does that mean for what will happen next? Here are a few predictions for 2015:
2015 will be the year of SQL: SQL is hot again.
The last several years saw a near-deafening amount of discussion about noSQL. Originally the discussion was whether “no SQL” platforms would dominate and start to crowd out traditional SQL databases, but more recently that conversation has shifted. In 2015 the market will get smarter, starting with a rediscovered appreciation for the differences between the transactional database and analytical database worlds. While SQL will continue to dominate transactional databases for the majority of business applications, noSQL databases with their ability to scale out and provide high availability will take permanent root for cloud applications that need these characteristics. On the analytical database front, the innovation and focus will swing full circle, and SQL will again reign as king. This shift will be driven by the recognition of the extreme difficulty in finding specialized data scientists and infrastructure teams that can deploy and use noSQL platforms, in sharp contrast to the availability of millions of people and thousands of tools that understand SQL.
There will be a long overdue consolidation wave of cloud application vendors.
Consolidation of cloud application vendors is inevitable and, frankly, long overdue. In 2015 we’ll see large, traditional on-premises vendors snatch up cloud application vendors in their core and complementary product categories in an attempt to capture a share of the accelerating market shift to cloud. We already saw a taste of this in 2014 with, for example, SAP’s $8.3B acquisition of Concur. It’s actually surprising that we haven’t seen far more acquisitions of cloud companies already–the large enterprise players have largely failed to transition to the cloud and won’t be able to hold on to maintenance revenues forever.
The war between public and private cloud will be over. The public cloud will be victorious.
The window of opportunity for private cloud is closing fast and 2015 will be the year of the public cloud. I have long said that I expected to see around 500 private clouds and I thought many of these to be operational by now. To be clear, a private cloud is distinguished from a virtualized datacenter by the level of automation–with a private cloud, no person is involved in allocating system resources. Given this definition, there are almost no production-level private clouds within enterprises today. While some very large companies will continue to pursue efforts to develop private clouds, the numbers will be much smaller than many (myself included) were predicting. Amazon, Google and Microsoft are simply too far ahead in maturity, cost, and functionality. This means that virtualized data centers will become the new legacy and the vast majority of companies will aggressively move to public clouds. The opportunity for private clouds was there, but almost no one was able to execute on it successfully, and in 2015 the window of opportunity for a future in which private clouds are commonplace will officially close.
2015 will be the turning point where SaaS truly bridges the data divide.
The first wave of “big data” innovation delivered tools for the 1%–those organizations with the resources and patience to build complex systems and highly specialized teams to work with them. The result was opportunity for the elite, but near-insurmountable obstacles for the masses separated by the “data divide”. For example, the requirements and complexities of Hadoop have meant that only a handful of the most sophisticated companies, with access to the right resources and talent, have been able to make Hadoop a viable option. 2015 will be a turning point in which SaaS applications for data processing and analytics will bridge the data divide. These SaaS applications will deliver flexibility and ease of deployment that will allow the masses to catch up, as the expertise and resources required will no longer be an insurmountable barrier to value.
Machine data will fuel the next wave of data analytics innovation.
Huge investments have been made to date in analysis of the transactional, structured data that makes up the “what” of understanding business activity. However, analyzing structured data alone is not enough. Huge and rapidly growing amounts of machine-generated data are now available—application-generated data, log data, device data, Internet of Things (IoT) data, and more. To date these have been largely collected and analyzed in isolated silos with special-purpose tools. However, there is an emerging recognition of the enormous value in analyzing machine data together with transactional data—transactional data provides the “what”, machine data provides the “why” needed for deeper understanding. This recognition will catalyze a rapid shift to data processing and analytics that will enable business analysts to gain deeper understanding of their business by analyzing structured and semi-structured data together.
Now comes the fun part: we all get to be part of not only witnessing but also creating what really happens in 2015. I’m looking forward to it.