Sergey Nivens - Fotolia
Darkness is turning to light in the database management system market, which had been in a deep sleep in recent years. As enterprises shift to modern system infrastructures, limitations of database management systems have become more pronounced. In response, a bevy of start-up suppliers are rousing the market, giving users more choices.
Relational Database Management Systems (RDBMS) emerged in the 1970s to provide enterprises a place to store business information. Three companies -- Oracle with 42%, Microsoft with 20%, and IBM with 18% -- dominate the database management system (DBMS) market, according to Gartner. These vendors were among early market entrants and delivered DBMS that support a range of applications, from online processing to data analytics. But despite their popularity, there was little innovation.
"At one point, MIT actually shut down their DBMS research group," said Barry Morris, founder and executive chairman at NuoDB Inc.
As new requirements, such as the need to support more unstructured data, emerged from advances like social networking, start-up vendors approached DBMS concepts differently. "The DBMS market had become quite boring, but market changes recently have made it much more interesting," said Donald Feinberg, vice president, analyst at Gartner.
Traditional system revenue has flattened out, but sales of new tools -- although small in relation to the total market -- are increasing by double digits. So, buying a DBMS is no longer a simple choice among Oracle, Microsoft and IBM.
One reason for the recent attention is the market's size and growth. The database market is expected to increase from $40 billion in 2015 to $50 billion by 2017, according to IDC. A number of factors are fueling the growth and market shake-up.
Information influx drives new DBMS products
First, traditional systems, which rely on SQL, are not well suited to the information influx. These suppliers have essentially used 40-year-old technology and constantly enhanced it. Much like continually sprucing up a 1970 Chevy Camaro, eventually flaws emerge. For instance, the popular systems were designed to work with textual, structured data -- items that translate well to rows and columns. But they do not work as well with unstructured information, such as video.
The differences leave companies with unappealing choices. "To consolidate information in one central system would take years, and the chances are slim that all of the different types of information would work with one data model," said Marc Linster, senior vice president, products and services at EnterpriseDB.
Businesses have new demands, as recent technical advances created new data sources. The number of media types has grown; many companies now rely on YouTube videos for internal and external communications. Enterprises track social media interactions in various ways. Firms are deploying internet of things end points and gaining more visibility into their supply, distribution and fulfillment chains. As a result, IDC expects the data volume to double every two years until 2020.
In addition, corporations want real-time information, but the distributed nature of information nowadays creates problems. RDBMSs have trouble updating one record stored on multiple systems. Whenever an update is made, the change must be relayed to all systems simultaneously. The concept is simple to understand but difficult to implement as systems store terabytes and even petabytes of information and databases run on multiple servers scattered across the country or the globe.
When DBMS emerged, uptime demands were less stringent than they are today. If IT delivered four nines of uptime, which translates to less than one hour of downtime per year, top management was satisfied. Nowadays, some organizations stretch systems uptime to nine nines, which measures annual downtime in milliseconds.
Finally, the traditional systems are largely based on proprietary technology and can be expensive.
New kind of DBMS
A number of new DBMS products have emerged to address these limitations.
As noted, an RDBMS works with structured data. Much of the new information gathered today is unstructured -- and the variety of information is constantly growing. For example, companies such as Facebook track and correlate friends' locations during the day. These new features spawn developer and business managers' imaginations, so new interactions -- and the supporting applications -- are possible. Consequently, databases need to store items like geospatial information.
In response, vendors have worked on multimodel systems, a database that handles multiple types of data and data models. In fact, Gartner predicts that by 2017, all leading operational DBMSes will offer multiple data models.
How the DBMSes are built is also different. As noted, the established RDBMSes, were forged from proprietary technology, but the new systems have open source foundations. By 2018, more than 70% of new in-house applications will be developed on an open source DBMS, and 50% of existing commercial RDBMS instances either will have been converted or will be in the process of being converted, according to Gartner.
The end result is companies find themselves with more choices. Dozens of new options have emerged, and the lines between them are murky. A good place to start is with NoSQL DBMS. These systems, which have also been called Non-SQL tools, were conceived to hold non-tabular information. In other words, they were designed to overcome RDBMS shortcomings when working with unstructured information, like video. A number of vendors, including Cloudant, Couchbase, DataStax, MarkLogic and Oracle offer NoSQL products.
PostgreSQL is an open source object oriented RDBMS also designed for multiple data types. This runs on numerous operating systems, including Linux, most flavors of UNIX, Apple Mac OS X and Microsoft Windows. Command Prompt Inc., Dalibo, EnterpriseDB, Red Hat and Salesforce support PostgreSQL.
Big data finds big growth
Big data has been one major area of emphasis in the last few years. IDC projects that the big data technology and services market will grow at a compound annual growth rate of 23.1% from 2014 to 2019, when it predicts annual spending will reach $48.6 billion.
Recently, Hadoop emerged as a viable open source big data repository. Cloudera, Hortonworks, MapR Technologies and Splice Machine are some of the suppliers building on that foundation. MemSQL, which was delivered in April 2013, offers businesses another data analytics option. The distributed, in-memory system automatically compiles SQL into machine code.
Another newcomer, NuoDB, was designed to support high transaction volumes. Its SQL database was built to support server clusters, many nodes operating in large data centers. That functionality appealed to the WO On Demand Group, a subset of WideOrbit Inc., a provider of advertising management technology for cable networks, local television stations and radio stations. WO On Demand Group delivers tools that enable clients to manage, monetize and measure the reach of on-demand audio content.
In the fall of 2015, Stephen Jazdzewski, lead developer at On Demand, needed a distributed DBMS that completed updates in real time. After considering leading MySQL DBMS products, Jazdzewski opted for NuoDB. Using NuoDB's software, WO On Demand applications stay up when the company completes maintenance tasks, such as system updates and data backups.
Challenges do arise if a firm opts for one of the fledgling DBMS products. These often lack the amenities found in established systems. Organizations may have to do a lot of manual programming because robust development tools are lacking. They may also have difficulty finding sophisticated programming and management tools.
In addition, the market keeps evolving, so vendors need to update their systems with the latest bells and whistles, such as adding support for containers. Smaller suppliers may have trouble keeping pace with ever changing customer demands.
Since many vendors recently entered the market, questions about eventual market consolidation loom. In some cases, the new players will be purchased by larger suppliers and the technology somehow melded into existing suites. But some products -- and probably a few vendors -- will not survive as the market continues to mature, so risk arises whenever a business bets on a new player.
What was old has become new again. Recent changes have breathed life into tired and stodgy market, and a variety of new DBMS products have emerged. On the plus side, businesses are now in a better position to find a product that meets their application needs. However, questions remain about the long-term future of the nascent startups.
Explore DBMS options before a purchase
What type of RDBMS does your company need?
Manage big data with a Hadoop distribution