As never before, online transaction processing strategies should be high on major organizations' IT priority lists.
Transactional systems were once the key database technology for large enterprises. Now, with big data, online transaction processing
Transactional systems that involve high numbers of updates are ever more critical to keep the business running and to deliver nearly real-time access to vital data, including data that will give a business the competitive edge.
Online transaction processing extends analytics to near-real-time data, especially customer data; supplies a virtual data warehouse with nearer to real time; load-balances updates in master data management and increases flexibility for the enterprise information architecture.
To use OLTP for these new tasks successfully, redefine its place in the enterprise information architecture. OLTP is no longer just a legacy set of applications to handle orders. Analytics are rarely performed simultaneously with the massive stream of updates that the typical OLTP system handles, because supposedly that's not what OLTP is for. Yet data warehouse updates are inherently delayed, preventing near-real-time analytics on fresh data. For organizations to react quickly to changes in their data, IT must accord OLTP a larger role than ever before.
Near-real-time data access
When IT applies data warehousing to OLTP data stores, reporting and analytics occur as the data hits the OLTP system, detecting patterns in the arriving data in nearly real time.
To do this, a query combines data from an operational data store and from the data warehouse. The quickest method is to recruit your enterprise's data virtualization solution to achieve a virtual data warehouse that accesses multiple data stores without differentiating between them to the user. Failing that, there are veneers from database vendors that make several data stores appear as one. The long-term solution for nearly instantaneous analytics is to implement a global metadata store that gathers information from OLTP and the data warehouse -- data virtualization is perfect for this. As new types of data arrive, they are included and handled semi-automatically and cost-effectively.
Master data management
Master data management for customer information, popular in today's large enterprises, offers its own scalability problems. Many implementations seek to maintain one central record -- creating bottlenecks when several operational systems simultaneously update the record.
Distributed, load-balanced updating with multiple record copies, or variants, prevents delays in customer order processing that result from logjams at the central master data server. Edge OLTP data stores enable load balancing, which improves the performance and scalability of distributed copy updates.
While data virtualization makes multiple data copies easier to handle, any solution should be part of a more extensive master data management system that covers global metadata, as well as data governance.
Redesigning the enterprise information architecture
The enterprise information architecture of the past two decades focused on putting as much data in the data warehouse as possible, causing subpar reporting and query performance. OLTP performs reporting and querying, load balancing the data warehouse. If we also throw in big social-media data culled from the Internet, content-rich-media and in-house data, IT can afford far greater flexibility in allocating data to particular stores and management systems and reallocating in real time.
With OLTP, the enterprise information architecture will see better performance. Freed from some of the constraints of a central data warehouse, the architect has greater freedom to put the data, and access to it, in the best place. Add data virtualization, and you can realize more cost-effective and easier data management, lower development costs, and higher visibility for more users.
Juicing up OLTP
IT can add even more scalability to OLTP, with improvements such as shared-disk clustering. Shared-disk clustering operates as if disk (or all storage below the main-memory tier) looks like one giant disk array, shared by multiple physical-processor and main-memory pairs. Practically, software on every physical server coordinates their process scheduling and allocation of the disk "pool." Main memory on each physical server belongs to its own set of processors, and that sets limits for how well shared-disk clustering can scale relative to other architectures, but this inflection point is well beyond the scale of symmetric multiprocessing (SMP), such as that on today's Wintel servers.
Basic clustering doesn't do load balancing between servers. Single system image clustering can approximate load balancing by ignoring the location of software and fine-tuning workload balancing at the single-system level. Recent IBM and Oracle products tuned for transactional workloads don't just load-balance transactional code indirectly; they load-balance like a transaction-processing monitor or some Web app-server software, directly and dynamically.
The result, say IBM and database company Oracle, is near-linear scalability of these clustered systems, or OLTP scalability well beyond the limits of today's SMP nonclustered systems.
Clustering has long been known for adding an extra degree of robustness by performing rapid failover and reboot when a database node fails, in effect "isolating and healing" the node with little effect on ongoing performance.
About the author:
Wayne Kernochan is president of Infostructure Associates, an affiliate of Valley View Ventures, with more than 20 years of experience in IT industry analysis. This document is the result of Infostructure Associates-sponsored research. Infostructure Associates believes that its findings are objective and represent the best analysis available at the time of publication.
This was first published in October 2013