In theory, companies are now capable of capturing and ana­lyzing the details of every minute transaction and event that occurs within their walls. Although businesses are being inundated with data, much of it is the wrong data. It’s not timely, and it’s not get­ting to the right end- users. This is perhaps one of the most vexing challenges to “ competing on analytics,” now seen as a key strategy for attaining competitive differ­entiation, and well- document- ed in popular books by indus­try experts such as Tom Davenport of Babson College.

“In the old days, you could more or less rely on your competitors being at about the same level of efficiency, but analytics changes the playing field dramatically,” Joe Pusztai, director of product marketing for Applix, told DBTA. “ Business process automation is important too, but it essentially only enables you to execute strategy; ana­lytics is what enables you to set the strategy in the first place, for example, by detect­ing trends and ‘ seismic shifts’ in your industry early.”

How can such shifts be accu­rately and quickly detected? Some leaders in competing on analytics have employed multi-faceted approaches that leverage a wide range of data sources, and they extend this capability to as many end­users as possible. One such organization, BlueCross BlueShield of Tennessee ( BCBST), offers account reporting to its largest groups, which allows the company to respond more effectively to RFPs to acquire new business and retain existing clients, Frank Brooks, senior manager of data resource management and chief data architect for BCBST, told DBTA. Analytical capabilities cur­rently delivered via the Internet to BCBST clients include utilization manage­ment through interactive reports and OLAP data cubes. BCBST plans to provide additional analytical capabili­ties for its account reporting packages, including national and regional benchmarking data from Blue Health Intelligence ( a national data warehouse of BlueCross BlueShield Plans), Brooks said. “ We’re now in the process of enhancing our busi­ness intelligence and analyti­cal infrastructure to also sup­port instant access to the results of text analytics and predictive analytics process­ing.” BCBST is taking a multi­pronged approach involving traditional business intelli­gence tools, as well as data

mining, text analytics, and enterprise search to sift through a variety of com­pany data sources to spot trends and pat­terns in service, claims, and utilization.

Clearly, the industry is moving into a new generation of tools that focus more on real-time delivery of operational data, as well as extending reporting capabilities to corporate performance management dashboard systems to pro­vide a picture of the health of the busi­ness.

However, many companies are inun­dated with data, and are still mired in earlier generations of query and report­ing products. “Most organizations are barely at the toddler stage when it comes to analytics,” Eric Blankenburg, vice president of application and integration solutions at Avanade, told DBTA. “We are drowning in information. It’s past the point where it is even possible for us to interpret the data and make reasoned decisions without some significant level of analytical support.”

A recent survey of 296 data applica­tions managers, conducted by Unisphere Research for the Oracle Applications Users Group (OAUG) in partnership with Cognos, found that a paradox exists in most organizations today. Decision- makers are over­whelmed by information overload, but at the same time, there isn’t enough of the right information available. The study found that 91 percent of companies said that their decision-making capabilities were stymied by a lack of complete information. Yet, three out of four also report they suffer from ‘information overload.’ Identifying and separating out the pieces of data that have the most value may be like looking for a particular piece of straw in a haystack. Add to this the fact that most end-users do not have access to the latest BI tools, and still have to go through IT or other depart­ments. The majority of respondents to the OAUG survey, in fact, report that it takes more than three to five days to get a report out of IT. Overall, the survey found, fewer than 10 percent of employ­ees have access to BI and corporate per­formance management tools.

“We’re still only touching the surface of business intelligence,” Marc Andrews, director of strategy and busi­ness development for unstructured information at IBM, told DBTA. “The number of business processes and the number of users across the organization that are leveraging the technologies is still only the fraction of the population potential.”

Other industry experts strongly agree that BI has not proliferated as thorough­ly as it should. “Companies have yet to find an effective way to deliver BI capa­bilities to more than a handful of ‘power users’ who have the technical expertise to leverage BI tools,” Mark Lorion, director of product marketing for the Spotfire Division of TIBCO, told DBTA. “Instead, their employees are using spreadsheets and other packaged applications because the BI platforms are not flexible enough to suit their analysis needs or pace. BI tools fre­quently are not intuitive, and require heavy IT involvement to reconfigure cubes or generate new reports. Because they require IT involvement, they do not work at the speed of front­line decision-makers.”

How does a company leverage such overwhelming data stores and learn how to compete on analytics? To suc­cessfully compete on analytics, compa­nies need to embed analytic functional­ity in every mission-critical application across the enterprise, IBM’s Andrews pointed out. “Most companies are using BI for traditional querying and report­ing, not for real-time operational busi­ness intelligence. They’re not using it as part of their business applications – as part of processing a claim, as part of helping a customer resolve a problem, or as part of processing a transaction. The future is enabling people to access business intelligence within a call cen­ter application – not as a standalone application that they have to go to for querying and reporting.”

Data quality also takes on greater urgency as companies turn on opera­tional analytics. Mary Crissey, analytics marketing manager at SAS Institute, sounded a note of caution that many companies may rush too fast to rely on real-time or near real-time data without vetting it for accuracy or timeliness. “With business intelligence, there’s a data integration piece, which involves the storage and cleansing of the data. We’re all putting data together from dif­ferent sources – some people are key­stroking it in, some people are collect­ing it in from the Web, some people are getting it over the phone. You get all this data coming in, lots of times, different formats, and you have to merge it all together. Cleansing of that data real­time is critical.”

Eventually, prices of sophisticated analytical tools – still out of the reach of many companies – may begin to come down as capabilities become more widespread. This will dramatically improve the availability of such tools and capabilities. “There has not been enough innovation in the BI industry for years,” agreed Scott Yara, co­founder and president of Greenplum, who posited that more powerful com­modity systems and open source soft­ware are poised to disrupt the entire BI industry. “Only very recently has it become possible to buy a high-perform­ance database for large-scale BI for under $1 million per terabyte. By com­parison, you can go to the store today and buy a terabyte of storage for well under a thousand dollars. It’s the cost and performance of the traditional solu­tions that have made it difficult for companies to adopt BI to analyze all ­or any significant portion – of their data.”

In the meantime, data managers need to sharpen their selling skills to cost-justi­fy BI expenditures to skeptical corpo­rate management. Demonstrating ROI on new BI technologies was the greatest challenge for BCBST, Brooks related. “Our biggest issue is the justification of new technology where the value cannot be easily quantified,” he explained. “ Unlike operational systems where projects or enhancements provide cost reductions in the form of increased effi­ciency and productivity, information management infrastructure enhance­ments often enable a more effective organization where cost savings or increased revenue are difficult to corre­late.” Many of the potential uses for enterprise data warehouses, for exam­ple, “are difficult to forecast a return on investment,” he said.

This requires greater understanding and education provided to the business as a whole. “Analytics at the strategic and competitive level of decision-mak­ing in enterprises is typically under­resourced, misunderstood, and doesn’t lend itself as well to digital solutions as the kind of tactical and day-to-day deci­sions that analytics, BI and knowledge management solutions are most com­monly applied to,’ observed Craig Fleisher, co-author of Business and Competitive Analysis (Financial Times Press) and professor at the University of Windsor. “ Many companies are, to some degree, competing on analytics, but the bigger issue is to what degree are they competing on analytics? Since the field of analytics, particularly whereby databases, systems, solutions and applications are concerned, is still in its early stages, companies are in var­ious stages of moving up their analytics learning curves,” he told DBTA.

Ultimately, Pusztai observed, the best pitch for greater analytics comes from the late management thinker Peter Drucker, who said, “We have to stop counting and start measuring.” “This means that many business analysts out there are actually ‘counting,’ not analyz­ing. BI and analytics puts an enormous amount of power into people’s hands, but they have to learn how to leverage it better,” Pusztai said.

Overall, fewer than 10 per­cent of employees have access to BI tools.

It’s now been several years since the most significant com­pliance mandates swept through U.S.-based businesses, up-ending long baked-in rou­tines and causing untold num­bers of sleepless nights.

The 800-pound gorilla of mandates, Sarbanes-Oxley, is marking its fifth year of existence, and it’s been eight years since the Financial Modernization Act (Gramm-Leach-Bliley) arrived. It’s been more than 11 years since the Health Insurance Portability and Accountability Act (HIPAA) was passed and put into action. Most organiza­tions and the auditors charged with policing these mandates have had some time and experi­ence, then, to develop or recog­nize best practices in data man­agement, security, and account­ability.

Probably the most potent response to date has been evolving under the banner of governance, risk and compli­ance (GRC) management. With GRC, these distinct categories are taken together as one, with the goal of transforming bur­densome information sharing and reporting processes. Effective compliance with reg­ulations such as Sarbanes­Oxley requires a governance structure that incorporates input from various parts of the enterprise, with the ability to recognize the risks inherent in failing to establish proper con­trols over information that is reported.

Governance, Risk and Compliance

Awareness of GRC runs high, a recent survey 392 members of the Oracle Applications Users Group (OAUG), in partnership with Unisphere Research and LogicalApps, found. GRC has particularly gained traction among larger firms, in which many are working proactively to improve the effectiveness of their compliance management and risk mitigation efforts.

Information technology and data management play a critical role in this process, the survey also confirmed. Four out of 10 companies reported making headway with automating and providing continuous monitor­ing of their internal controls environments, but there is still much work to be done. Only a handful, 15 percent, said that the majority of their critical processes are automated, mean­ing their internal controls envi­ronments are well-documented and continuously and automati­cally enforced to the point where violations are immedi­ately caught and remedied. Another 42 percent said that while they have well- docu­mented controls environments, these controls are subject to regular evaluations, with ensu­ing remediation cycles to address whatever issues may be identified. Another 30 percent said controls and enforcement are erratic at best.

Lately, there’s been a change in thinking and tactics as to what, exactly, needs to have such con­trols. In the early days of the compli­ance era, companies tended to focus on application- level controls. However, after attempts to police access to poten­tially hundreds of different applications, many companies recognized that they needed to drill down to a more funda­mental level – to the database itself. “People have gotten over the initial hump of looking at internal controls at the application level; now there’s a broader view that the foundation of compliance needs to address the data repositories that contain the application data,” Harald Collet, director of risk assurance solutions at Oracle, told DBTA. “There was this false sense of security that this was taken care of in the database environment. Now, internal auditors are finding deficiencies in the data management processes.”

The ability to manage user access, protect data and monitor transactions is key to compliance efforts. Technology needs to be employed to better automate compliance processes, as well as enforce controls. GRC is “much broad­er than Sarbanes-Oxley. SOX has creat­ed an environment where people start to evaluate risks much more rigorously within their company,” said Collet. “Customers are starting to get hold of their internal controls process, and are starting to look at all their controls.”

Trust but Verify

Other industry observers agree that there has been a shift in auditing from applications to databases themselves. “The impact of compliance mandates on the area of database auditing has been significant – where there was none, now the effort to monitor and manage user activity on the database is real,” Murray Mazer, co-founder and vice president of Lumigent Technologies, told DBTA. “Previously, most organiza­tions did not have auditing controls in place that allowed them to validate the actions of privileged users. Many organizations employed a ‘trust my users’ approach when really a ‘trust but verify’ approach was required.”

Such was the challenge at Coldwater Creek, which needed to provide data­base access to a widely fluctuating sea­sonal workforce. “The first kind of compliance challenge ends up being, how do you manage requests for user accounts, and how do you authorize access and things of that nature?” Michael Carper, vice president of tech­nology operations for Coldwater Creek, said in an interview with DBTA. “Although we do have a formal process for requesting user accounts, it’s not automated, so it’s prone to human error. We’re going to be tightening that fairly soon, but that’s been kind of a chal­lenge.” Carper said that this is especial­ly important to Coldwater Creek, since its staff size can double during the fourth quarter to handle holiday sales.

To a large degree, mandates such as Sarbanes-Oxley have benefited compa­nies in ways beyond simply meeting the letter of the law. Effective governance, in which professionals from various functions are brought together through oversight committees, regular meetings, and project initiatives not only serve to better plan and manage compliance activities, but also build more bridges between formerly siloed departments.

“SOX forced everybody to follow procedure,” remarked Arup Nanda, director of database engineering and architecture for Starwood Hotels & Resorts Worldwide, which maintains more than 500 Oracle 10g databases at sites across the globe. “Three years ago, we had no single data management strategy,” he told DBTA. “Each busi­ness area or functional area had their own database or their own DBA staff, or application DBAs. With SOX, we com­bined them into a single DBA group and architecture group to develop a sin­gle strategy and a single data model. I also got budget to buy additional hard­ware, and hire additional bodies. Our organization as a whole benefited from SOX.”

Phil Neray, vice president at Guardium, noted that “ SOX gave organizations the board-level visibility ­and IT budgets – they needed to imple­ment best practices controls around cor­porate financial data, such as imple­menting real-time security and auto­mated monitoring of privileged users to prevent unauthorized changes.” Such standards and controls are being extended to other areas where sensitive data is managed, he told DBTA.

Where’s the Data?

The first step in this engagement is to develop a data map to pinpoint where the most sensitive business data resides, Collet explained. “Where does it sit? What repositories contain this data? Then they start looking at what kinds of preventive controls, and what kinds of detective controls can they put in place on those data repositories. That means, for example, making sure you can pre­vent DBAs and super users from modi­fying critical data, implementing sepa­ration of duties for administrators and detect inappropriate behavior.”

“Compliance mandates can be divid­ed into two parts: the ‘do this’ part and the ‘or else’ part,” Jim Doherty, chief marketing officer of CipherOptics, told DBTA. “The ‘or else’ part changes based on regulation and industry, and over time, has become more and more demanding. However, the ‘do this’ part is pretty consistent across all regula­tions and industries: control your data. Make sure that the only people who see your data are the ones who are sup­posed to see it. If you can control your data end to end then you will be com­pliant.”

Less Firefighting

Coldwater Creek is rolling out BMC’s Identity Management Suite as a means to automate account management, and therefore address SOX-related compli­ance requirements. “Our goal is to spend less time on unplanned work, or firefighting,” said Carper. “If we spend so much time firefighting, we’re absolutely not able to build a new devel­opment environment for SAP or other development or business projects.”

Automation will reduce this fire­fighting, especially when it comes to compliance, Carper continued. “Maintaining our position with regards to compliance and being able to get through an audit cleanly and quickly decreases the amount of unplanned work that we do, and therefore gives us more time to be working on more inno­vative projects for the business.” Carper noted that over the summer, “we will roll out BMC’s identity manage­ment suite, concurrent with our imple­mentation of SAP for HR and finance. With the implementation of those prod­ucts, we will completely have automat­ed the tasks that in the first quarter took up 70 percent of one person’s time.”

The Unisphere Research-OAUG sur­vey found that on a monthly basis, most organizations commit 30 hours or more in documenting, testing, or reporting on internal controls. At least 18 percent reported that they commit at least half a week (20 hours) or more in staff time for the effort. Another 15 percent reported spending 10 to 20 hours of staff time each month. Another 29 per­cent said they simply did not know what type of time investment was involved.

“People are still using the same tools to manage the data itself, but they’ve gotten smarter about how they manage the audit data,” said Neray. “Instead of manually examining reams of tradition­al log data, many are leveraging automation and data mining techniques to identify unauthorized or suspicious access to sensitive databases. People are also more aware of data governance issues such as ‘Where is my sensitive data stored?’ and are looking for tools that can help them find sensitive data in their environments, especially after mergers and acquisitions or if they’re still using legacy systems.”

Hundreds of Repositories

Ultimately, GRC and compliance automation helps systemize compliance management while reducing the com­plexity built up over the years in many companies’ systems.

“For you to put proper internal con­trols in place that are not manual, it forces you to take a view on your sys­tems,” said Collet. “You need to look at how you can streamline. You might ask, ‘Why do I have 25 different content repositories? Why do I have 10 different business identities? Why do I have hun­dreds and hundreds of data reposito­ries? Can I consolidate, can I take all that and have a single source of truth?’ That in itself, weeding complexity out of the IT systems, means that less of your IT budget is going to be tied up in non-discretionary spending. You’ll have more discretionary spending to make the IT department more agile, so you can effect more change with the same budget.”

Automation of compliance reporting through GRC management represents a great untapped opportunity. For Coldwater Creek, such efforts represent the beginning of new opportunities, Carper said. “With the kind of automa­tion we’ll be doing, we won’t spend so much time on unplanned or mundane work. We’ll be spending more time on innovation.”

There’s no doubt that open source’s time has come. Enterprises are openly embracing open source solutions up and down the stack, to the point where it has become the “ new normal.” But what’s the true cost of this kind of software?

A Unisphere Research study of 434 companies, commissioned by IBM last fall, confirmed that Linux had clearly become an enterprise-class operating system for supporting mission-critical applications, such as ERP. Other open source solutions are popular within the surveyed enterprises as well – however, while many compa­nies are or will soon be running mission-critical enterprise systems such as ERP on Linux, the applications themselves are still dominated by commercial vendors.

A new survey of 500 executives released by Unisys Corp. says this may be chang­ing as well. More than half of the respondents – 58 percent – stated that they now use open source software for mission-critical applications. More than 79 percent report­ed using open source in the application infrastructure – databases, Web servers and application servers – that provides the underpinning for mission-critical applications. For many enterprises, the value proposition of open source seems to be about cost savings. About 77 percent of the study respondents called open source important or very important for improving IT efficiency and delivering more with less.

However, the economics of open source are a tricky proposition, and one that companies and IT professionals are just beginning to understand. Of course, there already have been raucous industry debates about open source TCO in recent years. There are some estimates that staff skills and maintenance costs – which form the bulk of IT costs – are the same for open source systems as they are for their com­mercial counterparts.

In our own survey work for Unisphere Research, we found adoption of open source in many parts of the stack – except databases. Our statistics from across var­ious user groups finds leading open source DB products such as MySQL firmly embedded in about a third of enterprises, but showing no further signs of growth. That’s because the level and costs of skills needed to manage an open source data­base amount to the same as for commercial databases, and companies we’ve spoken with prefer to remain with their commercial vendors.

The open source industry is actually built upon two foundations at this point: communities of volunteers, and developers working within supporting organiza­tions. Dirk Riehle, who leads the open source research group at SAP Research, recently published a paper that takes a hard look at the impact of open source on developers’ market rates, including professionals that work for vendors, systems integrators, and end-user companies. While many open source solutions have been built and are maintained by volunteers, there’s also an impressive base of develop­ers who contribute time to open source projects on company time (and are encour­aged to do so). Riehle concludes that such “committers” are likely to be perceived by their employers as having more value, as well as having skills that are in greater demand in the marketplace.

Riehle observes that “a developer who chooses the right project can gain and maintain a position that will increase salary-negotiation power and job prospects. The developer will enjoy those benefits as long as the project is of significance to potential employers.”
In addition, Riehle writes, “open source reinforces the trend toward employees becoming ‘free agents,’ ” adding that “committers who rationally follow their eco­nomic interests are likely to be more loyal to the open source project than to their current employer because that’s where their market value lies.” However, attaining “committer” status to the point where companies will fund your time requires a prominent role in an open source project.

Still, the move to open source represents a substantial shift in where corporate IT dollars are being spent. At one time, software licenses were a big part of the pie – and still are. But with open source, as well as software as a service deliv­ered on a metered basis, software has become as commoditized as the air around us. IT suppliers are now scrambling to beef up their service and support rev­enues as the value- add.

There haven’t been any studies that I’m aware of that have measured the costs to end-user companies for supporting open source committers. While this is likely a far cry from the costs of maintaining commercial software licenses, there is still cost that is being absorbed in terms of compensation and support. And, to an extent, since code is contributed back to the communities, companies are supporting devel­opment that eventually benefits other companies, even competitors.

The bottom line is that the advantage of open source solutions has little to do with the fact that this software is “ free.” The advantage is in the robustness and flexibility of the software, and the ability of the community ( or supporting ven­dor) to provide support on a timely basis. And, as is the case with commercial products, companies bear some risk that open source solutions will lose support in the marketplace.