July 12, 2016
These four innovations can help companies achieve an optimal mind-machine balance when it business decision making.
As organizations strive to find the right mind-machine balance, human judgment still plays a big role in decision-making. In fact, according to PwC’s Global Data and Analytics Survey 2016: Big Decisions™ only 39% of respondents say their organizations are highly data-driven. Executives often make critical decisions based on insights from limited data sets and predetermined analytics rules—overlooking important correlations and context that could be vital to shaping questions and making informed decisions.
The emergence of machine learning is ushering in a new wave of intelligent, context-aware data and analytics technologies that identify correlations across numerous data sets in seconds—and provide “smart” recommendations that become increasingly precise over time. Executives gain the peripheral vision that empowers them to make better decisions and avoid costly mistakes.
A major retailer with hundreds of physical stores and a major online presence noticed a massive spike in web traffic on a weekday. Traffic soared from the typical five million visitors to ten million. A senior executive displayed this statistic in a presentation to the board—assuming that recent new web features were attracting millions of new potential customers.
During the presentation, a board member recalled that on that particular day, many stores faced stock-outs due to a holiday promotion. The board member speculated that customers who couldn’t find the on-sale items in stores turned to the web site to try and redeem their coupons.
The senior executive’s success story quickly turned into a red-faced argument in self-defense.
This new breed of ‘smart’ data and analytics technologies could rapidly analyze the retailer’s online and offline data and uncover correlations between the holiday promotion, inventory stock-outs, and the spike in web traffic. The intelligent technologies also provide recommendations that help executives turn data-driven insights into action.
These innovations can enhance productivity, increase accuracy, speed and sophistication of decisions, and reduce errors. Here are four areas to watch:
1. Data Ingestion
Most data ingestion technologies are cumbersome, expensive, and unable to manage complex data in different formats, speeds, and sizes. We’ll start to see the emergence of automated, smart data ingestion technologies. Like a smart water valve, these technologies will self-control and guide data from source to destination. They will also manage network congestion and make intelligent real-time recommendations based on variables like frequency of use, user, location, data values, server utilization, energy use, business issue, costs, and desired outcomes.
2. Data modeling and integration
Rigid enterprise data models, hard-coded integration rules, and the poor interpretation of data can lead to billions in losses. Data integration software with embedded machine learning capabilities are on the rise. These tools can analyze data sets and recommend how to tag and structure data and write business logic. It’s not unrealistic to think about an advanced, holistic end-to-end intelligent data modeling and integration assistant in the new smart data integration era.
3. Intelligent workload management assistant
As data grows exponentially and high-performance cluster computing becomes the norm—it will become critical to enable applications that can solve complex data and analytics challenges and run concurrent workloads of varying frequency (batch, micro batch and real time) with different computing engines (disk, in-memory streaming, etc.). An intelligent ‘workload management’ assistant that can adapt to information flow, infrastructure, and computing workloads in real-time—and recalibrate performance to overcome bottlenecks, will be a boon to large enterprises that are investing millions to manage complex data and analytics infrastructure environments.
4. Mind-machine balance
Solving analytical problems requires the right balance of human judgment, data, and business skills—made more difficult by vast differences in knowledge among those conducting the analysis. Executives may end up with solid answers—but to the wrong questions—which can derail business decisions. Intelligent, context-aware technologies are leveling the playing field by exposing the intricacies of relationships between the data sets, and guiding data scientists to ask the right questions while developing models and visualizations. These tools recommend a “base” analytics model, or visualizations, from predefined libraries based on the specific problem domain and relevant data sets. This enables data scientists to augment their skills and capabilities with automated intelligence to accelerate the analytics and insights process.
A thermostat’s value is in its ability to self-adjust to the temperature in the room. Increasingly we’ll see data and analytics technologies becoming more intelligent and self-aware—and the balance between mind and machine will become stronger. If you are new to machine learning, Anand Rao’s series is a must read. If you are among those already assessing or using these technologies, we’d like to hear about your experiences.