The capabilities and limitations of video analytics

June 1, 2015



Video analytics promise to help retailers better understand customers. Here are three issues to keep in mind.

In part one of this series on sensors and analytics in retail, we explored how retailers can use beacon technologies to create a customized shopping experience and gather rich data about a shopper’s habits and interests. Another powerful tool in understanding the customer journey through a store is video analytics.

While video as a technology is nothing new, the maturity of computer vision algorithms has enabled automated tracking of objects appearing within a video. With the application of parallel processing platforms such as Hadoop, these process intensive tasks can be performed at scale. Combined, these tools empower retailers with an understanding of the types of customers who are entering their stores, their precise movements and how they can direct their attention. Below are just a few of the capabilities retailers could utilize with video analytics:

  • Traffic flow: Define virtual tripwires to understand conversion rates from sidewalk to store or whether the majority of customers turn left or right upon entering a store
  • Dwell times: Determine the effectiveness of an advertisement or endcap display by tracking what percentage of customers stop to notice the ad and what percentage do not
  • Demographics: Understand the age and gender breakdown of customers entering a store
  • Heat maps: Develop a visual representation of the activity within a store to optimize store layout and the sale of high margin items
  • Queue analysis: Determine relative queue size to optimize staffing for both normal and peak shopping periods.
  • Security and safety: Use machine learning algorithms to automatically detect suspicious or out of the ordinary behavior in real time.

Video analytics implementation considerations

PwC’s Emerging Technology Lab developed a simulated shopping environment to understand the capabilities and limitations of video analytics. Below are some of our lessons learned:

Calibration: Video analytics rarely perform well out of the box. Environmental conditions such as variable light conditions from an open window, reflective surfaces, shadows and the relative distance and angle of view must all be carefully considered to reliably and consistently identify a customer. As a result each camera requires a period of configuration and testing, which must be repeated if the camera is moved or adjusted. In particular, anomaly detection only works after running multiple-week baseline training periods for the system to learn normal behaviors.

Data accuracy: During our tests of traffic and dwell counts, we found that vendor results produced a high degree of precision but were only able to accurately identify people 70 – 80% of the time after proper calibration. Based on these results, we recommend the use of these metrics to identify relative trends and patterns but not for exact counts. For demographics identification, we found that gender detection was highly accurate when provided a clear view of the face while age detection performed poorly, often producing different results for the same person. It seems age detection is particularly susceptible to slight deviations in lighting conditions and viewing angles. Unless you can expect a photo booth-like environment, we wouldn’t rely on it.

Data access: Each of the vendors we tested provided access to the data in several formats including online dashboards, configurable reports and downloadable media such as activity heat maps, video clips, and video snapshots for scene analysis. The tools to define and configure view areas of interest took some level of expertise to do properly as there were many configuration options that are not intuitively obvious without some level of training. The vendors also provided API access but this seemed much less developed than their dashboard interfaces. Our experience with the APIs included frequent stability issues, outdated or missing documentation and a largely reduced set of functions as compared to the dashboards. These problems may be resolved as these young companies mature, but at this point we suggest relying on the provided vendor dashboards as opposed to the APIs.

As the Internet of Things (IoT) movement continues to extend the digital domain into the physical world, brick and mortar retailers have an opportunity to reclaim the digital high ground. By leveraging emerging sensor platforms and advanced analytics they can effectively optimize the customer journey and create customized experiences tailored to the needs and desires of each individual. Let us know how you are using sensors and analytics to transform your retail environment.

Curious to see how these technologies could be put into practice? Take a look at “Symphoni”, which imagines how retailers and consumer packaged goods companies might orchestrate activities in the physical world by deploying smart sensors to transmit up-to-the minute information.

Niko Pipaloff contributed to this post.



Chris Curran

Principal and Chief Technologist, PwC US Tel: +1 (214) 754 5055 Email

Vicki Huff Eckert

Global New Business & Innovation Leader Tel: +1 (650) 387 4956 Email

Mark McCaffery

US Technology, Media and Telecommunications (TMT) Leader Tel: +1 (408) 817 4199 Email