Beyond AI snake oil: Red flags in vendor claims

Armen Ghambaryan, Ph.D.

Lead Deep Learning Engineer

Introduction

Despite industry consolidation, there are still many vendors offering AI powered video surveillance solutions for the security industry. Not all AI, and solutions, are created equal, and it is important to be able to distinguish between valuable solutions and worthless AI snake oil.

This quick guide is to help educate security executives so that they can evaluate vendors and make the right purchasing decision.

Understanding the technology

Video surveillance is well known, but the amount of data in large camera networks can easily overwhelm human operators. Studies show that operators can start missing events after only 20 minutes.

AI powered video surveillance can evaluate multiple video feeds in real time to identify critical events of interest such as shoplifting, trespassing, fighting, trips and falls, drawn guns and other weapons and fires.

This works as a filter, automatically ignoring routine activity but highlighting unusual activity for human evaluation. In practice, AI can remove over 99% of all false positives, letting operators focus on actionable events.

AI is always going to let through some “false positives”. There is always a trade-off between showing borderline events and incorrectly suppressing something that might be critical (false negative). AI is designed to work with and complement human security officers, who should always make the final decision.

AI is a maturing technology and some marginal vendors are still in the market. As such, there are some vendors still making unsupported claims. It is important to look for good AI, not snake oil AI.

Good AI reduces false positives significantly while maintaining high threat detection (low false negatives) and augments human decision making rather than claiming to replace it.

It acknowledges uncertainty and flags edge cases and works in complex environments rather than claiming “one size fits all” ability to understand context.

Red flags in vendor claims

This section highlights some red flags - beware if a vendor makes any of these claims:

“Our AI has no false positives” ● This is mathematically dubious, given the nature of machine learning (which is probabilistic) ● What is more important is false negatives (missed critical events) ● A claim of no false positives means that the vendor is likely misrepresenting the performance of their product.

“Our AI understands context perfectly” ● Context is multidimensional and ambiguous. ● Even humans (the ultimate decision makers) can easily misinterpret context. ● Don’t use a product that uses “context” as a filter.

No published accuracy metrics or methodology ● An honest vendor will have measured accuracy in different scenarios or worked with a third party to do this. ● They will also be able to explain their methodology / testing protocols. ● Look for metrics like false positives, false negatives, sample size.

Only lab results, no real world tests ● Real world conditions include suboptimal lighting, weather, contrast, camera angles and distance from the camera. ● This is very different from lab results with perfect lighting and camera placement, indoors

Absolute language without caveats ● Be skeptical of terms such as “always”, “never”, “perfect”, “eliminates”. ● Good vendors will explain limitations.

Powered by ChatGPT or similar ● ChatGPT and other LLMs are not designed for high criticality security environments ● Look for a vendor that has their own models and training data, not one that tries to take shortcuts with LLMs.

More focus on marketing than on AI development ● Look at the size of the marketing team vs AI developers. ● A marketing driven organisation means that there is a low focus on improving the product.

“Secret” or very unusual approaches ● There are no “secret” approaches (such as trying to understand context). ● Most mainstream vendors use similar approaches. ● The differentiator is in execution and AI training, not on some proprietary secret new approach.

Questions for vendors

This section is a practical framework so that you can ask vendors the hard questions:

Technical due diligence ● What is the tradeoff between false positives and false negatives in your system? ● Can you show us your ROC curves or precision-recall curves? ● What scenarios does your AI struggle with? ● How does your solution work with outdoors cameras and in different weather conditions? ● How do you define and measure “context” ● What happens when your AI is uncertain? ● Do you have your own training data and models, or are you using an LLM? ● What third party integration options do you provide? ● What other complementary products do you provide (so that you can grow into other solutions)? ● What issues are there with scaling to use many more cameras? ● What other deployment options do you provide (look for on premises, cloud, edge devices)? ● Can we see case studies with actual numbers, not just testimonials?

Business due diligence ● How long have you been deploying at this scale? ● What is your customer retention rate? ● How can I scale usage up and down? ● What is your product roadmap and how do you plan to deliver this? ● Can we speak to customers who have been using this for 2+ years? ● What is your company’s financial stability?

Conclusion

When choosing a video surveillance solution, it is important to choose a partner, not just a product

A good partner values transparency and accurately represents the capabilities of their product’s AI. Their product integrates with standard third party solutions and offers multiple deployment options and scalability.

If a claim seems impossible, it probably is. Exercise caution when you see any of the red flags above or when a vendor can’t answer technical or business due diligence questions.

It is vital to be able to cut through marketing hype and AI snake oil to understand the real world performance of a solution. A good vendor will have existing reference sites as well as published performance statistics.

About Scylla

Scylla’s mission is to use advanced technology to protect human life across the globe.

Scylla’s proven AI-powered security technology powers leading Security Operations Centres and is trusted to safeguard public infrastructure, military installations, schools, retailers and other critical sites.