WESEE – DETECTING SHOPLIFTING THANKS TO ARTIFICIAL INTELLIGENCE

About the product

WeSee is a product imagined and developed by ARHS Spikeseed. When elaborating the solution, ARHS Spikeseed teams had to go through several steps:

• Create custom datasets targeting specific behaviors

• Train tailor-made ML models to detect desired behaviors

• Install, configure, and operate IoT devices at client premises

• Design and implement the global platform, ranging from the development of the mobile application to the automated remote deployment of the solution running on IoT device.

About the customer

Our typical customers are leaders in the retail industry, such as stores and hypermarkets, who are victims of recurrent theft activities. By leveraging WeSee, our customers want to reduce shoplifting by providing a reliable and cost-effective supporting tool to their human workforce:

Security teams are automatically altered in case a suspicious activity is detected. A video recording is immediately made available via the dedicated mobile application, supporting their interventions.

The challenges

Our teams faced and went over multiple challenges from the elaboration until the rollout of the product:

• Creating custom datasets specific to shoplifting use cases is challenging because video surveillance images and videos are not publicly available

• Analyzing in real-time massive streams from +100 cameras requires an efficient hardware scaling, but also intensive optimization of the application

• Reidentify people across different cameras and camera angles is a difficult task that needs to implement state-of-the-art research

Project Solution

When designing the solution, ARHS Spikeseed teams evaluated the AWS services portfolio, and identified the following building blocks:

• 01.

  The deployment and management of IoT devices leverages AWS Greengrass

• 02.

  The database layer features AWS DynamoDB

• 03.

  AWS CloudWatch monitors model training and IoT device activity

• 04.

  Custom image classifier models are trained via AWS EC2 instances

• 05.

  Images, recordings of suspicious activities and Machine Learning models are stored on AWS S3

• 06.

  The WeSee altering system is implemented thanks to AWS SNS

• 07.

  Machine Learning models are created thanks to PyTorch framework, either from scratch or leveraging transfer learning, and their generation is boosted via TensorRT

• 08.

  Custom WeSee hardware decodes RTSP streams

• 09.

  OpenCV and PyTorch retrieve the video streams, then resize, convert and normalize the frames

Project Results

As a result, the WeSee computer platform provides retailers with on-the-shelve tools to automatically detect shoplifting:

1. Dedicated mobile application, available on Google Play and Apple Store
2. Automated installation and configuration of WeSee IoT devices, connected to existing retailer’s camera
3. Remote deployment and update of IoT software
4. Configuration of retailer opening hours for the edge running applications, optimizing the cost of the solution, and preventing false-positive detections
5. Remote monitoring of model performances
6. Remote maintenance of IoT devices
7. Deployment in various environments

Success factors

ARHS Spikeseed extensive experience in developing Cloud native and state-of-the-art Artificial Intelligence solutions let to develop an efficient and low-cost solution adapted to its clients in the retail market. It is clear the WeSee platform helps to reduce the costs related to theft.

What the customer says...

The WeSee platform is efficient and provides a real assistance to our detectives. It also allows us to improve and optimise our internal practices.