What’s Clerk? Clerk empowers retailers and brands with resources to improve the in-store performance and meet the customers’ needs. This includes: out-of-stock tracking, planogram management, category analysis and planning, or in-store marketing. Clerk’s digital display network, Grocery TV, uses AI-enabled camera technology to verify ad impressions and keep track of products in supermarkets.
Clerk came to KUNGFU.AI with some unique challenges. The Clerk team had already built a planogram verification solution that informed clients about how the paid magazine slots were managed at grocery checkout display racks. A planogram is a schematic tool that retailers use to plan their store layout to maximize sales and customer experience. Typically, planograms highlight product placement, displays, and point of sale locations. However, Clerk’s team was spending too much time and resources on manually detecting and classifying hundreds of thousands of images. Clerk wanted to find a way to be more efficient and label these planograms much faster.
Clerk wanted to be able to automate the labeling process but lacked the proper resources. Their team created an R-CNN model – a two stage detection algorithm. However, in order to fully achieve the goals, KUNGFU.AI chose to use a RetinaNet model, which offers great performance, latency, accuracy, and ability to detect a wide range of object sizes. In addition, KUNGFU.AI implemented best practices which include:
KUNGFU.AI and Clerk also had a series of brainstorming sessions to identify core and supplementary labels that enhance planogram comprehension. The study resulted in identifying six distinct labels, several of which were vital in unlocking performance improvements. One example was an empty magazine rack label, which was necessary for inferring planogram layouts when vacant placements exist.
KUNGFU.AI helped build a hybrid learning solution that pre-populates labels for review, allowing Clerk’s team to focus on reviewing rather than labeling. The solution gets more accurate as users interact with it, meaning the time savings will continue to increase.
KUNGFU.AI applied a vision transformer to Clerk’s labeled data and achieved an overall top-1 accuracy exceeding 80% (a more than 20% improvement on the baseline), with magazine title specific scores exceeding 99% in many cases. The team did this by implementing a vector similarity magazine classifier, where GPU-optimized neighbor searches were performed using an open source library (faiss). Overall, KUNGFU.AI’s work with Clerk has cut the labeling time in half for Clerk’s labeling team.
Upon completing our original project with Clerk, their Chief Technology Officer sent us over some updated results. He added kornia feature matching as a confidence check and it increased their guess confidence.
.png)
To explain this graph a little further,
And with this, every piece of what KUNGFU.AI delivered to Clerk is in their live pipeline!
What’s Clerk? Clerk empowers retailers and brands with resources to improve the in-store performance and meet the customers’ needs. This includes: out-of-stock tracking, planogram management, category analysis and planning, or in-store marketing. Clerk’s digital display network, Grocery TV, uses AI-enabled camera technology to verify ad impressions and keep track of products in supermarkets.
Clerk came to KUNGFU.AI with some unique challenges. The Clerk team had already built a planogram verification solution that informed clients about how the paid magazine slots were managed at grocery checkout display racks. A planogram is a schematic tool that retailers use to plan their store layout to maximize sales and customer experience. Typically, planograms highlight product placement, displays, and point of sale locations. However, Clerk’s team was spending too much time and resources on manually detecting and classifying hundreds of thousands of images. Clerk wanted to find a way to be more efficient and label these planograms much faster.
Clerk wanted to be able to automate the labeling process but lacked the proper resources. Their team created an R-CNN model – a two stage detection algorithm. However, in order to fully achieve the goals, KUNGFU.AI chose to use a RetinaNet model, which offers great performance, latency, accuracy, and ability to detect a wide range of object sizes. In addition, KUNGFU.AI implemented best practices which include:
KUNGFU.AI and Clerk also had a series of brainstorming sessions to identify core and supplementary labels that enhance planogram comprehension. The study resulted in identifying six distinct labels, several of which were vital in unlocking performance improvements. One example was an empty magazine rack label, which was necessary for inferring planogram layouts when vacant placements exist.
KUNGFU.AI helped build a hybrid learning solution that pre-populates labels for review, allowing Clerk’s team to focus on reviewing rather than labeling. The solution gets more accurate as users interact with it, meaning the time savings will continue to increase.
KUNGFU.AI applied a vision transformer to Clerk’s labeled data and achieved an overall top-1 accuracy exceeding 80% (a more than 20% improvement on the baseline), with magazine title specific scores exceeding 99% in many cases. The team did this by implementing a vector similarity magazine classifier, where GPU-optimized neighbor searches were performed using an open source library (faiss). Overall, KUNGFU.AI’s work with Clerk has cut the labeling time in half for Clerk’s labeling team.
Upon completing our original project with Clerk, their Chief Technology Officer sent us over some updated results. He added kornia feature matching as a confidence check and it increased their guess confidence.
To explain this graph a little further,
And with this, every piece of what KUNGFU.AI delivered to Clerk is in their live pipeline!
