Posts in category: Blog

High performance biometric data provides the knowledge required to manage and control the movement of individuals and people in medical and environmental crises.

The COVID-19 pandemic has demonstrated that face recognition can play a key role in stopping the spread of epidemics in cities and large enterprises, such as commercial areas and industrial facilities. The technology has shown a great deal of effectiveness in identifying those who violate quarantines, essential to preventing the spread of the virus, while tracking their social interactions and providing notifications to respective authorities. This identifying and flagging has undoubtedly prevented infections and saved lives.

The main challenge faced by every country exposed to medical crises is the sudden surge of infected people placing immense pressure on the healthcare system and risking total shutdowns of over-burdened infrastructure. Being able to set up an intelligent surveillance system that decreases manpower and person-to-person contact requirements is crucial in fighting infection rates.

Face recognition technology offers an unprecedented capability to cities, and local authorities to ensure quarantine is maintained, and infection spread curtailed, by employing face recognition along with a number of associated technologies. CCTV cameras detect and identify people in the streets in real time allowing their identification for an immediate relevant response, while AI analyses social connections.

Face recognition software is the only system in the world that can reliably track and trace contact’s made by infected persons. Apps that use the geolocation and Bluetooth functions have many flaws, as the geolocation function can be extremely inaccurate, Bluetooth function can be turned off, and in many cases mobile phones are shared by multiple users or simply left at home. Our system has a social connections analysis feature that can precisely detect contacts between individuals of less than two metres. This function alone can sufficiently reduce the number of people put under the stresses of quarantine and medical examination, as it can effectively illuminate their chances of being contaminated when proximate to someone who is known to have it.

The convergence of massive volumes and variety of images with advances in computer vision software made affordable for cities to deploy video intelligence capabilities on a variety of architectures, from core datacenters, to cloud to embedding computer vision in edge computing. As a result, cities have been able to expand the public safety use cases, in which they can surveil, detect, recognize people, objects and events, interpret patterns, and empower better decisions with high accuracy and speed. These use cases include crowd monitoring, searching for criminals, identifying missing people in case of emergencies, improving access control, enhancing physical security in schools, hospitals, airports, and sport arenas, and, of course in the COVID-19 aftermath, monitoring behavior that could increase the risk of spreading of viruses.

Video surveillance systems have ceased to be just cameras monitored by security officers making sure that someone who ‘forgot’ to pay does not take something out of the store or a negligent cashier does not cheat the customer. Today, these platforms use big data and machine learning, neural networks, and much more.

What is video analytics? It’s far more than shoplifting prevention

Video analytics today is a complex layer of technologies based on computer vision and machine learning. And this technology is becoming increasingly commonplace. For instance, four out of every ten (41%) of England-based medium and large-sized businesses which are running CCTV systems have already deployed facial recognition analytics in their systems to capture human faces and compare images to databases with a view to identifying matches for access control, event security or for public safety purposes.

Retail is one of the business fields that makes successful use of video analytics systems and you can argue with conviction needs it the most. The pace of adoption in retail allows us to say with confidence that video analytics will become an integral part of our life and are likely to be ubiquitous within three or four years.

Many retail uses, many benefits

The main use of video analytics in retail is to combat loss and theft. In the US, retailers’ losses due to theft, fraud, and other causes totalled nearly $62 billion in 2019, up from nearly $51 billion the previous year said the National Retail Federation, the world’s largest retail trade association.

Video analytics helps not only to register a deliberate theft but also to identify a forgetful buyer, when a person accidentally passes by the cash register without paying for a purchase. A security officer stops him at the exit and asks him to pay. At this point, the forgetful customer gets watchlisted, this is a basic function in some video analytics systems. When this customer comes to the store again, security receives an alert monitors the customer closely.

The use of video analytics with overhead CCTV observation of the sales counter can be a real-time deterrent to incidents of internal shrink according to the National Retail Federation. Video analytics is the capability of automatically analyzing video to detect and determine if an anomaly has taken place based on a set of instructions built into the video software. However, its use can go far beyond identifying theft.

Keep customers coming back

For instance, modern algorithms can identify someone who forgot something in the store. The system sees when a customer enters, for example, with a bag, puts it down and leaves without it.

In video analytics systems, there are also «whitelists» that can be used to improve loyalty programs. The client uploads a photo for his account and at the checkout or the entrance, the system recognizes the customers and sends a notification to staff. VIP customers can be greeted by name and then offered tailored product recommendations.

In addition, video analytics systems enable the creation of an ID for personalized advertising in the area near the cash register for a loyalty program participant. Retail is not actively using this idea yet, but restaurants already have, for example, the CaliBurger chain in California allows registered customer to pay using their face. In a step further, face recognition also enables personalized offers and speeds up ordering.

Farewell to plastic

In the medium term, the use of video analytics will likely allow us to completely abandon plastic cards. In 2020, in Russia, a Koshelek application (which stores loyalty cards) led to over 300 million cards being transferred to this method. Plastic cards were willingly abandoned.

Video analytics systems also make it easier to keep track of employees’ working hours and also their location in a particular department, at lunch or other breaks. At a general level this may have slightly sinister connotations but for companies who suffer from poor productivity it can be a great leveller allowing the retailer to ‘reclaim’ its workforce. The data from this system can be combined with information from ERP platforms.

Queue no more

Video analytics is also useful when it comes to queues. For instance, a system can notify employees when people are unattended and stacking up at the checkout, in the fitting room, and so on. At the same time, it can collect information about queues, such as the number of people in the queue. This enables more effective resource management and can also increase average daily revenue by attending to people up were set to leave without making a purchase because of queues. A UK study by Honeywell revealed the queue prevention also increases customer loyalty by 35%.

Video analytics also helps deliver sophisticated, revenue-increasing management of products on the shelves. According to IHL Group, global retail loses approximately €900 billion a year because goods are not on shelves when customers are looking for them. A video analytics monitors shelves and sends notifications when products are running short, or a shelf has been emptied. In addition, the system also recognises when a product is in the wrong place.

Sophisticated planning

Another potential use is crowd analytics to create market reports to inform better management and planning based on data obtained from cameras. The system determines gender, age (with an accuracy of up to two years), calculates total number of visitors, including unique and returning ones, and helps to create a customer load schedule. It enables customer behavior tracking for movement through a store. This helps create store planning for customers. This idea can be scaled up for an entire shopping center.
IBM, in a study Video Analytics for Retail, spelt out clearly the benefits for retailers. It said store operations encompasses a wide variety of activities, many of which can be aided by video analytics, from planning store layouts based on customer path statistics to staff planning based on historical and instantaneous customer counts, at store entrances, departments and check-out queues. Merchandising activities can also be planned based on similar analytics choosing the location of a display based on customer paths, as well as measuring the effectiveness of a display based on customer counts coupled with sales figures.
A number of high profile retailers are already well down this route. Using audience analysis and advertising communication through strategically located media players they’ve increased sales substantially. Retail giant WalMart is going even further and building its own advertising platform to improve the user experience of customers, partly through video analytics. The company’s strategy includes media activity via TV sets in stores and outdoor screens, improving digital advertising in partnership with third-party agencies, and much more.

The roadblocks for retail

The main difficulties of using big data-powered analytics are related to the lack of the infrastructure for collecting information and the lack of historical data. For example, in video analytics, many retail projects were implemented during the COVID-19 pandemic, and the customer behavior model may change when retail returns to pre-pandemic levels of shopping.

Further incomplete or insufficient coverage of shopping areas due to the lack of cameras is an issue. That said, the cost of video cameras is set to decrease in the next five years, and solutions based on video analytics will become increasingly ubiquitous.

Of course, data protection is of supreme importance. It’s extremely important to ensure the security and confidentiality of customer data. As such camera information must remain on a store’s local server and be well protected. And face images should not be stored as images but rather as a digital description, that is, a type of code that corresponds to the image. Further, to comply with data regulations the system should be configured so that collected data is deleted every day and only summary reports are saved.
The introduction of video analytics into retail is growing, consumer trust in smart solutions and the spread of cameras and sensors are increasing, and the requisite back-office infrastructure is improving. Retailers are also recognizing that video analytics technology helps reduce the number of customers who leave a store without buying and when used for planning can not only reduce losses but also grow revenue. As such within the next five years expect to see retail video analytics platforms become the norm and not the exception.

3 tips for the introduction of video analytics in retail

• Decide on the budget and key tasks you want video analytics to perform. A golden rule of thumb is that the bigger the feature set, the faster the return on investment especially when used to boost sales.
• Consider the activity segments. In grocery chains it is important to work with queues, analyze visitor data and use loyalty programs at the checkout. In food only retailers there can be an acute problem with customers forgetting to pay for goods. Non-food retailers benefit from personalized offers, sales area analytics and automation of marketing tools.
• Inform customers about the introduction of video analytics and ensure you are complying with the relevant data protection laws. The consumer has the right to know that a store, for example, is running a facial recognition system.

There are already over 700 million CCTV cameras installed in cities worldwide, with 20 cities having over 10 cameras installed per every 1,000 residents. Furthermore, video data captured by privately owned cameras, body-worn cameras, smartphones, drones, and dashcams offer additional precious sources of intelligence that cities can use to improve efficiency, effectiveness, and safety of policing.

The fast technological evolution, widespread adoption, and affordability of video hardware – including CCTV, IP cameras, HD 4K cameras, 360° cameras, drone cameras, dash cams, smartphone cameras, wearable cameras – provide a broad variety and high volume of data that national and city public safety agencies use as a force multiplier, to radically rethink workflows and operations to increase productivity. However, the huge volume of camera feeds must be monitored continuously to improve the productivity of policing, an activity that human cannot carry out effectively. As a result, it is crucial that public safety authorities invest in video intelligence. That is not only the ability to capture more video surveillance images, faster and in a more accurate manner, but most importantly to apply advanced artificial intelligence capabilities to:

• Detect people, objects, and events with high accuracy and speed.
• Recognize people, objects, and events even in challenging conditions.
• Organize data and consistently apply appropriate governance policies.
• Analyze and interpret data to find trends, patterns, and correlations.
• Deliver 360° intelligent insights to improve public safety outcomes.

Advances in video analytics and artificial intelligence have drastically improved the accuracy and speed of detection and recognition of people’s characteristics, such as faces, body attributes, gender, age, as well as objects, such as vehicle make, body type, and color. Increasingly, the performance of humans in recognizing and interpreting images is being surpassed by computer vision. In essence, city public safety authorities can let the software do the heavy lifting and alert human operators when it detects anomalies, thus reducing the lag time needed to dispatch first responders. Or it can flag suspects by matching millions of images with criminal records, thus speeding up the work of case investigators on the thousands of petty crimes that consume most of police time and resources and instead free up their time to focus on more complex investigation. For instance, Beijing Public Security Bureau estimated that thanks to surveillance cameras, officers detained 5 percent more suspects in 2015, compared with 2014.

The convergence of massive volumes and variety of images with advances in computer vision software made affordable for cities to deploy video intelligence capabilities on a variety of architectures, from core datacenters, to cloud to embedding computer vision in edge computing. As a result, cities have been able to expand the public safety use cases, in which they can surveil, detect, recognize people, objects and events, interpret patterns, and empower better decisions with high accuracy and speed. These use cases include crowd monitoring, searching for criminals, identifying missing people in case of emergencies, improving access control, enhancing physical security in schools, hospitals, airports, and sport arenas, and, of course in the COVID-19 aftermath, monitoring behavior that could increase the risk of spreading of viruses.

There are three main usage scenarios in which artificial intelligence algorithms improve the productivity of day-by-day policing in cities:

• Recognize faces (or body attributes) rapidly by processing video streams from the city’s CCTV system at a fast rate, rather than waste time of human operators staring at millions of frames. The computer-recognized face’s feature vector and metadata (e.g. time, location) can then be stored as an object in a database for future usage. The stored object is totally anonymous and cannot be used to reconstruct the original video frame, thus inherently protects individuals’ privacy.
• Match faces captured by the city CCTV system with video footage from criminal records to establish identity and involvement of suspects in a crime, hence speed up criminal investigations. In fact, academic research demonstrates that when video footage is used, crime clearance rates improve14. Besides matching a face (or body shape) of a known criminal to identify him/her as the person that committed a crime at a certain location and time, the face recognition metadata also allows for more complex investigations, such as placing a certain person near (in time and space) other people that may be involved in the crime. These analyses must take place in compliance with regulation on police investigative power and authorizations granted by magistrates, thus it is important that the matching algorithms are processed within police-protected IT systems.
• Conduct real-time searches of suspects to identify their current location to enforce arrests. Accurate and current information on the location of the suspect is important to ensure not only the efficacy of the arrest, but also the safety of officers. Similar real-time searches can be used to identify individuals that, for example, have been banned from entering certain facilities, such as sports arenas, thus block them before they do so, while not imposing burdensome personal inspections to every legitimate spectator.
  The combination of massive amount of video with these artificial intelligence capabilities bears the potential for city public safety authorities to work much more efficiently and effectively, thus reducing criminal behavior, by showing how much faster police can punish bad behavior.

Furthermore, cities that have invested in face, body shape, vehicle recognition can use the capabilities in other city domains, such as traffic intelligence to identify congested roads and recommend alternative routes, infrastructure condition monitoring for preventive maintenance, and flood or other natural disaster monitoring to enhance safety and search of missing persons.

What we offer:

• Off-the-shelf face, body attributes, and object, in particular vehicle, recognition capabilities that are primarily applied to public safety use cases, and the agility to extend and re-train algorithms for other use cases, such as traffic intelligence.
• High-performing algorithms that can work with data from real-time video streams or large video databases.
• Beyond object detection and recognition, the software can carry out analysis finding trends, incidents, and correlations to build video intelligence solutions.

Advances in video analytics and artificial intelligence have drastically improved the accuracy and speed of detection and recognition of people’s characteristics, such as faces, body attributes, gender, age, as well as objects, such as vehicle make, body type, and color. Increasingly, the performance of humans in recognizing and interpreting images is being surpassed by computer vision.

City public safety leaders, such as chief investigators, real-time crime centers commanders, and traffic management operation center managers that want to leverage the power of video intelligence must:

• Inventory existing video surveillance cameras and video management systems to assess their costs and their connectivity, data transmission, archiving, and cybersecurity capabilities. The inventory should be matched against the city historical data of crime patterns to understand which cameras need to be upgraded to feed data into advanced video intelligence AI systems, which ones can be discarded to save money to be invested in new hardware and software.
• Prioritize investment in people/object detection AI algorithms that can increase the productivity and effectiveness of police officers in charge of analyzing video footage. The investment in AI must align technology innovation with key public safety goals, such as crowd safety at mass events, search and find of criminals, search and find of missing people, time to respond to emergencies, perimeter control in critical locations.
• Select video AI software platforms that offer best-in-class performance in people/object detection and recognition, including in challenging conditions (e.g. people wearing face masks) to avoid false positives that can create an overload of activities for public safety officers and investigators and, of course, avoid false negatives that put citizens and officers at risk. Procure solutions that are open and interoperable so that they can ingest data from multiple sources, including legacy cameras.
• Invest in GPU processing and video management software and storage capabilities to optimize the performance of face/object recognition AI software platforms.
• Leverage the modularity of video AI software platforms to configure, deploy, and administer them in a way that secures compliance with data protection regulation. For instance, face recognition software must archive only objects with feature vectors and metadata, and not the actual images, the processing of objects to match them against video archived in criminal records must take place in police-protected systems and can only be carried out by authorized officers, and the queries that enable to intake the data from the face recognition object database for matching purposes must also be logged only in police-protected systems.
• Deploy video intelligence capabilities in combination with adequate change management resources, including engaging and training public safety officers that will eventually have to make the critical decisions, based on the insights from the AI algorithms.
• Combine video intelligence with a holistic approach to improving the city safety, such as engaging with community organizations that take care of neighborhood watch activities or collaborating with correctional and social service agencies that work to reduce re-offending rates.

Ongoing research and innovation constantly enhance our expertise to better support our customers and deliver 360° intelligent insights to improve public safety outcomes.

The world is urban. According to the UN, almost 70 percent of people will live in cities by 2050. City leaders that want to deliver on the promise of UN Sustainable Development Goal 11 to «Make cities and human settlements inclusive, safe, resilient and sustainable» must make the city economic growth inclusive, the infrastructure more resilient, the mobility of people and goods more convenient, affordable, and environmentally sustainable, and, last but not least, their communities safe.

In fact, according to studies like the EIU Safe City Index, there is a virtuous cycle between economic growth that depends on a secure environment and higher income that makes it affordable for cities to make safety-increasing investments. Studies conducted across U.S. cities proved, for example, that cutting homicides by 10 percent increases housing values by 0.83 percent.

Increasing investment in public safety personnel can improve city safety; for instance, estimates indicate that an additional 10 to 17 officers hired prevented one new homicide per year and more officers and investigators can reduce the time that is required to solve a violent crime such as homicide. But cities and public safety authorities are always dealing with resource constraints. So, increasing the number of police officers is not affordable, or not enough. Cities need to improve productivity of the public safety officers that they have available. One of the keys to improve productivity is to invest in technology innovation to extract intelligent insights from data that empower city leaders to improve public safety policies, services, and operations.

According to INTERPOL, «data is at the heart of international policing. When the right data is in the right hands, it can give law enforcement a comprehensive global picture of crime trends to help them tackle emerging crimes more effectively.»

City safety globally is undergoing a process of digital reinvention, some facets of which have been further catalyzed by the impact of COVID-19. City leaders across the globe are working with police forces and other first respondents to realize the benefits of next-generation 911 systems, early warning systems, real-time crime centers, digital evidence management, digital forensic and intelligence, and data sharing platforms. For instance:

• Data must not only be collected from CCTV cameras, radiation, chemical and gunshot detection sensors, 911 logs and fed into integrated command centers, but must also be analyzed with advanced AI algorithms to recognize people, objects and events, detect anomalies, and promptly alert command center operators. Cities and police forces that do not integrate multiple sources of data will have a siloed view of the risks. Those that do not apply advance analytics will overwhelm operators with visual information that is impossible to interpret, hence paralyzes their ability to respond to real emergencies.
• Data sharing platforms are also providing investigators with digital evidence management and analysis capabilities to speed up forensic investigation and capture wanted criminals. Investigators that do not have access to easy-to-use archival, search, and analysis of evidence are slowed down by a backlog of cases and run higher risks of breaking chain of custody that must be auditable from ingestion to case closure.
• Public safety authorities are also investing to deliver relevant information at the fingertip of officers in the field. Mobile policing gives officers access to portable command centers, investigative databases, and notification capabilities that provide real-time information on location of emergencies and suspect criminals, which increases operational efficiency, effectiveness, and officer safety.

Research and innovation to develop new automated systems for surveillance, such as drone fleets that can fly automatically, acquire data, detect and locate people, objects, hazards, events, and provide real-time reports along with suggested actions, such as the number of required police patrols, ambulances, or fire trucks to be mobilized, are continuing. Therefore, city public safety authorities must invest in advanced analytical and AI capabilities that enable them to interpret the data that they have today to promptly capture criminals, and easily integrate and interpret new sources of data in the near future.

As a team of engineers, we are firm believers in technology for good, however face recognition companies must actively monitor the deployment of their technologies to ensure that that value is upheld, irrespective its deployment.

In consideration of smart cities – it is generally the police who would use face recognition software. The criminal databases they check video sources against shall remain entirely within the police systems and are not shared with any third parties. In this capacity, the software is a hyper-efficient update to the traditional policing methods that police camera operators would have been manually doing for a number of years.

At a software level, the algorithms must work with unique face features which makes it impossible to restore the original image of the face; something that is of distinct importance in protecting privacy. This means that when a person passes a camera linked to a system running our software, the image of their face is transformed into a digital imprint consisting of several lines of code, and only this digital imprint is compared against a database of similar digital imprints. It is impossible to restore the original image of the face from this digital imprint.

In a scenario where the software is being used for crime prevention, once a match is found between a database of wanted criminals, and the digital imprint of someone passing a camera, then the designated police officers receive an alert. Digital imprints of other people passing cameras are deleted once cross referenced against databases.

For commercial applications face recognition software collects anonymised data, limited to relative age and gender in order to generate consumer data and insight that enables businesses to improve brand communication and customer journeys. In cases where a business is required to react to certain people (for example, the VIP-scenario, where businesses use the system to recognise special guests to offer them VIP services) we expect businesses to receive written confirmation from individuals authorising the use of their images for such purposes.

Face recognition companies invest a lot of resource into maintaining a system that is both fully reliable and secure from vulnerabilities. The software becomes highly configurable and modular, which allows to make substantial additions or turn some features off for different markets – for example, customers can blur the faces of unknown people in the system, so that the operators will never receive any of their footage.

How do we ensure your software is not being misused

In every country we work in strict compliance with local laws and regulations, and we thoroughly check and vet our clients. Once our software is deployed, it is imperative that we ensure privacy by having no contact with the process of ongoing implementation, and clients do not need to use servers owned or operated by Recognito, allowing them to operate our software on their own secure servers.

Questions have been raised around misidentification and bias, and we are aware that other face recognition systems have displayed such issues in the past. However, unlike organisations that have encountered this problem, we trained our software’s neural networks with stimuli from different ethnicities, skin tones, and genders in equal proportions. The distinct way our neural network is built, combined with our engineering talent has ensured we mitigate against biases.

We are strong supporters of the deployment of face recognition, as long as it is done in a controlled and responsible manner that serves the public good first and foremost.