Understand video analytics metadata, how it is generated at the edge or on servers, how VMS and dashboards use it, and how metadata supports intelligent video surveillance.
Check it out!
In the context of Video Surveillance, metadata are structured descriptions that characterize the visual content of video.
Metadata can provide a wide range of useful information. They can describe a situation in detail, identify relevant visible objects, and provide information about specific characteristics associated with a scene.
Metadata add context to events, allowing large volumes of recordings to be organized and searched quickly. They may include details such as vehicle and clothing colors, precise object locations, or direction of movement.
For this reason, understanding how to work with metadata has become increasingly important for improving security, protection, and operational efficiency.
This article discusses metadata in the context of surveillance and operational efficiency, explaining how metadata work and how they are used to add intelligence to surveillance systems.
Read on!
[elementor-template id=”24446″]
What Is Video Metadata?
Metadata are structured sets of information that describe, locate, or facilitate the retrieval, use, or management of an information resource.
They can be classified into three main types: administrative, structural, and descriptive:
- Administrative metadata provide information used to manage a resource, such as when and how it was created, its file type, and who may access it;
- Structural metadata describe how the components of a resource are organized;
- Descriptive metadata provide information that helps users discover and identify other data.
In a CCTV system, metadata provide a structured description of video content, identifying objects of interest or providing a detailed description of the scene.
How Is Video Metadata Generated?
Metadata are generated in real time through Video Analytics. Analytics are algorithms that can run directly on the camera or on a dedicated server.
Until relatively recently, video analytics were performed almost exclusively on servers because they required processing capacity that edge devices could not support.
As camera processing capacity increased, it became possible to perform advanced analytics directly at the edge.
Edge analytics have access to uncompressed video and extremely low latency. This enables fast, real-time operation while avoiding the additional costs and complexity of sending all video content elsewhere for processing.
However, cameras with sufficient processing capacity for advanced edge analytics generally cost more. This is an important consideration when designing a video surveillance system.
The designer must evaluate whether to invest in cameras with greater processing capacity or allocate computing resources to dedicated servers for video analytics.
This decision should be based on careful return-on-investment analysis, considering factors such as required system performance, available budget, and project-specific needs.
The most appropriate architecture varies according to the circumstances of each project.
Where Is Metadata Used?
The main metadata consumers can be grouped as follows:
- Edge Applications;
- Hybrid Processing Applications;
- Video Management Systems (VMS);
- Dashboards.
Edge Applications
As Edge Applications se referem ao uso de ferramentas de análise que são executadas diretamente na câmera. Essas ferramentas podem aplicar filtros e regras lógicas para processar as informações relacionadas aos objetos detectados na cena.
These edge analytics can trigger specific actions based on predefined events or detected behavior. For example, they can control a PTZ (Pan-Tilt-Zoom) camera to follow a person moving through the scene.
The evolution of edge computing has enabled advanced technologies such as Artificial Intelligence (AI), Machine Learning, and Deep Learning to run inside cameras. This significantly changes how video data can be processed and interpreted.
In addition, cameras with edge-processing capability can store video locally on SD cards, improving system resilience and enabling high-performance applications in remote locations.
Hybrid Processing Applications
As Hybrid Processing Applications representam um modelo onde o processamento na borda (na câmera) e o processamento no servidor são combinados para realizar análises mais avançadas.
In this model, preprocessing is typically performed on the camera, including tasks such as initial object detection, filtering, and basic analytics.
Additional processing is then performed on the server, where more complex analytics can use greater computing power or storage capacity than the camera can provide.
Examples include correlating data from multiple cameras, performing long-term analysis, or applying more advanced machine-learning algorithms.
Video Management Systems (VMS)
A Video Management System (VMS) plays a central role in intelligent video surveillance. It acts as the system’s control layer, receiving, processing, and storing images and video from IP cameras.
A VMS can integrate advanced capabilities into a single platform, including technologies such as facial recognition, motion detection, and behavior analytics.
One of the main VMS capabilities is advanced search. Search filters allow operators to quickly locate specific events in recorded video using metadata such as date, time, camera, event type, object color, and direction of movement.
A VMS also provides robust user and permission management, allowing administrators to create profiles with different levels of access and control so that only authorized users can access cameras and recordings.
Finally, a VMS can integrate with other security systems such as access control and alarms, providing a unified view of events and improving security-system effectiveness.
Dashboards
Dashboards are Business Intelligence platforms that receive and organize metadata to support historical and real-time trend analysis.
These dashboards can use statistical analysis based on collected data, such as customer flow or customer experience, to generate insights that support data-driven decisions and more efficient operations.
Metadata used by dashboards may range from customer-behavior information to system-performance metrics. The data are processed and presented visually so users can quickly identify patterns, trends, and anomalies.
Dashboards can also be customized to each organization’s needs, tracking specific metrics, generating real-time alerts, and integrating multiple data sources for a more complete operational view.
How Is Metadata Transmitted?
Metadata generated in CCTV systems can be delivered in two main ways, depending on the context and the needs of the consuming system:
Real-Time Transmission
In this method, a complete description of the scene is provided for every frame, even when no activity or objects are present.
Metadata are continuously transmitted and made available on demand. This is essential in situations requiring immediate response and precise situational awareness.
The figure illustrates a metadata stream in which consecutive camera frames provide real-time information about the scene. Each frame describes the scene at a specific instant independently of prior events.
In Frame 1, objects A and B are detected: A is classified as a person wearing red clothing and B as a person wearing blue clothing.
In Frame 2, the camera updates the classification, determining that object A is actually wearing blue clothing and object B yellow clothing. Although they are the same objects as in Frame 1, their color attributes change and this is reflected in the metadata.
Frame 3 shows object B no longer present, with the camera tracking only object A, still classified as a person wearing blue clothing.
Optimized Delivery
In this approach, metadata associated with each specific object in the scene are consolidated into a single entity, a process that can be described as aggregation.
Instead of treating every observation of an object as a separate entity, all observations of the same tracked object are consolidated. This can significantly reduce the volume of data that must be stored and processed.
In optimized delivery, metadata are provided only when objects are present in the scene. This avoids unnecessary transmission and ensures that the most relevant information is delivered.
The figure demonstrates optimized metadata delivery, where the camera provides a unified representation based on object tracking. Each object record contains the known details collected throughout the object’s tracked lifetime.
The first record presents details about object B, including first and last detection, trajectory summary, and attributes observed during tracking. Object B had a 50% probability of wearing yellow clothing and a 50% probability of wearing blue clothing.
The second record uses the same format for object A, showing a 33% probability of red clothing and a 67% probability of blue clothing.
Understanding the advantages and disadvantages of each approach is essential when designing the system architecture.
Metadata can also be delivered through different communication protocols and data formats according to the needs of the consuming system.
Combining Metadata from Different Sources
Integrating metadata from multiple sources is a powerful strategy for maximizing its operational value.
When applied across visual, audio, activity, and process data sources, metadata can provide valuable insights for effective site management.
RFID tracking, GPS coordinates, alarm events, meter readings such as temperature or chemical levels, noise detection, and point-of-sale transaction data are examples of sources that can be integrated. A key requirement is aligning data from all sources through consistent timestamps.
Combining metadata from different sources produces a richer and more complete view than any source can provide alone, enabling better insights and more efficient decision-making.
Conclusion
Metadata play a crucial role in improving security and operational management.
Whether through real-time delivery for immediate response, optimized delivery for efficient analysis, or the combination of multiple sources for broader context, metadata are transforming how information is managed and interpreted.
As these technologies evolve, video surveillance systems will continue to gain efficiency, analytical capability, and operational value.
