This monitor is a replacement for the SMNP component of Viz Engine. It uses Opentelemetry metrics and Opentelemetry logs to provide the information from OS shared memory, written by a Viz Engine instance if configured to do so.

  • Application Name: Viz Engine Monitor

  • Enterprise ID: 27566.3.1.7.%N, where %N is the instance number without leading zeros.

Installation

msiexec /i "VizEngineMonitor-xxx.msi" /qn

Using the installer installs the application as a service. The service is installed as first instance by default. For further instances please refer to Install Service.

Before running the application, the configuration files must be adjusted for OpenTelemetry endpoints and logging destination.

Configuration

Viz Engine Configuration

Viz Engine needs to be configured to write the OS shared memory. Please make sure shm_system_status_enabled is set to 1 in the Viz Engine configuration file.

OpenTelemetry Endpoints

The Viz Engine Monitor expects end endpoint for logs, and an endpoint for metrics for OpenTelemetry HTTP transport. They usually look like:

Viz Engine Monitor Configuration

The configuration files are located at %PROGRAMDATA%\\Vizrt\\Engine_Monitor\\cfg. The location may be changed with application arguments. See also usage below.

The configuration is done in vizeng_monitor%N.json, or, if it cannot find this file, in vizeng_monitor.json. The %N is replaced by the instance number with a two digit format.

For the first instance, the configuration file is searched at vizeng_monitor01.json and then at vizeng_monitor.json. If the instance number is not found, the first instance is used.

Example configuration file vizeng_monitor01.json:

{
"version": "1.0",
"sleep_time_ms": 100,
"verbose_messages": false,
"report_expected_messages": false,
"metric_config": "otel_metrics_1"
}

Parameters for Engine Monitor Configuration

  • version: The version of the configuration file. This is used to check if the configuration file is compatible.

  • sleep_time_ms: The time in milliseconds between two runs. Default is 100 milliseconds.

  • verbose_messages: If set to true it log more details in each log message. Default is false.

  • report_expected_messages: If set to true then expected failures, are emitted as well as regular info logs. Default is false.

  • metric_config: The id to use in the metrics array of viz component logger configuration. Default is otel_metrics_1.

Usually an instance of this application is installed for each Viz Engine to monitor.

Viz Component Log Configuration

The configuration for logging is located in vcl_config%N.json, or, if it cannot find this file, in vcl_config.json. The %N is replaced by the instance number with a two digit format. If the instance number is not found, the first instance is used.

For example, for the third instance, the configuration file is searched at vcl_config03.json and then at vcl_config.json.

OpenTelemetry Logs Endpoint

In the default configuration, the metrics endpoint is located at backend/concrete/otel_log/exporter/http/endpoint.

"backend": [
{
"id": "backend_id_otel",
"concrete": {
"otel_log": {
"exporter": {
"type": "http",
"http": {
"endpoint": "http://127.0.0.1:4318/v1/logs",
...

OpenTelemetry Metrics Endpoint

In the default configuration the metrics endpoint is located at metrics/concrete/otel_metrics/exporter/http/endpoint.

"metrics": [
{
"id": "otel_metrics_1",
"concrete": {
"otel_metrics": {
"exporter": {
"type": "http",
"http": {
"endpoint": "http://localhost:4318/v1/metrics",
...

Usage

"%ProgramFiles%\Vizrt\Engine_Monitor\vizeng_monitor.exe" -h

The application can be started as a service or as a console application. The service is by default installed as first instance, by the installer.

For debugging, the console application can be started with the -c parameter. Usually, the console mode is combined with a path to the configuration and log files, -p.

vizeng_monitor.exe -c -p "some\where\" -n 3

In this case, the configuration files are searched at some\\where\\cfg\\license_monitor03.json, some\\where\\cfg\\license_monitor.json and the log files are stored at some\\where\\log\\.

The service may also be installed, uninstalled, started, or stopped.

Install Service

To install a service manually (for example, for multiple instances on one host), the -n parameter is used to specify the instance number.

vizeng_monitor.exe -i [-n instance]

  • vizeng_monitor.exe -i -n 3 registers a service with the name.

  • Vizrt Viz Engine Monitor_3 for the third instance.

Uninstall Service

To uninstall a service manually (for example, for multiple instances on one host), the -n parameter is used to specify the instance number.

vizeng_monitor.exe -u [-n instance]

  • vizeng_monitor.exe -u unregisters the service with the name.

  • Vizrt Viz Engine Monitor_1 for the first instance.

Start Installed Service

To start a registered service manually (for example, for multiple instances on one host), the -n parameter is used to specify the instance number.

vizeng_monitor.exe -r [-n instance]

  • vizeng_monitor.exe -r -n 2 starts the registered service with the name.

  • Vizrt Viz Engine Monitor_2 for the second instance.

Stop Installed Service

To stop a registered service manually, the -n parameter is used to specify the instance number.

vizeng_monitor.exe -k [-n instance]

  • vizeng_monitor.exe -k -n 2 stops a running service with the name.

  • Vizrt Viz Engine Monitor_2 for the second instance.

Troubleshooting

The application uses a bootstrap logger that writes to the console, and this logger is used to log errors during startup. In case the logs are not emitted where they should be, the bootstrap logger shows any log configuration errors. For this to work, the application must be started in console mode. A registered service does not need to be unregistered first, but it is recommended to stop the service before starting the application in console mode.

vizeng_monitor.exe -c ...

Using a log file instead of Opentelemetry is a good way to check if the application is running correctly. See configuration above.

Logged Information

For each iteration, the following log records are logged.

Common to all log records, are these key/value pairs in the default group:

  • host.name: The hostname of the origin of the message.

  • service.instance.id: The instance number of the application.

  • service.name: The name of the application, license_monitor.

  • app_version: The version of the application.

  • vizrt.service.oid: The OID of the application, 27566.3.1.7.%N, where %N is the instance number without leading zeros.

Shared Memory Information Logs

Logged as shm metric in the vizrt.details group:

  • metrics_for_oid: The OID for the app to which this log belongs to. 27566.3.3.1.%N, where %N is the instance number without leading zeros.

  • TCPPort: TCP port the application is listening to.

  • ComputerType: String identifying the computer type.

  • SystemID: String identifying the system ID.

  • VizVersionStr: Version of Viz Engine.

  • SceneFront: Scene loaded into the front layer.

  • SceneMid: Scene loaded into the middle layer.

  • SceneBack: Scene loaded into the back layer.

  • ScenePostFront: Scene loaded into the front layer in post mode.

  • ScenePostMid: Scene loaded into the middle layer in post mode.

  • ScenePostBack: Scene loaded into the back layer in post mode.

Shared Memory Information Metrics

Emitted with %N replaced by the instance id.

  • shm.27566.3.3.1.%N.OnAir_gauge: True if On-Air.

  • shm.27566.3.3.1.%N.ConnectedToDb_gauge: True if connected to a Graphic Hub.

  • shm.27566.3.3.1.%N.GenlockStatus_gauge: Status of Genlock.

  • shm.27566.3.3.1.%N.CurrentFrameRate_gauge: Current framerate in frames per second.

  • shm.27566.3.3.1.%N.VideoIn.1_gauge: Status of Video Input 1.

  • shm.27566.3.3.1.%N.VideoIn.2_gauge: Status of Video Input 2.

  • shm.27566.3.3.1.%N.VideoIn.3_gauge: Status of Video Input 3.

  • shm.27566.3.3.1.%N.VideoIn.4_gauge: Status of Video Input 4.

  • shm.27566.3.3.1.%N.VideoIn.5_gauge: Status of Video Input 5.

  • shm.27566.3.3.1.%N.VideoIn.6_gauge: Status of Video Input 6.

  • shm.27566.3.3.1.%N.VideoIn.7_gauge: Status of Video Input 7.

  • shm.27566.3.3.1.%N.VideoIn.8_gauge: Status of Video Input 8.

  • shm.27566.3.3.1.%N.ClipIn.1_gauge: Status of Clip Input 1.

  • shm.27566.3.3.1.%N.ClipIn.2_gauge: Status of Clip Input 2.

  • shm.27566.3.3.1.%N.ClipIn.3_gauge: Status of Clip Input 3.

  • shm.27566.3.3.1.%N.ClipIn.4_gauge: Status of Clip Input 4.

  • shm.27566.3.3.1.%N.ClipIn.5_gauge: Status of Clip Input 5.

  • shm.27566.3.3.1.%N.ClipIn.6_gauge: Status of Clip Input 6.

  • shm.27566.3.3.1.%N.ClipIn.7_gauge: Status of Clip Input 7.

  • shm.27566.3.3.1.%N.ClipIn.8_gauge: Status of Clip Input 8.

  • shm.27566.3.3.1.%N.TextureMemory.Free_gauge: Free texture memory in KB.

  • shm.27566.3.3.1.%N.TextureMemory.Total_gauge: Total texture memory in KB.

  • shm.27566.3.3.1.%N.TextureMemory.Used_gauge: Used texture memory in %.

  • shm.27566.3.3.1.%N.Post_gauge: Post mode is active.

  • shm.27566.3.3.1.%N.IsNLE_gauge: NLE mode is active.

  • shm.27566.3.3.1.%N.ReadyForConnect_gauge: Viz Engine is initialized and ready to receive commands.

  • shm.27566.3.3.1.%N.RetraceCounterMoving_gauge: true if retrace counter changed since the last run.

Where Status for Genlock, Video and Clip Inputs is one of the following:

  • 0: OK

  • 1: Wrong Format

  • 2: Bad

  • 3: Dropped Frames

  • 255: Not Available

Complete Engine Configuration File

Example configuration file vcl_config.json:

{
"async": [
{
"id": "async_1_id",
"num_items": 1200,
"num_workers": 4,
"overflow_policy": "overrun_oldest"
}
],
"attribute": [
{
"id": "vcl_config_logger_attributes_id",
"multi_attributes": [
{
"sdid_name": "config",
"single_attributes": [
{
"param_name": "facility",
"param_value": "vcl config"
}
]
}
]
}
],
 
"formatter": [
{
"id": "formatter_txt",
"concrete": {
"formatter_simple": {
"attr_prefix": "\n ",
"attr_assign": "=",
"attr_postfix": "",
"attr_separator": "",
"attr_name_escape": "\n\r\t\"\\{} ",
"attr_value_escape": "\n\r\t\"",
"group_prefix": "\n ",
"group_assign": ": ",
"group_postfix": "",
"group_separator": "",
"group_name_escape": "\n\r\t\"\\{} ",
"groups_prefix": "",
"groups_postfix": "",
"payload_prefix": " <<",
"payload_postfix": ">>",
"payload_escape": "\n\r\t\""
}
}
},
{
"id": "formatter_compact",
"concrete": {
"formatter_simple": {
"attr_prefix": "{\"",
"attr_assign": "\"=\"",
"attr_postfix": "\"}",
"attr_separator": ", ",
"attr_name_escape": "\n\r\t\"\\{} ",
"attr_value_escape": "\n\r\t\"",
"group_prefix": "{ \"",
"group_assign": "\": ",
"group_postfix": " }",
"group_separator": "; ",
"group_name_escape": "\n\r\t\"\\{} ",
"groups_prefix": "[",
"groups_postfix": "]",
"payload_prefix": " <<",
"payload_postfix": ">> ",
"payload_escape": "\n\r\t\""
}
}
}
],
 
"backend": [
{
"id": "collection_id_console",
"concrete": {
"backend_collection": {
"backend_refs": [
"backend_cout",
"backend_id_logfile",
"backend_id_otel"
]
}
}
},
{
"id": "collection_id_service",
"concrete": {
"backend_collection": {
"backend_refs": [
"backend_id_logfile",
"backend_id_otel"
]
}
}
},
{
"id": "collection_id_control",
"concrete": {
"backend_collection": {
"backend_refs": [
"backend_id_control"
]
}
}
},
{
"id": "backend_vcl_config",
"concrete": {
"spdlog": {
"attribute_ref": "",
"formatter_ref": "formatter_compact",
"spdlog_format": "%v",
"spdlog_time_type": "utc",
"spdlog_eol": "default",
"dup_filter_max_skip_duration": 10,
"spdlog_log_level": "debug",
"spdlog_flush_level": "debug",
"rotating_file_sink": {
"base_filename": "vcl_config_%N",
"max_size_MB": 1024,
"max_files": 20,
"rotate_on_open": false
}
}
}
},
{
"id": "backend_cout_terse",
"concrete": {
"cout_terse": {
"attribute_ref": ""
}
}
},
{
"id": "backend_cout",
"concrete": {
"cout": {
"formatter_ref": "formatter_txt"
}
}
},
{
"id": "backend_id_logfile",
"concrete": {
"spdlog": {
"attribute_ref": "",
"formatter_ref": "formatter_txt",
"spdlog_format": "%v",
"spdlog_time_type": "utc",
"spdlog_eol": "default",
"dup_filter_max_skip_duration": 10,
"spdlog_log_level": "debug",
"spdlog_flush_level": "debug",
"rotating_file_sink": {
"base_filename": "id_%N_monitor",
"max_size_MB": 1024,
"max_files": 20,
"rotate_on_open": false
}
}
}
},
{
"id": "backend_id_otel",
"concrete": {
"otel_log": {
"exporter": {
"type": "http",
"grpc": {
"endpoint": "localhost:4317",
"use_ssl_credentials": false,
"ssl_ca_cert_path": "",
"max_threads": 4,
"timeout_seconds": 1
},
"http": {
"endpoint": "http://127.0.0.1:4318/v1/logs",
"content_type": "",
"use_json_name": false,
"console_debug": false,
"timeout_seconds": 1,
"max_concurrent_requests": 0,
"max_requests_per_connection": 0,
"ssl_insecure_skip_verify": true,
"ssl_ca_cert_path": "",
"ssl_ca_cert_string": "",
"ssl_client_key_path": "",
"ssl_client_key_string": "",
"ssl_client_cert_path": "",
"ssl_client_cert_string": "",
"ssl_min_tls": "",
"ssl_max_tls": "",
"ssl_cipher": "",
"ssl_cipher_suite": "",
"compression": "",
"retry_policy_max_attempts": 0,
"retry_policy_initial_backoff": 1.0,
"retry_policy_max_backoff": 60.0,
"retry_policy_backoff_multiplier": 1.5,
"api_key": ""
}
}
}
}
},
{
"id": "backend_id_control",
"concrete": {
"spdlog": {
"attribute_ref": "",
"formatter_ref": "formatter_compact",
"spdlog_format": "%v",
"spdlog_time_type": "utc",
"spdlog_eol": "default",
"dup_filter_max_skip_duration": 10,
"spdlog_log_level": "debug",
"spdlog_flush_level": "debug",
"rotating_file_sink": {
"base_filename": "id_%N_control",
"max_size_MB": 1024,
"max_files": 20,
"rotate_on_open": false
}
}
}
},
{
"id": "backend_id_mapping",
"concrete": {
"spdlog": {
"attribute_ref": "",
"formatter_ref": "formatter_compact",
"spdlog_format": "%v",
"spdlog_time_type": "utc",
"spdlog_eol": "default",
"dup_filter_max_skip_duration": 10,
"spdlog_log_level": "debug",
"spdlog_flush_level": "debug",
"rotating_file_sink": {
"base_filename": "id_%N_mapping",
"max_size_MB": 1024,
"max_files": 20,
"rotate_on_open": false
}
}
}
}
],
 
"xlate_logger": [
{ "": "default_logger_id_%N" },
{ "vizrt/vlc_config": "vcl_config_logger" },
{ "xx_vizrt/os_lib": "logger_id_control" },
{ "xx_vizrt/wibu": "logger_id_control" },
{ "vizrt/app/viz_engine/monitor": "logger_id_control" },
{ "vizrt/app/viz_engine/monitor/": "logger_id_control" },
{ "vizrt/app/viz_engine/monitor/console": "logger_id_console" },
{ "vizrt/app/viz_engine/monitor/service": "logger_id_service" },
{ "vizrt/app/viz_engine/monitor/console/mapping": "logger_id_mapping" },
{ "vizrt/app/viz_engine/monitor/service/mapping": "logger_id_mapping" }
],
 
 
 
"logger": [
{
"id": "null_logger",
"async_ref": "",
"concrete": {
"logger_simple": {
"backend_ref": "",
"attribute_ref": "",
"should_log_filter_ref": "",
"repeat_filter_ref": ""
}
}
},
{
"id": "vcl_config_logger",
"async_ref": "",
"concrete": {
"logger_simple": {
"backend_ref": "backend_vcl_config",
"attribute_ref": "vcl_config_logger_attributes_id",
"should_log_filter_ref": "",
"repeat_filter_ref": ""
}
}
},
{
"id": "logger_id_control",
"async_ref": "",
"concrete": {
"logger_simple": {
"backend_ref": "collection_id_control",
"attribute_ref": "",
"should_log_filter_ref": "minimum_debug_id",
"repeat_filter_ref": ""
}
}
},
{
"id": "logger_id_console",
"async_ref": "",
"concrete": {
"logger_simple": {
"backend_ref": "collection_id_console",
"attribute_ref": "",
"should_log_filter_ref": "minimum_debug_id",
"repeat_filter_ref": ""
}
}
},
{
"id": "logger_id_service",
"async_ref": "",
"concrete": {
"logger_simple": {
"backend_ref": "collection_id_service",
"attribute_ref": "",
"should_log_filter_ref": "minimum_info_id",
"repeat_filter_ref": ""
}
}
},
{
"id": "logger_id_mapping",
"async_ref": "",
"concrete": {
"logger_simple": {
"backend_ref": "backend_id_mapping",
"attribute_ref": "",
"should_log_filter_ref": "minimum_info_id",
"repeat_filter_ref": ""
}
}
}
],
 
 
"repeat_filter": [
{
"id": "repeat_filter_id",
"concrete": {
"repeat_error_code": {
"minimum_level": "error",
"max_repeat_count": 100,
"max_age": 10
}
}
},
{
"id": "repeat_param_value_id",
"concrete": {
"repeat_error_code": {
"minimum_level": "warning",
"max_repeat_count": 20,
"max_age": 30
}
}
}
],
 
 
"should_log_filter": [
{
"id": "minimum_debug_id",
"concrete": {
"minimum_severity": {
"minimum_level": "debug"
}
}
},
{
"id": "minimum_debug_fixed_id",
"concrete": {
"minimum_severity_fixed": {
"minimum_level": "debug"
}
}
},
{
"id": "minimum_info_id",
"concrete": {
"minimum_severity": {
"minimum_level": "info"
}
}
},
{
"id": "minimum_info_fixed_id",
"concrete": {
"minimum_severity_fixed": {
"minimum_level": "info"
}
}
}
],
"metrics": [
{
"id": "otel_metrics_1",
"concrete": {
"otel_metrics": {
"export_interval_ms": 1000,
"export_timeout_ms": 500,
"exporter": {
"type": "http",
"grpc": {
"endpoint": "localhost:4317",
"use_ssl_credentials": false,
"ssl_ca_cert_path": "",
"max_threads": 4,
"timeout_seconds": 1,
"aggregation_temporality": "cumulative"
},
"http": {
"endpoint": "http://localhost:4318/v1/metrics",
"content_type": "",
"use_json_name": false,
"console_debug": false,
"timeout_seconds": 1,
"max_concurrent_requests": 0,
"max_requests_per_connection": 0,
"ssl_insecure_skip_verify": true,
"ssl_ca_cert_path": "",
"ssl_ca_cert_string": "",
"ssl_client_key_path": "",
"ssl_client_key_string": "",
"ssl_client_cert_path": "",
"ssl_client_cert_string": "",
"ssl_min_tls": "",
"ssl_max_tls": "",
"ssl_cipher": "",
"ssl_cipher_suite": "",
"compression": "",
"retry_policy_max_attempts": 0,
"retry_policy_initial_backoff": 1.0,
"retry_policy_max_backoff": 60.0,
"retry_policy_backoff_multiplier": 1.5,
"api_key": "",
"aggregation_temporality": "cumulative"
}
}
}
}
}
],
"version": "1.0"
}

Parameters for Viz Component Log Configuration

The loggers used are found under xlate_logger:

  • vizrt/vlc_config logger for viz component logger. Leave it to vcl_config_logger to have a bootstrap log file.

  • vizrt/os_lib logger for os lib component. This entry is missing by default to inherit the logger.

  • vizrt/wibu logger for wibu component. This entry is missing by default to inherit the logger.

  • vizrt/app/viz_engine/monitor and vizrt/app/viz_engine/monitor/ logger for the control mode operations, such as install as a service.

  • vizrt/app/viz_engine/monitor/console logger when running in console mode.

  • vizrt/app/viz_engine/monitor/service logger when running in service mode.

  • vizrt/app/viz_engine/monitor/console/mapping logger to map obstructed sensitive data as hash to real data when running in console mode. Not yet used.

  • vizrt/app/viz_engine/monitor/service/mapping logger to map obstructed sensitive data as hash to real data when running in service mode. Not yet used.

The usual customization points are then:

  • vizrt/app/viz_engine/monitor/console This logs by default to console and log files.

  • vizrt/app/viz_engine/monitor/service This logs by default to log files and Opentelemetry endpoints.

Look up the translated logger and follow any backend_ref or backend_refs until you reach a concrete backend, such as otel, ... and configure their endpoints.

Example Backend otel Log

{
"id": "backend_id_otel",
"concrete": {
"otel_log": {
"exporter": {
"type": "http",
"http": {
"endpoint": "<http://127.0.0.1:4318/v1/logs>",
"timeout_seconds": 1,
"max_concurrent_requests": 0,
"max_requests_per_connection": 0,
"ssl_insecure_skip_verify": true,
}
}
}
}
}