Skip to content

VehicleAnalytics

Pipeline Overview

VehicleAnalytics pipeline provides vehicle and ALPR detection and classification. The pipeline can utilize a variety of input and output methods, as well as, provides a set of parameters to fine-tune it for needs of a particular application.

The following modules are specific entry-points

  • VehicleAnalyticsFile.yaml - runs the pipeline on a video or image input file
  • VehicleAnalyticsRTSP.yaml - runs the pipeline in an RTSP stream
  • VehicleAnalyticsGstRTSP.yaml - runs the pipeline in an RTSP stream (GStreamer backend, only available on Jetson)
  • VehicleAnalyticsFolderWatch.yaml - using a configured watched folder as an input
  • VehicleAnalyticsDirect.yaml - pipeline used by applications embedding SIO as an SDK
  • VehicleAnalyticsImageLoop.yaml - pipeline feeds a pre-configured image in a loop; useful for performance testing.

Examples for RTSP and Folder Watcher are available here.

Pipeline Parameters

Common

  • detectionMinConfidence - detection confidence threshold. Default: 0.3
  • detectionOverlapThreshold - parameter to help with filtering detection that have bounding boxes overlapping at least that much. Default: 0.3
  • detectionModel - specifies the detection model to use. This will affect both performance and accuracy. Values will match the models available in ./models folder, and have a form of generation+[server or embedded]+[standard or fast or best]. So, for example, 6th generation embedded best model will be gen6eb. Default: gen5es
  • enabledClasses - list of classes enabled in the object detector. Default: ["car", "licenseplate", "bus", "truck", "motorbike"]
  • imageSaveDir - folder name and prefix where to save analyzed frames corresponding to the generated analytics output. Default: empty (do not generate images)
  • imageWriterFormat - If set, images will be saved in the specified format. Default: 'jpg'
  • sourceId - a "callback context" : reported results will include this ID. Useful if gathering outputs from multiple cameras/pipelines at a single point. Default: lpr-1
  • useTracker - by default, tracker is not utilized, resulting in detected vehicles assuming unique object ID in every frame, unless they are correlated with a license plate. Setting this value to true enables the tracker, at a significant performance cost. We recommend it is never set to true for a stream from a moving source. Default: false
  • trackerType - choose which algorithm of tracker to use. Options are [cpuBest, cpuFast, cpuFaster, cpuFastest]. Currently, cpuFaster is not in a great state and performance is very low, not advised to use.
  • debugSettings - comma-separated list that enables debugging in multiple modules. Supported options: eventAnalytics: Print input and output of event analytics. OSD: Enable OSD streamView events. dumpJson: Save frame_data to /data/tmp. jsonPrint: Print frameData to stdout. json: Add debug section in frameData. log: Enable debug logging in python modules. filteringDebug: Enable OutputFilter debug. Default: ""
  • splitMakeModel - if true, the "value" of a car object will be represented as JSON dictionary with values for 'make', 'model', 'generation'. If false, it will be represented as a comma-separated string. Default: false

Update strategy

  • updateAllowEmptyJSON - if true, output analytics result, even if no objects are contained in it. When false, empty JSON may still be emitted in the case the update is forced due to any of the other update related parameters described here. Default: false
  • updateFilterUnchangedObjects - sets whether or not OutputFilter removes entries that are not updated. If false, the objects that haven't been updated in the frame are no longer removed from the JSON output, even if updateOnlyOnChange=true. Default: true
  • updateMinFrequencyMs - send an output refresh even if no updates had been seen within the specified time from the last output. The update will be sent even if the output is empty. This can be used as a keep-alive mechanism for the client. Default: 0 (Never force updates based on time)
  • updateOnlyOnChange - if true, only emit new or updated detections (that includes previously detected objects with an improved score). Default: true
  • updateOnlyOnEvents - if true, only emit frameData with events ( "sensorEvents" ). Default: false
  • updateOnMetaclassCountChange - comma-separated list of metaclasses (licensePlates,vehicles) to trigger an update whenever count of that class changes in the output. Default: '' (no updates are triggered by count changes). Special value all means update on count changing for any classes supported by the pipeline.
  • updateOnStart - send an update at the very start of processing, even if no objects are detected in that frame. Default: true
  • updateResetStateOnNewInput - relevant for FolderWatch pipeline only; if true, treat each new input as a separate stream. Default: true

License Plate Detection

  • lptEnableCascading - run a secondary detection on car crops to attempt finding license plates. This can potentially have a significant performance cost. Default: true
  • lptModel - specifies the character detection model to use. Same nomenclature applies to this parameter as detectionModel. Default: gen5ss.
  • lptEnabledRegions - list of LP regions to be considered when classifying license plates. Supported values: "us", "eu" or combination. Default: [ "us" ]
  • lptMaxCharacters - license plates with a higher number of characters will be ignored. Default: 8
  • lptMinCharacters - license plates with a lower number of characters will be ignored. Default: 3
  • lptMinConfidence - confidence score threshold used to filter weightedScore inside the LicensePlateTracker node. Default: 0.9
  • lptSkipCarsWithoutLPs - do not report vehicles that aren't associated with a license plate. Default: true
  • lptStabilizationDelay - Delay output by that many frames, to see if improved result for license plates is available. Default: 0 (do not delay)
  • lptStabilizationStrategy - What to do, when better result is encountered during stabilization phase: "remove" or "update". Default: "" (system default, 'remove' when updateOnlyOnChange=true, 'update' otherwise)
  • lptStringMergeWindow - maximum frames to consider when merging license plates objects. Default: 30
  • lptStringScoring - a way to convert individual character scores to a string score. There are three scoring methods available "min", "max" and "average". The "min" method uses the minimum value in a list of character detection scores as the detection score for a specific license plate string, "max" uses the maximum value, "average" uses the median value. Default: min
  • lptWeightDetScore - weights for detection and character scores. weightCharScore = 1 - weightDetScore. With default value the pipeline relies more on the character scores. Default: 0.0

Box filtering

Allows to filter out detection boxes based on bounding box dimensions.

  • boxFilterClasses - specifies classes to filter. Classes not mentioned here won't be filtered out. To disable, set to [ ]. Default: [ "bus", "car", "licenseplate", "truck" ]
  • boxFilterMin - specifies minimum dimensions of objects in boxFilterClasses. Must contain the same number of elements. Default: [ 35, 35, 30, 35 ]
  • boxFilterMax - specifies maximum dimensions of objects in boxFilterClasses. Default [ 0, 0, 0, 0 ] (objects are accepted no matter how big
  • boxFilterType - approach to use for filtering. Can be dimensionAbsolute, dimensionRelative, widthAbsolute, widthRelative, heightAbsolute, heightRelative, areaAbsolute, areaRelative, aspectRatio. dimension* compares any dimension of a bounding box, height/width compares specific dimension. Absolute uses the provided value as pixel count, Relative compares the value in the range [0,1] relative to the image dimensions. aspectRatio compares provided value to width/height ratio. Default: dimensionAbsolute (object needs to fulfill the constrain on either of the dimensions, in pixels)
  • boxFilterROI - ensure boxes with labels specified in boxFilterClasses are contained in specified ROIs. Example: [ [ 10, 10, 100, 100 ], [ 20, 20, 120, 120 ] ]. The box coordinates are interpreted as [ x, y, w, h ]; those coordinates may be absolute pixel coordinates, or relative to the size of the image, with each value being in range of [0,1]. Default: [] (no ROI filtering)
  • boxFilterROIMethod - method to apply box filter ROI. boxInRoi or roiInBox. The former will require full containment of bounding box in ROI to pass, the latter will require full containment of ROI in bounding box to pass. Can also be specified with a percentage: for example boxInRoi75 will require that at least 75% of box area is within ROI to pass.

This is also useful to turn off unnecessary classifications. For example, setting boxFilterMin of licensePlate class to 5000 ensures that no license plate is ever reported out of the detector, and thus never analyzed, effectively turning off ALPR. Similarly, setting all the vehicle constraints to similarly large value can be used to turn off MMC (make model color) analytics.

Extensibility

  • extensionModules - comma-separated list of file paths pointing to Python modules to be used, in order specified, to post-process and potentially modify the JSON output emitted by the pipeline. Default: empty
    • Each module specified needs to implement two methods:
      • def configure(configPath) - will be called once, while loading the module. Path to specified configuration will be provided as the parameter.
      • def process(tick, data, frame) - will be called for each frame processed. Parameters are sequential frame ID, json output and RGB frame object. Must return the new value for data.

Debugging/Performance

  • debugSettings - enables various debugging facilities. Default: ''. Formatted as a comma separated list of the following values:
    • json a debug section is added to the JSON output.
    • log enable log output from the pipeline. history periodically (and on pipeline termination) log all the reports emitted so far.
  • debugRawResultsPrefix - when set to a path, will generate raw results useful for post-processing and tuning in the path specified. Default: '' (disabled)
  • performanceInfo - report performance metrics. Default: false
  • includeAverageHardware - report average hardware usage information. Default: false
  • performanceCSVFile - save path for performance metrics CSV, if empty (default) no file will be saved. Default: ''
  • debugOutputPath - Save path for debug files. Default: '/data/tmp'
  • jsonDumpFormat - save format for output JSONs. Default: 'json'. Options: 'json' or 'csv'

EventAnalytics

  • sensorsConfigFile - path of configuration sensors file, if empty (default) the Event Analytics module will be disabled and input will be forwarded to output. Default: ''

AMQP

  • amqpUseSsl - option to specify if SSL is used. Default: false.
  • amqpHost - host address for the AMQP output. AMQP output generation will be skipped if this is empty. Default: ''
  • amqpPort - connection port. Host address and port number make a complete address. Default: 0
  • amqpMaxRetries - maximum number of connection retries before producing an error message. Default: 5
  • amqpUser - username that will be used to establish a connection with the host address. Default: ''
  • amqpPassword - password that will be used to establish a connection with the host address. Default: ''
  • amqpExchange - used to configure the AMQP Exchange. Default: ''
  • amqpRoutingKey - used to configure the AMQP Routing key. Default: "sio"
  • amqpErrorOnFailure - generate an error on AMQP connection/message post failure. Default: false

Pipeline Specific

VehicleAnalyticsFile

Used for processing video files.

Required Parameters: VIDEO_IN='Video_Path'

Pipeline Specific Options:

  • outputPrefix - save path and image filename prefix. When set the pipeline wil save images with detection boxes drawn. Default: ''
  • VIDEO_IN - file to process.

VehicleAnalyticsRTSP

Used for processing videos streamed using Real Time Streaming Protocol (RTSP).

Required Parameters: VIDEO_IN='RTSP_URL'

Pipeline Specific Options:

  • hardwareDecoderType - provides control over the hardware video decoder. Default value results in the use of system default. This can be completely turned off by setting the value to "None". Default: ''
  • resizeToSize - the size of the images that the pipeline ingests can be controlled using this option. The larger the image the more resource intensive it becomes to process it. A larger source image might also result in more frames dropped during video processing. This value will be used as the maximum height of the frame when it is being resized. Default: 720
  • maxQueueSize - the maximum number of frames to hold in the frame queue. The default setting for the RTSP pipeline is lossy. Lossy queue will drop frames, rather than stop consuming input when full. When set to 0, asynchronous reading of frames is disabled. Default: 200
  • fpsLimit - the maximum number of frames that are let through the frame queue. This limits incoming FPS to process to a specific value. This can be used to control resource usage in cases where several streams need to be processed simultaneously. Default: 20
  • VIDEO_IN - RTSP URI to process.
  • recordTo - Folder to record the incoming video to. Default: '' (recording disabled)
  • recordInterval - Interval, in milliseconds, after which a new file will be opened for a recording (may take longer, depending on I-frame distance). Default: 5000
  • rtspTimeoutMs - RTSP timeout in milliseconds (actual timeout may be a double, if RTSP client attempts to reconnect with a different authentication mechanism). Default: 5000
  • rtspRetryCount - Number of time RTSP connection will be retried before giving up. Default: 3

VehicleAnalyticsGstRTSP

Used for processing videos streamed using RTSP. This uses the GStreamer backend which is only available on Jetson.

Required Parameters: VIDEO_IN='RTSP_URL'

Pipeline Specific Options:

  • fpsLimit - same as the VehicleAnalyticsRTSP pipeline fpsLimit option.
  • VIDEO_IN - RTSP URI to process.
  • rtspTimeoutMs - RTSP timeout in milliseconds (actual timeout may be a double, if RTSP client attempts to reconnect with a different authentication mechanism). Default: 5000
  • rtspRetryCount - Number of time RTSP connection will be retried before giving up. Default: 3

VehicleAnalyticsGstFile

Used for processing video files; same as VehicleAnalyticsFile, but using GStreamer rather than ffmpeg-based intake

Required Parameters: VIDEO_IN='Video_Path'

Pipeline Specific Options:

  • outputPrefix - save path and image filename prefix. When set the pipeline wil save images with detection boxes drawn. Default: ''
  • VIDEO_IN - file to process.

VehicleAnalyticsFolderWatch

Used for processing images using a configured "watched" folder as an input.

Required Parameters: folderPath='Valid_Folder_Path'

Pipeline Specific Options:

  • folderPath - this is the folder path that will be used to get the input images.
  • folderPollInterval - this options sets the time interval in milliseconds to wait before checking for new files. Default: 200
  • folderPollAgeMin - this options sets the time interval in milliseconds for which files are not used, to ensure they're not still being written to storage. Default: 1000
  • folderPollPrefix - only use files with the prefix set with this option, default is an empty string.
  • folderPollExtensions - list of image extensions to consider. Default: "["jpg"]"
  • folderGenerateAnalyticsFile - this option creates a JSON file with the frame data results for each image/video processed in "processed" subfolder. Default: false
  • folderRemoveSourceFiles - this option when set to true removes source files upon completion, when false, it will move them to "processed" subfolder. Default: false

VehicleAnalyticsDirect

This pipeline is used by applications which use SIO as an embedded SDK.

Required Parameters: Pipeline Specific Options:

VehicleAnalyticsImageLoop

Used for feeding a pre-configured image in a loop. This is useful for performance testing.

Required Parameters: VIDEO_IN='Image_Path'

Pipeline Specific Options:

  • VIDEO_IN - path to image to feed as input.