The previous articles detailed the theoretical approach to identifying the Vision Targets on the 2014 FRC Field. This article details the implementation in the LabVIEW code that matches this theoretical approach.
Integrating into Program
The elements of the 2014 Vision example can be incorporated into the robot program, dashboard, or contain elements in both. For more information about integrating the code into a program, see Tutorial 8 included with your LabVIEW install by clicking Tutorials -> Tutorial 8 from the Splash screen.
Finding the Example
The 2013 LaVIEW example vision code is bundled along with the other LabVIEW examples. To find the example from the LabVIEW splash screen, click Support >> Find FRC Examples then open the Vision Folder and locate 2014 Target Detection.lvproj
The Front Panel
2014 Target Detection.vi is the main VI of the project. Double clicking on this VI in the Project Explorer will show the front panel shown above. The front panel contains a variety of controls and indicators used in the example
- The File/Camera controls: This section contains the controls for selecting the camera to connect to or file to process. On the File Images tab, clicking the folder icon will allow you to browse to a directory containing the desired images, then click the image in the box to select it. On the Live Camera tab you can enter the IP address of the camera to connect to and set the resolution, framerate, and compression.
- The Camera Type: The camera type control changes the view angle used in the distance calculation based on the camera used.
- The Color Threshold Controls: This section contains the controls used for the initial thresholding operation on the image. The Color tab has controls for the Hue, Saturation and Value and margins for each (to set the width of the range). Note that these values can be set automatically by clicking and dragging a line on the desired color in the Original Image section. Remember that to save modified values, after the program has stopped running right click on the desired control and select "Data Operations" >> " Make Current Value Default"
- Area Size Limit: This controls limits the minimum particle size to be scored as a percentage of the total image size.
- Score Cutoff: This section sets the minimum limit for each score used to determine if a target is a hot target. A target pair must exceed this minimum for the Tape Width and Vertical Score and one of the two directional scores to be considered a Hot Target.
- Std Devs: This section is used to tweak the behavior of the automatic color selection. The numbers in the boxes represent the number of standard deviations from the mean that will be included in the range created when a line is drawn across the image.
- Original Image: This section contains the original unprocessed image. Clicking and dragging anywhere in this image will set the color threshold values to detect that color. Detected particles will be annotated depending on whether they are determined to be a target or not. Particles which are not targets or a pair of targets which are not considered a Hot Target will be surrounded by a red box. Non-Targets will include the Aspect ratio and Rect score highlighting in red the scores which do not meet the cutoff, not-Hot pairs will include the Left, Right, Tape Width and Vertical Scores, highlighting in red scores which do not meet the cutoff. Particles which are vertical targets determined to be not hot targets are surrounded by a teal box with the Aspect Ratio and Rectangularity scores in teal. Hot targets will be surrounded by a green box with the Left, Right, Tape Width and Vertical scores in green (the unused Left or Right score will be in grey).
- Debugging Tab: This tab shows the processed image. This is the image after thresholding, but does not show the results of the particle filtering.
- Details Tab: Clicking on the Details tab will show an array indicator (click the arrows near the top right of the tab to view other particles) containing details on each detected particle/target.
- Loop Time: This indicator shows the rate at which the loop is running.
Acquiring an Image and Clearing Overlays
The left hand side of the LabVIEW code handles image acquisition and clearing overlays. It will typically not be necessary to modify this code.
Set Threshold Values
The top right section of the code sets the HSV threshold values on a click and drag event or passes through the values set in the controls.
The Process Image for Target VI is responsible for thresholding the image, filtering out small particles and determining which particles are believed to be horizontal or vertical targets.
Target Search Details - Threshold
This section of the code performs the threshold operation on the image, creating a binary image with only the pixels which match the specified color range. You can also see the math converting the Area Cutoff % to a pixel value in the bottom center of this section.
Target Search Details - Particle Filter and Report
This section of the code filters out small particles and creates a report for each remaining particle (up to 8) which contains the specified measurements.
Target Search Details - Scoring
This section of the code scores the particles on Rectangularity and Aspect Ratio and indicates whether the particle is larger horizontally or vertically. It uses the orientation to pack up the Equivalent Rectangle measurements, along with the other values, into a cluster which is output from the VI.
Score and Rank Targets
The Score and Rank Target Objects VI iterates through each vertical target and determines the best matching horizontal target then compares the Left or Right Score, Tape Width Score and Vertical Score to determine if the target is a Hot Target. This VI also annotates the image and sorts the detected targets in the order Left Targets, Right Targets, Not Hot Targets.
The Rank HV Combos VI iterates through each Horizontal Target and computes the Left Score, Right Score, Tape Width Score and Vertical Score. It then sorts the Horizontals by Total Score (sum of Left, Right, Width and Vertical) to select the best matching Horizontal for the Vertical.
Score and Rank Targets Details - Score Compare
This section of code compares the scores to the Score Limit and determines if the target is a Hot target.
Score and Rank Target Details - Annotation
This section annotates the images with the boxes and scores for each particle.
Score and Rank Target Details - Sorting
This section of the code sorts the targets placing Left Targets first in the array, then Right Targets, then Not Hot Targets
The code in the Compute Target Distances VI implements the formula described in the previous article. The top part of the VI normalizes the target bounding box to provide a coordinate pair for the target center on the (-1,1) scale. The bottom part of the VI computes the distance.
A number of sample images are provided in the example Project Directory. The provided images are broken up into groups, one group is of a pseudo-target in the Hot position (reflective tape attached directly to the polycarbonate in the correct locations). Another group is of the pseudo-target in the Not hot position. The last group is images of the actual target setup from the kickoff filming field. All lit images were taken with a pair of green LED ring lights that nest one inside the other. While these images should help teams test algorithms quickly, it is highly recommended to utilize the reflective material provided in the Kit of Parts to create a target to test the camera and lighting setup that will be used on the robot. Note that the measurements in the image filenames are very rough and should not be taken as accurate measurements to be used for distance calibration calculations.