What PhotonVision Is

PhotonVision is an open-source vision processing platform built specifically for FRC. It runs as a service on a dedicated coprocessor — typically a Raspberry Pi 4, Orange Pi 5, or similar single-board computer — separate from the roboRIO. The coprocessor connects to one or more USB cameras, runs the image processing pipeline, and publishes detection results to NetworkTables on the robot's network.

The separation from the roboRIO is intentional and important. AprilTag detection is computationally expensive. Running it on the roboRIO would consume CPU time needed for the 20 ms control loop. By offloading detection to a dedicated coprocessor, the roboRIO only receives the finished results: tag IDs, transforms, and ambiguity scores. The roboRIO's loop time remains unaffected.

PhotonVision exposes a browser-based web dashboard for configuration — you connect to it from your laptop over the robot network. No SSH, no command line required for basic setup. Once configured, the settings persist through power cycles. The Java library (PhotonLib) provides the API your robot code uses to read results.

Installation

Installing PhotonVision on the Coprocessor

Download the latest PhotonVision release from github.com/PhotonVision/photonvision/releases. Choose the image file for your specific coprocessor hardware (Raspberry Pi 4, Orange Pi 5, etc.). Flash the image to a microSD card using Balena Etcher or a similar tool. Insert the card into the coprocessor, connect it to the robot's network switch, and power it on.

After approximately 30–60 seconds of boot time, PhotonVision is accessible at http://photonvision.local:5800 from any laptop on the robot network. If mDNS resolution isn't working, try the IP address directly — check your router's DHCP table or use a network scanner to find the coprocessor's IP.

Adding PhotonLib to Your Robot Project

// In VS Code: W key → "Manage Vendor Libraries" → "Install new libraries (online)"
// Paste this URL for the current release:
// https://maven.photonvision.org/repository/internal/org/photonvision/
//         PhotonLib-json/1.0/PhotonLib-json-1.0.json
//
// The exact URL changes each season — always get the current one from:
// docs.photonvision.org/en/latest/docs/programming/photonlib/adding-vendordep.html
//
// After installing, verify the dependency appears in vendordeps/photonlib.json
// and that your build.gradle shows it in the dependencies block.

// Verify the install compiled cleanly by adding this import to any Java file:
import org.photonvision.PhotonCamera;
// If VS Code resolves it without errors, PhotonLib is installed correctly.

Configuring an AprilTag Pipeline

A PhotonVision pipeline is a named configuration that tells the coprocessor how to process images from one camera. Each camera can have multiple pipelines (e.g., one for AprilTags, one for game pieces), and you switch between them from your robot code or the dashboard. Walk through the configuration steps below — each one is a required setting for reliable AprilTag detection.

Critical Camera Settings for AprilTag Detection

These four settings have the largest impact on detection quality. PhotonVision exposes them in the camera settings panel (the gear icon next to each camera in the dashboard). These are hardware-level settings that apply to all pipelines on that camera.

Camera Calibration: Why It's Required

In Lesson 1 we learned that solvePnP converts the tag's corner pixel coordinates into a 3D pose using the camera's intrinsic calibration. The calibration tells solvePnP how the camera's lens maps 3D world points to 2D pixel coordinates — specifically: focal length (how much the lens zooms), principal point (where the optical axis intersects the sensor), and distortion coefficients (how the lens bends straight lines).

Without calibration, solvePnP uses inaccurate lens parameters. The result is systematic distance error: the algorithm thinks the camera sees the world from a slightly different geometry than it actually does. A 10% error in focal length produces roughly 10% error in distance measurements — at 4 meters, that's 40 cm of systematic bias in every pose estimate. This bias is not random; it's consistent and predictable, which makes it look like the vision system is "almost working" while being persistently wrong.

The Calibration Workflow

Reading Detections with PhotonLib

Once PhotonVision is configured and the coprocessor is running, your robot code reads detection results using the PhotonLib Java library. The interface is straightforward: create a PhotonCamera with the camera's name (as set in the PhotonVision dashboard), poll getLatestResult() in periodic(), and iterate the detected targets.

import org.photonvision.PhotonCamera;
import org.photonvision.targeting.PhotonPipelineResult;
import org.photonvision.targeting.PhotonTrackedTarget;

public class VisionSubsystem extends SubsystemBase {

    // Camera name must EXACTLY match the name in PhotonVision's dashboard.
    // Case-sensitive. Check the camera name in the top-left dropdown of the UI.
    private final PhotonCamera m_frontCamera = new PhotonCamera("FrontCamera");

    private PhotonPipelineResult m_latestResult = new PhotonPipelineResult();

    @Override
    public void periodic() {
        // getLatestResult() returns the most recent detection frame from NetworkTables.
        // Call this once per loop and cache the result — calling it multiple times
        // in one loop can return different frames if processing is fast.
        m_latestResult = m_frontCamera.getLatestResult();

        // Log detection count for debugging
        SmartDashboard.putBoolean("Vision/HasTargets", m_latestResult.hasTargets());
        SmartDashboard.putNumber("Vision/TargetCount",
            m_latestResult.getTargets().size());
    }

    public PhotonPipelineResult getLatestResult() {
        return m_latestResult;
    }

    // Example: get the best target (highest decision margin / lowest ambiguity)
    public Optional<PhotonTrackedTarget> getBestTarget() {
        if (!m_latestResult.hasTargets()) {
            return Optional.empty();
        }
        return Optional.of(m_latestResult.getBestTarget());
    }
}

// Given a PhotonTrackedTarget from getLatestResult().getTargets():
PhotonTrackedTarget target = result.getBestTarget();

// ── Identity ─────────────────────────────────────────────────────────────
int    tagId     = target.getFiducialId();
// Returns -1 if the target is not an AprilTag (e.g., a colored target)

// ── Pose confidence ──────────────────────────────────────────────────────
double ambiguity = target.getPoseAmbiguity();
// 0.0 = two solvePnP solutions equally likely (don't trust this detection)
// 1.0 = one solution strongly preferred (high confidence)
// Reject detections where ambiguity < 0.2 (covered in Lesson 8)

// ── Camera-relative geometry ──────────────────────────────────────────────
Transform3d camToTag = target.getBestCameraToTarget();
double distance = camToTag.getTranslation().getNorm();
// getNorm() = Euclidean 3D distance in meters

double yaw   = target.getYaw();    // degrees, positive = tag is to the right
double pitch = target.getPitch();  // degrees, positive = tag is downward
double area  = target.getArea();   // % of frame area, useful for distance estimate

// ── All targets in this frame ─────────────────────────────────────────────
for (PhotonTrackedTarget t : result.getTargets()) {
    if (t.getFiducialId() == 7) {
        // Found tag 7 — process it
    }
}

// ── Timestamp — important for vision-odometry fusion ─────────────────────
double captureTimestamp = result.getTimestampSeconds();
// This is when the IMAGE WAS CAPTURED, not when your code received it.
// SwerveDrivePoseEstimator.addVisionMeasurement() needs this timestamp
// to correctly weight the measurement against historical odometry data.

Three Common PhotonVision Configuration Mistakes

These are the three mistakes that produce hard-to-diagnose failures — the code runs without errors but the robot doesn't behave correctly.

Mistake 1: Wrong camera name in robot code

new PhotonCamera("FrontCamera") must exactly match the name shown in PhotonVision's dashboard dropdown — including capitalization, spacing, and special characters. A mismatch produces a camera object that silently returns empty results. getLatestResult().hasTargets() always returns false. Check with SmartDashboard.putBoolean("Camera/Connected", m_camera.isConnected()) — if it returns false and the coprocessor is running, the name is wrong.

Mistake 2: No calibration, or calibration at wrong settings

An uncalibrated camera produces getBestCameraToTarget() transforms where the distance is systematically wrong. The camera detects tags and returns transforms, but every distance estimate is off by a consistent factor. If you compute robot poses from these and they're always too close to or too far from the tag's actual position, check the calibration. Verify the RMS error is shown in the Calibration tab and is below 0.5.

Mistake 3: Wrong tag family in the pipeline

If PhotonVision is configured for tag16h5 (a legacy family) but the field uses 36h11, the pipeline will either detect nothing at all or produce false positives from random dark patterns that accidentally match the smaller 4×4 bit grid. Verify the tag family in the pipeline settings: open the pipeline's configuration, find the "AprilTag Family" dropdown, and confirm it shows "TAG_36H11".

🔌 System Check

Knowledge Check