## Camera The corridor camera is used by the system itself, to let only one animal at a time into the box (see [Animal Detection & Tracking](../system_operation/detection.md)). The box camera records every session and, while a task runs, tracks the animal's position. This page explains how to use the box camera from your code: reacting to where the animal is, changing the shape of the detection areas, and drawing on the image. --- ### Setup The box camera has up to four detection areas. Their position and their usage — `ALLOWED`, `NOT_ALLOWED`, `TRIGGER` or `OFF` — are set in `MONITOR` → `DETECTION SETTINGS` (see [Detection in the Operant Box](../system_operation/detection.md#detection-in-the-operant-box)). The animal's position is only tracked when `CAM_BOX_TRACKING_POSITION` is ON (`SETTINGS` → `CAMERA SETTINGS`), which is needed for `TRIGGER` areas and for reacting to the animal's position. The lens distortion of the cameras can be measured with the [Camera Calibration](../calibrations/camera.md). --- ### Using It in Tasks The simplest way to overlay text on the camera feed is to call `write_text()` from the task. The text is rendered on every subsequent frame at a fixed position in the status bar and stays visible until it is changed or cleared. ```python # Inside the task (self.cam_box is the box camera): self.cam_box.write_text("reward delivered") # persists across frames until changed self.cam_box.write_text("") # clear ``` From a trigger, the box camera is `self.task.cam_box`. From a direct function, or from any other file, import it: ```python from village.devices.camera import cam_box cam_box.write_text("reward delivered") ``` The annotation is also saved frame by frame in the CSV file of the camera, saved next to the video in the videos directory (column `annotation`, together with the time and the position of the animal in each frame), so it is useful for marking task events for post-hoc analysis. For anything beyond plain text — shapes, coloured overlays, position-dependent graphics — see [Custom Camera Drawing](#custom-camera-drawing) below. --- ### Reacting to the Animal's Position While a task is running, the `trigger` method of `CameraTriggerBase` is called on every frame of the box camera. Before each call the camera updates, for each of the four areas: - `cam.areaN_is_triggered` — `True` if area N is a TRIGGER area and the tracked (x, y) position of the animal is inside it. It needs `CAM_BOX_TRACKING_POSITION` ON, and it is `False` on frames where no animal is tracked. - `cam.areaN_animals` — how many animals are in area N, from its pixel count and the two limits of `DETECTION_OF_MOUSE_BOX`: `0` (below `empty_limit`), `1` (below `subject_limit`), `2` (two or more) or `-1` (the area is OFF). It works for any active area (ALLOWED, NOT_ALLOWED or TRIGGER) and does not need tracking. `trigger` is only called once the `start` of the task has finished, so it can read any attribute `start` creates, and never in calibrations. An error raised inside it sends an "Error in the camera trigger" alarm (at most one camera alarm per hour, with the traceback in the events log), and the frame is still processed and recorded. The default implementation checks each of the four configurable areas (the rectangles defined in the GUI). For any area set to TRIGGER status, if the animal is tracked inside it, the text "Area X triggered" is overlaid on the camera feed. You can override this behavior by creating a custom class in your project. Create a file inside your project's `code` directory (e.g. `camera_trigger.py`) and define a class (e.g. `CameraTrigger`) that inherits from `CameraTriggerBase`. The file and class names are up to you: the system detects the class by what it inherits from, and uses it instead of the default base class. Only one such class is used per project. In the following example, a subclass is created that writes a text when the subject is detected in Area1, executes direct function number 2 when it enters Area2, and triggers function number 6 when it enters a circular region at the center of the image. ```{admonition} trigger runs on every frame :class: warning `trigger` is called on every frame (`CAM_BOX_FRAMERATE` times per second, 30 by default), and `cam.areaN_is_triggered` stays `True` the whole time the animal is inside the area. To act only once when the animal **enters**, remember whether it was already inside, as the example below does (or as [camera_trigger.py](../examples/camera_trigger.md) does with a signal the task resets every trial). ``` ```python from village.custom_classes.camera_trigger_base import CameraTriggerBase from village.devices.camera import Camera class CameraTrigger(CameraTriggerBase): """Implement specific actions when an animal enters a detection area or reaches a specific position in the camera feed. You have access to self.task, so any variable or function defined in the task can be used to specify the actions to perform. """ def __init__(self) -> None: super().__init__() self.in_area2 = False self.in_center = False def trigger(self, cam: Camera) -> None: """Evaluates camera triggers and performs corresponding actions. Called on every frame of the box camera while the task is running, also when no animal is detected. Available area triggers (True if the area is a TRIGGER area and the tracked position of the subject is inside it; False when no animal is tracked): - cam.area1_is_triggered ... cam.area4_is_triggered Available area occupancy (any active area, from its pixel count): - cam.area1_animals ... cam.area4_animals: 0, 1, 2 (two or more) or -1 (the area is OFF) Available position properties: - cam.x_position, cam.y_position (subject position in pixels, -1 if no animal is tracked) - cam.width, cam.height (camera feed dimensions in pixels) Other: - cam.write_text("text") to overlay text on the camera feed - self.task.system_state: the current state of the Village system Args: cam (Camera): The camera instance providing trigger status and position data. """ # Check if the animal is in some of the defined areas if cam.area1_is_triggered: cam.write_text("Area 1 triggered") # write an annotation text to the cam # Execute the direct function number 2 once, when the animal enters Area2 if cam.area2_is_triggered and not self.in_area2: self.task.execute_function(2) self.in_area2 = cam.area2_is_triggered # Execute the direct function number 6 once, when the animal gets within a # 50-pixel radius of the center distance = ((cam.x_position - cam.width / 2) ** 2 + (cam.y_position - cam.height / 2) ** 2) ** 0.5 in_center = cam.x_position != -1 and distance < 50 if in_center and not self.in_center: self.task.execute_function(6) self.in_center = in_center ``` ```{admonition} Note :class: note `cam.areaN_is_triggered` also works transparently for a BOX area whose shape has been replaced by a `CameraAreaBase` (see [Custom Detection Area](#custom-detection-area)) — it checks the actual polygon/circle, not just its bounding box. ``` --- ### Custom Detection Area The 4 BOX areas are rectangles by default, positioned and resized from the GUI. If one of them needs to track the animal through a non-rectangular region (e.g. an L-shaped corridor between two zones, or a circular zone), subclass `CameraAreaBase` to **replace that area's shape** with a union of polygons and/or circles defined in code. ```{admonition} Note :class: note This replaces the shape of one of the 4 existing areas — it does not add a 5th area. Its status (`ALLOWED`/`NOT_ALLOWED`/`TRIGGER`/`OFF`) and its threshold keep working exactly as before, still set from the GUI (`MONITOR` -> `DETECTION SETTINGS`). Only its position controls are hidden from the GUI, since the shape is no longer a plain rectangle — a "defined with code" label is shown in their place instead, listing the area's `polygons`/ `circles` coordinates (there's no draggable position control for them). ``` Create a file inside your project's `code` directory and define a class that inherits from `CameraAreaBase`, setting `area_index` to the area it replaces (1-4). The system detects it automatically. Create the area by adding polygons and/or circles — any number of each (zero or more), in any combination. ```python from village.custom_classes.camera_area_base import CameraAreaBase class LCorridor(CameraAreaBase): name = "L_CORRIDOR" area_index = 2 # replaces AREA2_BOX # Polygons of [x, y] pixel vertices. Concave shapes are fine, # and several polygons make a disjoint area (here: an L as two rectangles). polygons = [ [[100, 50], [300, 50], [300, 100], [100, 100]], # top bar [[100, 100], [150, 100], [150, 250], [100, 250]], # side ] # Circles, each (x, y, radius). Combined with polygons in the same area. circles = [(200, 300, 40)] ``` ```{admonition} Note :class: note Only one `CameraAreaBase` per `area_index` is used — if two subclasses target the same area, only the first one found is registered and an error is logged. An invalid `area_index` (not 1-4) is also rejected and logged. ``` #### Moving the shape at runtime `polygons`/`circles` don't have to stay fixed for the whole session — from within a task, reach the registered instance through `self.custom_areas[area_index]` and reassign them (e.g. to relocate a "home zone" the animal must enter to start each trial). After changing either one, call `update_area()` so the change is picked up on the next frame: ```python area = self.custom_areas[2] area.circles = [(new_x, new_y, 40)] area.update_area() ``` ```{admonition} Note :class: note The mask/bounding box/outline are cached and only rebuilt automatically when the camera's frame size changes — `update_area()` marks that cache stale so it rebuilds from the new `polygons`/`circles` instead. Without it, the shape keeps reacting to its old position until something else happens to trigger a rebuild. ``` --- ### Custom Camera Drawing `CameraDrawBase` exposes two methods that are called automatically on every frame: | Method | Renderer | Destination | |--------|----------|-------------| | `draw(cam)` | cv2 (modifies `cam.frame`) | **disk and screen** | | `draw_preview(cam, painter)` | QPainter (widget overlay) | **screen only** | The default implementation of `draw` writes the status bar texts and pixel counts for both cameras, and for the CORRIDOR camera also draws the thresholded detection mask and area rectangles (if VIEW_DETECTION is active). The default `draw_preview` draws those same overlays for the BOX camera via QPainter so they appear on screen but are not encoded into the video file. `draw` runs on every frame of both cameras, also when no task is running. An error raised inside it sends an "Error in the camera draw" alarm (at most one camera alarm per hour, with the traceback in the events log), and the frame is still recorded, without the drawing. Create a file inside your project's `code` directory (e.g. `camera_draw.py`) and define a class (e.g. `CameraDraw`) that inherits from `CameraDrawBase`. The file and class names are up to you: the system detects the class by what it inherits from, and uses it instead of the default base class. Only one such class is used per project. **Overriding `draw`** — runs on every frame, changes are saved to disk: ```python from village.custom_classes.camera_draw_base import CameraDrawBase import cv2 class CameraDraw(CameraDrawBase): def __init__(self) -> None: super().__init__() def draw(self, cam) -> None: super().draw(cam) # keep default status bar and detection overlays # Add a red circle at the animal's position (BOX only, task running) if cam.name == "BOX" and cam.task_is_running and cam.x_position != -1: cv2.circle( cam.frame, (cam.x_position, cam.y_position), radius=20, color=(0, 0, 255), # BGR thickness=2, ) ``` **Overriding `draw_preview`** — runs at preview framerate, screen only: ```python from PyQt5.QtCore import QRect from PyQt5.QtGui import QBrush, QColor from village.custom_classes.camera_draw_base import CameraDrawBase class CameraDraw(CameraDrawBase): def draw_preview(self, cam, painter) -> None: super().draw_preview(cam, painter) # keep default detection overlays # Highlight the reward zone on screen (never saved to video) zone = cam.items_to_draw.get("reward_zone") if zone and cam.name == "BOX": device = painter.device() sx = device.width() / cam.width sy = device.height() / cam.height x, y, w, h = zone painter.setPen(QColor(0, 255, 0)) painter.setBrush(QBrush()) painter.drawRect(QRect(int(x*sx), int(y*sy), int(w*sx), int(h*sy))) ``` Both methods can be overridden in the same class. ````{admonition} Passing data from the task :class: note `cam.items_to_draw` is a plain `dict` the task can populate at any point to pass coordinates, labels, or any other data (like `"reward_zone"` in the example above). ```python # Inside the task, at any point: self.cam_box.items_to_draw["reward_zone"] = (200, 150, 80, 80) # x, y, w, h self.cam_box.items_to_draw["reward_zone"] = None # remove ``` ```` A complete example, which draws circles requested by the task (saved in the video) and the animal's recent trail (on screen only): [camera_draw.py](../examples/camera_draw.md). ```{admonition} draw must never fail :class: warning As with `trigger`, an error raised inside `draw` is not caught: that frame loses its row in the CSV file of the camera. Check that every value you read exists (e.g. `cam.items_to_draw.get("circles") or []` instead of `cam.items_to_draw["circles"]`). ``` Both methods `draw` and `draw_preview` receive these attributes on `cam` (updated every frame): **Identity & state** - `cam.name` — `'BOX'` or `'CORRIDOR'`. - `cam.task_is_running` — `True` if a task is active, `False` if not. - `cam.view_detection` — whether detection overlays are enabled in the GUI. - `cam.tracking` — whether animal position tracking is active (BOX only). - `cam.color` — `Color.BLACK` or `Color.WHITE`: which pixels to detect. **Frame data** - `cam.frame` — current frame as a BGR numpy array (read/write, cv2 only). - `cam.width`, `cam.height` — frame dimensions in pixels. - `cam.frame_number` — frames captured since recording started, saved in the `frame` column of the CSV file of the camera. In the BOX camera it is 0 whenever no task is running, so the last frames of a session (after the animal is back home) are saved with 0: use the `timestamp` column to align the video. - `cam.timing` — milliseconds elapsed since recording started. **Session info** - `cam.filename` — base name of the current video file (empty if not recording). - `cam.trial` — current trial number (0 if no task running or CORRIDOR camera). - `cam.annotation` — current `write_text()` string; persists until changed. - `cam.items_to_draw` — plain `dict` for passing arbitrary data from the task (see above). **Detection areas** - `cam.number_of_areas` — number of configurable areas (always 4). In the CORRIDOR camera with `CORRIDOR_AREA_EXTRA` ON, the lists below (`cam.areas`, `cam.areas_active`... `cam.masks`, `cam.counts`) have a 5th element: the extra area. - `cam.areas` — list of `[x1, y1, x2, y2]` pixel coordinates per area. For a BOX area overridden by a `CameraAreaBase`, this is the shape's bounding box, not a real detection rectangle. - `cam.areas_active` — bool list, whether each area is enabled (not OFF). - `cam.areas_allowed` — bool list, `True` only for ALLOWED areas. - `cam.areas_not_allowed` — bool list, `True` only for NOT_ALLOWED areas. - `cam.areas_trigger` — bool list, `True` only for TRIGGER areas. - `cam.custom_areas` — `dict[int, CameraAreaBase]`, BOX area index (1-4) to the `CameraAreaBase` overriding that area's shape, if any (see [Custom Detection Area](#custom-detection-area)). **Detection results** - `cam.masks` — grayscale numpy arrays with the thresholded mask per area. - `cam.counts` — detected pixel count per area (`-1` if the area is OFF). - `cam.area1_is_triggered` ... `cam.area4_is_triggered` — tracked position inside a TRIGGER area. BOX only. - `cam.area1_animals` ... `cam.area4_animals` — animals in each area from its pixel count: `0`, `1`, `2` (two or more) or `-1` (OFF). BOX only. - `cam.x_position`, `cam.y_position` — tracked animal position in pixels (`-1` if not detected). **Task access** - `self.task` — the current task instance. ```{toctree} :hidden: Camera Draw Example <../examples/camera_draw.md> Camera Trigger Example <../examples/camera_trigger.md> ```