We are pleased to announce the release of Pupil Core software v3.4!

Download the latest bundle (scroll down to the end of the release notes and see Assets).

Please feel free to get in touch with feedback and questions via the #pupil channel on Discord! 😄

Overview

Pupil v3.4 introduces a new color scheme for pupil detection algorithms, improved gaze-estimation stability for frozen eye models, and a new IMU visualization and export plugin for Pupil Invisible recordings within Pupil Player.

New

• Dynamic pye3d model confidence

• Pupil detection color scheme

• Added IMU Timeline plugin

Improved

• Gaze-estimation stability with frozen eye models

• Annotation log message

• Prevent log flood in Surface Tracker related to finding homographies

• Support upgrading legacy recordings

New

Dynamic pye3d model confidence

Pupil Core software version v3.4 ships the new pye3d version 0.1.1.

In the previous versions of pye3d model_confidence was fixed to a value of 1.0. Starting with version 0.1.0, pye3d assigns different model_confidence values based on the long-term model's parameters. Specifically, model_confidence is set to 1.0 by default and set to 0.1 if at least one model output exceeds its physiologically reasonable range. If the ranges are exceeded, it is likely that the model is either not fit well or the 2d input ellipse was a false detection.

The assumed physiological bounds are:

• phi: [-80, 80] degrees

• theta: [-80, 80] degrees

• pupil diameter: [1.0, 9.0] mm

• eyeball center x: [-10, 10] mm

• eyeball center y: [-10, 10] mm

• eyeball center z: [20, 75] mm

Phi and theta ranges are relative to the eye camera's optical axis. The eye ball center ranges are defined relative to the origin of the eye camera's 3d coordinate system.

The model_confidence will be set to 0.0 if the gaze direction cannot be calculated.

Pupil detection color scheme

The new pupil detection color scheme makes use of the dynamic model confidence values by visualizing within-bounds and out-of-bounds models in different colors.

Color legend:

• #FAC800: Pupil ellipse (2d)

• #F92500: Pupil ellipse (3d)

• #0777FA: Long-term model outline (within-bounds)

• #80E0FF: Long-term model outline (out-of-bounds)

Note: Even though the detected pupil ellipses fit the pupil well on the right-side picture, the model is outside of the physiological ranges defined above. This can lead to errors in gaze-direction and pupil-size estimates. You can improve such a model by looking into different directions.

While the 3d debug window is opened, pye3d also visualizes the short- and ultra-long-term model outlines:

• #FFB876: Short-term model outline

• #FF5C16: Ultra-long-term model outline

Added IMU Timeline plugin - #2151

This is a new plugin designed to visualize and export IMU data available in Pupil Invisible recordings.

In addition to the raw angular velocity and linear acceleration, the plugin also uses Madgwick's algorithm to estimate pitch and roll.

To export the data, the plugin must be enabled and a raw data export must be triggered. Once the export is complete, there should be a imu_timeline.csv file with the following columns

Improved

Improve gaze-estimation stability with frozen eye models

Freezing the eye models is the best way to reduce noisy data when in a controlled environment. Prior to this release, freezing the eye model only affected eye ball center and pupil diameter measurements. Starting with this version, freezing the eye model also stabilizes gaze direction estimates.

In order to adapt to slippage, pye3d estimates eyeball positions on different time scales. These serve specific downstream purposes. By default, the short-term model, which integrates only the most recent pupil observations, is used for calculating raw gaze-direction estimates.

In highly controlled environments with few eye movements (e.g. subject uses head rest and is asked to fixate a static target), the most recent pupil observations might not be sufficient to build a stable short-term model. If this is the case, the gaze direction varies more than one would expect given the controlled environment.Starting in pye3d version 0.1.0, the gaze direction will be inferred from the long-term model if it is frozen.

Freezing the eye model is only recommended in controlled environments as this prevents the model from adapting to slippage. In turn, it is able to provide very stable data.

Improve annotation log message - #2148

When creating a new annotation, the log message will no longer show the Pupil Timestamp. Instead, Pupil Capture will show how old the annotation is, while Pupil Player will show the frame index of the world video.

This change should help avoid the confusion associated with displaying Pupil Timestamp.

Prevent log flood in Surface Tracker related to finding homographies - #2150

Previously, failing to locate a surface because of missing homographies would print an error message in the main window. In some cases, this would result in a flood of error logs.

This change ensures that failures of such type are only issued once, and are written to the log file directly.

Support upgrading legacy recordings - #2161

Pupil Core software version v1.16 is no longer necessary to upgrade deprecated recording formats. Instead, Pupil v3.4 performs all necessary upgrades by itself. This release also includes a bug fix for an issue with pre v0.7.4 recordings.

In v1.16, recordings made with Pupil Capture v1.2 or earlier, andPupil Mobile r0.22.x or earlier have been deprecated due to the fact that these recordings are missing meta information that is required for the upgrade to the Pupil Player Recording Format 2.0. For details see "Missing Meta Information" in the v1.16 release notes.

Fixed

• Use UTF-8 decoding for known UTF-8 encoded text files - #2146

• Update circle drawing to display correctly on Apple Silicon devices - #2147

• Fix crash on surface tracker export if filling the marker cache is not complete - #2157

• Fix surface tracker marker export to use export range - #2158

Developer notes

Dependencies updates

Python dependencies can be updated using pip and the requirements.txt file:

python -m pip install --upgrade pip wheel
pip install -r requirements.txt

pyglui v1.30.0

This new version of pyglui includes the menu icon for the new IMU timeline plugin as well as the new Color_Legend UI element.

from pyglui import ui
color_rgb_red = (1.0, 0.0, 0.0)
label = "red color label"
ui.Color_Legend(color_rgb_red, label)

pye3d v0.1.1

This version improves gaze-estimation stability for frozen eye models. For details see above and the pye3d changelog.

Downloads

To open the RAR-archive on Windows, you will need to use decompression software, such as WinRAR or 7-Zip (both are available for free).

Supported operating systems

• Windows 10

• Ubuntu 16.04

• macOS Mojave 10.14

• macOS Monterey 12 is not supported by this release