Skip to main content
Version: Release

API Docs (C++)

Overview

ClassDescription
GazeTrackerClass that generates gaze tracking data
IGazeCallbackGaze data listener
ICalibrationCallbackCalibration process listener
IUserStatusCallbackUser status data listener
CoordConverterV2Matrix-based coordinate converter
ErrorReporterError listener base class
SeeSoBooleanSubstitution for bool type
SeeSoStatusModuleOptionsEnum with flag for each module when using StatusModule
SeeSoCalibrationPointNumEnum with number of points for calibration
SeeSoCalibrationAccuracyEnum containing options that manage calibration process and accuracy
SeeSoTrackingStateEnum containing tracking state types used in OnGaze callback
SeeSoEyeMovementStateEnum containing eye movement state types used in OnGaze callback
UserStatusOption(deprecated) Class containing user status option information for GazeTracker

global_init


void global_init(const char* file = "seeso_core.dll")

This function sets up the program environment that SeeSo SDK needs.
This function must be called at least once within a program before the program calls any other function in SeeSo SDK.
It's a dll loader in actual. On other platforms except Windows, this function is no-op.

Exceptions

Failed to load a dll

SeeSo SDK requires the dll files to execute(which are included in the SDK files), including seeso_core.dll and several third party library dlls.
See seeso/bin or error logs for such dll files.

seeso::global_init() will throw if one or more of these files are missing.
Make sure that they're in the same directory with an executable file, or use add_custom_command in CMakeLists.txt as the following.
If add_custom_command isn't working, copy and paste the dll files to the directory(where an executable file is located) manually.

Failed to find XXX from dll

Dll files in the SeeSo SDK are broken, or wrong version is used.
Re-download the SDK from SeeSo Console and try again.


getVersionStr

std::string getVersionStr()

return SeeSo SDK version information in string.

ReturnTypeDescription
version_strstd::stringSeeSo SDK version in std::string

Example

const std::string version = seeso::getVersionStr();

getVersionInt

int32_t getVersionInt()

Return SeeSo SDK version as 9 digit integer.
Version forms XXYYZZZZZ, where XX is a major version, YY is a minor version, and ZZZZZ is a patch version.

ReturnTypeDescription
version_intint32_tSeeSo SDK version in 9 digit integer

Example

int32_t version = seeso::getVersionInt();

GazeTracker


class GazeTracker

GazeTracker is the class that generates GazeTracking data.

Summary

Constructor and Destructor
GazeTracker(const converter_type&)
~GazeTracker()
Functions
initialized
deinitialized
isTrackerInitialized
setFaceDistance
getFaceDistance
setGazeCallback
setCalibrationCallback
setUserStatusCallback
removeGazeCallback
removeCalibrationCallback
removeUserStatusCallback
setTrackingFPS
addFrame
startCalibration
stopCalibration
startCollectSamples
setCalibrationData
setAttentionRegion
getAttentionRegion
removeAttentionRegion
setAttentionInterval
getAttentionScore
converter

GazeTracker

explicit GazeTracker(const converter_type& coord_converter = makeNoOpConverter<float>())
ParametersTypeDescription
coord_converterconverter_typeCoordinate system converter. See Custom Coordinate

Example

// Coordinate system is camera-centered millimeters coordinate(default)
GazeTracker gaze_tracker();

// Coordinate system is display pixel coordinate
const auto camera2display = seeso::makeDefaultCameraToDisplayConverter<float>(...);
GazeTracker gaze_tracker_custom(camera2display);

~GazeTracker

~GazeTracker()

Effectively calls GazeTracker::deinitialize() and releases internal object.


initialize

int GazeTracker::initialize(
const std::string& licenseKey,
const SeeSoStatusModuleOptions& status_option = {kSeeSoFalse, kSeeSoFalse, kSeeSoFalse},
ErrorReporter* error_reporter = DefaultErrorReporter())

⚠️ Below function has been removed since

int GazeTracker::initialize(
const std::string& licenseKey,
const UserStatusOption& option = UserStatusOption())

GazeTracker construction requires authentication.

This example will pass SeeSoStatusModuleOptions to activate User Status Detector in GazeTracker.

After initialization, authentication result will be displayed as a log with status code.

ParametersTypeDescription
licenseKeyconst std::string&Key generated from SeeSo Console
status_optionconst SeeSoStatusModuleOption&User-status-module option
error_reporterErrorReporter*Optional ErrorReporter. GazeTracker does not own the object.
Return TypeDescription
intAuthentication code. See Authentication for more details.

Example

SeeSoStatusModuleOptions status_options = {};
userStatusOption.use_attention = kSeeSoTrue;
userStatusOption.use_blink = kSeeSoTrue;
userStatusOption.use_drowsiness = kSeeSoTrue;

const int errCode = gaze_tracker.initialize('licenseKey', status_options);
if (errCode == 0) {
// Initialization success
} else {
// Initialization failed
}

deinitialize

void GazeTracker::deinitialize()

Deinitialize internal object. Calling other member functions(except for initialize) after calling deinitialize() is undefined behavior.


isTrackerInitialized

bool GazeTracker::isTrackerInitialized() const;
Return TypeDescription
boolReturns true if initialized, false otherwise.

setFaceDistance

void GazeTracker::setFaceDistance(int cm)

Set face distance from the camera in cm.

To get better GazeTracking performance, accurate face distance should be set by this function.

Default valule is 60 (cm).

ParametersTypeDescription
faceDistanceintDistance between user face and camera in cm.

getFaceDistance

int getFaceDistance() const
Return TypeDescription
intDistance between user face and camera in cm.

setGazeCallback

void GazeTracker::setGazeCallback(seeso::IGazeCallback *listener)

Set Gaze callback for GazeTracking.
Note that GazeTracker does not own the instance.

ParametersTypeDescription
listenerIGazeCallback*Address of gaze listener instance

Example

seeso::IGazeCallback gaze_callback;
gaze_tracker.setGazeCallback(&gaze_callback);

setCalibrationCallback

void GazeTracker::setCalibrationCallback(seeso::ICalibrationCallback* listener);

Set Calibration callback for GazeTracking.
Note that GazeTracker does not own the instance.

ParametersTypeDescription
listenerICalibrationCallback*Address of calibration listener instance.

Example

seeso::ICalibrationCallback calibration_callback;
gaze_tracker.setCalibrationCallback(&calibration_callback);

setUserStatusCallback

void GazeTracker::setUserStatusCallback(seeso::IUserStatusCallback* listener)

Set User Status callback for GazeTracking.
Note that GazeTracker does not own the instance.

ParametersTypeDescription
listenerIUserStatusCallback*Address of status callback instance.

Example

seeso::IUserStatusCallback status_callback;
gaze_tracker.setUserStatusCallback(&status_callback);

removeGazeCallback

void GazeTracker::removeGazeCallback()

Remove gaze callback interface for GazeTracking.


removeCalibrationCallback

void GazeTracker::removeCalibrationCallback()

Remove calibration callback interface for GazeTracking.


removeUserStatusCallback

void GazeTracker::removeUserStatusCallback()

Remove user status callback interface for GazeTracking.


setTrackingFPS

void GazeTracker::setTrackingFps(int fps)

Set the maximum InputFrames-per-second of the GazeTracking.
FPS can be lower than the set value due to the device spec.

It limits the calculation by omitting the given frame internally.

ParametersTypeDescription
fpsintMaximum FPS(frames per seconds) for gaze tracking

Example

gaze_tracker.setTrackingFps(15);

addFrame

bool GazeTracker::addFrame(int64_t time_stamp, uint8_t *buffer, int width, int height)

Add image frame into GazeTracker for GazeTracking.

addFrame should be called continuously, and add image frame from the camera stream.

The image should be in RGB color format.

Note that some frames can be ignored inside SeeSo SDK, thus not calculated

  • If frames are input very often(Don't worry; SeeSo SDK will perform 30+ FPS in real-time on most of the latest hardware).
  • Even if the hardware specs are good, some frames can be ignored to guarantee OnGazeListener::OnGaze()'s timestamp is in increasing order. This happens rarely.

For each calculated input image, OnGazeListener::OnGaze() will be called with the input timestamp and the calculated values.
Image data is copied inside the function, so it's safe to destroy an image object after calling addFrame.

ParametersTypeDescription
time_stampint64_tTimestamp of the given frame
bufferuint8_t*Image data buffer in RGB format. It is safe to release buffer after passing to GazeTracker since the buffer is deep-copied.
widthintWidth of the image
heightintHeight of the image
Return TypeDescription
boolReturns false if the input frame is omitted, true otherwise.

Example

cv::Mat frame, input;
while(true) {
video >> frame; // read frame
if (frame.empty()) break;

cv::cvtColor(frame, input, cv::COLOR_BGR2RGB); // convert color format

gaze_tracker.addFrame(current_time(), input.data, input.cols, input.rows);
}

startCalibration

void GazeTracker::startCalibration(
SeeSoCalibrationPointNum point_num,
SeeSoCalibrationAccuracy criteria,
float left, float top, float right, float bottom);

⚠️ Below function has been removed since .

void GazeTracker::startCalibration(
TargetNum num,
CalibrationAccuracy criteria,
float left, float top, float right, float bottom);

Start calibration process with CalibrationMode and Criteria.

calibrationMode is number of calibration points.

criteria is option that manage calibration process and accuracy.

ParametersTypeDescription
point_numSeeSoCalibrationPointNumNumber of points used for calibration
criteriaSeeSoCalibrationAccuracyAccuracy option of the calibration
leftfloatDistance to left edge of the calibration ROI
topfloatDistance to top edge of the calibration ROI
rightfloatDistance to right edge of the calibration ROI
bottomfloatDistance to bottom edge of the calibration ROI

Example

gaze_tracker.startCalibration(
kSeeSoCalibrationPointFive,
kSeeSoCalibrationAccuracyDefault,
0.0f, 0.0f, 1280.0f, 720.0f);

stopCalibration

void GazeTracker::stopCalibration()

Stop the current calibration process.


startCollectSamples

void GazeTracker::startCollectSamples()

Start collect next calibration point data.

This should be called in onCalibrationProgress when need to collect new calibration point data.

The onCalibrationProgress will give progress (0.0 ~ 1.0), which represents the data collecting progress of current calibration point.

Example

if (isCalibrating) {
gaze_tracker.startCollectSamples();
}

setCalibrationData

void GazeTracker::setCalibrationData(const std::vector<float>& calibrationData)

⚠️ Below function has been removed since .

void GazeTracker::setCalibrationData(std::vector<float> calibrationData)

Set calibration data without calibrating manually.

ParametersTypeDescription
calibrationDataconst std::vector<float>&Stored calibration data

Example

  gaze_tracker.setCalibrationData(calibrationData);

setAttentionRegion

void GazeTracker::setAttentionRegion(float left, float top, float right, float bottom);

Set Status ROI.

ParametersTypeDescription
leftfloatSet distance to the left edge of the ROI rectangle
topfloatSet distance to the top edge of the ROI rectangle
rightfloatSet distance to the right edge of the ROI rectangle
bottomfloatSet distance to the bottom edge of the ROI rectangle

getAttentionRegion

std::vector<float> GazeTracker::getAttentionRegion() const;

Get Status ROI.

Return TypeDescription
std::vectorcontains the value from setAttentionRegion if set, empty std::vector otherwise.

removeAttentionRegion

void GazeTracker::removeAttentionRegion();

Remove Status ROI.


setAttentionInterval

void GazeTracker::setAttentionInterval(int interval)

Set time interval for the UserStatus Attention callback.

The UserStatus Attention score will be calculated for the given time interval.

The beginning and ending timestamps are passed through the onAttention callback as timestampBegin and timestampEnd.

The interval range is 10 to 60 (seconds), and the default value is 30 seconds.

ParametersTypeDescription
intervalintTime interval for the UserStatus Attention score in seconds.

Example

// IUserStatusCallback::OnAttention() will be called every 10 seconds
gaze_tracker.setAttentionInterval(10);

getAttentionScore

float GazeTracker::getAttentionScore()

Get current Attention score from the GazeTracker.

This API does not provide timestamp of the Attention score data.

Use onAttention callback, unless the Attention score is required at a specific time/location.

Return TypeDescription
floatCurrent attention score

Example

float currentAttention = gaze_tracker.getAttentionScore();     

converter

converter_type& GazeTracker::converter()
const converter_type& GazeTracker::converter() const

Access or modify GazeTracker's coordinate converter. See Custom Coordinate.


IGazeCallback


class IGazeCallback {
public:
IGazeCallback() = default;
virtual ~IGazeCallback() = default;

virtual void OnGaze(/* See below */) = 0;
};

Summary

Abstract Functions
onGaze

OnGaze

virtual void IGazeCallback::OnGaze(
std::uint64_t timestamp,
float x, float y,
SeeSoTrackingState tracking_state,
SeeSoEyeMovementState eye_movement_state)

⚠️ Below function has been removed since .

virtual void IGazeCallback::OnGaze(
std::uint64_t timestamp,
float x, float y,
TrackingState tracking_state,
EyeMovementState eye_movement_state)

Gaze callback.

ParametersTypeDescription
timestampuint64_tTimestamp of the iamge from GazeTracker::addFrame
xfloatx coordinate of gaze point
yfloaty coordinate of gaze point
tracking_stateSeeSoTrackingStateTracking state value
eye_movement_stateSeeSoEyeMovementStateEye movement state value

ICalibrationCallback


class ICalibrationCallback {
public:
ICalibrationCallback() = default;
virtual ~ICalibrationCallback() = default;

virtual void OnCalibrationProgress(float progress) = 0;
virtual void OnCalibrationNextPoint(float x, float y) = 0;
virtual void OnCalibrationFinish(const std::vector<float>& data) = 0;
};

Summary

Abstract Functions
onCalibrationNextPoint
onCalibrationProgress
onCalibrationFinished

OnCalibrationNextPoint

void ICalibrationCallback::OnCalibrationNextPoint(float x, float y)

Coordinate of the next calibration point.

x and y are current calibration point coordinate.

Point (x, y) should be focused during the calibration process.

ParametersTypeDescription
timestampuint64_tTimestamp of the iamge from GazeTracker::addFrame
xfloatx coordinate of the current calibration point
yfloaty coordinate of the current calibration point

Example

void OnCalibrationNextPoint(float x, float y) {
this->render_point_x = x;
this->render_point_y = y;
}

OnCalibrationProgress

void ICalibrationCallback::OnCalibrationProgress(float progress)

Calibration progress callback.

progress is the calibration progress of the current calibration point (0.0 ~ 1.0).

Next calibration point will be guided when the progress reaches 1.0.

startCollectSamples shuold be called to collect next calibration data.

Return 0.0 when start calibration progress.

Return 1.0 when finish calibratino progress.

ParametersTypeDescription
progressfloatCalibration progress of the current calibration point

Example

void OnCalibrationProgress(float progress){
if(progress >= 1.0){
// Collect next calibration point data
gaze_tracker.startCollectSamples();
} else {
// Collect current calibration point data
}
}

OnCalibrationFinished

void ICalibrationCallback::OnCalibrationFinished(const std::vector<float>& calibrationData)

Calibration finish callback.

calibrationData is the result calibration data.

ParametersTypeDescription
calibrationDataconst vector<float>&Result of the calibration data

IUserStatusCallback


class IUserStatusCallback {
public:
IUserStatusCallback() = default;
virtual ~IUserStatusCallback() = default;

virtual void OnAttention(uint64_t timestampBegin, uint64_t timestampEnd, float score) = 0;
virtual void OnDrowsiness(uint64_t timestamp, bool isDrowsiness) = 0;
virtual void OnBlink(uint64_t timestamp, bool isBlinkLeft, bool isBlinkRight, bool isBlink, float eyeOpenness) = 0;
};

Summary

Abstract Functions
onAttention
onDrowsiness
onBlink

The user status includes Attention, Drowsiness, and Blink.

Attention: How much the user attention is focused on the screen content for interval time (0.0 ~ 1.0)

  • Timestamp range of the data will be passed as timestampBegin and timestampEnd in onAttention callback.
  • The default time interval is 30 seconds.
  • If the user attention level is low, score in onAttention callback will be closed to 0.0.
  • If the user attention level is high, score in onAttention callback will be closed to 1.0.

Drowsiness: If the user feel drowsiness (True/False)

  • Timestamp of the data will be passed as timestamp in onDrowsiness callback.
  • If the user feel Drowsiness, isDrowsiness in onDrowsiness callback will be true.
  • Otherwise, isDrowsiness will be false.

Blink: If the user blink eyes (left eye, right eye, general(both eyes))

  • Timestamp of the data will be passed as timestamp in onBlink callback.
  • If the user blink left eye, isBlinkLeft in onBlink callback will be true.
  • If the user blink right eye, isBlinkRight in onBlink callback will be true.
  • If the user blink eyes, isBlink in onBlink callback will be true (This is a general blink condition).
  • If the user's eyes are wide, eyeOpenness in onBlink callback will be closed to 1.0 (not available yet).
  • If the user's eyes are narrow, eyeOpenness in onBlink callback will be closed to 0.0 (not available yet).

OnAttention

void IUserStatusCallback::OnAttention(uint64_t timestampBegin, uint64_t timestampEnd, float score)

Attention User Status callback.

ParametersTypeDescription
timestampBeginuint64_tBeginning timestamp of the data
timestampEnduint64_tEnd timestamp of the data
scorefloatUser attention rate score between the timestamps

OnDrowsiness

void IUserStatusCallback::OnDrowsiness(uint64_t timestamp, bool isDrowsiness)

Drowsiness User Status callback.

ParametersTypeDescription
timestampuint64_tTimestamp of the data
isDrowsinessboolUser drowisness flag

void IUserStatusCallback::OnBlink(
uint64_t timestamp,
bool isBlinkLeft, bool isBlinkRight, bool isBlink,
float eyeOpenness)

Blink User Status callback.

ParametersTypeDescription
timestampuint64_tTimestamp of the data
isBlinkLeftboolLeft blink flag
isBlinkRightboolRight blink flag
isBlinkboolBlink flag
eyeOpennessfloatOpen ratio of eyes (not available yet)

CoordConverterV2


template<typename T = double>
class CoordConverterV2 {
public:
using value_type = T;
using transform_type = Matrix<value_type, 2, 2>;
using translate_type = Matrix<value_type, 2, 1>;
using coordinate_type = Matrix<value_type, 2, 1>;

/** default constructor: R=I, T=Zeros */
constexpr CoordConverterV2() = default;
CoordConverterV2(const transform_type& r, const translate_type& t);

CoordConverterV2& transform(const transform_type& t);
transform_type& transform();
const transform_type& transform() const { return transform_; }

CoordConverterV2& translate(const translate_type& t);
translate_type& translate();
const translate_type& translate() const;

coordinate_type convert(const coordinate_type& m) const;
coordinate_type revert(const coordinate_type& m) const;
};

Helper functions

CoordConverterV2<T> seeso::makeCameraToDisplayConverter(const Point<float>& display_tl, const Point<float>& display_pixels, const Point<float>& display_mm)
CoordConverterV2<T> seeso::makeDefaultCameraToDisplayConverter(const Point<float>& display_pixels, const Point<float>& display_mm)
CoordConverterV2<T> seeso::makeDefaultCameraToDisplayConverter(float px_width, float px_height, float mm_width, float mm_height)
CoordConverterV2<T> seeso::makeNoOpConverter()

See the header file comment


ErrorReporter


class ErrorReporter {
public:
virtual ~ErrorReporter() = default;
virtual int report(const char* message) = 0;

int report(const std::string& message);
};

Summary

Constructor and Destructor

| ~ErrorReporter

Functions
report

Base class of error reporter.
This feature is experimental.


report

virtual int report(const char* message) const noexcept; 
ParametersTypeDescription
messagechar*Message got from the caller
Return TypeDescription
intUnspecified (Experimental)

Helper Functions

ErrorReporter* DefaultErrorReporter();

Get a pointer to the default error reporter in shared memory.
Releasing the returned object is undefined behavior.

ReturnTypeDescription
error_reporterErrorReporter*Pointer to the default error reporter.

SeeSoBoolean

typedef enum SeeSoBoolean {
kSeeSoFalse = 0,
kSeeSoTrue = 1,
} SeeSoBoolean;

Substitution of bool type.

SeeSoStatusModuleOptions

typedef struct SeeSoStatusModuleOptions {
SeeSoBoolean use_attention;
SeeSoBoolean use_blink;
SeeSoBoolean use_drowsiness;
} SeeSoStatusModuleOptions;

⚠️ Below enum has been removed since .

enum StatusOptions {
STATUS_ATTENTION = 1,
STATUS_BLINK = 2,
STATUS_DROWSINESS = 3
};

Flag of each module when using Status Module

NameDescription
use_attentionInclude attention score
use_blinkInclude blink detection
use_drowsinessInclude drowsiness detection

SeeSoCalibrationPointNum

typedef enum SeeSoCalibrationPointNum {
kSeeSoCalibrationPointOne = 1,
kSeeSoCalibrationPointTwo = 2,
kSeeSoCalibrationPointFive = 5,
} SeeSoCalibrationPointNum;

⚠️ Below enum has been removed since .

enum TargetNum {
ONE = 1,
TWO = 2,
FIVE = 5,
};

The number of points for calibration.

NameDescription
kSeeSoCalibrationPointOneOne-point calibration mode
kSeeSoCalibrationPointTwoTwo-point calibration mode
kSeeSoCalibrationPointFiveFive-point calibration mode

SeeSoCalibrationAccuracy

typedef enum SeeSoCalibrationAccuracy {
kSeeSoCalibrationAccuracyDefault = 0,
kSeeSoCalibrationAccuracyLow,
kSeeSoCalibrationAccuracyHigh,
} SeeSoCalibrationAccuracy;

⚠️ Below enum has been removed since .

enum class CalibrationAccuracy {
DEFAULT = 0,
LOW,
HIGH,
};

The enum containing options that manage calibration process and accuracy.

NameDescription
kSeeSoCalibrationAccuracyDefaultStarts default calibration
kSeeSoCalibrationAccuracyLowStart calibration with lower threshold for each step. (May result in lower accuracy)
kSeeSoCalibrationAccurarcyHighStart calibration with higher threshold for each step. (Likely to result in higher accuracy)

SeeSoTrackingState

typedef enum SeeSoTrackingState {
kSeeSoTrackingSuccess = 0,
kSeeSoTrackingLowConfidence,
kSeeSoTrackingUnsupported,
kSeeSoTrackingFaceMissing,
} SeeSoTrackingState;

⚠️ Below enum has been removed since .

enum class TrackingState {
SUCCESS = 0,
LOW_CONFIDENCE,
UNSUPPORTED,
FACE_MISSING,
};

The enum containing tracking state types used in OnGaze callback.

NameDescription
kSeeSoTrackingSuccessFace alignment is in the best position, tracking is successful with valid accuracy
kSeesoTrackingLowConfidenceFace alignment is not in the best position, but tracking is successful with lower accuracy
kSeeSoTrackingUnsupportedFace alignment is not suitable for tracking (Gaze tracking failed with invalud x and y)
kSeeSoTrackingFaceMissingFace is missing (Gaze tracking has failed)

SeeSoEyeMovementState

typedef enum SeeSoEyeMovementState {
kSeeSoEyeMovementFixation = 0,
kSeeSoEyeMovementSaccade = 2,
kSeeSoEyeMovementUnknown = 3,
} SeeSoEyeMovementState;

⚠️ Below enum has been removed since .

enum class EyeMovementState {
FIXATION = 0,
SACCADE = 2
};

The enum containing eye movement state types used in OnGaze callback.

NameDescription
kSeeSoEyeMovementFixationEye movement has fixation.
kSeeSoEyeMovementSaccadeEye movement has saccade.
kSeeSoEyeMovementUnknownEye movement is unknown.

UserStatusOption (removed)

⚠️ Initializing GazeTracker with UserStatusOption has been removed in . Please use SeeSoStatusModuleOptions instead.

class UserStatusOption {
UserStatusOption& useAttention();
UserStatusOption& useBlink();
UserStatusOption& useDrowsiness();
UserStatusOption& useAll();

bool isUseAttention() const;
bool isUseBlink() const;
bool isUseDrowsiness() const;
};

The class contains User Status options information for GazeTracker