API Docs (C++)

Overview

Class	Description
GazeTracker	Class that generates gaze tracking data
IGazeCallback	Gaze data listener
IFaceCallback	Face data listener
ICalibrationCallback	Calibration process listener
IUserStatusCallback	User status data listener
CoordConverterV2	Matrix-based coordinate converter
ErrorReporter	Error listener base class
SeeSoBoolean	Substitution for bool type
SeeSoStatusModuleOptions	Enum with flag for each module when using StatusModule
SeeSoCalibrationPointNum	Enum with number of points for calibration
SeeSoCalibrationAccuracy	Enum containing options that manage calibration process and accuracy
SeeSoTrackingState	Enum containing tracking state types used in OnGaze callback
SeeSoEyeMovementState	Enum containing eye movement state types used in OnGaze callback

global_init

void global_init(const char* file = "seeso_core.dll")

This function sets up the program environment that SeeSo SDK needs.
This function must be called at least once within a program before the program calls any other function in SeeSo SDK.
It's a dll loader in actual. On other platforms except Windows, this function is no-op.

Exceptions

Failed to load a dll

SeeSo SDK requires the dll files to execute(which are included in the SDK files), including seeso_core.dll and several third party library dlls.
See seeso/bin or error logs for such dll files. seeso::global_init() will throw if one or more of these files are missing.
Make sure that they're in the same directory with an executable file, or use add_custom_command in CMakeLists.txt as the following.
If add_custom_command isn't working, copy and paste the dll files to the directory(where an executable file is located) manually.

Failed to find XXX from dll

Dll files in the SeeSo SDK are broken, or wrong version is used.
Re-download the SDK from SeeSo Console and try again.

getVersionStr

std::string getVersionStr()

return SeeSo SDK version information in string.

Return	Type	Description
version_str	std::string	SeeSo SDK version in std::string

Example

const std::string version = seeso::getVersionStr();

getVersionInt

int32_t getVersionInt()

Return SeeSo SDK version as 9 digit integer.
Version forms XXYYZZZZZ, where XX is a major version, YY is a minor version, and ZZZZZ is a patch version.

Return	Type	Description
version_int	int32_t	SeeSo SDK version in 9 digit integer

Example

int32_t version = seeso::getVersionInt();

GazeTracker

class GazeTracker

GazeTracker is the class that generates GazeTracking data.

Summary

Constructor and Destructor
GazeTracker(const converter_type&)
~GazeTracker()

Functions
initialized
deinitialized
isTrackerInitialized
setFaceDistance
getFaceDistance
setGazeCallback
setFaceCallback
setCalibrationCallback
setUserStatusCallback
removeGazeCallback
removeFaceCallback
removeCalibrationCallback
removeUserStatusCallback
setTrackingFPS
addFrame
startCalibration
stopCalibration
startCollectSamples
setCalibrationData
setAttentionRegion
getAttentionRegion
removeAttentionRegion
setAttentionInterval
getAttentionScore
converter

GazeTracker

explicit GazeTracker(const converter_type& coord_converter = makeNoOpConverter<float>())

Parameters	Type	Description
coord_converter	converter_type	Coordinate system converter. See Custom Coordinate

Example

// Coordinate system is camera-centered millimeters coordinate(default)
GazeTracker gaze_tracker();

// Coordinate system is display pixel coordinate
const auto camera2display = seeso::makeDefaultCameraToDisplayConverter<float>(...);
GazeTracker gaze_tracker_custom(camera2display);

~GazeTracker

~GazeTracker()

Effectively calls GazeTracker::deinitialize() and releases internal object.

initialize

int GazeTracker::initialize(
    const std::string& licenseKey,
    const SeeSoStatusModuleOptions& status_option = {kSeeSoFalse, kSeeSoFalse, kSeeSoFalse},
    ErrorReporter* error_reporter = DefaultErrorReporter())

GazeTracker construction requires authentication. This example will pass SeeSoStatusModuleOptions to activate User Status Detector in GazeTracker. After initialization, authentication result will be displayed as a log with status code.

Parameters	Type	Description
licenseKey	const std::string&	Key generated from SeeSo Console
status_option	const SeeSoStatusModuleOption&	User-status-module option
error_reporter	ErrorReporter*	Optional ErrorReporter. GazeTracker does not own the object.

Return Type	Description
int	Authentication code. See Authentication for more details.

Example

SeeSoStatusModuleOptions status_options = {};
userStatusOption.use_attention = kSeeSoTrue;
userStatusOption.use_blink = kSeeSoTrue;
userStatusOption.use_drowsiness = kSeeSoTrue;
const int errCode = gaze_tracker.initialize('licenseKey', status_options);
if (errCode == 0) {
  // Initialization success
} else {
  // Initialization failed
}

deinitialize

void GazeTracker::deinitialize()

Deinitialize internal object. Calling other member functions(except for initialize) after calling deinitialize() is undefined behavior.

isTrackerInitialized

bool GazeTracker::isTrackerInitialized() const;

Return Type	Description
bool	Returns true if initialized, false otherwise.

setFaceDistance

void GazeTracker::setFaceDistance(int cm)

Set face distance from the camera in cm. To get better GazeTracking performance, accurate face distance should be set by this function. Default valule is 60 (cm).

Parameters	Type	Description
faceDistance	int	Distance between user face and camera in cm.

getFaceDistance

int getFaceDistance() const

Return Type	Description
int	Distance between user face and camera in cm.

setGazeCallback

void GazeTracker::setGazeCallback(seeso::IGazeCallback *listener)

Set Gaze callback for GazeTracking.
Note that GazeTracker does not own the instance.

Parameters	Type	Description
listener	IGazeCallback*	Address of gaze listener instance

Example

seeso::IGazeCallback gaze_callback;
gaze_tracker.setGazeCallback(&gaze_callback);

setFaceCallback

void GazeTracker::setFaceCallback(seeso::IFaceCallback *listener)

Set Face callback for GazeTracking.
Note that GazeTracker does not own the instance.

Parameters	Type	Description
listener	IFaceCallback*	Address of face listener instance

Example

seeso::IFaceCallback face_callback;
gaze_tracker.setFaceCallback(&face_callback);

setCalibrationCallback

void GazeTracker::setCalibrationCallback(seeso::ICalibrationCallback* listener);

Set Calibration callback for GazeTracking.
Note that GazeTracker does not own the instance.

Parameters	Type	Description
listener	ICalibrationCallback*	Address of calibration listener instance.

Example

seeso::ICalibrationCallback calibration_callback;
gaze_tracker.setCalibrationCallback(&calibration_callback);

setUserStatusCallback

void GazeTracker::setUserStatusCallback(seeso::IUserStatusCallback* listener)

Set User Status callback for GazeTracking.
Note that GazeTracker does not own the instance.

Parameters	Type	Description
listener	IUserStatusCallback*	Address of status callback instance.

Example

seeso::IUserStatusCallback status_callback;
gaze_tracker.setUserStatusCallback(&status_callback);

removeGazeCallback

void GazeTracker::removeGazeCallback()

Remove gaze callback interface for GazeTracking.

removeFaceCallback

void GazeTracker::removeFaceCallback()

Remove face callback interface for GazeTracking.

removeCalibrationCallback

void GazeTracker::removeCalibrationCallback()

Remove calibration callback interface for GazeTracking.

removeUserStatusCallback

void GazeTracker::removeUserStatusCallback()

Remove user status callback interface for GazeTracking.

setTrackingFPS

void GazeTracker::setTrackingFps(int fps)

Set the maximum InputFrames-per-second of the GazeTracking.
FPS can be lower than the set value due to the device spec. It limits the calculation by omitting the given frame internally.

Parameters	Type	Description
fps	int	Maximum FPS(frames per seconds) for gaze tracking

Example

gaze_tracker.setTrackingFps(15);

addFrame

bool GazeTracker::addFrame(int64_t time_stamp, uint8_t *buffer, int width, int height)

Add image frame into GazeTracker for GazeTracking. addFrame should be called continuously, and add image frame from the camera stream. The image should be in RGB color format. Note that some frames can be ignored inside SeeSo SDK, thus not calculated

• If frames are input very often(Don't worry; SeeSo SDK will perform 30+ FPS in real-time on most of the latest hardware).
• Even if the hardware specs are good, some frames can be ignored to guarantee OnGazeListener::OnGaze()'s timestamp is in increasing order. This happens rarely. For each calculated input image, OnGazeListener::OnGaze() will be called with the input timestamp and the calculated values.
  Image data is copied inside the function, so it's safe to destroy an image object after calling addFrame.

Parameters	Type	Description
time_stamp	int64_t	Timestamp of the given frame
buffer	uint8_t*	Image data buffer in RGB format. It is safe to release buffer after passing to GazeTracker since the buffer is deep-copied.
width	int	Width of the image
height	int	Height of the image

Return Type	Description
bool	Returns false if the input frame is omitted, true otherwise.

Example

cv::Mat frame, input;
while(true) {
  video >> frame; // read frame
  if (frame.empty()) break;
  
  cv::cvtColor(frame, input, cv::COLOR_BGR2RGB); // convert color format
  gaze_tracker.addFrame(current_time(), input.data, input.cols, input.rows);
}

startCalibration

void GazeTracker::startCalibration(
    SeeSoCalibrationPointNum point_num,
    SeeSoCalibrationAccuracy criteria,
    float left, float top, float right, float bottom);

Start calibration process with CalibrationMode and Criteria. calibrationMode is number of calibration points. criteria is option that manage calibration process and accuracy.

Parameters	Type	Description
point_num	SeeSoCalibrationPointNum	Number of points used for calibration
criteria	SeeSoCalibrationAccuracy	Accuracy option of the calibration
left	float	Distance to left edge of the calibration ROI
top	float	Distance to top edge of the calibration ROI
right	float	Distance to right edge of the calibration ROI
bottom	float	Distance to bottom edge of the calibration ROI

Example

gaze_tracker.startCalibration(
    kSeeSoCalibrationPointFive,
    kSeeSoCalibrationAccuracyDefault,
    0.0f, 0.0f, 1280.0f, 720.0f);

stopCalibration

void GazeTracker::stopCalibration()

Stop the current calibration process.

startCollectSamples

void GazeTracker::startCollectSamples()

Start collect next calibration point data.

This should be called in onCalibrationProgress when need to collect new calibration point data.

The onCalibrationProgress will give progress (0.0 ~ 1.0), which represents the data collecting progress of current calibration point.

Example

if (isCalibrating) {
  gaze_tracker.startCollectSamples();
}

setCalibrationData

void GazeTracker::setCalibrationData(const std::vector<float>& calibrationData)

Set calibration data without calibrating manually.

Parameters	Type	Description
calibrationData	const std::vector<float>&	Stored calibration data

Example

  gaze_tracker.setCalibrationData(calibrationData);

setAttentionRegion

void GazeTracker::setAttentionRegion(float left, float top, float right, float bottom);

Set Status ROI.

Parameters	Type	Description
left	float	Set distance to the left edge of the ROI rectangle
top	float	Set distance to the top edge of the ROI rectangle
right	float	Set distance to the right edge of the ROI rectangle
bottom	float	Set distance to the bottom edge of the ROI rectangle

getAttentionRegion

std::vector<float> GazeTracker::getAttentionRegion() const;

Get Status ROI.

Return Type	Description
std::vector	contains the value from setAttentionRegion if set, empty std::vector otherwise.

removeAttentionRegion

void GazeTracker::removeAttentionRegion();

Remove Status ROI.

setAttentionInterval

void GazeTracker::setAttentionInterval(int interval)

Set time interval for the UserStatus Attention callback. The UserStatus Attention score will be calculated for the given time interval. The beginning and ending timestamps are passed through the onAttention callback as timestampBegin and timestampEnd. The interval range is 10 to 60 (seconds), and the default value is 30 seconds.

Parameters	Type	Description
interval	int	Time interval for the UserStatus Attention score in seconds.

Example

// IUserStatusCallback::OnAttention() will be called every 10 seconds
gaze_tracker.setAttentionInterval(10);

getAttentionScore

float GazeTracker::getAttentionScore()

Get current Attention score from the GazeTracker.

This API does not provide timestamp of the Attention score data.

Use onAttention callback, unless the Attention score is required at a specific time/location.

Return Type	Description
float	Current attention score

Example

float currentAttention = gaze_tracker.getAttentionScore();     

converter

converter_type& GazeTracker::converter()
const converter_type& GazeTracker::converter() const

Access or modify GazeTracker's coordinate converter. See Custom Coordinate.

IGazeCallback

class IGazeCallback {
 public:
  IGazeCallback() = default;
  virtual ~IGazeCallback() = default;

  virtual void OnGaze(/* See below */) = 0;
};

Summary

Abstract Functions
onGaze

OnGaze

virtual void IGazeCallback::OnGaze(
    std::uint64_t timestamp,
    float x, float y,
    float fixation_x, float fixation_y,
    SeeSoTrackingState tracking_state,
    SeeSoEyeMovementState eye_movement_state)

Gaze callback.

Parameters	Type	Description
timestamp	uint64_t	Timestamp of the iamge from GazeTracker::addFrame
x	float	x coordinate of gaze point
y	float	y coordinate of gaze point
fixation_x	float	x coordinate value of last fixation point.
fixation_y	float	y coordinate value of last fixation point.
left_openness	float	openness degree of left eye(0.0~1.0). value will only return when userStatusOption in on
right_openness	float	openness degree of right eye(0.0~1.0). value will only return when userStatusOption in on
tracking_state	SeeSoTrackingState	Tracking state value
eye_movement_state	SeeSoEyeMovementState	Eye movement state value

IFaceCallback

---

class IFaceCallback {
 public:
  IFaceCallback() = default;
  virtual ~IFaceCallback() = default;

  virtual void OnFace(/* See below */) = 0;
};

Summary

Abstract Functions
onFace

OnFace

virtual void IFaceCallback::OnFace(
    std::uint64_t timestamp,
    float score,
    float left, float top, float right, float bottom
    float pitch, float yaw, float roll
    const float* center_xyz, std::uint32_t center_xyz_size)

Face callback.

Parameters	Type	Description
timestamp	uint64_t	Timestamp of the iamge from GazeTracker::addFrame
score	float	Value of facial recognition confidence (0.0 ~ 1.0).
left	float	Left of the face.
top	float	Top of the face.
right	float	Right of the face.
bottom	float	Bottom of the face.
pitch	float	Pitch is the rotation around the X-axis of an object. It is also known as the attitude angle. In aviation, it is often referred to as the nose-up or nose-down angle.
yaw	float	Yaw is the rotation around the Y-axis of an object. It is also known as the heading angle. In aviation, it is often referred to as the bearing or compass angle.
roll	float	Roll is the rotation around the Z-axis of an object. It is also known as the bank angle. In aviation, it is often referred to as the wing-up or wing-down angle.
center_xyz	const float*	The x,y,z distance of the center of the face from the camera. The unit is mm.
center_xyz_size	std::uint32_t	center_xyz is size

ICalibrationCallback

---

class ICalibrationCallback {
 public:
  ICalibrationCallback() = default;
  virtual ~ICalibrationCallback() = default;

  virtual void OnCalibrationProgress(float progress) = 0;
  virtual void OnCalibrationNextPoint(float x, float y) = 0;
  virtual void OnCalibrationFinish(const std::vector<float>& data) = 0;
};

Summary

Abstract Functions
onCalibrationNextPoint
onCalibrationProgress
onCalibrationFinished

OnCalibrationNextPoint

void ICalibrationCallback::OnCalibrationNextPoint(float x, float y)

Coordinate of the next calibration point.

x and y are current calibration point coordinate.

Point (x, y) should be focused during the calibration process.

Parameters	Type	Description
timestamp	uint64_t	Timestamp of the iamge from GazeTracker::addFrame
x	float	x coordinate of the current calibration point
y	float	y coordinate of the current calibration point

Example

void OnCalibrationNextPoint(float x, float y) {
  this->render_point_x = x;
  this->render_point_y = y;    
}

OnCalibrationProgress

void ICalibrationCallback::OnCalibrationProgress(float progress)

Calibration progress callback.

progress is the calibration progress of the current calibration point (0.0 ~ 1.0).

Next calibration point will be guided when the progress reaches 1.0.

startCollectSamples shuold be called to collect next calibration data.

Return 0.0 when start calibration progress.

Return 1.0 when finish calibratino progress.

Parameters	Type	Description
progress	float	Calibration progress of the current calibration point

Example

void OnCalibrationProgress(float progress){
  if(progress >= 1.0){
    // Collect next calibration point data
    gaze_tracker.startCollectSamples();
  } else {
    // Collect current calibration point data
  }
}

OnCalibrationFinished

void ICalibrationCallback::OnCalibrationFinished(const std::vector<float>& calibrationData)

Calibration finish callback.

calibrationData is the result calibration data.

Parameters	Type	Description
calibrationData	const vector<float>&	Result of the calibration data

IUserStatusCallback

class IUserStatusCallback {
 public:
  IUserStatusCallback() = default;
  virtual ~IUserStatusCallback() = default;

  virtual void OnAttention(uint64_t timestampBegin, uint64_t timestampEnd, float score) = 0;
  virtual void OnDrowsiness(uint64_t timestamp, bool isDrowsiness, float intensity) = 0;
  virtual void OnBlink(uint64_t timestamp, bool isBlinkLeft, bool isBlinkRight, bool isBlink, float leftOpenness, float rightOpenness) = 0;
};

Summary

Abstract Functions
onAttention
onDrowsiness
onBlink

The user status includes Attention, Drowsiness, and Blink. Attention: How much the user attention is focused on the screen content for interval time (0.0 ~ 1.0)

• Timestamp range of the data will be passed as timestampBegin and timestampEnd in onAttention callback.
• The default time interval is 30 seconds.
• If the user attention level is low, score in onAttention callback will be closed to 0.0.
• If the user attention level is high, score in onAttention callback will be closed to 1.0. Drowsiness: If the user feel drowsiness (True/False)
• Timestamp of the data will be passed as timestamp in onDrowsiness callback.
• If the user feel Drowsiness, isDrowsiness in onDrowsiness callback will be true.
• Otherwise, isDrowsiness will be false. Blink: If the user blink eyes (left eye, right eye, general(both eyes))
• Timestamp of the data will be passed as timestamp in onBlink callback.
• If the user blink left eye, isBlinkLeft in onBlink callback will be true.
• If the user blink right eye, isBlinkRight in onBlink callback will be true.
• If the user blink eyes, isBlink in onBlink callback will be true (This is a general blink condition).

OnAttention

void IUserStatusCallback::OnAttention(uint64_t timestampBegin, uint64_t timestampEnd, float score)

Attention User Status callback.

Parameters	Type	Description
timestampBegin	uint64_t	Beginning timestamp of the data
timestampEnd	uint64_t	End timestamp of the data
score	float	User attention rate score between the timestamps

OnDrowsiness

void IUserStatusCallback::OnDrowsiness(uint64_t timestamp, bool isDrowsiness, float intensity)

Drowsiness User Status callback.

Parameters	Type	Description
timestamp	uint64_t	Timestamp of the data
isDrowsiness	bool	User drowisness flag
intensity	float	Level of drowsiness intensity (0 to 1)

OnBlink

void IUserStatusCallback::OnBlink(
    uint64_t timestamp,
    bool isBlinkLeft, bool isBlinkRight, bool isBlink,
    float leftOpenness, float rightOpenness)

Blink User Status callback.

Parameters	Type	Description
timestamp	uint64_t	Timestamp of the data
isBlinkLeft	bool	Left blink flag
isBlinkRight	bool	Right blink flag
isBlink	bool	Blink flag
leftOpenness	float	Open ratio of left eye.
rightOpenness	float	Open ratio of right eye.

CoordConverterV2

---

template<typename T = double>
class CoordConverterV2 {
 public:
  using value_type = T;
  using transform_type = Matrix<value_type, 2, 2>;
  using translate_type = Matrix<value_type, 2, 1>;
  using coordinate_type = Matrix<value_type, 2, 1>;

  /** default constructor: R=I, T=Zeros */
  constexpr CoordConverterV2() = default;
  CoordConverterV2(const transform_type& r, const translate_type& t);

  CoordConverterV2& transform(const transform_type& t);
  transform_type& transform();
  const transform_type& transform() const { return transform_; }

  CoordConverterV2& translate(const translate_type& t);
  translate_type& translate();
  const translate_type& translate() const;

  coordinate_type convert(const coordinate_type& m) const;
  coordinate_type revert(const coordinate_type& m) const;
};

Helper functions

CoordConverterV2<T> seeso::makeCameraToDisplayConverter(const Point<float>& display_tl, const Point<float>& display_pixels, const Point<float>& display_mm)
CoordConverterV2<T> seeso::makeDefaultCameraToDisplayConverter(const Point<float>& display_pixels, const Point<float>& display_mm)
CoordConverterV2<T> seeso::makeDefaultCameraToDisplayConverter(float px_width, float px_height, float mm_width, float mm_height)
CoordConverterV2<T> seeso::makeNoOpConverter()

See the header file comment

ErrorReporter

class ErrorReporter {
 public:
  virtual ~ErrorReporter() = default;
  virtual int report(const char* message) = 0;

  int report(const std::string& message);
};

Summary

Constructor and Destructor
~ErrorReporter

Functions
report

Base class of error reporter.
This feature is experimental.

report

virtual int report(const char* message) const noexcept; 

Parameters	Type	Description
message	char*	Message got from the caller

Return Type	Description
int	Unspecified (Experimental)

Helper Functions

ErrorReporter* DefaultErrorReporter();

Get a pointer to the default error reporter in shared memory.
Releasing the returned object is undefined behavior.

Return	Type	Description
error_reporter	ErrorReporter*	Pointer to the default error reporter.

SeeSoBoolean

typedef enum SeeSoBoolean {
  kSeeSoFalse = 0,
  kSeeSoTrue = 1,
} SeeSoBoolean;

Substitution of bool type.

SeeSoStatusModuleOptions

typedef struct SeeSoStatusModuleOptions {
  SeeSoBoolean use_attention;
  SeeSoBoolean use_blink;
  SeeSoBoolean use_drowsiness;
} SeeSoStatusModuleOptions;

Flag of each module when using Status Module

Name	Description
use_attention	Include attention score
use_blink	Include blink detection
use_drowsiness	Include drowsiness detection

SeeSoCalibrationPointNum

typedef enum SeeSoCalibrationPointNum {
  kSeeSoCalibrationPointOne = 1,
  kSeeSoCalibrationPointTwo = 2,
  kSeeSoCalibrationPointFive = 5,
} SeeSoCalibrationPointNum;

The number of points for calibration.

Name	Description
kSeeSoCalibrationPointOne	One-point calibration mode
kSeeSoCalibrationPointTwo	Two-point calibration mode
kSeeSoCalibrationPointFive	Five-point calibration mode

SeeSoCalibrationAccuracy

typedef enum SeeSoCalibrationAccuracy {
  kSeeSoCalibrationAccuracyDefault = 0,
  kSeeSoCalibrationAccuracyLow,
  kSeeSoCalibrationAccuracyHigh,
} SeeSoCalibrationAccuracy;

The enum containing options that manage calibration process and accuracy.

Name	Description
kSeeSoCalibrationAccuracyDefault	Starts default calibration
kSeeSoCalibrationAccuracyLow	Start calibration with lower threshold for each step. (May result in lower accuracy)
kSeeSoCalibrationAccurarcyHigh	Start calibration with higher threshold for each step. (Likely to result in higher accuracy)

SeeSoTrackingState

typedef enum SeeSoTrackingState {
  kSeeSoTrackingSuccess = 0,
  kSeeSoTrackingLowConfidence,
  kSeeSoTrackingUnsupported,
  kSeeSoTrackingFaceMissing,
} SeeSoTrackingState;

The enum containing tracking state types used in OnGaze callback.

Name	Description
kSeeSoTrackingSuccess	Face alignment is in the best position, tracking is successful with valid accuracy
kSeesoTrackingLowConfidence	Face alignment is not in the best position, but tracking is successful with lower accuracy
kSeeSoTrackingUnsupported	Face alignment is not suitable for tracking (Gaze tracking failed with invalud x and y)
kSeeSoTrackingFaceMissing	Face is missing (Gaze tracking has failed)

SeeSoEyeMovementState

typedef enum SeeSoEyeMovementState {
  kSeeSoEyeMovementFixation = 0,
  kSeeSoEyeMovementSaccade = 2,
  kSeeSoEyeMovementUnknown = 3,
} SeeSoEyeMovementState;

The enum containing eye movement state types used in OnGaze callback.

Name	Description
kSeeSoEyeMovementFixation	Eye movement has fixation.
kSeeSoEyeMovementSaccade	Eye movement has saccade.
kSeeSoEyeMovementUnknown	Eye movement is unknown.