Unity pico4 pro开发踩坑记录（1）眼动
Anybox_fanfande
编辑于 2024年03月12日 06:00

pico4 支持眼动模块，但是官方文档不够清晰，且不存在演示demo。记录以下流程和坑点。

1.首先在场景中的XR Origin物体上的PXR_Manager脚本，勾选Eye tracking（注意只有pico4 pro和Neo 3支持眼动）。此时，便可以通过脚本获取到眼动的数据了。

2.在创建的脚本中，输入如下代码

代码块
C#
自动换行
复制代码
using System;
using System.Collections;
using System.Collections.Generic;
using TMPro;
using Unity.XR.PXR;
using UnityEngine;
using UnityEngine.SceneManagement;
using UnityEngine.UI;



public class EyeTrackingManager : MonoBehaviour
{
    public TMP_Text ui_text;
    public Slider slider;
    public float zCoordinate =14;

    private bool trackingSupported = false;
    private bool tracking = false;
    private EyeTrackingData eyeTrackingData;

    public Transform ray;
    public Transform point_gaze;
    public Transform root;


    // Start is called before the first frame update
    void Start()
    {
        var trackingState = (TrackingStateCode)PXR_MotionTracking.WantEyeTrackingService();


        if (trackingState == TrackingStateCode.PXR_MT_SUCCESS)
        {
            ui_text.text = "Want Success";

            EyeTrackingMode eyeTrackingMode = EyeTrackingMode.PXR_ETM_NONE;

            int supportedModesCount = 0;
            trackingState = (TrackingStateCode)PXR_MotionTracking.GetEyeTrackingSupported(ref trackingSupported, ref supportedModesCount, ref eyeTrackingMode);


            if (trackingSupported)
            {
                ui_text.text = "Supported!!";

                EyeTrackingStartInfo info = new EyeTrackingStartInfo();
                info.needCalibration = 1;
                info.mode = EyeTrackingMode.PXR_ETM_BOTH;
                trackingState = (TrackingStateCode)PXR_MotionTracking.StartEyeTracking(ref info);
            }
            else
            {
                ui_text.text = "Not Supported!!";
            }
        }
        else
        {
            ui_text.text = "Want Fail";
        }
    }

    void Update()
    {

        if (trackingSupported)
        {
            EyeTrackingState eyeTrackingState = new EyeTrackingState();
            var trackingState = (TrackingStateCode)PXR_MotionTracking.GetEyeTrackingState(ref tracking, ref eyeTrackingState);

            if (tracking)
            {
                EyeTrackingDataGetInfo getInfo = new EyeTrackingDataGetInfo();
                getInfo.displayTime = 0;
                getInfo.flags = EyeTrackingDataGetFlags.PXR_EYE_DEFAULT
                                | EyeTrackingDataGetFlags.PXR_EYE_POSITION
                                | EyeTrackingDataGetFlags.PXR_EYE_ORIENTATION;

                trackingState = (TrackingStateCode)PXR_MotionTracking.GetEyeTrackingData(ref getInfo, ref eyeTrackingData);

                var eyeData = eyeTrackingData.eyeDatas[2].pose;


                var localorigin = new Vector3(eyeData.orientation.y, eyeData.orientation.z, -eyeData.orientation.w) + root.localPosition;
                var origin = root.TransformPoint(localorigin);


                Quaternion direction = new Quaternion(-eyeData.position.x, -eyeData.position.y, eyeData.position.z, eyeData.orientation.x) ;

         
                var point = origin + (root.rotation * direction * Vector3.forward* zCoordinate);
                point_gaze.position = point;
 
                ray.position = origin;
                ray.rotation = root.rotation*direction ;

                ui_text.text = eyeTrackingData.ToString();


            }
        }
    }
复制成功

代码来自    https://zhuanlan.zhihu.com/p/678956811  

            PICO4 pro眼动追踪开发实战 - 知乎 (zhihu.com)

坑点1：结构体 EyeTrackingData中存放Pico运行时的眼动数据。eyeDatas[0]，eyeDatas[1]分别是左眼和右眼单独的数据，只有企业版能够读取，故数据为空。eyeDatas[2]则为两者混合后的数据

代码块
C#
自动换行
复制代码
var eyeData = eyeTrackingData.eyeDatas[2].pose;

var localorigin = new Vector3(eyeData.orientation.y, eyeData.orientation.z, -eyeData.orientation.w) + root.localPosition;
var origin = root.TransformPoint(localorigin);


Quaternion direction = new Quaternion(-eyeData.position.x, -eyeData.position.y, eyeData.position.z, eyeData.orientation.x) ;
复制成功

比较逆天的是，其中position和rotation的数据是乱的。

Pico官网论坛

坑点二：Pico获取的position以XR Origin 或者 Camera offset物体为参考系。然而Unity

pico 执行旋转时会直接对Xr Origin进行旋转操作，总而言之，需要先使用本地坐标系确定眼动原点位置，再转换为世界坐标。

UNity XR tookit 提供的预制体 

代码块
C#
自动换行
复制代码
 var localorigin = new Vector3(eyeData.orientation.y, eyeData.orientation.z, -eyeData.orientation.w) + root.localPosition;
var origin = root.TransformPoint(localorigin);
复制成功

对于rotation数据，类似的，需要再加上XR Origin的旋转，注意旋转的叠加应该使用四元数的乘法，在四元数中，乘法顺序是从右到左的，

代码块
C#
自动换行
复制代码
var point = origin + (root.rotation * direction * Vector3.forward* zCoordinate);
复制成功

cv33127515