Getting the Most from Eye-Tracking: User-Interaction Based Reading Region Estimation Dataset and Models

Ruoyan Kong; Ruixuan Sun; Charles Chuankai Zhang; Chen Chen; Sneha Patri; Gayathri Gajjela; Joseph A. Konstan

doi:10.1145/3588015.3588404

Getting the Most from Eye-Tracking: User-Interaction Based Reading Region Estimation Dataset and Models

Ruoyan Kong, Ruixuan Sun, Charles Chuankai Zhang, Chen Chen, Sneha Patri, Gayathri Gajjela, Joseph A. Konstan

Administration (CSENG)

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

A single digital newsletter usually contains many messages (regions). Users' reading time spent on, and read level (skip/skim/read-in-detail) of each message is important for platforms to understand their users' interests, personalize their contents, and make recommendations. Based on accurate but expensive-to-collect eyetracker-recorded data, we built models that predict per-region reading time based on easy-to-collect Javascript browser tracking data. With eye-tracking, we collected 200k ground-truth datapoints on participants reading news on browsers. Then we trained machine learning and deep learning models to predict message-level reading time based on user interactions like mouse position, scrolling, and clicking. We reached 27% percentage error in reading time estimation with a two-tower neural network based on user interactions only, against the eye-tracking ground truth data, while the heuristic baselines have around 46% percentage error. We also discovered the benefits of replacing per-session models with per-timestamp models, and adding user pattern features. We concluded with suggestions on developing message-level reading estimation techniques based on available data.

Original language	English (US)
Title of host publication	Proceedings - ETRA 2023
Subtitle of host publication	ACM Symposium on Eye Tracking Research and Applications
Editors	Stephen N. Spencer
Publisher	Association for Computing Machinery
ISBN (Electronic)	9798400701504
DOIs	https://doi.org/10.1145/3588015.3588404
State	Published - May 30 2023
Event	15th Annual ACM Symposium on Eye Tracking Research and Applications, ETRA 2023 - Tubingen, Germany Duration: May 30 2023 → Jun 2 2023

Publication series

Name	Eye Tracking Research and Applications Symposium (ETRA)

Conference

Conference	15th Annual ACM Symposium on Eye Tracking Research and Applications, ETRA 2023
Country/Territory	Germany
City	Tubingen
Period	5/30/23 → 6/2/23

Bibliographical note

Funding Information:
This work was supported by the National Science Foundation under grant CNS-2016397. We thank the University of Minnesota’s Usability Lab’s User Experience Analysts, Nick Rosencrans, and David Rosen, for their assistance with eye-tracking technology.

Publisher Copyright:
© 2023 ACM.

Keywords

bulk email
eye-tracking
neural network
personalization
reading region estimation
reading time
recommendation

Access

10.1145/3588015.3588404

OpenUrl availability

Full text

Cite this

Kong, R., Sun, R., Zhang, C. C., Chen, C., Patri, S., Gajjela, G., & Konstan, J. A. (2023). Getting the Most from Eye-Tracking: User-Interaction Based Reading Region Estimation Dataset and Models. In S. N. Spencer (Ed.), Proceedings - ETRA 2023: ACM Symposium on Eye Tracking Research and Applications Article 10 (Eye Tracking Research and Applications Symposium (ETRA)). Association for Computing Machinery. https://doi.org/10.1145/3588015.3588404

Getting the Most from Eye-Tracking: User-Interaction Based Reading Region Estimation Dataset and Models. / Kong, Ruoyan; Sun, Ruixuan; Zhang, Charles Chuankai et al.
Proceedings - ETRA 2023: ACM Symposium on Eye Tracking Research and Applications. ed. / Stephen N. Spencer. Association for Computing Machinery, 2023. 10 (Eye Tracking Research and Applications Symposium (ETRA)).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Kong, R, Sun, R, Zhang, CC, Chen, C, Patri, S, Gajjela, G & Konstan, JA 2023, Getting the Most from Eye-Tracking: User-Interaction Based Reading Region Estimation Dataset and Models. in SN Spencer (ed.), Proceedings - ETRA 2023: ACM Symposium on Eye Tracking Research and Applications., 10, Eye Tracking Research and Applications Symposium (ETRA), Association for Computing Machinery, 15th Annual ACM Symposium on Eye Tracking Research and Applications, ETRA 2023, Tubingen, Germany, 5/30/23. https://doi.org/10.1145/3588015.3588404

Kong R, Sun R, Zhang CC, Chen C, Patri S, Gajjela G et al. Getting the Most from Eye-Tracking: User-Interaction Based Reading Region Estimation Dataset and Models. In Spencer SN, editor, Proceedings - ETRA 2023: ACM Symposium on Eye Tracking Research and Applications. Association for Computing Machinery. 2023. 10. (Eye Tracking Research and Applications Symposium (ETRA)). doi: 10.1145/3588015.3588404

Kong, Ruoyan ; Sun, Ruixuan ; Zhang, Charles Chuankai et al. / Getting the Most from Eye-Tracking : User-Interaction Based Reading Region Estimation Dataset and Models. Proceedings - ETRA 2023: ACM Symposium on Eye Tracking Research and Applications. editor / Stephen N. Spencer. Association for Computing Machinery, 2023. (Eye Tracking Research and Applications Symposium (ETRA)).

@inproceedings{72610c97406f4cc58fd3408015d44aea,

title = "Getting the Most from Eye-Tracking: User-Interaction Based Reading Region Estimation Dataset and Models",

abstract = "A single digital newsletter usually contains many messages (regions). Users' reading time spent on, and read level (skip/skim/read-in-detail) of each message is important for platforms to understand their users' interests, personalize their contents, and make recommendations. Based on accurate but expensive-to-collect eyetracker-recorded data, we built models that predict per-region reading time based on easy-to-collect Javascript browser tracking data. With eye-tracking, we collected 200k ground-truth datapoints on participants reading news on browsers. Then we trained machine learning and deep learning models to predict message-level reading time based on user interactions like mouse position, scrolling, and clicking. We reached 27% percentage error in reading time estimation with a two-tower neural network based on user interactions only, against the eye-tracking ground truth data, while the heuristic baselines have around 46% percentage error. We also discovered the benefits of replacing per-session models with per-timestamp models, and adding user pattern features. We concluded with suggestions on developing message-level reading estimation techniques based on available data.",

keywords = "bulk email, eye-tracking, neural network, personalization, reading region estimation, reading time, recommendation",

author = "Ruoyan Kong and Ruixuan Sun and Zhang, {Charles Chuankai} and Chen Chen and Sneha Patri and Gayathri Gajjela and Konstan, {Joseph A.}",

note = "Funding Information: This work was supported by the National Science Foundation under grant CNS-2016397. We thank the University of Minnesota{\textquoteright}s Usability Lab{\textquoteright}s User Experience Analysts, Nick Rosencrans, and David Rosen, for their assistance with eye-tracking technology. Publisher Copyright: {\textcopyright} 2023 ACM.; 15th Annual ACM Symposium on Eye Tracking Research and Applications, ETRA 2023 ; Conference date: 30-05-2023 Through 02-06-2023",

year = "2023",

month = may,

day = "30",

doi = "10.1145/3588015.3588404",

language = "English (US)",

series = "Eye Tracking Research and Applications Symposium (ETRA)",

publisher = "Association for Computing Machinery",

editor = "Spencer, {Stephen N.}",

booktitle = "Proceedings - ETRA 2023",

}

TY - GEN

T1 - Getting the Most from Eye-Tracking

T2 - 15th Annual ACM Symposium on Eye Tracking Research and Applications, ETRA 2023

AU - Kong, Ruoyan

AU - Sun, Ruixuan

AU - Zhang, Charles Chuankai

AU - Chen, Chen

AU - Patri, Sneha

AU - Gajjela, Gayathri

AU - Konstan, Joseph A.

N1 - Funding Information: This work was supported by the National Science Foundation under grant CNS-2016397. We thank the University of Minnesota’s Usability Lab’s User Experience Analysts, Nick Rosencrans, and David Rosen, for their assistance with eye-tracking technology. Publisher Copyright: © 2023 ACM.

PY - 2023/5/30

Y1 - 2023/5/30

N2 - A single digital newsletter usually contains many messages (regions). Users' reading time spent on, and read level (skip/skim/read-in-detail) of each message is important for platforms to understand their users' interests, personalize their contents, and make recommendations. Based on accurate but expensive-to-collect eyetracker-recorded data, we built models that predict per-region reading time based on easy-to-collect Javascript browser tracking data. With eye-tracking, we collected 200k ground-truth datapoints on participants reading news on browsers. Then we trained machine learning and deep learning models to predict message-level reading time based on user interactions like mouse position, scrolling, and clicking. We reached 27% percentage error in reading time estimation with a two-tower neural network based on user interactions only, against the eye-tracking ground truth data, while the heuristic baselines have around 46% percentage error. We also discovered the benefits of replacing per-session models with per-timestamp models, and adding user pattern features. We concluded with suggestions on developing message-level reading estimation techniques based on available data.

AB - A single digital newsletter usually contains many messages (regions). Users' reading time spent on, and read level (skip/skim/read-in-detail) of each message is important for platforms to understand their users' interests, personalize their contents, and make recommendations. Based on accurate but expensive-to-collect eyetracker-recorded data, we built models that predict per-region reading time based on easy-to-collect Javascript browser tracking data. With eye-tracking, we collected 200k ground-truth datapoints on participants reading news on browsers. Then we trained machine learning and deep learning models to predict message-level reading time based on user interactions like mouse position, scrolling, and clicking. We reached 27% percentage error in reading time estimation with a two-tower neural network based on user interactions only, against the eye-tracking ground truth data, while the heuristic baselines have around 46% percentage error. We also discovered the benefits of replacing per-session models with per-timestamp models, and adding user pattern features. We concluded with suggestions on developing message-level reading estimation techniques based on available data.

KW - bulk email

KW - eye-tracking

KW - neural network

KW - personalization

KW - reading region estimation

KW - reading time

KW - recommendation

UR - http://www.scopus.com/inward/record.url?scp=85161163180&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85161163180&partnerID=8YFLogxK

U2 - 10.1145/3588015.3588404

DO - 10.1145/3588015.3588404

M3 - Conference contribution

AN - SCOPUS:85161163180

T3 - Eye Tracking Research and Applications Symposium (ETRA)

BT - Proceedings - ETRA 2023

A2 - Spencer, Stephen N.

PB - Association for Computing Machinery

Y2 - 30 May 2023 through 2 June 2023

ER -

Getting the Most from Eye-Tracking: User-Interaction Based Reading Region Estimation Dataset and Models

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access

OpenUrl availability

Other files and links

Fingerprint

Cite this