In the online world, user engagement refers to the quality of the user experience that emphasizes the positive aspects of the interaction with a web application and, in particular, the phenomena associated with wanting to use that application longer and frequently. This definition is motivated by the observation that successful web applications are not just used, but they are engaged with. Users invest time, attention, and emotion into them.

User engagement is measured in many ways, through methods of self-reporting (e.g., questionnaires), observer methods (e.g., facial expression analysis, speech analysis, desktop actions, etc.), neuro-physiological signal processing methods (e.g., respiratory and cardiovascular accelerations and decelerations, muscle spasms, etc.), and from a web analytics perspective (through online behavior metrics that assess users’ depth of engagement with a site). These methods represent various tradeoffs between scale of data and depth of understanding (for instance, surveys are small-scale but deep, whereas clicks are large-scale but shallow in understanding). Little work has been done to integrate these various measures into a coherent understanding of engagement success.  We address this problem by combining techniques from web analytics and mining, information retrieval evaluation, and existing works on user engagement coming from the domains of information science, multimodal human computer interaction and cognitive psychology. In this way, we can combine insights from big data with deep analysis of human behavior in the lab or through crowd-sourcing experiment.

This talk comprises three "inter-woven" parts:

(1) I will define user engagement, list its many characteristics as identified in the research and analytic literature, and discuss the challenges associated with measuring user engagement.

(2) I will describe data-driven approaches looking at user engagement through the development of models that allow for a better representation of how users engage within and across different digital services.

(3) I will describe how looking at affect and cognition is providing additional insights into measuring user engagement.

This work was done in collaboration with Ioannis Arapakis, Ricardo Baeza-Yates, Georges Dupret, Janette Lehmann, Lori McCay-Peet, Vidhya Navalpakkam, David Warnock and Elad Yom-Tov.