Last week, Boys & Men Online co-hosted a webinar with the Psychology of Technology Inst. to bring together researchers using new tools to study digital interactions.
Many of the most important questions about technology—from sports betting and pornography to social media and AI companions—cannot be answered through recall surveys alone. People often misremember what they do online and struggle to reconstruct the context surrounding their digital behavior.
The five featured researchers share how they combine behavioral data, AI tools, and real-time surveys to study how digital technologies shape our lives.
Alexandra Rodman & Varun Mishra described the Connect Study’s approach to analyzing how digital experiences affect adolescent mental health using objective behavioral data rather than relying solely on surveys. Rodman described her motivation for the study: mental health disorders often emerge during adolescence, when teenagers are highly sensitive to online and offline social experiences.
The Connect Study follows adolescents for eight months and collects:
Smartphone usage data
GPS location data
Physical activity data
App usage data
Smartwatch physiological data
fMRI scans
Repeated surveys and interviews
Instead of sending surveys at random times, the software monitors behavior patterns, identifies unusual behaviors, and immediately asks participants what is happening in the moment. For example, if a teenager who rarely uses Instagram late at night suddenly spends hours on the platform on a school night, the system can detect that anomaly and ask follow-up questions about what they were doing, who they were interacting with, and how they were feeling.
This allows Rodman and her colleagues to move beyond simple measures of screen time and instead understand digital behavior in context.
Matthew Brown collected actual betting data alongside surveys to ask: Are sports bettors making informed choices, or are they systematically misunderstanding the risks?
Study participants predicted they would roughly break even. In reality, they lost about 7.5 cents for every dollar wagered.
The study participants underestimated the true financial cost of betting, which may strengthen the case for stronger consumer-protection policies.
Brown suggested that behavioral data and surveys should be viewed as complements. Behavioral data tells us what happened, while surveys can help explain why.
He also highlighted two challenges using sensitive observational data:
People willing to share account data are not representative of all bettors. (External validity)
Researchers must manage attrition so that participants who drop out do not bias the representativeness. (Internal validity)
Fiona Baker provided an overview of the Adolescent Brain Cognitive Development (ABCD) Study, one of the largest longitudinal studies ever conducted:
Over 11,000 participants
Began when participants were ages 9–10 in 2016-2018
Has followed them for about a decade
Combines neuroimaging, surveys, health measures, and digital behavior data
Just recently released data through the participants’ seven-year follow-up visit.
Using the EARS app to capture smartphone data, researchers found that adolescents averaged about 70 minutes of phone use during school hours — mostly on social media, streaming video, and games. During the 10 PM–6 AM period, adolescents averaged roughly 50 minutes of phone use, with social media the dominant activity.
Future studies will combine smartphone usage, sleep data, heart rate measurements, and physical activity data to explore how digital behavior affects sleep, health, and development.
Richard Landers introduced QUAIL (Qualtrics AI Link), an open-source system that allows researchers to embed AI conversations directly into Qualtrics surveys.
Rather than asking participants how they think they would respond in a hypothetical situation, researchers can simulate realistic conversations and observe actual behavior.
For example, participants could be randomly assigned to interact with different AI personas:
Self-reliance framing
Anti-stigma framing
Neutral framing
Researchers could then compare how these different conversational styles influence disclosure, help-seeking, or emotional expression.
Landers argued that AI-based interactions could become valuable tools for studying loneliness, stigma, mental health disclosure, and other sensitive topics that are difficult to capture through traditional surveys alone. At the same time, he cautioned that AI systems evolve rapidly, models change over time, and rigorous validation (by humans) remains essential.
Three Big Takeaways
Digital behavior should be measured directly whenever possible. Surveys remain useful, but people are often inaccurate when recalling their online behavior. Researchers increasingly need tools that capture behavior as it occurs.
Context matters more than raw screen time. Beyond raw screentime numbers, researchers are now asking: “What were you doing, with whom, under what circumstances, and how did it affect you?”
The next generation of digital interaction research is multimodal. Researchers increasingly combine behavioral data, wearable devices, real-time surveys, and AI-powered interviews.
During the Q&A, panelists emphasized that collecting better data requires building greater trust. Researchers ought to ensure easy-to-understand disclosure, participant ownership of data, and giving participants value in return for sharing their data.
They also noted the need for better tools, more collaboration, shared methodologies, and better ways to track behavior across websites and platforms.
If we want better answers about how technology shapes attitudes and behaviors, we need more accurate methods of observing how people actually behave.
