[CSD MS Thesis] 5th Year M.S. Thesis Presentation - Tianyue Ruby Sun - December 2, 2025

[Artur Dubrawski]

Dear Autonians,

Come see Tianyue present her masters work tomorrow morning!

Cheers
Artur

On Mon, Dec 1, 2025 at 10:51 AM Angy Malloy <amalloy at cs.cmu.edu> wrote:

> Tianyue Ruby Sun
>
> Tuesday, December 2, 2025
>
> 9:30 AM
>
> Gates Building, Room 8102
>
> Title:
>
> Remote Photoplethysmography: Spatiotemporal Architecture
>
> Abstract:
> Remote photoplethysmography (rPPG) enables contactless measurement of
> physiological signals such as heart rate and respiratory rate from videos,
> offering a practical alternative to traditional contact-based sensor
> measurements. Recent deep learning methods have achieved strong rPPG
> accuracy, but these approaches often depend on controlled settings and
> struggle to generalize to real-world environments with motion and varying
> lighting. These limitations are in part due to the reliance on techniques
> such as manual parameter tuning and the need for large labelled datasets
> that are often captured under clean conditions.
>
> This research thesis presents an exploration of the end-to-end rPPG
> pipeline. The primary contribution is a novel spatiotemporal architecture
> for rPPG that combines DINOv2, a vision transformer, and Chronos, a time
> series model. This represents the first multimodal rPPG framework that
> leverages a combination of spatial and temporal representations from
> foundation models for physiological measurement. The two foundation models
> are kept frozen, and a lightweight prediction head is trained.
>
> The proposed model achieves strong performance on the synthetic SCAMPS
> dataset for heart rate estimation, establishing benchmarks for future rPPG
> research. On real-world datasets, including PURE and UBFC-rPPG, the model
> demonstrates effective learning of blood volume pulse (BVP) waveforms and
> heart rate estimation, despite the increased errors reflecting the
> difficulty of more challenging conditions. Extensions of the model to
> respiratory rate illustrate the generalizability of the architecture across
> different physiological measurement tasks. Overall, the results show that
> foundation models can improve rPPG robustness and generalization, offering
> a promising path towards practical rPPG systems with applications in
> inpatient monitoring, telehealth, and emergency response.
>
> In addition to model development, this thesis analyzes components of the
> full rPPG pipeline, including signal processing and ground truth
> extraction. It is shown that common signal processing methods applied to
> the same BVP signal can lead to discrepancies in the estimation of the
> scalar heart rate value. Moreover, the method of obtaining the ground truth
> from data affects the reported performances. These insights motivate the
> need to further discuss reliable signal processing and evaluation
> procedures to ensure reliable comparisons and interpretations of rPPG
> methods.
>
> Thesis Summary:
>
> https://drive.google.com/drive/folders/1HFwJZlk0jV-FC6h4TDbuCu2PzQgnD7FK
>
> Thesis Committee:
> Artur Dubrawski, Chair
> Laszlo Jeni
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20251201/abd89963/attachment.html>