Ammarah Hashmi

Postdoctoral Researcher | Institute of Information Science, Academia Sinica

Google Scholar DBLP ORCID GitHub LinkedIn X

About

I am a postdoctoral researcher specializing in Artificial Intelligence Generated Content (AIGC), multimedia forensics, audiovisual deepfake detection, multimodal AI, and human-centered deepfake analysis. My research focuses on building practical systems for video forgery detection and analyzing how human judgments differ from AI models in real-world scenarios.

My recent research interests include:

Multimedia forensics and deepfake detection, with a focus on audiovisual forgery analysis in real-world video content.
Audiovisual deepfake detection, including multimodal modeling of facial, visual, acoustic, and speech-related cues.
Human-centered deepfake analysis, especially how human perception, decision-making, and behavior differ from AI models when evaluating manipulated media.
Multimodal AI systems that combine vision, audio, speech, language models, retrieval, tools, and memory for practical interactive applications.

News

2026.07 — Served as a reviewer for the MDPI journal Journal of Imaging.
2026.06 — Served as a reviewer for the MDPI journal Future Internet.
2026.06 — Served as a reviewer for IEEE Transactions on Consumer Electronics.
2026.03 — Served as a reviewer for International Journal of Human-Computer Interaction.
2026.03 — Served as a reviewer for IEEE Transactions on Circuits and Systems for Video Technology.
2026.06 — Served as a reviewer for IEEE Transactions on Systems, Man and Cybernetics: Systems.
2026.01 — Joined Academia Sinica as a postdoctoral researcher .
2026.01 — 🎓Officially received my PhD. degree.
2025.12 — 📃Paper accepted in IEEE Computational Intelligence Magazine.
2025.12 — Successfully defended my Ph.D. dissertation at Academia Sinica.
2026.06 — Served as a reviewer for IEEE Transactions on Systems, Man and Cybernetics: Systems.
2025.11 — Received an AS postdoctoral fellowship [Jan 2026 - Dec 2027].
2025.10 — 📃Paper accepted in IEEE Transactions on Human-Machine Systems.
2025.09 — Received an NII travel grant to attend IEEE IJCB 2025 (8-11 Sept), held in Osaka, Japan.
2025.08 — Received Research and Travel Grant from NII, Japan [July - Dec 2025].
2025.07 — Joined Yamagishi Lab as an Invited Researcher [June - Dec 2025].
2025.07 — Successfully defended my Ph.D. dissertation at NTHU.
2025.03 — Completed all Ph.D. graduation requirements and chose to gain practical research experience before officially receiving the degree.
2025.03 — 📃Paper accepted in IEEE Transactions on Cognitive and Developmental Systems.
2024.11 — Selected as one of three candidates at NTHU for the international NII internship opportunity.
2024.10 — Served as a reviewer for IEEE Transactions on Emerging Topics in Computational Intelligence.
2024.07 — Received research grant from NTHU [July 2024 - Dec 2025].
2024.02 — Served as a reviewer for IEEE Transactions on Emerging Topics in Computational Intelligence.

Education

PhD in CS (Joint PhD Program), [2019 – 2025]
Department of Computer Science, National Tsing Hua University
Institute of Information Science, Academia Sinica
MS in Software Engineering, [2017 – 2019]
Department of Software Engineering, Beijing Institute of Technology
BS in Software Engineering, [2013 – 2017]
Department of Software Engineering, Comsats University

Work Experience

Postdoctoral Researcher

Institute of Information Science, Academia Sinica, Taipei, Taiwan
Jan, 2026 – present

[Conducts advanced, independent research in areas such as cybersecurity or multimedia forensics, aiming to publish findings in high-impact journals and conferences].
[Collaborate with academic partners, mentor junior researchers or students, and contribute to developing research proposals and funded projects].

Visiting Scholar

National Institute of Informatics(Yamigishi Lab), Tokyo, Japan
July, 2025 – Dec, 2025

[Conducted collaborative research with faculty and research team].
[Engaged in academic exchange through seminars, discussions, and knowledge sharing in areas related to machine learning, multimedia forensics, and AI research].

Research Scholar (in collaboration with NTHU)

Academia Sinica, Taipei, Taiwan
Sept, 2019 – Dec, 2025

[Conducted independent research, designed and implemented experiments, and contributed to publications in peer-reviewed venues].
[Engaged in coursework, seminars, and academic collaboration while developing expertise in a specialized research area].

Research Interests

Multimedia forensics and deepfake detection
Audiovisual deepfake detection using multimodal learning
Human-centered deepfake analysis and perception studies
Multimodal AI systems for real-world applications

Selected Projects

Human Perception of Audiovisual Deepfakes and Cognitive Modeling

This project investigates how humans perceive audiovisual deepfakes and how perceptual cues influence detection performance. It better bridges machine learning and human cognition to understand the limitations of both human and AI-based detection systems.

The work highlights perceptual vulnerabilities, decision biases, and alignment between human judgment and automated deepfake detection models.

Related work:

Unmasking Illusions: Understanding Human Perception of Audiovisual Deepfakes paper
Understanding Audiovisual Deepfake Detection: Techniques, Challenges, Human Factors, and Perceptual Insights paper

Keywords: human-AI interaction, perception study, deepfake detection, cognitive modeling, explainability, multimedia forensics

Ensemble and Hybrid Architectures for Robust Deepfake Detection

This line of work focuses on improving generalization in deepfake detection using ensemble learning strategies, multi-expert architectures, and hybrid feature representations across audio and visual modalities.

The goal is to enhance robustness under compression, unseen manipulations, and cross-dataset evaluation settings.

Related work:

Multimodal Forgery Detection Using Ensemble Learning paper
AVTENet: A human-cognition-inspired audio-visual transformer-based ensemble network for video deepfake detection paper

Keywords: ensemble learning, robustness, deepfake detection, multimodal fusion, video forensics

Audio-Visual Synchronization for Forgery Detection

This project explores audio-visual synchronization as a key signal for detecting manipulated videos. It focuses on lip-sync consistency, cross-modal temporal alignment, and speech-visual correspondence.

It forms the basis of several lip-sync–based forgery detection models designed for real-world deepfake scenarios.

Related work:

Lip Sync Matters: A Novel Multimodal Forgery Detector paper
AV-Lip-Sync+: Leveraging AV-HuBERT for Multimodal Inconsistency Detection paper

Keywords: lip synchronization, AV-HuBERT, temporal alignment, audiovisual consistency, deepfake detection

Publications

[2026] SAVe: Self-Supervised Audio-visual Deepfake Detection Exploiting Visual Artifacts and Audio-visual Misalignment paper
[2026] Understanding Audiovisual Deepfake Detection: Techniques, Challenges, Human Factors and Perceptual Insights paper
[2025] AVTENet: Audio-Visual Transformer-based Ensemble Network Exploiting Multiple Experts for Video Deepfake Detection paper
[2025] AV-Lip-Sync+: Leveraging AV-HuBERT to Exploit Multimodal Inconsistency for Video Deepfake Detection paper
[2024] Unmasking Illusions: Understanding Human Perception of Audiovisual Deepfakes paper
[2024] How good is chatgpt at audiovisual deepfake detection: A comparative study of chatgpt, ai models and human perception paper
[2022] Multimodal Forgery Detection Using Ensemble Learning paper
[2022] Lip Sync Matters: A Novel Multimodal Forgery Detector paper
[2022] ResViT: A Framework for Deepfake Videos Detection paper
[2022] A single smartwatch-based segmentation approach in human activity recognition paper
[2019] An Intelligent Wellness Assessment System for the Elderly Healthcare paper
[2018] A Survey on Vision-Based Hand Gesture Recognition paper

Talks and Presentations

PhD defense practice talk at the National Institute of Informatics, Japan, [2025]

Awards and Honors

Received an invited research grant from NII, Japan. [July 2025 - Dec 2025]
Received a research grant from NTHU. [July 2024 - Dec 2025]
Taiwan Gov. Fellowship for PhD studies, Taiwan Government, [2019-2025]
Chinese Gov. Fellowship for MS studies, Chinese Government, [2017-2019]
Pakistan Gov. Fellowship for BS studies, Pakistan Government, [2013-2017]
Certification by Zhongguancun the Belt and Road Industrial Promotion Association (ZBRA) for participation in a 3-day tour, China. [2018]
Best Volunteer Award Certificate by Beijing Institute of Technology, China. [Sept 2018]
Certificate of Android workshop attended at Comsats University, Pakistan. [2014]
Bronze medal for the International Kangaroo Mathematics Contest. [2007]
Bronze medal for the International Kangaroo Mathematics Contest. [2006]
3 Certificates of participation in the International Kangaroo Mathematics Contest. [2006-2008]

Services

Reviewer: IEEE Transactions on Emerging Topics in Computational Intelligence, Future Internet (MDPI), IEEE Transactions on Systems, Man, and Cybernetics: Systems, International Journal of Human-Computer Interaction, IEEE Transactions on Circuits and Systems for Video Technology, IEEE Transactions on Consumer Electronics

Contact

Ammarah Hashmi
Academia Sinica, Taiwan
Email: [hashmiammarah0[at]gmail[dot]com]