As the Internet of Things (IoT) continues to grow, smartphones have become an integral part of IoT systems. However, with the increasing amount of personal information stored on smartphones, users’ privacy is at risk of being compromised by malicious attackers. Malware detection engines are commonly installed on smartphones to defend against these attacks, but new attacks that can evade these defenses may still emerge. In this paper, we present EavesDroid, a new side-channel attack on Android smartphones that allows an unprivileged attacker to accurately infer fine-grained user behaviors (e.g. viewing messages, playing videos) through the on-screen operations. Our attack relies on the correlation between user behaviors and the return values of system calls. The fact that these return values are affected by many factors, resulting in fluctuation and misalignment, makes the attack more challenging. Therefore, we build a CNN-GRU classification model, apply min-max normalization to the raw data and combine multiple features to identify the fine-grained user behaviors. A series of experiments on different models and systems of Android smartphones show that, EavesDroid can achieve an accuracy of 98% and 86% for already considered user behaviors in test set and real-world settings. To prevent this attack, we recommend malware detection, obfuscating return values or restricting applications from reading vulnerable return values.