In the view of escalating threat in homeland security, it is imperative to have an automated detection system that can pick up suspicious patterns of human movement in physical environments. It can give a forewarning before a planned attack happens or an ultimate security is breached. In the past, significant research on Intrusion Detection Systems was established, but limited to virtual environments like computer networks and operating systems. In this paper, we proposed a general mining model for detecting suspicious patterns in physical environment. Suspicious patterns are subtle, relatively difficult to define and detect. Experiments are currently being conducted.