In this paper, an efficient technique for anomalous pedestrian activity detection in the academic institution is proposed. At the pixel and block levels, the proposed method elicits motion components that accurately represent pedestrian action, velocity, and direction, as well as along a frame. We also adopted these motion features to detect anomalous actions. The detection of anomalous behavior in academic environments is not available at the moment. Similarly, the existing method produces a high number of false positives. An anomaly detection dataset and a newly designed proposed student behavior database were used to validate the proposed framework. A significant improvement in anomalous activity recognition has been demonstrated in experimental results. Based on motion features, the proposed method reduces false positives by 3% and increases true positives by 5%. A discussion of future research directions concludes the paper.