In today’s world having a smart reliable surveillance system is very much in need. In fact in many public places like banks, jewellery stores, malls, schools and colleges it is basic necessary to have a surveillance system (CCTV). Most of today’s implementations are not smart and they record videos during night even when there is no motion. This will lead to unnecessary storage usage and difficult to get the important part of the footage. And also, most of the today’s implementations are stationary, they can’t track the moving object. This report will outline a naive approach to implement a smart video surveillance system using object motion detection and tracking. Here we are using conventional Background subtraction model to detect motion and we estimate the direction of motion of object by comparing the centroid of the moving object in subsequent frames and track the moving object by rotating the camera using servo. Video recording takes place only when there is movement in the frame which helps in storage efficiency. We are also improving the speed of email alert delivery by using multithreading.