In this paper, we have proposed an Automatic Aerial Video Processing System for analyzing land surface features. Analysis of aerial video is done in three steps a) Image pre-processing b) Image registration and c) Image segmentation. Using the proposed system, we have identified Land features like Vegetation, Man-Made Structures and Barren Land. These features are identified and differentiated from each other to calculate their respective areas. Most important feature of this system is that it is an instantaneous video acquisition and processing system. In the first step, radial distortions of image are corrected using Fish-Eye correction algorithm. In the second step, the image features are matched and then images are stitched using Scale Invariant Feature Transform (SIFT) followed by Random Sample Consensus (RANSAC) algorithm. In the third step, the stitched images are segmented using Mean Shift Segmentation and different structures are identified using RGB model. Here we have used a hybrid system to identify Man-Made Structures using Fuzzy Edge Extraction along with Mean Shift segmentation. The results obtained are compared with the ground truth data, thus evaluating the performance of the system. The proposed system is implemented using Intel's OpenCV.