Edge detection is the most basic problem in the process of image processing. The precision of traditional edge detection algorithm is not very high, it unable to meet the high precision need of modern industrial test technology. In order to overcome the deficiency, this text proposed subpixel edge detection algorithm based on the function curve fitting-Gauss fitting of gradient direction sub-pixel edge detection algorithm. According to the gradient distribution of the image, this text use gauss curve fitting the edge in order to realize the sub-pixel location. This text compared this algorithm with sub-pixel edge detection based on the LOG operator and sub-pixel edge detection based on the quadratic, and draw that this algorithm not only have the short running time and high efficiency, but also has proved that the algorithm has rotation invariant through the experiment. It is that pattern recognition and picture measure the important pretreatment means in the course to follow the method at the border, contradiction at accuracy and speed that but follow the method and exist at the traditional border. To above-mentioned problems, this text proposes following algorithms at the border based on model, and then try to get the diameter of the cutter. The experiment shows this algorithm at the realization border that can be very good and follows, measure the comparison of the algorithm through two kinds of diameters, drawing the running time of least square method shorter, efficiency is relatively high.