The aim of image segmentation in imaging science is to solve the problem of partitioning an image into smaller disjoint homogeneous regions that share similar attributes. The improvement of level set method (LSM) based on Chan-Vese (C-V) model with initialization mask for vector image segmentation in multiple color spaces is studied here. And simultaneously, the final segmentation is completed by a simple labeling scheme. Then the comparative study of the refined C-V model is done in multiple color spaces. The experimental results illustrate that the optimized C-V model leads faster and better segmentation results with robustness to noise and good adaptability in RGB, CIE XYZ, and YCbCr color spaces where the results of test image changes little. But it has made mistakes in HSV and CIE L*a*b* color model. Moreover, these color spaces, i.e. h1h2h3, produce poor segmentation on the reliability and accuracy of a set of test images by performance analysis with evaluation indicators.