Abstract : The growth of digital image and video archives is increasing the need for tools that effectively filter and efficiently search through large amounts of visual data. Towards this goal we suggested new method to automatically extract the color for the image to form a class of meta-data that is easily indexed. The algorithm of indexing the color based on the binary image to extract color regions from image. To achieve this goal the image segmented to many parts after converting color image to binary image based on Y component from the YCbCr color space. Color factors determined for each part by first convert color image for each part to HSV color space and then from the H component define the dominant color for that part. The proposed algorithm tested with many different images and gives very promised results.