Let the colors get useds on the important parts of the image. Edit your starting image background and make it a sepia tone, black and white, blur the colors out or remove it completely. If there are lots of colors your images it may be going to max colors befor it makes the parts of the image you care about. colorful object in  background that keep using up all the colors