There is already a system in place for refining the results of images that need it. When you come across an image that isn't well sorted there will be an option near the top that says "Would you like to help us improve these results?" which will allow you to pick between two sets of results to show which is most accurate which will help refine the results.