Hi everyone, I am Cofounder / CTO of Coco Labs. We are the creators of VybBench. A complaint we kept hearing with image generation models was how quickly they would drift from the user’s taste. As subjective human reviewers did not reliably detect this issue, we built an objective benchmark, VybBench, to quantify the issue numerically and released it on Hugging Face: https://huggingface.co/datasets/cocolabs/vybbench.

Full disclosure: We also created a new model, Vyb which is designed to preserve fidelity to the user’s taste.