Hacker News (curated)new | past | comments | ask | show | jobs| show hidden

I wish model providers would stop committing chart crimes in their releases.

- if you're gonna order the rest of the bar chart by rank, order your model accordingly.

- if you're gonna highlight a winner in a table of benchmarks, don't highlight your entire model row in the table.

Etc etc



I'd bet there's a correlation between benchmaxxing and chart crimes. Companies who try to deceive perceptions via the charts are more likely to cheat at the benchmarks too, I'm sure. That's assuming ill intent, of course - which is often the case for charts related to model releases, but not necessarily always the case.

I wonder if this is being reinforced via LLM because they see every other modeler doing the same thing.

Read websites through llm.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact | github