Show HN: JevBench, a reproducible benchmark for typed decision models

(benchmarkheaven.com)

34 points | by florianstandhar 8 hours ago

3 comments

sean_pedersen 3 minutes ago

Good project but this one also exists https://huggingface.co/spaces/multimodalart/jev-decision-ind... and the results do not seem to add up and also model sets are different... still needs time to mature likely
swyx 58 minutes ago

jev ceo on why he eschewed benchmarking: https://www.latent.space/i/216783460/privacy-benchmarking-an...

[-]
- jldugger 2 minutes ago
  
  Interesting; was curious how this didn't fall into trouble with ToS. Apparently the "no benchmarks" clause was intended for "limited preview" audiences and didn't get removed at launch on accident.