Bad benchmarks and evals: Senior SWE-Bench, napkin math, and winter tires

3 comments

• stephantul 2 hours ago
  The (relevant) segue into tires elevates this post so much. I don’t understand why, but it does
  [-]
  • vintagedave 1 hour ago
    Maybe that it's an example of the same pattern in a real-world, non-digital area? It makes it feel more grounded.
• quietraster 15 minutes ago
  [dead]
• alexpran 2 days ago
  [flagged]