: I'd be nervous about the benchmarks used. It's too easy to optimize for a
: specific set of circumstances, and getting blind to the "larger picture".
Yes, very nervous.
Twenty years ago I wrote some checkers playing programs.
First a primitive one that did reasonably well.
Then a much more sophisticated one, with lots of tunable
parameters, and let it play against itself many times, in
order to optimize the parameters.
In the end it had become much worse than the primitive program -
the only thing it was good at, was beating other incarnations
of itself.
Tuning is dangerous.