Tennison wrote:You are right, there is a problem ... ;-)
But it's always on moves 7,17,27,... so statistically I think it will affect the 2 engines the same way. So results should be nearly the same.
But I prefer stop using this time control to have the most correct results as possible.
I have to try other time controls. ;-)
Thanks a lot Effel ;-)
But even if you don't find any elo gain with your test on T3, I'm surprised that on all test I made I have a plus for T3. How can we explain that ?
The impact could be the same on both engines, but results are still meaningless.
My best advice is:
1) Don't use a normal GUI for running ultra-fast engine matches. Use cutechess-cli or LittleBlitzer that have been designed for this task.
2) Verify the engine matches. Replay some games. Make sure everything is perfect. Don't *assume* everything is correct (as most people seem to do).
3) Know what to expect. If you're trying to change some Houdini parameters, you'll be very fortunate to end up with 10 Elo improvement. If you measure 50 Elo improvement, something is very wrong.
4) Play the number of games that can demonstrate the improvement. If you want to measure 5 to 10 Elo, expect to play 10,000 games per engine. If you're not ready to do this, don't even bother to start.