the next-gen random tourney: extreme world's best engine test

on Tue Dec 03, 2013 11:32 pm by · on Tue Dec 03, 2013 11:32 pm by

there is a thread about random tourney
and that first random tourney is like
the traditional random tourney

there are matches for a specific random FEN position

that's it
it is a simple and plain random tourney
this is the 1st gen random tourney
this random tourney was loconic and lucid
with no added on things

***

then there is another thread about
the 2nd gen random tourney
that is the tactical and strategical random tourney

for this 2nd gen random tourney
we have FENs with specific purposes
in more specific situations
so engines could be tested under specific conditions
(tested for zugzwang,
bottleneck,
opening,
ending phase,
combinational moves in terms of defence and attack)

also this 2nd gen random tourney was strategical
in a sense that
the FENs are conceived in a way
that is resembling the historical events

for example,
if you take a look at that Iliad map
the FEN random position is similar to that of the actual fight scene

this is the 2nd gen random tourney

***

then this is the new next-gen random tourney
this is the 3rd gen random tourney

this is like the following

I'll run a plethora of matches
for the best engines
under extreme conditions

the engines will be tested for specific FEN random positions

but the purpose is holistically different from the goals of
1st and 2nd gen random tourneys

for this next-gen random tourney
I'll run a number of matches
for the engines for specific FEN positions

then I can find that
certain engines are weak in certain positions
for instance,
some engines show and exhibit weakness
in zugzwang and/or bottleneck positions
these engines can't make good moves well
when many pawns with diff colors
are intermixed and correlated

also I can see that
certain engines can't do well
in mass pawns versus mass pawns situation

moreover we will know that
which engines are weak in this situation
combinational moves of many generals
versus combinational moves of many generals
(generals: Q R N B)

some engines can't do well
to form a platoon
and make combo attacks and combo defence moves

in addition, we can see
which engines are weak in king defence
some engines are not doing well
when they have to protect their kings
under siege and under heavy attacks

also, we can find something like
which engines are very adroit in mate searching
some engines can find mate moves very well
they know how to win the opponent's king

and there are many other possible senarios
when we run this next-gen random tourney
for exteme engine tests

***

so the purpose of this next-gen tourney is
once we find out
which engines are showing the weakness under certain situations
those engines can be improved
in those situations
then those engines will be better and more powerful
after improving upon these weaknesses

so even common and normal ppl, fans, supporters and enthusiasts
can contribute for the betterment of engines
and that is the purpose of this next-gen random tourney

when compared to the 1st and 2nd random tourneys
this next-gen random tourney is
more thorough,
more detailed,
more specific,
better testing
and putting the engines under extreme severe FEN conditions
for the fierce performance testing

so, in essence
this next-gen random tourney is an upgrade

***

[pgn]
[Event "Computer Chess Game"]
[Site "USER01"]
[Date "2013.12.04"]
[Round "-"]
[White "Houdini 3 Pro w32"]
[Black "Stockfish 241113"]
[Result "1-0"]
[TimeControl "60/60"]
[FEN "1r1b3k/3r1pp1/3p1R2/p1pPp2P/PpP1P1R1/1P4P1/2N2P1K/8 w - - 0 1"]
[SetUp "1"]

{--------------
. r . b . . . k
. . . r . p p .
. . . p . R . .
p . p P p . . P
P p P . P . R .
. P . . . . P .
. . N . . P . K
. . . . . . . .
white to play
--------------}
1. Rf3 {+1.06/18} Kh7 {-1.57/20 1.0} 2. Rd3 {+0.99/19} g6 {-1.91/18} 3.
hxg6+ {+1.22/18} fxg6 {-1.95/23} 4. Kg2 {+1.30/19} Bf6 {-1.95/24} 5. Rd1
{+1.27/19} g5 {-1.79/23} 6. Rh1+ {+1.64/16} Kg6 {-2.56/17} 7. f4 {+1.75/17}
Rf8 {-2.60/20} 8. Ne3 {+1.86/16} Be7 {-2.42/21} 9. fxg5 {+1.87/17} Bxg5
{-2.96/20} 10. Nf5 {+1.96/18} Rxf5 {-3.19/20} 11. exf5+ {+2.10/19} Kxf5
{-3.15/23} 12. Kf3 {+2.01/20} Rd8 {-3.23/22} 13. Rh5 {+2.29/19} Rg8
{-2.85/21} 14. Rgh4 {+2.36/17} Rd8 {-2.93/22} 15. Rh1 {+2.72/17} e4+
{-3.53/18} 16. Ke2 {+2.83/17} Kg4 {-4.08/19} 17. Rh8 {+3.23/17} Rxh8
{-5.02/21} 18. Rxh8 {+3.97/17} Kxg3 {-5.39/23} 19. Ra8 {+6.31/17} Kf4
{-10.62/18} 20. Rxa5 {+8.80/16} Bd8 {-12.68/19} 21. Ra8 {+10.54/18} Bc7
{-15.49/20}
{Xboard adjudication} 1-0

[/pgn]

last moves:
21. Ra8 {+10.54/18} Bc7 {-15.49/20}

[pgn]
[Event "Computer Chess Game"]
[Site "USER01"]
[Date "2013.12.04"]
[Round "-"]
[White "Stockfish 241113"]
[Black "Houdini 3 Pro w32"]
[Result "1/2-1/2"]
[TimeControl "60/60"]
[FEN "1r1b3k/3r1pp1/3p1R2/p1pPp2P/PpP1P1R1/1P4P1/2N2P1K/8 w - - 0 1"]
[SetUp "1"]

{--------------
. r . b . . . k
. . . r . p p .
. . . p . R . .
p . p P p . . P
P p P . P . R .
. P . . . . P .
. . N . . P . K
. . . . . . . .
white to play
--------------}
1. Rf5 {+1.65/23} Kh7 {-0.93/17 1.0} 2. Kg2 {+1.65/24} Bf6 {-0.93/18} 3.
Ne1 {+1.65/23} Rbd8 {-0.71/17} 4. Kf3 {+1.55/21} Rb7 {-0.71/18} 5. Ke2
{+1.65/22} Kh6 {-0.71/16} 6. Ng2 {+1.65/20} Rc7 {-0.74/16} 7. Kd3
{+1.65/20} Kh7 {-0.93/17} 8. Nh4 {+2.20/17} Rcd7 {-0.93/15} 9. f4
{+2.36/20} Re8 {-1.01/16} 10. Nf3 {+2.52/20} exf4 {-0.91/16} 11. gxf4
{+2.42/19} Rde7 {-1.33/15} 12. Ng5+ {+2.42/20} Kh6 {-1.43/16} 13. Rg1
{+2.42/21} Ra7 {-1.22/15} 14. Rf1 {+2.48/16} Bxg5 {-1.38/15} 15. Rxg5
{+2.74/19} Rae7 {-1.42/16} 16. Re1 {+2.98/17} Kh7 {-1.37/15} 17. e5
{+3.09/19} f6 {-1.21/17} 18. Rf5 {+3.37/21} Kg8 {-1.25/18} 19. e6
{+3.13/24} Rf8 {-1.25/20} 20. Rg1 {+3.11/30} Kh7 {-1.27/21} 21. Rh1
{+3.11/31} Kh6 {-1.25/22} 22. Ke4 {+3.11/34} Rg8 {-1.25/22} 23. Rd1
{+3.11/35} Rge8 {-1.25/23} 24. Rd2 {+3.11/35} Rh8 {-1.25/22} 25. Ra2
{+3.11/34} Rb8 {-1.25/23} 26. Re2 {+3.11/36} Rbe8 {-1.25/23} 27. Rc2
{+3.11/37} Rf8 {-1.25/23} 28. Rc1 {+3.11/37} Rh8 {-1.25/23} 29. Re1
{+3.11/35} Rhe8 {-1.25/23} 30. Rg1 {+3.11/38} Rf8 {-1.25/23} 31. Ra1
{+3.11/36} Rb8 {-1.25/24} 32. Rb1 {+3.11/36} Rc8 {-1.25/24} 33. Rb2
{+3.11/39} Ra8 {-1.25/23} 34. Rf2 {+3.11/40} Rc8 {-1.25/23} 35. Rh2
{+3.11/40}
{Xboard adjudication: 50-move rule} 1/2-1/2
[/pgn]

last moves:
35. Rh2 {+3.11/40}

on Wed Dec 04, 2013 7:25 am by · on Wed Dec 04, 2013 7:25 am by

For 32 bit H3 as 65% sin against any SF version. the end.

on Wed Dec 04, 2013 8:08 pm by · on Wed Dec 04, 2013 8:08 pm by

Program Elo + - Games Score Av.Op. Draws

1 Houdini 3 Pro w32 : 3250 30 30 364 53.0 % 3229 28.0 %
2 Strelka 5.1 : 3241 48 48 124 51.6 % 3229 38.7 %
3 Stockfish 241113 : 3228 20 20 732 54.6 % 3196 34.6 %
4 Gull 2.3 : 3141 46 47 121 37.6 % 3229 45.5 %
5 Critter 1.6a 32-bit : 3033 48 50 123 24.4 % 3229 39.0 %

we can analyze the performances of engines

after the perft
in these super complex zugzwang with bottleneck FEN positions

we can see that
H3 is doing very well

strelka is doing well
strelka got +20 elo or so
(when compared with the engine's normal rating)
we can see that strelka and H3 are the best engines
in zugzwang and bottleneck positions

SF got -40 elo down
we can see that
these kinds of zugzwang and bottleneck positions are hard for SF

gull has seen about -60 elo down
these kinds of zugzwang and bottleneck positions are very hard for gull

critter has got approx -170 elo down
these kinds of zugzwang and bottleneck positions are extremely hard for critter

along with detailed illustrations and analyses
the full pgn and result will be posted
when many more matches are done

on Thu Dec 05, 2013 10:42 am by · on Thu Dec 05, 2013 10:42 am by

Program Elo + - Games Score Av.Op. Draws

1 Houdini 3 Pro w32 : 3250 30 30 364 53.0 % 3229 28.0 %
2 Strelka 5.1 : 3241 48 48 124 51.6 % 3229 38.7 %
3 Stockfish 241113 : 3228 20 20 732 54.6 % 3196 34.6 %
4 Gull 2.3 : 3141 46 47 121 37.6 % 3229 45.5 %
5 Critter 1.6a 32-bit : 3033 48 50 123 24.4 % 3229 39.0 %

this is the same rating list
that is in the above post

I just changed the font to courier new
since that font supports something like
all the horizontal widths of letters have the same length

so if I use this font
it will be easy for viewing the rating list

just using this font is a lot faster
than uploading the picture of the rating list

on Fri Dec 06, 2013 6:47 am by · on Fri Dec 06, 2013 6:47 am by

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 241113 : 3281 42 41 172 58.1 % 3224 37.2 %
2 Critter 1.6a 32-bit : 3264 85 85 40 47.5 % 3281 40.0 %
3 Houdini 3 Pro w32 : 3250 66 67 44 45.5 % 3281 59.1 %
4 Gull 2.3 : 3234 87 87 44 43.2 % 3281 31.8 %
5 Strelka 5.1 : 3149 99 102 44 31.8 % 3281 18.2 %

this is the rating list of a completely different new random tourney
and SF did well in this random FEN position
also critter did so well
and strelka didn't

along with detailed illustrations and analyses
the full pgn and result will be posted
when many more matches are done

by