Chess2u
Would you like to react to this message? Create an account in a few clicks or log in to continue.

Chess2uLog in

descriptionDifferent engine matches and tournaments EmptyDifferent engine matches and tournaments

more_horiz
Hardware: AMD Ryzen 9 5900X CPU @ 3.70GHz, 12 cores, 64GB, gpu RTX 3080, 10GB
Gauntlet: SF-dev vs. LC0 384x30b nets
Spoiler :

Code:


   # PLAYER             :  RATING  ERROR    POINTS   PLAYED   (%)  CFS(%)       W       D       L  D(%)
   1 lc0.net.610024     :    46.8    3.0   11294.5    20000    56      75    5076   12437    2487    62
-  2 lc0.net.609958     :    45.3    3.0   11254.0    20000    56      54    5040   12428    2532    62
...
+ 16 lc0.net.610428     :    39.8    3.0   11103.5    20000    56      59    4831   12545    2624    63
...
- 66 Stockfish.dev      :     0.0   ----  588363.0  1300000    45     ---  185698  805330  308972    62

White advantage = 55.27 +/- 0.19
Draw rate (equal opponents) = 64.93 % +/- 0.05


PGN: https://pixeldrain.com/u/yPNWm6hZ

descriptionDifferent engine matches and tournaments EmptyRe: Different engine matches and tournaments

more_horiz
Network IDs: 609722, 609953, 609958, 609969, 610024, 610034, 610044, 610053, 610093, 610110, 610161, 610221, 610357, 610363, 610366, 610369, 610428
Book: book-ply8-unifen-Q-0.0-0.25.pgn and book-ply8-unifen-Q-0.25-0.40.pgn (10K openings in both books), order=sequential
Comment: Adding 40K games to tests with Chad's book. The aim of the test is to order T60 nets. The order is considered sure with 99% CFS (a bit more than 3.5 Elo). CFS matrix and detailed results in 7z file.

Code:


# PLAYER            :  RATING  ERROR    POINTS  PLAYED  (%)  CFS(%)      W      D      L  D(%)
  1 lc0.net.609958    :    47.9    1.8  33897.0    60000    56      86  17124  33546    9330    56
  2 lc0.net.609969    :    46.4    1.9  33778.0    60000    56      77  16922  33712    9366    56
  3 lc0.net.610024    :    45.4    1.8  33698.0    60000    56      57  16894  33608    9498    56
  4 lc0.net.610093
  5 lc0.net.610053
  6 lc0.net.610044
  7 lc0.net.610034
  8 lc0.net.609953
-  9 lc0.net.610110
- 10 lc0.net.610161
- 11 lc0.net.610221
  12 lc0.net.609722    :    41.0    1.9  33345.5    60000    56      61  16764  33163  10073    55
- 13 lc0.net.610363    :    40.7    1.8  33316.0    60000    56      60  16499  33634    9867    56
+ 14 lc0.net.610428    :    40.4    1.8  33289.0    60000    55      57  16341  33896    9763    56
- 15 lc0.net.610366    :    40.1    1.9  33271.0    60000    55      66  16453  33636    9911    56
- 16 lc0.net.610357    :    39.6    1.9  33225.0    60000    55      61  16301  33848    9851    56
- 17 lc0.net.610369    :    39.2    1.9  33194.5    60000    55    100  16494  33401  10105    56
  18 Stockfish.dev    :    0.0  ----  450527.5  1020000    44    ---  164841  571373  283786    56

White advantage = 74.69 +/- 0.23
Draw rate (equal opponents) = 60.15 % +/- 0.05


PGN: https://pixeldrain.com/l/RDYTRTXZ#item=0

descriptionDifferent engine matches and tournaments EmptyRe: Different engine matches and tournaments

more_horiz
Black: Stockfish 151121-nn-13406b1dcbe0.nnue
Intel Core i5-7600K, GHz:4.20 , Сore:4, Book:off, Endgame tables:Syzygy 3-4-5, Hash:128 MB, Ponder:On
ТС = 2 min+1sec


[pgn]
1. d4 Nf6 2. c4 e6 3. Nf3 d5 4. Nc3 Bb4 5. Qa4+ {(Сg5)}
Nc6 6. e3 O-O 7. Bd2 dxc4 8. Bxc4 Bd6 9. O-O {(Фc2)} e5 10. d5 {(d:e5)} Ne7 11.
Qc2 h6 12. e4 {(h3)} a6 13. Be3 {(h3)} b5 14. Bb3 {(Сd3)} Ng6 15. Rac1 {(a4)}
Bd7 16. Rfe1 {(a3)} Ng4 17. Bd2 Nh4 18. Qd3 b4 19. Ne2 {(Кd1)} Nxf3+ 20. gxf3
Nf6 21. Kh1 {(Кg3)} Nh5 22. Rg1 {(Кg3)} Kh8 23. f4 exf4 24. Nd4 {(Сc2)} Re8 25.
Bc2 Nf6 26. f3 Qe7 27. Be1 Qe5 28. Bh4 {(a3)} Nh7 29. Be1 {(Сf2)} Ng5 30. Rg2
Nh3 31. Bd2 {(Кc6)} a5 32. Rf1 {(Лd1)} Rab8 33. Bc1 {(a4)} Bc5 34. Rd1 Rb6 35.
Qc4 Bd6 36. Nf5 {(Фd3)} Bxf5 37. exf5 Qe1+ 38. Qf1 Qh4 39. f6 g6 40. Ba4 Rd8
41. Qe1 {(Лc2)} Qxf6 42. Qe4 {(Фf1)} Ng5 43. Qe2 Be5 44. Bc6 {(Фf1)} b3 45. a4
{(a3)} Rxc6 46. dxc6 Rxd1+ 47. Qxd1 Qxc6 48. Bxf4 Bxf4 49. Qd4+ Kg8 50. Qxf4
Nxf3 51. Rf2 {(Лg3)} Ng5+ 52. Rg2 Ne6 53. Qe3 {(Фe5)} g5 54. Kg1 Qxa4 55. Rd2 {
(h3)} Qg4+ 56. Kf2 a4 57. Qe5 Qc4 58. Qe3 {(Крg2)} Qb5 59. Qc3 Qf5+ 60. Qf3 Qe5
61. Qg3 {(Лe2)} Qa5 62. Qd3 a3 63. bxa3 Qxa3 64. Qc3 Qa4 65. Rb2 {(Крf1)} Qf4+
66. Ke1 {(Крg2)} Nd4 67. Kd1 c5 68. Qxc5 {(h4)} Qe3 69. Qb4 Nf3 70. Rxb3 Qg1+
71. Kc2 Nd4+ 72. Kd3 Nxb3 73. Qb8+ Kg7 74. Qe5+ Kg6 75. Kc4 {(Фe4+)} Qd4+ 76.
Qxd4 Nxd4 77. Kxd4 {(Крd3)} g4 78. Ke3 g3 79. hxg3 {(h3)} Kg5 80. Kf2 {(Крd3)}
Kg4 81. Kg2 h5 82. Kh2 {(Крf2)} Kf3 83. Kh3 f5 84. Kh2 Kf2 85. Kh3 Kg1 86. g4 {
(Крh4)} fxg4+ 87. Kg3 Kf1 88. Kh4 Kf2 89. Kxh5 g3 90. Kg4 {(Крg6)} g2 91. Kf5
g1=Q 92. Ke6 Qg6+ 93. Kd5 {(Крd7)} Ke3 94. Kc5 Qe6 95. Kb4 Qc6 96. Ka5 {(Крb3)}
Qb7 97. Ka4 Kd3 98. Ka5 {(Крa3)} Kc3 99. Ka4 Qa6# 0-1
[/pgn]

Game: https://pixeldrain.com/u/H8Epa1f2

descriptionDifferent engine matches and tournaments EmptyRe: Different engine matches and tournaments

more_horiz
Hardware: AMD Ryzen 9 5900X CPU @ 3.70GHz, 12 cores, 64GB, gpu RTX 3080, 10GB
Gauntlet: SF-dev vs. LC0 512x40b nets
Network IDs: 780001, 780048, 780856, 780900, 781032, 781128, 781474, 781589, 781740, 781932
LC0-version: v0.28.2
LC0-options: default
SF-version: SF-dev 22011322
SF-options: 8 Threads, 1024MB Hash
Software: Cutechess-CLI
Time control: LC0: 1,500 nodes/move, SF-dev: 500,000 nodes/move
Book: Chad's 8-ply book, book-ply8-unifen-Q-0.0-0.25.pgn and book-ply8-unifen-Q-0.25-0.40.pgn (10K openings in each book), order=sequential
Tablebases: 6-men
Adjudication: -draw movenumber=50 movecount=5 score=8 -resign movecount=5 score=1000
Ordo: -a 0  -A "Stockfish.dev" -W -D -U "0,1,2,3,4,5,6,7,8,9,10" -s 2000 -n 1
Comment: The aim of the test is to order T78 nets. The order is considered sure with 99% CFS (a bit more than 3.5 Elo). CFS matrix and detailed results in 7z file.

Code:


 # PLAYER            :  RATING  ERROR    POINTS  PLAYED   (%)  CFS(%)       W       D       L  D(%)
   1 Stockfish.dev     :     0.0   ----  321682.0  600000    54     100  143526  356312  100162    59
   2 lc0.net.781474    :   -20.2    1.8   28353.5   60000    47      85   10237   36233   13530    60
   3 lc0.net.781589    :   -21.5    1.8   28244.5   60000    47      55    9993   36503   13504    61
   4 lc0.net.781128    :   -21.7    1.8   28232.0   60000    47      89   10177   36110   13713    60
+  5 lc0.net.781932    :   -23.2    1.7   28107.5   60000    47      85   10235   35745   14020    60
   6 lc0.net.781032    :   -24.6    1.7   28000.5   60000    47      77    9938   36125   13937    60
-  7 lc0.net.781740    :   -25.5    1.7   27925.5   60000    47     100   10239   35373   14388    59
   8 lc0.net.780856    :   -30.3
   9 lc0.net.780900    :   -30.6
  10 lc0.net.780048    :   -32.4
  11 lc0.net.780001    :   -36.3

White advantage = 77.34 +/- 0.28
Draw rate (equal opponents) = 63.81 % +/- 0.07


https://pixeldrain.com/u/62Mbsg9t

Spoiler :

descriptionDifferent engine matches and tournaments EmptyRe: Different engine matches and tournaments

more_horiz
Different engine matches and tournaments 04-05-2022-060634

Intel Core i5-7600K, GHz:4.20 , Сore:1, Gui:Fritz-11, Games:100, Book:Off, Endgame tables:Off, Ponder:Off, Hash:128 MB.
ТС = 1 min+1 sec


Stockfish 030522 x64_bmi2 - nn-d0b74ce1e5eb.nnue
Sugar Al 2.50 - nn-ac07bd334b62.nnue

https://pixeldrain.com/u/kZ97oyXd

descriptionDifferent engine matches and tournaments EmptyRe: Different engine matches and tournaments

more_horiz
lisandr wrote:
Different engine matches and tournaments 04-05-2022-060634

Intel Core i5-7600K, GHz:4.20 , Сore:1, Gui:Fritz-11, Games:100, Book:Off, Endgame tables:Off, Ponder:Off, Hash:128 MB.
ТС = 1 min+1 sec


Stockfish 030522 x64_bmi2 - nn-d0b74ce1e5eb.nnue
Sugar Al 2.50 - nn-ac07bd334b62.nnue

https://pixeldrain.com/u/kZ97oyXd


Interestingly, the Sugar AI 2.50 is still a powerful engine! I was expecting a bigger difference.

descriptionDifferent engine matches and tournaments EmptyRe: Different engine matches and tournaments

more_horiz
Someone claimed that sugar is stronger, so I held a match to show the opposite.

descriptionDifferent engine matches and tournaments EmptyRe: Different engine matches and tournaments

more_horiz
Different engine matches and tournaments 04-05-2022-155030

Intel Core i5-7600K, GHz:4.20 , Сore:1, Gui:Fritz-11, Games:100, Book:Off, Endgame tables:Off, Ponder:Off, Hash:128 MB.
ТС = 1 min+1 sec


Classic versions without networks

https://pixeldrain.com/u/Qrbmbrc4

descriptionDifferent engine matches and tournaments EmptyRe: Different engine matches and tournaments

more_horiz
Different engine matches and tournaments Screenshot-20220506-071104Different engine matches and tournaments Screenshot-20220506-071104

Vivo V21e, GHz:2.30 , Сore:1, Gui:Chess for Android, Games:100, Book:Off, Endgame tables:Off, Ponder:Off, Hash:64 MB.
ТС = 1 sec+1 sec


Without networks


https://pixeldrain.com/u/5hJa1qdC

descriptionDifferent engine matches and tournaments EmptyRe: Different engine matches and tournaments

more_horiz
Different engine matches and tournaments 06-05-2022-101440
Different engine matches and tournaments 06-05-2022-102254

Intel Core i5-7600K, GHz:4.20 , Сore:1, Gui:Fritz-11, Games:100, Book:Off, Endgame tables:Off, Ponder:Off, Hash:64 MB.
ТС = 1 sec+1 sec


Without networks


https://pixeldrain.com/u/NLtGC1Sn

descriptionDifferent engine matches and tournaments EmptyRe: Different engine matches and tournaments

more_horiz
Match: lc0.net.801087 vs lc0.net.783333
LC0-version: v0.29.0-rc0, threads=1, backend=cuda-fp16
Hardware: RTX 2060
Software: cutechess-cli
Time control: 2s + 0.1s
Book: book-ply8-unifen-Q-0.0-0.25, sequential, color reversed
Tablebases: Syzygy 5-men
Adjudication: -draw movenumber=50 movecount=5 score=8 -resign movecount=5 score=1000
Comment:

Code:


# PLAYER        : RATING  ERROR  POINTS  PLAYED    W    L    D  D(%)  CFS(%)
1 lc0.net.783333 :    0.0  ----  264.0    500  81  53  366    73    100
2 lc0.net.801087 :  -19.9  15.1  236.0    500  53  81  366    73    ---

White advantage = 38.12 +/- 7.99
Draw rate (equal opponents) = 75.36 % +/- 2.01


https://pixeldrain.com/u/A2szEAKK

descriptionDifferent engine matches and tournaments EmptyRe: Different engine matches and tournaments

more_horiz
Hardware: AMD Ryzen 9 5900X CPU @ 3.70GHz, 12 cores, 64GB, gpu RTX 3080, 10GB
Gauntlet: SF-dev vs. LC0 192x15b nets
Network IDs: T75-32M-2.4M, 753723, 753893, 754042, 761343, 771143, 771482, 771632, 771708, 791971
LC0-version: v0.28.0 (v0.29.0-rc0 for T79)
LC0-options: minibatch-size=112
SF-version: SF-dev 21111518
SF-options: 4 Threads, 512MB Hash
Software: Cutechess-CLI
Time control: LC0: 9,000 nodes/move, SF-dev: 200,000 nodes/move
Book: Chad's 8-ply book, book-ply8-unifen-Q-0.0-0.25.pgn and book-ply8-unifen-Q-0.25-0.40.pgn (10K openings in each book), order=sequential
Tablebases: 6-men
Adjudication: -draw movenumber=50 movecount=5 score=8 -resign movecount=5 score=1000
Ordo: -a 0  -A "Stockfish.dev" -W -D -U "0,1,2,3,4,5,6,7,8,9,10" -s 2000 -n 1
Comment: The aim of the test is to order 192x15b nets. The order is considered sure with 99% CFS (a bit more than 5 Elo). CFS matrix and detailed results in 7z file.

Code:


   # PLAYER                  :  RATING  ERROR    POINTS  PLAYED   (%)  CFS(%)      W       D       L  D(%)
+  1 lc0.net.791971          :    79.2    1.8   36386.0   60000    61     100  19252   34268    6480    57
-  2 lc0.net.753723          :    49.4    1.8   34020.5   60000    57      95  16632   34777    8591    58
   3 lc0.net.754042          :    47.3    1.8
   4 lc0.net.753893          :    47.1    1.8
-  5 lc0.net.761343          :    45.1    1.8   33668.5   60000    56      99  16084   35169    8747    59
-  6 lc0.net.771632          :    42.2    1.8   33435.0   60000    56      76  15957   34956    9087    58
   7 lc0.net.771708          :    41.3    1.8
   8 lc0.net.771482          :    39.7    1.8
   9 lc0.net.771143          :    29.2    1.7
  10 lc0.net.T75-32M-2.4M    :    23.0    1.8
  11 Stockfish.dev           :     0.0   ----
White advantage = 74.97 +/- 0.29
Draw rate (equal opponents) = 62.90 % +/- 0.07


https://pixeldrain.com/u/KTkXEZYM

descriptionDifferent engine matches and tournaments EmptyRe: Different engine matches and tournaments

more_horiz
Comment: Stockfish vs Lc0 - 3 books.
Lc0: defaults ex. mbs=512 cuda-fp16 MoveOverheadsMs=0
Stockfish: Threads=14 Hash=8192 Move Overhead=0 Sy.50MR=false
Hardware: Ryzen 7 2700X 8c16t | RTX 3080 (10GB)
Time control: G/5 mins. + 3 secs.
Books: TCEC S21+S22 Infrafinal, DOTS2 (1st 50 of DOTS4), TCEC S22 SuFi
Tablebases: Syzygy-6
Adjudication: Draw: Move 35, count 2, 0.12; Resign: count 2, score 7.00, two-sided
Software: cutechess-cli timemargin=200, Ordo 1.2.6 -s 3000

Code:


#                        :  RATING  ERROR  POINTS  PLAYED (%)  CFS(%)  W    D    L  D(%)
1 Stockfish_22051520    :  63.2  24.5  175.0    300    58  100  83  184  33  61
2 Lc0-0.29.0-rc0_783277  :    0.0  ----  125.0    300    42  ---  33  184  83  61

 Book  games    Sf score  draws
Infra  1-100    58.0/100  =62
  S21  1-50      28.5/50  =33
  S22  51-100    29.5/50  =29
DOTS2  101-200  58.0/100  =62
SuF22  201-296  59.0/100  =60


https://pixeldrain.com/u/NwaqbenG

descriptionDifferent engine matches and tournaments EmptyRe: Different engine matches and tournaments

more_horiz
Gauntlet: 0.29.0-rc0 and dag-tt vs Komodo Dragon 3
Hardware: Ryzen 9 5900X + GeForce RTX 3080 Ti
Software: BanksiaGUI
LC0 version: 0.29.0-rc0 and dag-tt-2f05eab
LC0 network: 783752
LC0 settings: NNCacheSize=5000000
Komodo settings: Threads=20, Hash=4096, Contempt=0
Time control: 1m + 1s
Book: TCEC21+22
Tablebases: 3-4-5

Code:


  # PLAYER                          :  RATING  ERROR  POINTS  PLAYED  (%)  CFS(%)    W    D    L  D(%)
  1 Lc0 v0.30.0-dev-dag-tt 783752    :    2.0  30.9  100.5    200    50      55  50  101  49    51
  2 Dragon 3                        :    0.0  ----  203.5    400    51      84  95  217  88    54
  3 Lc0 v0.29.0-rc0 783752          :  -16.4  32.5    96.0    200    48    ---  38  116  46    58


https://pixeldrain.com/u/fPRErLPw

descriptionDifferent engine matches and tournaments EmptyRe: Different engine matches and tournaments

more_horiz
Comment: Lc0 nets vs contemporaneous Stockfish versions.
Lc0: --strict-uci-timing cuda-fp16 MoveOverheadsMs=0 , 0.28.2 multi_stream=true
Stockfish: Threads=14 Hash=8192 Move Overhead=0 Sy.50MR=false
Hardware: Ryzen 7 2700X 8c16t | RTX 3080 10GB
Time control: Game 2 mins + 1.5 secs
Book: DOTS4 https://open-chess.org/viewtopic.php?f=7&t=3329, all 200 seq. each match
Tablebases: Syzygy-6
Adjudication: Draw: Move 35, count 2, 0.12; Resign: count 2, score 5.00, two-sided
Software: cutechess-cli wait=1000 restart=on timemargin=200, Ordo 1.2.6 -s 4000

2019-07-26 = Last T40, 20x256, the only Sf pre-NNUE here

Code:


  # PLAYER                :  RATING  ERROR  POINTS  PLAYED  (%)  CFS(%)    W    D    L  D(%)

+  1 Lc0-0.28.2_42850      :    0.0  ----  236.0    400    59    100  160  152  88    38
  2 Stockfish_19072603    :  -63.9  27.2  164.0    400    41    ---  88  152  160    38


2020-12-07 = First 30x384 T60 , Sf @ 16ths others @ 14ths

Code:


  1 Stockfish_20120516    :    8.8  23.0  205.0    400    51      77  95  220  85    55
  2 Lc0-0.28.2_606512    :    0.0  ----  195.0    400    49    ---  85  220  95    55


2022-01-08 = Last T60

Code:


  1 Stockfish_22010808    :    63.0  23.3  235.5    400    59    100  129  213  58    53
-  2 Lc0-0.28.2_611246    :    0.0  ----  164.5    400    41    ---  58  213  129    53


2022-06-19 = recent T78 15x512, Lc0 https://github.com/Etcaqab/lc0/tree/dag-tt - commit 2f05eab3

Code:


  1 Stockfish_22061607    :    67.6  23.1  238.0    400    60    100  130  216  54    54
-  2 Lc0.dev-dag-tt_783822 :    0.0  ----  162.0    400    41    ---  54  216  130    54
 


https://pixeldrain.com/u/JXSL5f2e

descriptionDifferent engine matches and tournaments EmptyRe: Different engine matches and tournaments

more_horiz
Permissions in this forum:
You cannot reply to topics in this forum