Chess2u
Would you like to react to this message? Create an account in a few clicks or log in to continue.

Chess2uLog in

Android engines tests

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
Well folks, i was expecting better result. On my private tests on PC, engine was performing better without "Use classical eval for Bishop vs Pawns" and RDM's "Improve play for closed positions" patches.
From the other hand, on PC engine is definitely much stronger than 1.1b, despite the small regression (-3 elo on android, from mentioned patches), current engine has better understanding on closed positions.
Tablet - Teclast P80X_EEA (tPad, android 9) - 64bit - 2GB RAM - 8x Cortex-A55 - 1,6 GHz
GUI: CuteChess-cli / RR / 1 core per engine /TC: 60sec+0.6 / hash 16 / Book=TopGM_4moves.pgn - move order=sequential
/ repeat opening=yes / Concurrency=7 / Resignation=500cp

Android engines tests - Page 2 D3e836107669d6baa7330238b794914d
Games https://pixeldrain.com/u/X1UKGri4

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
On PC results are much better..
Windows 10 pro 64-bit, AMD Ryzen 7 1800X - 8 cores / 16 threads, 3.60 GHz, 16 GB RAM
GUI: Cutechess 1.2 / Round Robin / TC: 1min+0.6 / 1 core per engine / Hash 64 / Book:topGM_4moves.pgn, moves order=sequential /
For GUI syzygy 5 for draw adjudication / Ponder OFF / concurrency = 15 with CPU usage at 92-93%.

Code:

Score of Spider_1.1b_x64_ELTO_AVX2 vs Stockfish_13_x64_avx2: 342 - 317 - 5341 [0.502]
...      Spider_1.1b_x64_ELTO_AVX2 playing White: 298 - 35 - 2667  [0.544] 3000
...      Spider_1.1b_x64_ELTO_AVX2 playing Black: 44 - 282 - 2674  [0.460] 3000
...      White vs Black: 580 - 79 - 5341  [0.542] 6000
Elo difference: 1.4 +/- 2.9, LOS: 83.5 %, DrawRatio: 89.0 %
6000 of 6000 games finished.

Games https://pixeldrain.com/u/mAsrvVjD
===============================
Windows 10 pro 64-bit, AMD Ryzen 7 1800X - 8 cores / 16 threads, 3.60 GHz, 16 GB RAM
GUI: Cutechess 1.2 / Round Robin / TC: 1min+0.6 / 1 core per engine / Hash 64 / Book:topGM_4moves.pgn /
For GUI syzygy 5 for draw adjudication / Ponder OFF / concurrency = 15 with CPU usage at 92-93%.

Code:

Score of Spider_1.1C_x64_ELTO_AVX2 vs Stockfish_13_x64_avx2: 403 - 299 - 5298 [0.509]
...      Spider_1.1C_x64_ELTO_AVX2 playing White: 369 - 40 - 2591  [0.555] 3000
...      Spider_1.1C_x64_ELTO_AVX2 playing Black: 34 - 259 - 2707  [0.463] 3000
...      White vs Black: 628 - 74 - 5298  [0.546] 6000
Elo difference: 6.0 +/- 3.0, LOS: 100.0 %, DrawRatio: 88.3 %
6000 of 6000 games finished.

Games https://pixeldrain.com/u/DvuooU5N

Test of the newer network nn-ae5925b37cc9.nnue.

Code:

Score of Spider_1.1c_New_net_x64_ELTO_AVX2 vs Stockfish_13_x64_avx2: 393 - 263 - 5344 [0.511]
...      Spider_1.1c_New_net_x64_ELTO_AVX2 playing White: 337 - 33 - 2630  [0.551] 3000
...      Spider_1.1c_New_net_x64_ELTO_AVX2 playing Black: 56 - 230 - 2714  [0.471] 3000
...      White vs Black: 567 - 89 - 5344  [0.540] 6000
Elo difference: 7.5 +/- 2.9, LOS: 100.0 %, DrawRatio: 89.1 %
6000 of 6000 games finished.

Games https://pixeldrain.com/u/yEZmHJeG
===============================
Windows 10 pro 64-bit, AMD Ryzen 7 1800X - 8 cores / 16 threads, 3.60 GHz, 16 GB RAM
GUI: Cutechess 1.2 / Round Robin /
TC: 2min+1.2 / 1 core per engine / Hash 128 / Book:TCEC-14-20+40hardpositions+10lines from SALC 10moves.pgn /
For GUI syzygy 5 for draw adjudication / Ponder OFF / concurrency = 15 with CPU usage at 92-93%.

Code:

Score of Spider_1.1C_x64_ELTO_AVX2 vs Corchess_020521_x64_avx2: 208 - 138 - 854 [0.529]
...      Spider_1.1C_x64_ELTO_AVX2 playing White: 193 - 5 - 402  [0.657] 600
...      Spider_1.1C_x64_ELTO_AVX2 playing Black: 15 - 133 - 452  [0.402] 600
...      White vs Black: 326 - 20 - 854  [0.627] 1200
Elo difference: 20.3 +/- 10.5, LOS: 100.0 %, DrawRatio: 71.2 %
1200 of 2000 games finished.

Games https://pixeldrain.com/u/cZCr4j3z

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
Tablet - Teclast P80X_EEA (tPad, android 9) - 64bit - 2GB RAM - 8x Cortex-A55 - 1,6 GHz
GUI: CuteChess-cli / RR / 1 core per engine /TC: 60sec+0.6 / hash 16 / Book=TopGM_4moves.pgn - move order=sequential
/ repeat opening=yes / Concurrency=7 / Resignation=500cp

Android engines tests - Page 2 915716fac3490428b778f60934086346
Games https://pixeldrain.com/u/BoAuZEKi

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
Next test (with engine made from the same source code, will end in about 8 hours), which is compiled with newer Clang, will bring tremendous results. shok
Tablet - Teclast P80X_EEA (tPad, android 9) - 64bit - 2GB RAM - 8x Cortex-A55 - 1,6 GHz
GUI: CuteChess-cli / RR / 1 core per engine /TC: 60sec+0.6 / hash 16 / Book=TopGM_4moves.pgn - move order=sequential
/ repeat opening=yes / Concurrency=7 / Resignation=500cp

Android engines tests - Page 2 Fed9c60fe6d1a0b13b4027c135187e7d
Games https://pixeldrain.com/u/p3eVvpDT
-----------------------------------------------------
P.s. engine is still under development, about 4 patches need tests.

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
@ChessFan1 wrote:
Next test (with engine made from the same source code, will end in about 8 hours), which is compiled with newer Clang, will bring tremendous results. shok
Tablet - Teclast P80X_EEA (tPad, android 9) - 64bit - 2GB RAM - 8x Cortex-A55 - 1,6 GHz
GUI: CuteChess-cli / RR / 1 core per engine /TC: 60sec+0.6 / hash 16 / Book=TopGM_4moves.pgn - move order=sequential
/ repeat opening=yes / Concurrency=7 / Resignation=500cp

Android engines tests - Page 2 Fed9c60fe6d1a0b13b4027c135187e7d
Games https://pixeldrain.com/u/p3eVvpDT
-----------------------------------------------------
P.s. engine is still under development, about 4 patches need tests.

Wow, new Clang rocks! First test, +136 games over CF 030121, second test +233   games over CF 030121..
Android engines tests - Page 2 A60225c99e8f5038ec84aed0d09a346e
Games https://pixeldrain.com/u/Hwyf5At8

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
All arm7 engines are with NEON (in previous tests too).
Tablet - Lenovo android 4.4.2 - 32 bit - Quad Core (4x Cortex A7) - 1GB RAM - 1,3 GHz
GUI=Cfa - Hash 32 mb - 4 cores per engine - TC: 1min+1sec - Book=TCEC-17-18.pgn (300 pos) - Resign 600 cp

Code:

# PLAYER      :  RATING  ERROR  PLAYED  (%)    W    D    L  D(%)  CFS(%)
1 Fire 8.NN    :    3061    11    600  66.2  300  194  106  32.3    100
2 Fire 8.11    :    2939    11    600  33.8  106  194  300  32.3    ---

Games https://pixeldrain.com/u/upL9kY1j

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
Best of the best beasts. All SFs GCC compilations by SwiftSnips with embedded 40+ mb nets.  thumb up PART 1.
Tablet - Teclast P80X_EEA (tPad, android 9) - 64bit - 2GB RAM - 8x Cortex-A55 - 1,6 GHz
GUI: CuteChess-cli / RR /
2 cores per engine /TC: 120sec+1sec / hash 64 / Book=TopGM_4moves.pgn - move order=sequential
/ repeat opening=yes / Concurrency=3 / Resignation=500cp

Code:

# PLAYER                :  RATING  ERROR  PLAYED   (%)    W     D    L  D(%)  CFS(%)
1 Spider 1.1dT 64 NEON  :    3030     8    1429  56.9  268  1090   71  76.3     100
2 Stockfish 230521      :    3000    11    1000  50.3  101   804   95  80.4      68
3 Stockfish 220521      :    2997     8    1429  48.4  122  1140  167  79.8     100
4 Stockfish 180521      :    2973    11    1000  42.1   44   754  202  75.4     ---

Games till the moment https://pixeldrain.com/u/QERZv7xF

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
@ChessFan1 wrote:
Best of the best beasts. All SFs GCC compilations by SwiftSnips with embedded 40+ mb nets.  thumb up PART 1.
Tablet - Teclast P80X_EEA (tPad, android 9) - 64bit - 2GB RAM - 8x Cortex-A55 - 1,6 GHz
GUI: CuteChess-cli / RR /
2 cores per engine /TC: 120sec+1sec / hash 64 / Book=TopGM_4moves.pgn - move order=sequential
/ repeat opening=yes / Concurrency=3 / Resignation=500cp

Code:

# PLAYER                :  RATING  ERROR  PLAYED   (%)    W     D    L  D(%)  CFS(%)
1 Spider 1.1dT 64 NEON  :    3030     8    1429  56.9  268  1090   71  76.3     100
2 Stockfish 230521      :    3000    11    1000  50.3  101   804   95  80.4      68
3 Stockfish 220521      :    2997     8    1429  48.4  122  1140  167  79.8     100
4 Stockfish 180521      :    2973    11    1000  42.1   44   754  202  75.4     ---


PART 2

Code:

# PLAYER                  :  RATING  ERROR  PLAYED   (%)    W     D    L  D(%)  CFS(%)
1 Spider 1.1dT 64 NEON    :    3032      5    2000  56.3  362  1526  112  76.3     100
2 Stockfish 220521        :    2995      5    2000  47.5  163  1576  261  78.8     69
3 Stockfish 230521        :    2993      5    1864  50.6  188  1512  164  81.1     100
4 Stockfish 180521        :    2980      5    1864  45.3  113  1462  289  78.4     ---

Games till the moment https://pixeldrain.com/u/ZGDXq5Z9

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
@ChessFan1 wrote:
@ChessFan1 wrote:
Best of the best beasts. All SFs GCC compilations by SwiftSnips with embedded 40+ mb nets.  thumb up PART 1.
Tablet - Teclast P80X_EEA (tPad, android 9) - 64bit - 2GB RAM - 8x Cortex-A55 - 1,6 GHz
GUI: CuteChess-cli / RR /
2 cores per engine /TC: 120sec+1sec / hash 64 / Book=TopGM_4moves.pgn - move order=sequential
/ repeat opening=yes / Concurrency=3 / Resignation=500cp

Code:

# PLAYER                :  RATING  ERROR  PLAYED   (%)    W     D    L  D(%)  CFS(%)
1 Spider 1.1dT 64 NEON  :    3030     8    1429  56.9  268  1090   71  76.3     100
2 Stockfish 230521      :    3000    11    1000  50.3  101   804   95  80.4      68
3 Stockfish 220521      :    2997     8    1429  48.4  122  1140  167  79.8     100
4 Stockfish 180521      :    2973    11    1000  42.1   44   754  202  75.4     ---


PART 2

Code:

# PLAYER                  :  RATING  ERROR  PLAYED   (%)    W     D    L  D(%)  CFS(%)
1 Spider 1.1dT 64 NEON    :    3032      5    2000  56.3  362  1526  112  76.3     100
2 Stockfish 220521        :    2995      5    2000  47.5  163  1576  261  78.8     69
3 Stockfish 230521        :    2993      5    1864  50.6  188  1512  164  81.1     100
4 Stockfish 180521        :    2980      5    1864  45.3  113  1462  289  78.4     ---

Games till the moment https://pixeldrain.com/u/ZGDXq5Z9

The end.

Code:

# PLAYER                :  RATING  ERROR  PLAYED   (%)    W     D    L  D(%)  CFS(%)
1 Spider 1.1dT 64 NEON  :    3031      4    3000  55.7  519  2306  175  76.9     100
2 Stockfish 230521      :    2994      4    3000  48.9  264  2403  333  80.1      61
3 Stockfish 220521      :    2993      4    3000  48.7  268  2384  348  79.5     100
4 Stockfish 180521      :    2983      4    3000  46.8  212  2381  407  79.4     ---

Spoiler :

Code:

Individual statistics:

1 Spider 1.1dT 64 NEON : 3031 3000 (+519,=2306,-175), 55.7 %

Stockfish 220521       : 1000 (+160,=772,- 68), 54.6 %
Stockfish 230521       : 1000 (+157,=780,- 63), 54.7 %
Stockfish 180521       : 1000 (+202,=754,- 44), 57.9 %

2 Stockfish 230521     : 2994 3000 (+264,=2403,-333), 48.9 %

Stockfish 220521       : 1000 (+101,=804,- 95), 50.3 %
Stockfish 180521       : 1000 (+100,=819,- 81), 51.0 %
Spider 1.1dT 64 NEON   : 1000 (+ 63,=780,-157), 45.3 %

3 Stockfish 220521     : 2993 3000 (+268,=2384,-348), 48.7 %

Stockfish 230521       : 1000 (+ 95,=804,-101), 49.7 %
Stockfish 180521       : 1000 (+105,=808,- 87), 50.9 %
Spider 1.1dT 64 NEON   : 1000 (+ 68,=772,-160), 45.4 %

4 Stockfish 180521     : 2983 3000 (+212,=2381,-407), 46.8 %

Stockfish 220521       : 1000 (+ 87,=808,-105), 49.1 %
Stockfish 230521       : 1000 (+ 81,=819,-100), 49.0 %
Spider 1.1dT 64 NEON   : 1000 (+ 44,=754,-202), 42.1 %

Code:

Games        : 6000 (finished)

White Wins   : 922 (15.4 %)
Black Wins   : 341 (5.7 %)
Draws        : 4737 (79.0 %)
Unfinished   : 0

White Score  : 54.8 %
Black Score  : 45.2 %


Games https://pixeldrain.com/u/d4xzfdFL

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
Test vs more recent versions, SF and Cor are GCC compilations by SwiftSnips with embedded 40+ mb nets.
Tablet - Teclast P80X_EEA (tPad, android 9) - 64bit - 2GB RAM - 8x Cortex-A55 - 1,6 GHz
GUI: CuteChess-cli / RR /
2 cores per engine /TC: 120sec+1sec / hash 64 / Book=TopGM_4moves.pgn - move order=sequential

Code:


# PLAYER                    :  RATING  ERROR  PLAYED   (%)    W     D    L  D(%)  CFS(%)
1 Spider 1.1dT 64 NEON      :    3023      4    2800  54.8  417  2233  150  79.8     100
2 Stockfish 310521          :    2989      4    2800  47.6  195  2277  328  81.3     51
3 CorChess NNUE 1.3 310521  :    2989      3    2800  47.6  192  2282  326  81.5     ---

statistics :

Code:

Individual statistics:

1 Spider 1.1dT 64 NEON     : 3023 2800 (+417,=2233,-150), 54.8 %

Stockfish 310521           : 1400 (+207,=1114,- 79), 54.6 %
CorChess NNUE 1.3 310521   : 1400 (+210,=1119,- 71), 55.0 %

2 Stockfish 310521         : 2989 2800 (+195,=2277,-328), 47.6 %

Spider 1.1dT 64 NEON       : 1400 (+ 79,=1114,-207), 45.4 %
CorChess NNUE 1.3 310521   : 1400 (+116,=1163,-121), 49.8 %

3 CorChess NNUE 1.3 310521 : 2989 2800 (+192,=2282,-326), 47.6 %

Spider 1.1dT 64 NEON       : 1400 (+ 71,=1119,-210), 45.0 %
Stockfish 310521           : 1400 (+121,=1163,-116), 50.2 %

Code:

Games        : 4200 (finished)

White Wins   : 615 (14.6 %)
Black Wins   : 189 (4.5 %)
Draws        : 3396 (80.9 %)
Unfinished   : 0

White Score  : 55.1 %
Black Score  : 44.9 %


Games https://pixeldrain.com/u/QiBxaZsU

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
Quick test for fun.
Elo matching between older versions vs 13 version. Looks about the same, but Cfish performing a bit worse, i mean it is compiled with last Clang and it's speed is better (picture). Both SF versions has about the same speed at ~160 kNs. SF 311220 and CF 030121 made from "WeakUnopposed penalty for backwards on file A or H" patch.
Android engines tests - Page 2 2530aaacac8b11879b24027dad3cbfd9
Tablet - Teclast P80X_EEA (tPad, android 9) - 64bit - 2GB RAM - 8x Cortex-A55 - 1,6 GHz
GUI: CuteChess-cli / RR / 1 core per engine /TC: 60sec+0.6 / hash 16 / Book=TopGM_4moves.pgn - move order=sequential
/ repeat opening=yes / Concurrency=7 / Resignation=500cp

Code:

# PLAYER             :  RATING  ERROR  PLAYED   (%)    W    D    L  D(%)  CFS(%)
1 SF 13              :    3002      7     508  50.5   64  385   59  75.8      69
2 SF 311220          :    2998      7     508  49.5   59  385   64  75.8     ---

Code:

# PLAYER             :  RATING  ERROR  PLAYED   (%)    W    D    L  D(%)  CFS(%)
1 Cfish 13 64 NEON   :    3001      7     508  50.4   62  388   58  76.4      66
2 CF 030121 64       :    2999      7     508  49.6   58  388   62  76.4     ---

Games https://pixeldrain.com/u/1QfVXM1R

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
Permissions in this forum:
You cannot reply to topics in this forum