Chess2u
Would you like to react to this message? Create an account in a few clicks or log in to continue.

Chess2uLog in

Android engines tests

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
Well folks, i was expecting better result. On my private tests on PC, engine was performing better without "Use classical eval for Bishop vs Pawns" and RDM's "Improve play for closed positions" patches.
From the other hand, on PC engine is definitely much stronger than 1.1b, despite the small regression (-3 elo on android, from mentioned patches), current engine has better understanding on closed positions.
Tablet - Teclast P80X_EEA (tPad, android 9) - 64bit - 2GB RAM - 8x Cortex-A55 - 1,6 GHz
GUI: CuteChess-cli / RR / 1 core per engine /TC: 60sec+0.6 / hash 16 / Book=TopGM_4moves.pgn - move order=sequential
/ repeat opening=yes / Concurrency=7 / Resignation=500cp

Android engines tests - Page 2 D3e836107669d6baa7330238b794914d
Games https://pixeldrain.com/u/X1UKGri4

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
On PC results are much better..
Windows 10 pro 64-bit, AMD Ryzen 7 1800X - 8 cores / 16 threads, 3.60 GHz, 16 GB RAM
GUI: Cutechess 1.2 / Round Robin / TC: 1min+0.6 / 1 core per engine / Hash 64 / Book:topGM_4moves.pgn, moves order=sequential /
For GUI syzygy 5 for draw adjudication / Ponder OFF / concurrency = 15 with CPU usage at 92-93%.

Code:

Score of Spider_1.1b_x64_ELTO_AVX2 vs Stockfish_13_x64_avx2: 342 - 317 - 5341 [0.502]
...      Spider_1.1b_x64_ELTO_AVX2 playing White: 298 - 35 - 2667  [0.544] 3000
...      Spider_1.1b_x64_ELTO_AVX2 playing Black: 44 - 282 - 2674  [0.460] 3000
...      White vs Black: 580 - 79 - 5341  [0.542] 6000
Elo difference: 1.4 +/- 2.9, LOS: 83.5 %, DrawRatio: 89.0 %
6000 of 6000 games finished.

Games https://pixeldrain.com/u/mAsrvVjD
===============================
Windows 10 pro 64-bit, AMD Ryzen 7 1800X - 8 cores / 16 threads, 3.60 GHz, 16 GB RAM
GUI: Cutechess 1.2 / Round Robin / TC: 1min+0.6 / 1 core per engine / Hash 64 / Book:topGM_4moves.pgn /
For GUI syzygy 5 for draw adjudication / Ponder OFF / concurrency = 15 with CPU usage at 92-93%.

Code:

Score of Spider_1.1C_x64_ELTO_AVX2 vs Stockfish_13_x64_avx2: 403 - 299 - 5298 [0.509]
...      Spider_1.1C_x64_ELTO_AVX2 playing White: 369 - 40 - 2591  [0.555] 3000
...      Spider_1.1C_x64_ELTO_AVX2 playing Black: 34 - 259 - 2707  [0.463] 3000
...      White vs Black: 628 - 74 - 5298  [0.546] 6000
Elo difference: 6.0 +/- 3.0, LOS: 100.0 %, DrawRatio: 88.3 %
6000 of 6000 games finished.

Games https://pixeldrain.com/u/DvuooU5N

Test of the newer network nn-ae5925b37cc9.nnue.

Code:

Score of Spider_1.1c_New_net_x64_ELTO_AVX2 vs Stockfish_13_x64_avx2: 393 - 263 - 5344 [0.511]
...      Spider_1.1c_New_net_x64_ELTO_AVX2 playing White: 337 - 33 - 2630  [0.551] 3000
...      Spider_1.1c_New_net_x64_ELTO_AVX2 playing Black: 56 - 230 - 2714  [0.471] 3000
...      White vs Black: 567 - 89 - 5344  [0.540] 6000
Elo difference: 7.5 +/- 2.9, LOS: 100.0 %, DrawRatio: 89.1 %
6000 of 6000 games finished.

Games https://pixeldrain.com/u/yEZmHJeG
===============================
Windows 10 pro 64-bit, AMD Ryzen 7 1800X - 8 cores / 16 threads, 3.60 GHz, 16 GB RAM
GUI: Cutechess 1.2 / Round Robin /
TC: 2min+1.2 / 1 core per engine / Hash 128 / Book:TCEC-14-20+40hardpositions+10lines from SALC 10moves.pgn /
For GUI syzygy 5 for draw adjudication / Ponder OFF / concurrency = 15 with CPU usage at 92-93%.

Code:

Score of Spider_1.1C_x64_ELTO_AVX2 vs Corchess_020521_x64_avx2: 208 - 138 - 854 [0.529]
...      Spider_1.1C_x64_ELTO_AVX2 playing White: 193 - 5 - 402  [0.657] 600
...      Spider_1.1C_x64_ELTO_AVX2 playing Black: 15 - 133 - 452  [0.402] 600
...      White vs Black: 326 - 20 - 854  [0.627] 1200
Elo difference: 20.3 +/- 10.5, LOS: 100.0 %, DrawRatio: 71.2 %
1200 of 2000 games finished.

Games https://pixeldrain.com/u/cZCr4j3z

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
Tablet - Teclast P80X_EEA (tPad, android 9) - 64bit - 2GB RAM - 8x Cortex-A55 - 1,6 GHz
GUI: CuteChess-cli / RR / 1 core per engine /TC: 60sec+0.6 / hash 16 / Book=TopGM_4moves.pgn - move order=sequential
/ repeat opening=yes / Concurrency=7 / Resignation=500cp

Android engines tests - Page 2 915716fac3490428b778f60934086346
Games https://pixeldrain.com/u/BoAuZEKi

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
Next test (with engine made from the same source code, will end in about 8 hours), which is compiled with newer Clang, will bring tremendous results. shok
Tablet - Teclast P80X_EEA (tPad, android 9) - 64bit - 2GB RAM - 8x Cortex-A55 - 1,6 GHz
GUI: CuteChess-cli / RR / 1 core per engine /TC: 60sec+0.6 / hash 16 / Book=TopGM_4moves.pgn - move order=sequential
/ repeat opening=yes / Concurrency=7 / Resignation=500cp

Android engines tests - Page 2 Fed9c60fe6d1a0b13b4027c135187e7d
Games https://pixeldrain.com/u/p3eVvpDT
-----------------------------------------------------
P.s. engine is still under development, about 4 patches need tests.

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
ChessFan1 wrote:
Next test (with engine made from the same source code, will end in about 8 hours), which is compiled with newer Clang, will bring tremendous results. shok
Tablet - Teclast P80X_EEA (tPad, android 9) - 64bit - 2GB RAM - 8x Cortex-A55 - 1,6 GHz
GUI: CuteChess-cli / RR / 1 core per engine /TC: 60sec+0.6 / hash 16 / Book=TopGM_4moves.pgn - move order=sequential
/ repeat opening=yes / Concurrency=7 / Resignation=500cp

Android engines tests - Page 2 Fed9c60fe6d1a0b13b4027c135187e7d
Games https://pixeldrain.com/u/p3eVvpDT
-----------------------------------------------------
P.s. engine is still under development, about 4 patches need tests.

Wow, new Clang rocks! First test, +136 games over CF 030121, second test +233   games over CF 030121..
Android engines tests - Page 2 A60225c99e8f5038ec84aed0d09a346e
Games https://pixeldrain.com/u/Hwyf5At8

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
All arm7 engines are with NEON (in previous tests too).
Tablet - Lenovo android 4.4.2 - 32 bit - Quad Core (4x Cortex A7) - 1GB RAM - 1,3 GHz
GUI=Cfa - Hash 32 mb - 4 cores per engine - TC: 1min+1sec - Book=TCEC-17-18.pgn (300 pos) - Resign 600 cp

Code:

# PLAYER      :  RATING  ERROR  PLAYED  (%)    W    D    L  D(%)  CFS(%)
1 Fire 8.NN    :    3061    11    600  66.2  300  194  106  32.3    100
2 Fire 8.11    :    2939    11    600  33.8  106  194  300  32.3    ---

Games https://pixeldrain.com/u/upL9kY1j

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
Best of the best beasts. All SFs GCC compilations by SwiftSnips with embedded 40+ mb nets.  thumb up PART 1.
Tablet - Teclast P80X_EEA (tPad, android 9) - 64bit - 2GB RAM - 8x Cortex-A55 - 1,6 GHz
GUI: CuteChess-cli / RR /
2 cores per engine /TC: 120sec+1sec / hash 64 / Book=TopGM_4moves.pgn - move order=sequential
/ repeat opening=yes / Concurrency=3 / Resignation=500cp

Code:

# PLAYER                :  RATING  ERROR  PLAYED   (%)    W     D    L  D(%)  CFS(%)
1 Spider 1.1dT 64 NEON  :    3030     8    1429  56.9  268  1090   71  76.3     100
2 Stockfish 230521      :    3000    11    1000  50.3  101   804   95  80.4      68
3 Stockfish 220521      :    2997     8    1429  48.4  122  1140  167  79.8     100
4 Stockfish 180521      :    2973    11    1000  42.1   44   754  202  75.4     ---

Games till the moment https://pixeldrain.com/u/QERZv7xF

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
ChessFan1 wrote:
Best of the best beasts. All SFs GCC compilations by SwiftSnips with embedded 40+ mb nets.  thumb up PART 1.
Tablet - Teclast P80X_EEA (tPad, android 9) - 64bit - 2GB RAM - 8x Cortex-A55 - 1,6 GHz
GUI: CuteChess-cli / RR /
2 cores per engine /TC: 120sec+1sec / hash 64 / Book=TopGM_4moves.pgn - move order=sequential
/ repeat opening=yes / Concurrency=3 / Resignation=500cp

Code:

# PLAYER                :  RATING  ERROR  PLAYED   (%)    W     D    L  D(%)  CFS(%)
1 Spider 1.1dT 64 NEON  :    3030     8    1429  56.9  268  1090   71  76.3     100
2 Stockfish 230521      :    3000    11    1000  50.3  101   804   95  80.4      68
3 Stockfish 220521      :    2997     8    1429  48.4  122  1140  167  79.8     100
4 Stockfish 180521      :    2973    11    1000  42.1   44   754  202  75.4     ---


PART 2

Code:

# PLAYER                  :  RATING  ERROR  PLAYED   (%)    W     D    L  D(%)  CFS(%)
1 Spider 1.1dT 64 NEON    :    3032      5    2000  56.3  362  1526  112  76.3     100
2 Stockfish 220521        :    2995      5    2000  47.5  163  1576  261  78.8     69
3 Stockfish 230521        :    2993      5    1864  50.6  188  1512  164  81.1     100
4 Stockfish 180521        :    2980      5    1864  45.3  113  1462  289  78.4     ---

Games till the moment https://pixeldrain.com/u/ZGDXq5Z9

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
ChessFan1 wrote:
ChessFan1 wrote:
Best of the best beasts. All SFs GCC compilations by SwiftSnips with embedded 40+ mb nets.  thumb up PART 1.
Tablet - Teclast P80X_EEA (tPad, android 9) - 64bit - 2GB RAM - 8x Cortex-A55 - 1,6 GHz
GUI: CuteChess-cli / RR /
2 cores per engine /TC: 120sec+1sec / hash 64 / Book=TopGM_4moves.pgn - move order=sequential
/ repeat opening=yes / Concurrency=3 / Resignation=500cp

Code:

# PLAYER                :  RATING  ERROR  PLAYED   (%)    W     D    L  D(%)  CFS(%)
1 Spider 1.1dT 64 NEON  :    3030     8    1429  56.9  268  1090   71  76.3     100
2 Stockfish 230521      :    3000    11    1000  50.3  101   804   95  80.4      68
3 Stockfish 220521      :    2997     8    1429  48.4  122  1140  167  79.8     100
4 Stockfish 180521      :    2973    11    1000  42.1   44   754  202  75.4     ---


PART 2

Code:

# PLAYER                  :  RATING  ERROR  PLAYED   (%)    W     D    L  D(%)  CFS(%)
1 Spider 1.1dT 64 NEON    :    3032      5    2000  56.3  362  1526  112  76.3     100
2 Stockfish 220521        :    2995      5    2000  47.5  163  1576  261  78.8     69
3 Stockfish 230521        :    2993      5    1864  50.6  188  1512  164  81.1     100
4 Stockfish 180521        :    2980      5    1864  45.3  113  1462  289  78.4     ---

Games till the moment https://pixeldrain.com/u/ZGDXq5Z9

The end.

Code:

# PLAYER                :  RATING  ERROR  PLAYED   (%)    W     D    L  D(%)  CFS(%)
1 Spider 1.1dT 64 NEON  :    3031      4    3000  55.7  519  2306  175  76.9     100
2 Stockfish 230521      :    2994      4    3000  48.9  264  2403  333  80.1      61
3 Stockfish 220521      :    2993      4    3000  48.7  268  2384  348  79.5     100
4 Stockfish 180521      :    2983      4    3000  46.8  212  2381  407  79.4     ---

Spoiler :

Games https://pixeldrain.com/u/d4xzfdFL

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
Test vs more recent versions, SF and Cor are GCC compilations by SwiftSnips with embedded 40+ mb nets.
Tablet - Teclast P80X_EEA (tPad, android 9) - 64bit - 2GB RAM - 8x Cortex-A55 - 1,6 GHz
GUI: CuteChess-cli / RR /
2 cores per engine /TC: 120sec+1sec / hash 64 / Book=TopGM_4moves.pgn - move order=sequential

Code:


# PLAYER                    :  RATING  ERROR  PLAYED   (%)    W     D    L  D(%)  CFS(%)
1 Spider 1.1dT 64 NEON      :    3023      4    2800  54.8  417  2233  150  79.8     100
2 Stockfish 310521          :    2989      4    2800  47.6  195  2277  328  81.3     51
3 CorChess NNUE 1.3 310521  :    2989      3    2800  47.6  192  2282  326  81.5     ---

statistics :

Games https://pixeldrain.com/u/QiBxaZsU

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
Quick test for fun.
Elo matching between older versions vs 13 version. Looks about the same, but Cfish performing a bit worse, i mean it is compiled with last Clang and it's speed is better (picture). Both SF versions has about the same speed at ~160 kNs. SF 311220 and CF 030121 made from "WeakUnopposed penalty for backwards on file A or H" patch.
Android engines tests - Page 2 2530aaacac8b11879b24027dad3cbfd9
Tablet - Teclast P80X_EEA (tPad, android 9) - 64bit - 2GB RAM - 8x Cortex-A55 - 1,6 GHz
GUI: CuteChess-cli / RR / 1 core per engine /TC: 60sec+0.6 / hash 16 / Book=TopGM_4moves.pgn - move order=sequential
/ repeat opening=yes / Concurrency=7 / Resignation=500cp

Code:

# PLAYER             :  RATING  ERROR  PLAYED   (%)    W    D    L  D(%)  CFS(%)
1 SF 13              :    3002      7     508  50.5   64  385   59  75.8      69
2 SF 311220          :    2998      7     508  49.5   59  385   64  75.8     ---

Code:

# PLAYER             :  RATING  ERROR  PLAYED   (%)    W    D    L  D(%)  CFS(%)
1 Cfish 13 64 NEON   :    3001      7     508  50.4   62  388   58  76.4      66
2 CF 030121 64       :    2999      7     508  49.6   58  388   62  76.4     ---

Games https://pixeldrain.com/u/1QfVXM1R

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
Tablet - Teclast P80X_EEA (tPad, android 9) - 64bit - 2GB RAM - 8x Cortex-A55 - 1,6 GHz
GUI: CuteChess-cli / RR / 1 core per engine /TC: 60sec+0.6 / hash 16 / Book=TopGM_4moves.pgn - move order=sequential
/ repeat opening=yes / Concurrency=7 / Resignation=500cp

Code:

# PLAYER                   :  RATING  ERROR  PLAYED   (%)    W     D     L  D(%)  CFS(%)
1 Spider 1.1dT 64 NEON     :    3008      3    6000  51.5  873  4438   689  74.0      65
2 CfishU 240621 64 NEON    :    3007      3    6000  51.3  890  4381   729  73.0      84
3 Spider 080621 64 NEON    :    3004      3    6000  50.8  816  4467   717  74.5     100
4 Stockfish 220821         :    2980      3    6000  46.3  745  4066  1189  67.8     ---

Spoiler :

Games https://pixeldrain.com/u/6Yi3Tvxx

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
Still on top, but SF getting closer and closer, only 10 elo difference with SF 030721.  All engines with best suitability for Cortex A55.
Tablet - Teclast P80X_EEA (tPad, android 9) - 64bit - 2GB RAM - 8x Cortex-A55 - 1,6 GHz
GUI: CuteChess-cli / RR / 2 cores per engine /TC: 60sec+0.6 / hash 32 / Book=TopGM_4moves.pgn - move order=sequential
/ repeat opening=yes / Concurrency=3 / Resignation=500cp

Code:

# PLAYER                :  RATING  ERROR  PLAYED   (%)    W     D    L  D(%)  CFS(%)
1 Spider 1.1dT 64 NEON  :    3007      4    4000  51.4  553  3008  439  75.2     100
2 SF 030721             :    2997      4    4000  49.4  442  3068  490  76.7      63
3 Stockfish 14          :    2996      4    4000  49.2  415  3104  481  77.6     ---

Spoiler :

Games https://pixeldrain.com/u/io9GUJpP

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
Even after update, Chiron is miles away.  
Tablet - Lenovo android 4.4.2 - 32 bit - Quad Core A7 - 1GB RAM - 1,3 GHz
GUI=C4a - Hash 64 mb - 2min+1sec - Book=8move_v3 - 4 cores per engine - ponder=NO
syzygy=NO, Adjudication Rule=Resign - Move 25, Move Count 3, Score (in cp) 600
Draw=Move Number 30, Move count 4, Score (in cp) 10 - Calculator - Ordo 1.2.6

Code:

# PLAYER                 :  RATING  ERROR  PLAYED   (%)    W    D    L  D(%)  CFS(%)
1 Komodo 13.1 32-bit     :    3120     21     200  79.0  124   68    8  34.0     100
2 Chiron 5               :    2880     21     200  21.0    8   68  124  34.0     ---

games https://pixeldrain.com/u/udxokERA

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
Tablet - Lenovo android 4.4.2 - 32 bit - Quad Core (4x Cortex A7) - 1GB RAM - 1,3 GHz
GUI=Cfa - Hash 64 mb - 4 cores per engine - TC: 2min+1sec - Book=8moves.pgn (60 pos) - Resign 600 cp

ELO rating has been set to 3498 (Houdini 6.03 with 4 cores and TC: more then 2 min). Engines armv7.

Code:

# PLAYER           :  RATING  ERROR  PLAYED   (%)    W    D    L  D(%)  CFS(%)
1 Berserk 8.5.1    :    3601     22     360  71.3  183  147   30  40.8     100
2 Houdini 6.03     :    3498     22     360  53.8  115  157   88  43.6      91
3 Fire 8.2         :    3477     20     360  50.0  102  156  102  43.3     100
4 Tucano 10.00     :    3328     24     360  25.0   23  134  203  37.2     ---

Games https://pixeldrain.com/u/Kpa9xEJK
======================================
P.s. i also wanted to test last Koivisto and Ethereal, but unfortunately both engines resigns after one or two moves.

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
Tablet - Lenovo android 4.4.2 - 32 bit - Quad Core (4x Cortex A7) - 1GB RAM - 1,3 GHz
GUI=Cfa - Hash 64 mb - 4 cores per engine - TC: 2min+1sec - Book=8moves.pgn (60 pos) - Resign 600 cp

ELO rating has been set to 3498 (Houdini 6.03 with 4 cores and TC: more then 2 min). Engines armv7.

Code:

# PLAYER           :  RATING  ERROR  PLAYED   (%)    W    D    L  D(%)  CFS(%)
1 Berserk 8.5.1    :    3578     31     360  82.8  261   74   25  20.6     100
2 Houdini 6.03     :    3498     31     360  73.2  221   85   54  23.6     100
3 Minic 3.18       :    3158     30     360  29.6   64   85  211  23.6     100
4 Drofa 3.3.0      :    3029     34     360  14.4   23   58  279  16.1     ---

Games https://pixeldrain.com/u/Pxy2Sag5

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
Tablet - Lenovo android 4.4.2 - 32 bit - Quad Core (4x Cortex A7) - 1GB RAM - 1,3 GHz
GUI=Cfa - Hash 64 mb - 4 cores per engine - TC: 2min+1sec - Book=8moves.pgn (100 pos) - Resign 600 cp

ELO rating has been set to 3498 (Houdini 6.03 with 4 cores and TC: more then 2 min). Engines armv7.

Code:

# PLAYER            :  RATING  ERROR  PLAYED   (%)    W    D    L  D(%)  CFS(%)
1 Fire 8.NN.MC.3    :    3616     18     400  58.4  125  217   58  54.3      64
2 Berserk 8.5.1     :    3610     16     400  57.3  127  204   69  51.0     100
3 Houdini 6.03      :    3498     17     400  34.4   59  157  184  39.3     ---

Games https://pixeldrain.com/u/rDoSR6B8

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
Oh, ChessFan1, your tournaments are quite scientific. Mine are not. I just play two engines against each other (10 or 20 games) in Chess for Android.

I'll leave out all the Stockfish devs and nnue engines. I prefer independent engines. Of those, in rough descending order of strength:

1. Komodo 13 (but beaten by Houdini 6, a stockfish dev)
2. Andscacs 0.921 - amazingly strong.
3. Laser 1.08 - beats a lot of other independent engines.
4. Chiron 5 - a strong engine.
5. Texel 21115 - a favourite engine.
6. Arasan 23.2 - a good engine.
7. Cheng 4.39 - beat a lot of other engines.
8. Rodent IV - the classic.

For playing against a human, Chiron 5, Texel, Arasan, Cheng and Rodent all have significant elo weakening and can give a good game. Danasah 8.3ls goes down to 700 elo and is quite weak.

Komodo 13 has 25 skill levels which regulate search depth and nodes per second. So it's adaptable.

And there's also Maia, which I am trying out these days.

descriptionAndroid engines tests - Page 2 EmptyRe: Android engines tests

more_horiz
Permissions in this forum:
You cannot reply to topics in this forum