Chess2u
Would you like to react to this message? Create an account in a few clicks or log in to continue.

Chess2uLog in

descriptionHoudini Clone Tuning EmptyHoudini Clone Tuning

more_horiz
Hi!,

I made a Houdini clone tuning similar to Barnard's using HoudiniConfig.exe tool. My motivation is to find values that could be better than the original and to let Houdini give the approximate values of Rook and Queen. Values tuned are primarily the piece values with the following fixed: Pawn=100, Knight=320, Bishop=325, Bishop pair bonus=50. Variable values are Rook=500-535 and Queen=945-995.

The descriptive clone name on the table below indicates the Rook and Queen values used. The result of the test is random, but the top 4 beat the original Houdini with ELO margin of 3 to 18 points and the value of the Rook is 510 while the Queen is 965. I post this for help on further testing with other computer configurations because I felt that 21,000 games may not be enough when I found out that each computer I use show different results which could not be easily reproduced.

Code:


Rank    Name                 Elo    Win  Loss  Draw  Points Games  Score
01   Houdini 2.0 R510 Q965   2828   196   161   243   317.50   600   53%
02   Houdini 2.0 R505 Q965   2827   184   154   262   315.00   600   53%
03   Houdini 2.0 R515 Q975   2821   194   174   232   310.00   600   52%
04   Houdini 2.0 R510 Q970   2813   181   175   244   303.00   600   50%
05   Houdini 2.0 w32 Orig.   2810   6611   5907   8482   10852   21000   52%
06   Houdini 2.0 R530 Q975   2809   177   180   243   298.50   600   50%
07   Houdini 2.0 R500 Q945   2808   169   173   258   298.00   600   50%
08   Houdini 2.0 R505 Q970   2808   180   183   237   298.50   600   50%
09   Houdini 2.0 R525 Q975   2806   171   179   250   296.00   600   49%
10   Houdini 2.0 R515 Q960   2806   169   176   255   296.50   600   49%
11   Houdini 2.0 R510 Q960   2806   165   174   261   295.50   600   49%
12   Houdini 2.0 R520 Q980   2803   171   184   245   293.50   600   49%
13   Houdini 2.0 R505 Q950   2803   165   179   256   293.00   600   49%
14   Houdini 2.0 R530 Q980   2802   167   182   251   292.50   600   49%
15   Houdini 2.0 R510 Q955   2801   158   177   265   290.50   600   48%
16   Houdini 2.0 R525 Q980   2801   171   188   241   291.50   600   49%
17   Houdini 2.0 R520 Q985   2801   175   193   232   291.00   600   49%
18   Houdini 2.0 R505 Q955   2800   169   188   243   290.50   600   48%
19   Houdini 2.0 R515 Q965   2799   169   193   238   288.00   600   48%
20   Houdini 2.0 R505 Q960   2799   186   208   206   289.00   600   48%
21   Houdini 2.0 R520 Q965   2798   167   192   241   287.50   600   48%
22   Houdini 2.0 R500 Q950   2797   170   196   234   287.00   600   48%
23   Houdini 2.0 R520 Q975   2795   168   197   235   285.50   600   48%
24   Houdini 2.0 R500 Q960   2794   161   192   247   284.50   600   47%
25   Houdini 2.0 R515 Q980   2794   161   193   246   284.00   600   47%
26   Houdini 2.0 R530 Q990   2793   163   194   243   284.50   600   47%
27   Houdini 2.0 R525 Q985   2792   161   194   245   283.50   600   47%
28   Houdini 2.0 R515 Q970   2791   168   204   228   282.00   600   47%
29   Houdini 2.0 R525 Q970   2791   162   199   239   281.50   600   47%
30   Houdini 2.0 R535 Q990   2791   172   208   220   282.00   600   47%
31   Houdini 2.0 R500 Q955   2789   157   198   245   279.50   600   47%
32   Houdini 2.0 R520 Q970   2788   159   200   241   279.50   600   47%
33   Houdini 2.0 R535 Q995   2787   144   189   267   277.50   600   46%
34   Houdini 2.0 R535 Q980   2785   159   207   234   276.00   600   46%
35   Houdini 2.0 R530 Q985   2783   160   211   229   274.50   600   46%
36   Houdini 2.0 R535 Q985   2780   158   216   226   271.00   600   45%


For more information on the tournament conditions and download of complete games, you may follow this link: http://chessblitz.site40.net/wordpress/?p=887

Thanks.
drj

descriptionHoudini Clone Tuning EmptyRe: Houdini Clone Tuning

more_horiz
Thanks a lot! smile

descriptionHoudini Clone Tuning EmptyHoudin Clone Tuning + Pawn Storm

more_horiz
I just finished the gauntlet matches between the original Houdini and the top 3 finishers of the previous tuning. I added the Pawn Storm values 40-50-60 (outer-inner-middle file) to the tuning which are used by Tennison and Barnard. Just in case, I also mixed 2 additional boundary values 0f 35-45-55 and 45-55-65. The result shows that the computer seems not to like the 40-50-60 but instead have chosen the upper 45-55-65 values. The winning top combination will be used in the next 50 rounds of Houdini Clone Derby featuring the original Houdini with Tennison's T3, Barnard's S and Z and my chosen horses.

Here are the results:

Code:

Rank          Name                           Elo    +    -   games   score     oppo.    draws
  1    Houdini 2.0 R510 Q965 ST45-55-65      2822   23   23   600     53%     2803     33%
  2    Houdini 2.0 R505 Q965 ST45-55-65      2810   23   23   600     51%     2803     29%
  3    Houdini 2.0 R510 Q965 ST40-50-60      2809   23   23   600     51%     2803     32%
  4    Houdini 2.0 w32  - Original           2803    8    8  5400     51%     2800     33%
  5    Houdini 2.0 R505 Q965 ST35-45-55      2803   23   23   600     50%     2803     36%
  6    Houdini 2.0 R510 Q965 ST35-45-55      2799   23   23   600     49%     2803     35%
  7    Houdini 2.0 R515 Q975 ST45-55-65      2798   23   23   600     49%     2803     32%
  8    Houdini 2.0 R515 Q975 ST35-45-55      2797   24   24   600     49%     2803     30%
  9    Houdini 2.0 R515 Q975 ST40-50-60      2780   23   23   600     46%     2803     37%
  10    Houdini 2.0 R505 Q965 ST40-50-60      2778   23   23   600     46%     2803     33%


Standby for the Great Clone Race.
drj

descriptionHoudini Clone Tuning EmptyHoudini Clone Derby

more_horiz
Hi,

Here are the results of my Houdini Clone Derby:

Code:

Rank    Name                                 Elo    +    -       games    score   oppo.    draws
  1    Houdini 2.0 Z - Barnard              2809   20   20     750     52%     2798     40%
  2    Houdini 2.0 S - Barnard              2806   20   20     750     51%     2799     38%
  3    Houdini 2.0 w32                     2801   21   21     750     50%     2800     34%
  4    Houdini 2.0 R510 Q965 ST45-55-65      2797   21   21     750     49%     2801     34%
  5    Houdini 2.0 T3 - Tennison            2796   20   20     750     49%     2801     37%
  6    Houdini 2.0 R505 Q965                2791   20   20     750     48%     2802     38%


The surprise winner is Barnard's Z. Tennison's T3 is at the top of SCCT Rating list but was near the end of the bottom in this race. The clone winner's ELO rating advantage against the original is 8 points. Maybe this clone tuning exercise is not bad after all when the original is beaten.

Until the next cloning season, have a nice day everyone!
drj

Last edited by drj4759 on Thu Sep 13, 2012 5:31 am; edited 1 time in total

descriptionHoudini Clone Tuning EmptyRe: Houdini Clone Tuning

more_horiz
Could You,please,tell me more about this test;which CPU, No of cores, time control, opening book...

Thanks!

descriptionHoudini Clone Tuning EmptyHouding Cloning Conditions

more_horiz
Hi! Marijan,

For the Clone Derby the following are used:
Computer: 1 AMD Athlon Quad Core with 3 virtualized Windows XP
Time control: 1 minute + 1 second
Book: Perfect 2012c, up to 4 half moves
Number of rounds: 50 round robin x 3 virtualized computers

For more details, you may follow this link: http://chessblitz.site40.net/wordpress/?p=900

I did not include your Houdini Barracuda Clone in my test because my preliminaries showed it is at the bottom and also it uses contempt off which is not in the HoudiniConfig.exe.

Thanks.
drj

Last edited by drj4759 on Thu Sep 13, 2012 5:30 am; edited 1 time in total

descriptionHoudini Clone Tuning EmptyRe: Houdini Clone Tuning

more_horiz
Thanks for quick reply... Keep up the good work!

descriptionHoudini Clone Tuning EmptyHoudini Clone Tuning - Chess Piece Values - Part 2

more_horiz
Hi,

The second chess piece values search has just completed. It was based on the previous winner's values to shorten the range of possible combinations.

Tournament conditions:
Rounds = 100 x 4 virtualized computers
Time control = 10 seconds base + 1 second increment
Book = Perfect 2012c, up to 4 half-moves
Ponder = off

Descriptive names represents the values of Knight, Bishop, Rook and Queen. The Bishop pair is fixed at 50 centipawns.

Here is the result:


Code:

Rank    Chess Engine Name           Elo   Win   Loss   Draw   Points   Games   Score%
  1    H2 N320 B325 R510 Q965      2825   126   95    179   215.5     400      54%
  2    H2 N325 B330 R515 Q975      2808   130   122   148   204       400      51%
  3    H2 N330 B335 R525 Q975      2804   129   124   147   202.5     400      51%
  4    H2 N330 B335 R525 Q980      2802   126   123   151   201.5     400      50%
  5    Houdini 2.0 w32 – Orig.     2801   1082  1077  1441  1802.5   3600       50%
  6    H2 N325 B330 R520 Q970      2799   115   117   168   199       400      50%
  7    H2 N330 B335 R520 Q980      2793   111   120   169   195.5     400      49%
  8    H2 N325 B330 R515 Q970      2791   109   121   170   194       400      49%
  9    H2 N325 B330 R520 Q975      2790   118   131   151   193.5     400      48%
  10    H2 N330 B335 R520 Q975      2788   113   129   158   192       400      48%


The top finisher with a margin of 24 ELO points was also the first placer in the previous search competition. In the next phase, a tournament will be held with Houdini clones using the piece values of Stockfish, Komodo, Fire, Ivanhoe and also GM Kaufman's values.

Thanks and have a good day.

drj

Last edited by drj4759 on Mon Sep 24, 2012 5:33 am; edited 1 time in total

descriptionHoudini Clone Tuning EmptyRe: Houdini Clone Tuning

more_horiz
The error margin of a rating based on 400-games is approximately +/- 25 Elo.
This is larger than the differences you measure, which means that your test doesn't really demonstrate anything. You could run 10 strictly identical engines and obtain the results table above.

descriptionHoudini Clone Tuning EmptyHoudini Clone Tuning

more_horiz
@Effelsberg wrote:
The error margin of a rating based on 400-games is approximately +/- 25 Elo.
This is larger than the differences you measure, which means that your test doesn't really demonstrate anything. You could run 10 strictly identical engines and obtain the results table above.


Hi! Effelsberg,

Thank you for giving pointers. The gauntlet tournament was so short because I could see no reason to go on further upon seeing the initial results which showed no significant advantage of the clones over the original. I am just curious whether the established piece values and the ones used by the other strong chess engines are really optimized. The preceding tests were made so that I could pick at least one that will represent in the coming Houdini Clone Wars centered on the piece values.

I know that Houdini is very well optimized, otherwise it will not be at the top of world. There is no available information regarding the relative strength of each engine authors' chosen chess piece values, so perhaps this activity could provide one using the strongest chess engine Houdini to reveal it. It will be longer and will take maybe weeks to finish. Houdini may still be at the top, but it would also be interesting to know who is at the bottom.

Thanks again. Looking forward for more of your insights.

drj

descriptionHoudini Clone Tuning EmptyHoudini Clone Wars - Piece Values

more_horiz
Hi,

Here is the result of the Houdini Clone Wars contending the Chess Piece Values. Probably the best implementation of chess material piece values is from the winner Houdini. The worst implementation according to this small tournament is Komodo. Houdini's piece values is obfuscated which when normalized would look like 100, 337, 345, 528, 1070, 45. This is very much different than the classic "1 3 3 5 9" and may deserved investigation by other chess engine developers to improve the strength of their chess engines.

Code:

Rank             Chess Engine Name           ELO   Win   Loss   Draw   Points   Games   Score%
1   P93 N313 B321 R491 Q995 BP42 – Houdini   2824   818   612   970   1303.0   2400   54.29%
2   H2 N320 B325 R510 Q965 BP50 - Dolphin    2821   799   616   985   1291.5   2400   53.81%
3   H2 N330 B335 R500 Q1000 BP50-Stockfish   2814   802   681   917   1260.5   2400   52.52%
4   H2 N320 B335 R515 Q1000 BP50 – Fire --   2808   755   688   957   1233.5   2400   51.40%
5   H2 N300 B310 R500 Q950 BP45 – Ivanhoe    2804   724   693   983   1215.5   2400   50.65%
6   H2 N325 B325 R500 Q975 BP50 – Kaufman-   2797   704   738   958   1183.0   2400   49.29%
7   H2 N300 B315 R540 Q960 BP50 – Komodo -   2732   498   1072   830   913.0   2400   38.04%


Tournament conditions:
Computers: AMD Dual core, Quad core with Linux and virtualized Windows XP
Time control: 10 minutes base + 1 second increment
Number of rounds: 100 rounds x 4 computers
Ponder: off
Opening book: Perfect 2012c, up to 4 half-moves
Elo rating utility: Bayeselo with 2800 median value

descriptionHoudini Clone Tuning EmptyRe: Houdini Clone Tuning

more_horiz
Permissions in this forum:
You cannot reply to topics in this forum