DJ13-Test

Introduction

This is a test of the new engine Deep Junior 13 (by Amir Ban, Shay Bushinsky) against strong opponents at long time controls. The main goal is to see how the engine behaves (playing style, time management, attack / defend capabilites etc) and to have fun watching interesting games. But there’s also some interest in verifying / falsifying the big “myth” that Junior is much stronger on big hardware and / or playing with long time controls. Although it’s clear that this test can’t provide any statistical relevance I think it might be possible to read off a kind of trend after that match. Therefore the results will be compared to others played at shorter time controls.

The matches

Engine	Version	Threads	Hash MB	Tablebases
Deep Junior	13 (64bit)	6 (2 / 1)	4096 (1024 / 512)	5men Nalimov
vs.
Houdini	2.0c Pro (64bit)	6	4096	5men Gaviota
Critter	1.4a SSE42 (64bit)	6	4096	5men Gaviota
Stockfish	2.2.2 JA SSE42 (64bit)	6	4096	---
Deep Rybka	4.1 SSE42 (64bit)	6	4096	5men Nalimov
Naum	4.2 (64bit)	2	1024	5men Nalimov
Chiron	1.1a (64bit)	2	1024	5men Gaviota
Komodo	4 SSE (64bit)	1	512	---
Fritz	13 (32bit)	1	512	5men Nalimov

Hardware

CPU: AMD Phenom II (x6 / x4) @ 3.0 GHz

RAM: 8192 / 4096 MB

SSD: 60 GB

OS: Microsoft Windows 7 Professional x64

Fritz Benchmark for 1 thread: ~ 1900 kN/s
Matches 6 vs. 6 threads will be played on two different computers using NetChess

Time controls

125min / 50 moves + 100min / 40 moves + (15min+5s)
Ponder = ON

Opening books / Test positions

I truly thought long and hard about this. Opening books are somehow part of a program package – so Junior comes with an excellent own book made by GM Alon Greenfeld. Books should release the engines to positions they “like” and avoid those where they could have difficulties. But there are two arguments that kept me from using opening books in this test:

1) Some of Junior’s opponents have own books, others have not. Which books should I take for the others then?
2) Books give all kinds of tests some sort of random factor. If you want to measure something you have to test under more precise and reproducible conditions.

That’s why I decided to use a set of opening positions. I chose the Noomen Opening Suite 2012 (created by Jeroen Noomen). Jeroen sorted the positions by their classification (Sicilian, French, Ruy Lopez etc). I changed the sequence a bit to see more variety when following the test. Download the pgn with my order of the positions: Testset.pgn

The opponents have to play every one of the 30 position with both colors, so a single match includes a total of 60 games.

Live broadcast

The match series is already finished. You can download all the games here (thanks to Mike Scheidl): Games

Final results:

Junior vs.

- Houdini: 15 – 45 (25%)
- Critter: 20.5 – 29.5 (34.2%)
- Stockfish: 22 – 38 (36.7%)
- Rybka: 18 – 42 (30%)
- Naum: 28 – 32 (46.7%)
- Chiron: 28 – 32 (46.7%)
- Komodo: 18 – 42 (30%)
- Fritz: 29.5 – 30.5 (49.2%)

Best regards,
Timo