Q&A with Larry Kaufman

We have the pleasure to host a Q&A with Larry Kaufman, famous for his great contributions to the development of first Rybka and recently Komodo. Please be free to ask him any polite question you feel like. The best question will be awarded with a copy of Komodo TCEC, the newest version (Larry will choose).

Komodo recently won the World Championship in front of Stockfish and Houdini, so this is quite a catch for us to get this audience driven interview.

The rules are the same as always: all questions provided before the 11th of December will be read, though I cannot promise that Larry will answer them all. He will answer when he has the time, but before the end of the 15th December.

To kick this off, I will start with a few questions of my own (very eager for the free-bee!):

• What are the main reasons Komodo was able to edge out the competition in the World Championship?
• What are the different properties of Komodo, Stockfish, Houdini, Rybka and Fritz and how would the person who wants to improve their chess use these differences to his advantage?
• Is a person better off using one engine or several engines when analysing?

I hope this is a good start. Please provide your questions and make Larry feel welcome here!

51 thoughts on “Q&A with Larry Kaufman”

  1. Jesse Gersenson

    Hi Larry,

    I’ve always enjoyed the evaluations of Komodo and I understand they are mostly the result of your work. My question is, has your understanding of how a computer must value a position in order to play better chess improved your own chess playing? … and, if so, please give a few examples.

  2. – As most new engines today are released without an openingbook for engines, will Komodo be an exception?
    – I made recently a review of Stockfish compared with Houdini: http://chess-brabo.blogspot.be/2013/11/stockfish-4.html Between the 2 engines there are serious differences. Is Komodo more an endgame wizard like Stockfish or more a tactical moster like Houdini or still something else.
    -Which interfaces will support the engine, chessbase …?
    -How will the evaluations look like? Will they be tweaked for the end-users so better understandable or should we expect some high values like we get now for Stockfish?
    – Which tablebases does the engine support?
    – What will be the selling price? Is it downloadable or DVD to be bought via a shop?

    These are for me the questions which I ask when I am interested in a new program.

  3. Are there any types of positions that Komodo handles especially well compared to its rivals? Are there any that it is (relatively) weaker in? And, as I assume that Larry et al have tinkered with Houdini and Stockfish quite a bit, is there any feeling as to relative strengths and weaknesses of competitor engines?

  4. Larry, you prominently mention your work on chess engines and making use of them, in your two excellent opening repertoire books of 2003 and 2012. Easily accessible and affordable computational power and engine algorithm sophistication increased tremendously in the time between your two books. Do you think we have reached a point of diminishing returns, such that there will not be a Komodo-inspired brand new book in 2020?

  5. the_ModernPlayer

    Dear Larry,

    Congratulations on your team’s victory! My question is about the practical use of Komodo in tournament preparation and opening-middlegame study. When analysing/studying complex middlegames such as ones arising from KID: Mar del Plata and Grunfeld: Russian System, engines significantly vary in their assessments which greatly reveal game aspects they value most or their style.

    What is Komodo’s main approach to solving problems/decision-making in complex positions? Considering there’s a choice between two lines: one that leads to an equal ending (forcing or almost forcing way), another that leads to an unclear position that leaves chances for both sides (chances to still outplay the opponent!), which one will it pick/value more?

  6. @Franklin Chen
    To add to the previous question, do you think there will be a point at which the difference in strength between engines and humans is so big that further improvement of the engines might (who knows?) even be counterproductive? Would it make sense e.g. to have an engine with a rating of say 5000 given that at the chess board we’re on our own?

    Thanks for taking the time to answer our question! I really appreciate this opportunity which is offered by Quality Chess (and of course with the kind cooperation of the autors concerned).

  7. Vassil Vassilev

    Hi, Mr Kaufman!
    Congratulations for making Komodo TCEC such a strong chess engine – in fact, the best in the world for now (and I hope for ever)!
    My question is:
    How do you make evaluations to differ in the opening, middle game and endgame?
    For example, a rook in the opening cannot have the same value as a rook in the endgame. So, is there a special way for Komodo to recognize the real value of the pieces in different positions?
    (Don, R.I.P.)

  8. I’d just like to offer my condolences. (I think not all people on this blog know that the primary author of Komodo, Don Dailey, died ten days ago. Just when Komodo qualified for the final).

  9. Dear Mr. Kaufman, thank you for taking the time to answer our questions. One of the challenges when playing computer programs is that they play much different than humans. Even the so-called friend modes or handicap modes don’t simulate human play very well. Are there any promising approaches to change this? (I could imagine algorithms that simulate typical human mistakes like failing for in-between moves, overlooking seemingly quiet continuations in the calculation, over- or underestimating compensation for sacrificed material or in general over- / underestimating specific positional factors like bishop pair or space.)

  10. Hi Larry, I don’t remember if it was you that said that computer chess still has 300-400 rating points to gain. In which field do you think the gain will appear? In the search? In the evaluation? Opening, middlegame or endgame? Thanks

  11. Congratulations on Komodo, a very strong positional engine!

    What do you think is missing for engines to take the next leap? For example, as in understanding stalemate defences and that extra material doesn’t necessarily win if it’s badly placed and can’t be improved.

    Sometimes you hear strong players lamenting that engines have ‘destroyed the game’ as in removing some of the mystery and creating the basis for cheating, ‘the doping of chess’ so to speak. How would you react to that? And do you see a future for Chess after 50 or 100 years, with or without changes to the game?

  12. I have read many articles on the best way to beat a chess computer program.
    The 3 different ideas that I incorporate in my play against chess programs are:
    1) Take program out of opening early, so program leaves opening database and has to use time on the clock early.
    2) Seek to play a closed position on the board; and
    3) Try not to move into an early endgame where chess program is very powerful.
    Do you agree with these and are there any other ideas that I should use when playing against a chess program with the view of beating the program?
    Also, if I were to play a program as powerful as Komodo, would it be best just to play my normal game as it is so powerful it will beat me anyway and hence use program as training partner rather than thinking that I may possibly beat the program?

  13. Hi Larry,

    Congrats to Komodo on its recent victory
    It is sad to hear about Don (R.I.P.)

    I want to tell you that your recent repertoire book was amazing and I got great results with its recommendations.
    I am also looking forward to your book ‘Sabotage the Grunfeld’.
    What are your future plans? Are you thinking of writing more wonderful books like these?

  14. Larry,

    Let me ask a stupid question. What can the computers/engines teach us (humans)? I mean – they are completely stupid, doing only what we tell them to do – but are exceptional good at calulating lines. What have you learned and implemented into your own play by working on engines?

  15. Hi Mr kaufman
    I want to ask if it’s possible to make an engine that evaluate people chess abilities and style of play and help to improve the player in the weak areas ( like a virtual chess instructor)

  16. Larry, you are the author of several opening books, including your Kaufman Repertoire for White and Black. Many players in the past have said that using engines to analyze openings is not reliable due to the computer’s inability to foresee ramifications beyond the opening. Having owned your repertoire book, I know that engine analysis (especially Komodo) played a huge role in your recommendations. Have engines ‘evolved’ enough that nowadays they can be used to accurately evaluate opening positions?

  17. • What are the main reasons Komodo was able to edge out the competition in the World Championship?

    In my opinion it is due to spending more time in evaluating positions. This is not good for blitz chess where tactics dominate, but at long time limits it appears to pay off.

    • What are the different properties of Komodo, Stockfish, Houdini, Rybka and Fritz and how would the person who wants to improve their chess use these differences to his advantage?

    Komodo is best at evaluating middlegame positions accurately once the tactics are resolved. Stockfish seems to be best in the endgame and in seeing very deep tactics. Houdini is the best at blitz and at seeing tactics quickly. Rybka is just obsolete; I like to think of Komodo as its spiritual desceendant, since I worked on the evaluation for both, although the rest of the engines are not similar. Fritz is just too far below these top engines to be useful.

    • Is a person better off using one engine or several engines when analysing?

    I think it does make sense to use one core for each of the top three engines when you have a quad, partly because using all the cores for one engine is a bit inefficient, and also because the top 3 engines are not very similar.

  18. @Jesse Gersenson

    I won the World Senior and got the GM title right after finishing my work on the evaluation function of Rybka 3, despite being rated only 17th in the field. Coincidence? Possibly, but it seems unlikely. Mostly I learned that the subjective weights I gave to typical features of a position were sometimes pretty far off. For example, it seems that having fewer pawn islands is a much smaller edge than I thought. Also, the supposed advantage of queen and knight vs. queen and bishop is nearly impossible to prove based on computer results. There are hundreds of such things.

  19. brabo :
    – As most new engines today are released without an openingbook for engines, will Komodo be an exception?

    There are two free opening books for Komodo on Komodochess.com. These days books and engines are pretty much unrelated, though a few book authors do try to modify their books for optimum performance with a specific engine.

    – I made recently a review of Stockfish compared with Houdini: http://chess-brabo.blogspot.be/2013/11/stockfish-4.html Between the 2 engines there are serious differences. Is Komodo more an endgame wizard like Stockfish or more a tactical moster like Houdini or still something else.

    I think Komodo is best at middlegame evaluation. It spends a higher percentage of its time doing this than do Houdini or Stockfish, so it is tactically not quite as strong as Houdini. But in longer time control games positional play seems to be more important.

    -Which interfaces will support the engine, chessbase …?

    It’s UCI, so same as Stockfish, it will work with ChessBase, Fritz, Aquarium, and several free GUIs such as Arena.

    -How will the evaluations look like? Will they be tweaked for the end-users so better understandable or should we expect some high values like we get now for Stockfish?

    The evaluations are on a scale fairly close to Houdini, not to Stockfish. People who use Stockfish should just take 2/3 of the reported evals to compare to Komodo or Houdini.

    – Which tablebases does the engine support?

    So far none. Perhaps that will change in the future.

    – What will be the selling price?

    $49.95 in general; $29.95 if you already bought Komodo 6.

    Is it downloadable or DVD to be bought via a shop?

    So far download only.

  20. @John Hartmann
    Komodo’s assessment of positions is its strong point relative to the other top two, Houdini best for tactics, Stockfish for endgames and whenever great depth is required. Both Houdini and Stockfish overvalue the queen, Komodo has the best sense for relative piece values I think. Komodo is also best at playing the opening when out of book very early.

  21. @the_ModernPlayer
    Like most if not all other engines, Komodo doesn’t distinguish between an equal drawish position and an equal double-edged one; it just chooses the one it thinks is microscopically a tad better objectively. Maybe in the future we will try to make such distinctions.

  22. @Ray If improvement comes from better evaluation, it is always useful to the user, as the percentage of misevaluated positions will drop. If it comes from better (faster) search, the advantage is just getting your answer faster. At some point speed improvements will only be useful for correspondence players, for the reason you mention. But that’s still some years away.

  23. @Vassil Vassilev
    The way this is handled in most (all?) top engines is that for each feature, one value is assigned for a full board, another for a board with only king and pawns, and everything is interpolated. So for example if a knight is worth four pawns on a full board but only three in a pawn ending (plus the knight of course), then with half the material on the board it gets 3.5 pawns. Some features may have special rules that differ from this general scheme.

  24. @wok64
    Trying to simulate human error in chess is a problem some have worked on, but we have been focused pretty much just on making Komodo as strong as possible. Positional errors can be simulated by crude rounding of evaluations; tactical errors mostly just by severely limiting search depth, although of course there are many other approaches. Some combination of crude rounding and short seaarch might be a reasonable approximation to amateur human play.

  25. Dear Mr. Kaufman,

    Congratulations on winning the World Championship. And sorry for the loss of your friend and colleague.

    I’m a big fan of computer chess in general and Komodo in particular. Something I do fairly often is to run engine “gauntlet” tournaments, with my favorite engine (lately Komodo) playing my favorite openings. I think that over time, this process has corrected my understanding of chess openings and early middlegames quite a lot. But I still suck at endgames, mostly because I haven’t found a really congenial way to study them. Is there a computer-assisted method you would recommend?

  26. @garryk Nice gains are possible still in all the areas you mention. The endgame and the play in closed middlegames can be improved a huge amount; improved search will help all positions, and openings are just getting analyzed too deeply, so when we talk about rating gains we mean assuming only short opening books are allowed. Maybe an entirely different search will be found to work well with huge amounts of memory. There are many possibilities for improvement.

  27. @TalJechin Well, I can’t dispute that the possibility of cheating with computers is bad for chess. Other than that, they have greatly raised the level of play, but by giving everyone access to exhaustive opening analysis it has made chess more of a memory contest than it used to be, which is unfortunate. I think the future of chess lies in ideas that minimize the value of memorization and preparation, whether it be Chess960, balloting of opening moves in normal chess, or some other idea. I also think that rules changes are needed to make draws the exception rather than the norm. Ideas include making stalemate a win, making miinor piece vs king a win, and forbiddng perpetual check, and of course barring draws by agreement before some advanced move number. I think these rules taken together would cut draws in half.

  28. @Ed In general your suggestions are good for maximizing the chances to score against an engine (normally “score” means draw, not win) but not so good for improving your own game. Maybe on rare occasions you can trick the engine/book combination into allowing a super-closed position that you can then draw, but what’s the point? Someone should really write a book for the engines that just makes sure that a closed position will not arise from the opening. Then it would be rare for any human to draw a top engine.

  29. @Larry Kaufman
    Mr Kaufman, thanks for your answer. Using extensively all the engines, I find that Komodo is the most “human” and “knowledgeable” of the engines but I think it has one main drawback comparing to Houdini, it’s difficult to explain but I’ll try with an example.

    In a certain position there are only two good moves, move A and B. After long thought Houdini evaluates for example A +0.50 and B + 0.20. Instead Komodo evaluates B +0.80 and A +0.45. It seems both engines agree on evaluation of A but they disagree strongly on B. If I follow the main line of B, after executing 3-4 ply, Komodo evaluation drops to 0.20 because something new in the main line becomes “visible”.

    My question is this. As the choice between A and B seems decisive for the outcome of the game (A has winning chances, B is probably a draw), why doesn’t Komodo checks more accurately the main line that brings the evaluation of B?

    I know Komodo’s search is less deep than Houdini (in the same amount of time) but what I find strange is that the main point of B evaluation is often not so deep (just 3-4 ply of forced move) and other engines, far weaker than Komodo, seem not to suffer of the same problem.

    Did you notice this behavior? Are you planning to correct it?

    I think that if Komodo improves in this area it can overcome Houdini by 100 Elo points.

    Thanks again and best regards.

  30. @garryk
    I don’t believe what you describe is something typical for Komodo. You will see such behaviour in every program (so also Houdini) and is called the horizoneffect. When the horizoneffect pops up, depends on the type of position, how the program is built, the time used and the computer.
    In my blogarticle http://schaken-brabo.blogspot.be/2012/05/analyseren-met-de-computer.html (not translated yet to English so still in Dutch) I explain a method how to deal with the horizoneffect. Using Aquarium is another method how to deal with it.

  31. Komodo 5, the single processor version, is free and can be downloaded here
    http://komodochess.com/pub/Komodo5.zip

    Jacob’s analysis of one a TCEC game between Komodo and Stockfish

  32. @Larry Kaufman Does Komodo have any preference in its opening choices? If so, what openings does it seem to prefer against 1.d4/1.e4?

  33. @Multeplukker
    What I have learned is not from playing the engine or watching it play, but from seeing which things help its rating and which do not. One general observation; I thought before starting to work on Rybka that in general, material sacrifice for development, piece placement, etc. tend to work because humans make mistakes, but that they would not work with high class engine play. But it seems I was wrong about this.

  34. @Mario
    It is definitely possible to make an engine evaluate human play, but there are problems. Humans who choose simple, quiet openings tend to look better to engines than those who choose wild lines, because there just aren’t nearly as many possibilities to make big mistakes in boring positions. One study showed Capablanca to be much better than anyone else, including Fischer and Kasparov, but this was due to Capa playing simple chess. Maybe someone will work out a way around this problem.

  35. @Jonny In general, yes, the top engines can be reliably used for opening analysis. But some rather closed openings (French, Classical KI) are not ones that engines play well, so some judgment must be used to know when to trust the engines and when not to do so.

  36. Larry Kaufman :
    @Mario
    It is definitely possible to make an engine evaluate human play, but there are problems. Humans who choose simple, quiet openings tend to look better to engines than those who choose wild lines, because there just aren’t nearly as many possibilities to make big mistakes in boring positions. One study showed Capablanca to be much better than anyone else, including Fischer and Kasparov, but this was due to Capa playing simple chess. Maybe someone will work out a way around this problem.

    thanks Mr Kaufman ,another player who comes at my mind maybe the new world champion ¿?

  37. @Alan
    Engines are good at two types of endgames. First, if the number of men on the board is 6 or less, any engine with the proper tablebase support (not yet in Komodo) will play them perfectly. In more complicated endgames where both sides are doing things (as opposed to endgames where one side is basically “passing” most of the time in hopes of a draw) engines will play much better than GMs. But except for tablebase positions engines are very poor at judging which endgames are winning and which are drawn. Often they will think they are winning in a drawn position but of course cannot prove it.
    So I guess I would advocate getting 6 man TBs (they take a lot of space) and studying how the engine plays and evaluates (as win or draw) interesting 6 man endgames.

  38. @garryk
    No, I have not noticed the behavior of Komodo not seaching the PV thoroughly enough compared to other engines. One possibility is that this is a problem specific to multi-PV; that is a feature we have not tuned, because no one tests that way. In general I think all the top engines now do focus heavily on the PV, although due to certain extensions we may do so a bit less than others. We can easily focus more on the PV but it will reduce our rating a bit unless we find a better way.

  39. James :
    @Larry Kaufman Does Komodo have any preference in its opening choices? If so, what openings does it seem to prefer against 1.d4/1.e4?

    The opening book is totally separate from the engine. The choice of opening moves is made by the book, not the engine. Any book in proper format can be used with Komodo.

  40. Larry,

    Congratulations for the impressive Komodo victory in the last TCEC tournament.
    The tournament had an amazing participation, and Komodo’s victory was really well deserved. It really won playing beautiful positional chess!

    Honestly, I am quite impressed by the playing style of Komodo. His playing style is very human, but of course, it has an extraordinary-unparalled precision. I have no doubt that Komodo is the BEST positional engine that has been created so far. I think this is one of the things that make Komodo an unique engine, a gem. Thanks for this wonderful program!

    I hope that Mark Lefler and you continue making Komodo even more special. Don Dailey was an amazing programmer, but I am confident that Mark and you will further continue improving the program. And before I forget: Also THANKS for finally provide the TCEC version to the public!

    My BEST wishes to Mark and you, Larry.
    Gaмßito.

  41. @Mario
    Maybe Carlsen would also look better than he really is (which isn’t easy to do!) on an engine study of his moves, but he does play some rather sharp openings sometimes, even if they are not objectively so strong, so this might not apply.

  42. @PC
    Actually I had nothing to do with that book, I didn’t even download it, but the opening you mention is not a very sensible one so any coverage would be minimal. If Black plays …e6, what is the purpose of …c6? Maybe some super-subtle waiting move?!

  43. Hi Larry, I was wondering when it will be possible for an Engine to use anotations from publications like Chess life and New in Chess to give better coments rather then cant get the bull off the ice, or cant get the cat out of tree. This is humor however, some more meaningful and instuctive coments would really be a big plus. Also if an Engine could coment on opening recomendations as well and not just the use of eco anotations so even kids like me could learn from an engine would be very useful in chess camps. Do you see this type of engine possible to cross refrence an anotated game and inplement the coments of top grandmasters into the analysis sometime soon? Thanks Rick

  44. Larry – have you learned anything from working on Komodo – and computer chess in general – that you have applied to your own human play? As an amateur I often see things in chess books and tactics positions that are new to me, and then note them down in a little log book with ideas or ‘things to look out for’ that I try to add to my thinking techniques. I was just wondering if you have found new approaches or ways of thinking about positions based on the way you have seen Komodo discriminate on various positions? Thanks – Mike.

  45. Hello Larry, i’ve heard some GMs assert that it’s not very beneficial to play too much with computers given their unique anti- positional style. Is this different with Komodo?

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top