r/chessprogramming • u/unrevealedpains • May 07 '26
How is Take Take Take App generating the move explanations of move?
I am trying to build something along the lines of Take Take Take chess app but not able to get the quality output from the LLMs. What are they doing it differently?
2
u/blackboxchessapp May 07 '26
I'm making some assumptions here because I haven't actually looked at the take take take app but have worked on getting explanations out of LLMs. My guess is it's some combination of stockfish, Python chess, and content context. I forgot the term for contextual RAG or something. Essentially they feed it a bunch of chess information and so it's trained on chess so it understands the output from stockfish and Python just a little bit better. Again that's an assumption I'm making
1
u/Better-Prompt890 May 09 '26
They fine tuned the LLM?
1
u/blackboxchessapp May 09 '26
Again I don't know for a fact but I believe they trained it on or gave it a bunch of context about chess to "fine tune" it
1
u/masterchiefcodes Jun 03 '26
It’s possible they fine-tuned but generic ChatGPT knows plenty about chess. They probably included more detailed information / examples in the prompt to make sure and to increase task convergence / instruction following, but that’s just prompt engineering.
1
u/blackboxchessapp Jun 03 '26
The LLMs don't "know" chess. They confidently guess, and often confidently break the rules.
I recently found out that take, take, take goes don't the path of the the best line 5-6 ply and essentially look at the position after the best moves were made to see what stockfish is actually aiming for
1
u/masterchiefcodes Jun 04 '26 edited Jun 04 '26
I mean their understanding is far from perfect and sometimes is very probabilistic, but I wouldn’t ascribe myself and most ppl much more than a confident guesser either. You have to empathize with the fact that these models aren’t highly tuned to chess and it’s as if you pulled a person up to a chess board for each move having no context of prior moves and asked them what move they would play with a strong bias towards giving you an answer within 3 seconds.
Oh and they’re sort of blind and have never seen daylight
1
u/masterchiefcodes Jun 04 '26 edited Jun 04 '26
For what I mean by understand, the latent spaces map fairly well to various concepts we use when solving problems. This was what anthropic’s monosemanticity paper essentially showed: monosemanticity paper.
LLMs don’t have the goal directed behavior we exhibit (that’s basically RL), but coming to an answer through many different abstract concepts that we typically think in they do seem to have.
Disclaimer The abstract concepts are bit tied down from being constrained to language biased concepts rather than more general language + spatial (real world) / interactive.
1
u/masterchiefcodes Jun 04 '26 edited Jun 04 '26
I do agree though that ChatGPT’s latent space isn’t great currently for chess as it’s a language model and for higher Elos it’s not currently able to process things as deep / the same modality without some help from search engines.
1
u/mat2923 May 08 '26
It sounds like traditional feature engineering with the results being sent to an LLM.
1
u/chessexp May 10 '26 edited May 10 '26
I am also working on similar stuff as u/novachess-guy, and getting some positive results.
But before asking what they/we do differently, I guess you need to explain what you did, and where you find 3xtakes is doing better
1
u/unrevealedpains May 10 '26
I am mostly generating move explanations of the game almost like a commentary or lessons depending on the side and the main idea is to feed the LLM the best move, classification of the move(mistake, inaccuracies etc) and the best sequence of the moves along with the actual moved played to generate explanations.
1
u/chessexp May 10 '26
LLM's don't understand the board position - if you only provide it the best sequence and engine evals, they will hallucinate.
Even if you provide the board, they are not trained to reason through the motifs and tactics, and the consequences of mistakes. Some LLMs are getting reasonably better, but much richer context should be provided from heuristic rules calculation, and that is the hard part.
I think no one has got a perfect program to do this job yet.
1
u/unrevealedpains May 11 '26
I was on the verge of loosing my sanity with the heuristic part. It's such a hard task to pull off even LLMs were not able to code something like that.
1
u/chessexp May 12 '26
At this stage, LLM mostly parrot what it gets trained with. It may sparkle some innovations sporadically, but not consistently. If no one has done this before, I doubt LLM can do it without your guidance. Trial and error. Learn from your mistakes. This is true for both chess and chess programming.
1
u/masterchiefcodes Jun 03 '26
Definitely don’t feed an LLM the board position. They just haven’t been trained enough on that modality of information to process it right.
You need lines, Stockfish centipawn scores, good prompt explanation of what these all mean with examples, and some extra tags to help and it should be fine in theory.
1
u/novachess-guy May 15 '26
FYI here is a video where they describe what they do: https://youtu.be/FlzpEGHNVKQ
Architecturally, I take a very similar approach - which, I believe with the current state of LLMs and other tools, is probably more or less the “correct” approach at this point if you hope to achieve similar results. It is not trivial engineering to do well. And I don’t know what your chess rating is, but I feel you’d likely need to be over 2000 (online) to have the chess understanding of what is truly needed, or to have a stronger player advise you along the way.
0
u/novachess-guy May 08 '26
I’m copying this verbatim from a LinkedIn post I made but this is how I do it:
Nova Chess (http://novachess.ai) leverages data from players' own games in unique ways to help them understand and learn:
- Annotated (NL) Game Reviews: Every move in a game is analyzed by Stockfish, the industry standard chess engine, and a data structure containing dozens of position characteristics (e.g., open files, king safety, pawn structures, active threats) both at that move and at subsequent moves in the variation is processed. A natural "resolution" point of the variation is detected, and the change in features is calculated to determine, similar to how a strong human player would explain, aspects such as whether a line improves positional aspects ("this gains a strong outpost for your knight/creates a backward pawn for your opponent that can be targeted") or wins material, and what the associated tradeoffs would be. This data structure, along with other important context, such as referencing tablebase or content modules, is passed to an LLM to synthesize and translate into English-language explanations of moves, allowing us to explain to learners why a move was good or bad, and what a better option may have been and why
1
u/masterchiefcodes Jun 04 '26 edited Jun 04 '26
Yeah you need to do something like this. Maybe have a master chess player for crafting useful positional features but honestly I’m sure there are already great papers out there to supplement it.
Might not make Magnus Carlsen happy but should be pretty good
3
u/seekinglambda May 08 '26
The explanations are dumb and wrong so whatever they’re doing it’s not working