Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Castling-like moves by the A/B-helper is mis-interpreted as legacy encoded castling. #2

Closed
hans-ekbrand opened this issue Dec 10, 2021 · 0 comments

Comments

@hans-ekbrand
Copy link
Owner

This output from the A/B helper: "1210 18:19:52.086406 140709401196288 ../../src/mcts/auxengine.cc:302] pv:info depth 15 seldepth 26 multipv 1 score cp -180 time 17 nodes 89153 nps 5244294 tbhits 56 hashfull 10 pv e5b2 h6h7 e7e8 h7a7 b2c3 e6g7 e8d8 a7f7 c3b2 g7e6 d8e8 f7g7 f2f1 g7b7 b2a1 f5e4 f1e1 e4d3 e1c1 d3d2 c1c8"

Was interpreted as "debug info: length of PV given to helper engine: 4 position given to helper: position fen 8/4k3/3pNp1R/3PbK2/5P2/8/5r2/8 b - - 2 52 black to move at root, length of my_moves_from_the_white_side 25 my_moves_from_the_white_side: g5g7 f2f4 g7f7 h3h6 e4b7 h6h7 e2e1 h7a7 b7c6 e6g7 e1d1 a7f7 c6b7 g7e6 d1e1 f7g7 f7f8 g7b7 b7a8 f5e4 f8e8 e4d3 e8a8 d3d2 c8c1"

The problem is the move "e1c1" which is interpreted as "e8a8" (seen from white's perspective) (which is e1a1 from black's perspective) which is an application of convert legacy encoding of long castling to FRC encoding of long castling, where the move is a rook move, not a castling move. "e8c8" is the correct move here.

I thought my code for detecting legacy encoded castling (and converting to FRC encoding castling) was correct but apparently not. The code in question is this section from auxengine.cc

	// convert to Modern encoding, update the board and the position
	// For the conversion never flip the board. Only flip the board when you need to apply the move!
	Move m_in_modern_encoding = my_board.GetModernMove(m);

	if (my_board.flipped()) m_in_modern_encoding.Mirror();
	// Should the move applied be modern or legacy, or does it not matter?
	m_in_modern_encoding = my_board.GetModernMove(m_in_modern_encoding);
	// my_board.ApplyMove(m_in_modern_encoding); // Todo verify the correctness here, e.g. by printing a FEN.
	my_board.ApplyMove(m); // Todo verify the correctness here, e.g. by printing a FEN.	
	my_position = Position(my_position, m_in_modern_encoding);

	if (my_board.flipped()) m_in_modern_encoding.Mirror();
	my_board.Mirror();
hans-ekbrand pushed a commit that referenced this issue Dec 10, 2021
…like moves. For now, when a node cannot be added just don't add it, don't throw an exception.
hans-ekbrand pushed a commit that referenced this issue Feb 15, 2022
hans-ekbrand pushed a commit that referenced this issue Feb 16, 2022
hans-ekbrand pushed a commit that referenced this issue Apr 6, 2024
* Add a mode to turn lc0 into a chunk data rescorer powered by Tablebase.

* Add some stats.

* Add secondary rescoring using wdl to reduce back propigation of blunders a bit.

* Add policy distribution adjust support to rescorer.

* Track the game outcomes, and the change to the start of the game

* Add DTZ based assistance for secondary rescoring.

* Change move count to a moves remaining to potentially use for modulating target value.

* Use DTZ for pawnless 3 piece positions as a substitue for DTM to adjust move_count to be more correct

* another fix.

* More fixing.

* Getting things compiling again.

* Make rescorer more obvious.

* reorder to match struct order.

* Actually update the version when converting to v4 format.

* Implement the threading support.

* Fix compilation issues on some compilers.

* More compilation fixing.

* Fix off by one.

* Add support for root probe policy boosting for minimum dtz in winning positions.

* Fix test compile.

* Fix missing option.

* Add a counter.

* Log if policy boost is for a move labelled illegal.

* Add a histogram for total amount of boosted policy per boosted position.

* Distribute boost rather than apply to all - also log before and after dists.

* Add gaviotatb code for later use in dtm_boost

* Fix compile issue on linux.

* Prepare logic for dtm policy boost.

* Load gaviota tb if specified.

* Probe gaviota to decide which 'safe' moves are most deserving of boost based on dtm.

* First attempt at supporting arbitrary starting point training data for rescorer.

* Fix missing brackets.

* Some fixes.

* Avoid crashes from walking history before start of provided game information.

* Some more merge fixes.

* Fix some formatting.

* Only process .gz files, don't crash out on invalid files, don't create output until input has been read.

* Don't keep partially valid files.

* Add basic range validation for input data.

* Don't create writer any earlier than needed.

* Fix decoding castling moves for the new Move format.

* Validate game moves for legality.

* Also log illegal move if it passes probability check but fails the real check.

* Fix another merge error.

* Compile fix for linux.

* Plies left in rescorer (#1)

* Rescore move_count using Gaviota TBs

* Fix lczero-common commit

* Add condition for Gaviota move_count rescoring

* Post merge fixup for the kings/knights change in board.

* Rescore tb v5 (#2)

* Make lc0 output v5 training data.

* Finish merge of v5 data into rescorer tb.

* Fixes for rescoring v4 data.

* Revert some unneeded formatting changes.

* Support FRC input_format in rescoring.

* Add some very important missing break statements...

* Fix merge.

* Change movement decode to not rely on there being any history planes filled in.

Since that will not always be the case for input type 3.

* Minimum changes to make it compile again post merge.

* Input format 3 support.

* Fix data range checks were incorrect for format 3 and 2.

* Fix up bugs with chess 960 castle moves that leave a rook or king in place.

* Post merge compile fixups for renames.

* Add support for hectoplies and hectoplies armageddon to validate, and fixup the merge of latest code.

* More fixes for type 4 and 132.

* Add input format conversion support to rescorer.

* Better match for training.

* Add canonical v2 format to rescorer.

* Add a utility for substituting policy from higher quality data into main data.

* Fix missing option and add some commented out diagnostic code.

* More cleanup in comments.

* Handle empty policy-substitutions dir and input dir better.

* Don't keep chunks that are marked as not for training.

* More fixes for handling files with placeholder chunks.

* Add 'deblunderer'

Completely untested...

* Fix some bugs in deblunder.

* simplify windows rescorer build (#4)

Co-authored-by: borg323 <[email protected]>

* Tweak windows build file.

* Some updates for writer.h/cc for v6

* Update rescorer loop.cc for V6.

* Some additional validations to do with played_idx/best_idx.

* make appveyor build the rescorer (#7)

Co-authored-by: borg323 <[email protected]>

* subproject for gaviota tb files (#8)

Co-authored-by: borg323 <[email protected]>

* 'Fix' for build on windows

Probably should be fixed some other way...

* Fix my breakage. (#9)

* Update loop.cc

* Update meson.build

* Use the v6 field played_q to do a more direct blunder rescoring (#5)

* included the issue 1308 deblunder mechanism in loop.cc

* blunder detection now acts on missed proven wins and unforced proven losses

* added comment on missing activeM

* removed probabilistic randomization of result rescorer and worked with v6 data instead

* included moves left rescore, removed unneeded options

* doubled code not needed as final positions aren't special

* changed appveyor script to hopefully build rescorer.sln

* reverted failed attempt at fixing appveyor

* included minimal std::cout for blunders

* included blunder counter, added comment to visits v6 data checking

* checking for bit 3 of invariance info to make sure best_q is a proven win

* Fix v5 upgrading for decisive games.

* Additional safety.

* Add missing brackets.

* don't keep the first TB position for the deblundering pass. (#10)

* included the issue 1308 deblunder mechanism in loop.cc

* blunder detection now acts on missed proven wins and unforced proven losses

* added comment on missing activeM

* removed probabilistic randomization of result rescorer and worked with v6 data instead

* included moves left rescore, removed unneeded options

* doubled code not needed as final positions aren't special

* changed appveyor script to hopefully build rescorer.sln

* reverted failed attempt at fixing appveyor

* included minimal std::cout for blunders

* included blunder counter, added comment to visits v6 data checking

* checking for bit 3 of invariance info to make sure best_q is a proven win

* don't keep the first TB position for rescorer

* change recorer logo (#11)

Co-authored-by: borg323 <[email protected]>

* Make the deblunder transition soft through a width parameter (#13)

* included the issue 1308 deblunder mechanism in loop.cc

* blunder detection now acts on missed proven wins and unforced proven losses

* added comment on missing activeM

* removed probabilistic randomization of result rescorer and worked with v6 data instead

* included moves left rescore, removed unneeded options

* doubled code not needed as final positions aren't special

* changed appveyor script to hopefully build rescorer.sln

* reverted failed attempt at fixing appveyor

* included minimal std::cout for blunders

* included blunder counter, added comment to visits v6 data checking

* checking for bit 3 of invariance info to make sure best_q is a proven win

* don't keep the first TB position for rescorer

* added a deblunder width parameter to allow a soft transition

* clang formatting

* resolve merge conflict

* Add nnue plain file output (#12)

* GetFen() from pr834

* first version of nnue output

* flag to delete fils

* address review comments

* support pre v6 data

* fix sign

* correct nnue data misunderstanding

Co-authored-by: borg323 <[email protected]>

* fix copy-paste error (#15)

Co-authored-by: borg323 <[email protected]>

* add -t flag (#16)

Co-authored-by: borg323 <[email protected]>

* Post merge fixes.

* Missed cleanup.

* Fix input format change bug that can corrupt played_idx and best_idx

* Post merge fixes.

* fix merge

* remove unnecessary options

* split out rescore loop

* minimize rescorer build

* merge rescorer with master

* minimize syzygy diff

---------

Co-authored-by: Tilps <[email protected]>
Co-authored-by: Henrik Forstén <[email protected]>
Co-authored-by: borg323 <[email protected]>
Co-authored-by: Naphthalin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant