Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Placer divergence on bp_be/nangate45 #6542

Closed
povik opened this issue Jan 16, 2025 · 7 comments
Closed

Placer divergence on bp_be/nangate45 #6542

povik opened this issue Jan 16, 2025 · 7 comments
Assignees
@gudeh
Labels
gpl Global Placement

Comments

@povik
Copy link
Contributor

povik commented Jan 16, 2025 •
edited
Loading

Describe the bug

In an experiment to start inferring clock gates from Yosys, the global placer diverges on the design. The last lines are:

14:57:07  [NesterovSolve] Iter: 1640 overflow: 0.176 HPWL: 3556483454
14:57:08  [NesterovSolve] Iter: 1650 overflow: 0.155 HPWL: 3551918924
14:57:08  [NesterovSolve] Iter: 1660 overflow: 0.138 HPWL: 3549444690
14:57:08  [INFO GPL-0100] Timing-driven iteration 6/6, virtual: false.
14:57:08  [INFO GPL-0101] Iter: 1660, overflow: 0.144, keep rsz at: 0.3
14:57:34  [INFO GPL-0106] Timing-driven: worst slack -2.61e-10
14:57:34  [INFO GPL-0103] Timing-driven: weighted 5369 nets.
14:57:34  [INFO GPL-0107] Timing-driven: repair_design delta area: -126.244 um^2 (-0.11%)
14:57:34  [INFO GPL-0108] Timing-driven: new target density: 0.43047988
14:57:34  [NesterovSolve] Iter: 1670 overflow: 0.143 HPWL: 4368350234
14:57:34  [NesterovSolve] Iter: 1680 overflow: 0.113 HPWL: 4746991274
14:57:34  [NesterovSolve] Iter: 1690 overflow: 0.132 HPWL: 5348893280
14:57:34  [ERROR GPL-0307] RePlAce divergence detected. Re-run with a smaller max_phi_cof value.
14:57:34  Error: global_place.tcl, 40 GPL-0307

Expected Behavior

No divergence

Environment

[WARNING] Your current OpenROAD version is outdated.
It is recommened to pull the latest changes.
If problem persists, file a github issue with the re-producible test case.
kernel: Linux 6.5.0-1025-gcp
os: Ubuntu 22.04.4 LTS (Jammy Jellyfish)
cmake version 3.24.2
-- The CXX compiler identification is GNU 11.4.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- OpenROAD version: v2.0-17941-g7fb347f37
-- System name: Linux
-- Compiler: GNU 11.4.0
-- Build type: RELEASE
-- Install prefix: /usr/local
-- C++ Standard: 17
-- C++ Standard Required: ON
-- C++ Extensions: OFF
-- The C compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Performing Test C_COMPILER_SUPPORTS__-Wall
-- Performing Test C_COMPILER_SUPPORTS__-Wall - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wall
-- Performing Test CXX_COMPILER_SUPPORTS__-Wall - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-array-bounds
-- Performing Test C_COMPILER_SUPPORTS__-Wno-array-bounds - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-array-bounds
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-array-bounds - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-nonnull
-- Performing Test C_COMPILER_SUPPORTS__-Wno-nonnull - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-nonnull
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-nonnull - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-maybe-uninitialized
-- Performing Test C_COMPILER_SUPPORTS__-Wno-maybe-uninitialized - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-maybe-uninitialized
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-maybe-uninitialized - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-format-overflow
-- Performing Test C_COMPILER_SUPPORTS__-Wno-format-overflow - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-format-overflow
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-format-overflow - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-variable
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-variable - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-variable
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-variable - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-function
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-function - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-function
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-function - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-write-strings
-- Performing Test C_COMPILER_SUPPORTS__-Wno-write-strings - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-write-strings
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-write-strings - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-sign-compare
-- Performing Test C_COMPILER_SUPPORTS__-Wno-sign-compare - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-sign-compare
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-sign-compare - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-deprecated
-- Performing Test C_COMPILER_SUPPORTS__-Wno-deprecated - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-deprecated
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-deprecated - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-c++11-narrowing
-- Performing Test C_COMPILER_SUPPORTS__-Wno-c++11-narrowing - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-c++11-narrowing
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-c++11-narrowing - Failed
-- Performing Test C_COMPILER_SUPPORTS__-Wno-register
-- Performing Test C_COMPILER_SUPPORTS__-Wno-register - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-register
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-register - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-format
-- Performing Test C_COMPILER_SUPPORTS__-Wno-format - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-format
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-format - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-reserved-user-defined-literal
-- Performing Test C_COMPILER_SUPPORTS__-Wno-reserved-user-defined-literal - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-reserved-user-defined-literal
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-reserved-user-defined-literal - Failed
-- Performing Test C_COMPILER_SUPPORTS__-fpermissive
-- Performing Test C_COMPILER_SUPPORTS__-fpermissive - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__-fpermissive
-- Performing Test CXX_COMPILER_SUPPORTS__-fpermissive - Success
-- Performing Test C_COMPILER_SUPPORTS__-x
-- Performing Test C_COMPILER_SUPPORTS__-x - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__-x
-- Performing Test CXX_COMPILER_SUPPORTS__-x - Failed
-- Performing Test C_COMPILER_SUPPORTS__c++
-- Performing Test C_COMPILER_SUPPORTS__c++ - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__c++
-- Performing Test CXX_COMPILER_SUPPORTS__c++ - Failed
-- Performing Test C_COMPILER_SUPPORTS__-std=c++17
-- Performing Test C_COMPILER_SUPPORTS__-std=c++17 - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__-std=c++17
-- Performing Test CXX_COMPILER_SUPPORTS__-std=c++17 - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-but-set-variable
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-but-set-variable - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-but-set-variable
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-but-set-variable - Success
-- TCL library: /usr/lib/x86_64-linux-gnu/libtcl.so
-- TCL header: /usr/include/tcl/tcl.h
-- TCL readline library: /usr/lib/x86_64-linux-gnu/libtclreadline.so
-- TCL readline header: /usr/include/x86_64-linux-gnu
-- Found SWIG: /usr/local/bin/swig (found suitable version "4.1.0", minimum required is "4.0")  
-- Using SWIG >= 4.1.0 -flatstaticmethod flag for python
-- Found Boost: /usr/local/lib/cmake/Boost-1.80.0/BoostConfig.cmake (found version "1.80.0")  
-- boost: 1.80.0
-- Found GTest: /usr/local/lib/cmake/GTest/GTestConfig.cmake (found version "1.13.0")  
-- GTest: 1.13.0
-- Found Python3: /usr/include/python3.10 (found version "3.10.12") found components: Development Development.Module Development.Embed 
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.11") 
-- Found Threads: TRUE  
-- spdlog: 1.8.1
-- Found BISON: /usr/bin/bison (found version "3.8.2") 
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) 
-- STA version: 2.6.0
-- STA git sha: aa598a2f14c5c142e90391a69988523505e7db3d
-- System name: Linux
-- Compiler: GNU 11.4.0
-- Build type: RELEASE
-- Build CXX_FLAGS: -O3 -DNDEBUG
-- Install prefix: /usr/local
-- Found FLEX: /usr/bin/flex (found version "2.6.4") 
-- TCL library: /usr/lib/x86_64-linux-gnu/libtcl.so
-- TCL header: /usr/include/tcl/tcl.h
-- TCL readline library: /usr/lib/x86_64-linux-gnu/libtclreadline.so
-- TCL readline header: /usr/include/x86_64-linux-gnu/tclreadline.h
-- CUDD library: /usr/local/lib/libcudd.a
-- CUDD header: /usr/local/include/cudd.h
-- SSTA: 0
-- Found SWIG: /usr/local/bin/swig (found suitable version "4.1.0", minimum required is "3.0")  
-- STA executable: /home/mpoviser/repos/openroad-flow-scripts/tools/OpenROAD/src/sta/app/sta
-- Found re2: /opt/or-tools/lib/cmake/re2/re2Config.cmake (found version "9.0.0") 
-- Found Clp: /opt/or-tools/lib/cmake/Clp/ClpConfig.cmake (found version "1.17.7") 
-- Found Cbc: /opt/or-tools/lib/cmake/Cbc/CbcConfig.cmake (found version "2.10.7") 
-- Found SCIP: /opt/or-tools/lib/cmake/scip/scip-config.cmake (found version "8.0.1") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- Found OR-Tools: /opt/or-tools/lib/cmake/ortools (version: 9.5.2237)
-- TCL library: /usr/lib/x86_64-linux-gnu/libtcl.so
-- TCL header: /usr/include/tcl/tcl.h
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- Found OpenMP: TRUE (found version "4.5")  
-- GUI is not enabled
-- Found Boost: /usr/local/lib/cmake/Boost-1.80.0/BoostConfig.cmake (found version "1.80.0") found components: serialization 
-- Could NOT find VTune (missing: VTune_LIBRARIES VTune_INCLUDE_DIRS) 
-- Found Boost: /usr/local/lib/cmake/Boost-1.80.0/BoostConfig.cmake (found suitable version "1.80.0", minimum required is "1.78")  
-- TCL library: /usr/lib/x86_64-linux-gnu/libtcl.so
-- TCL header: /usr/include/tcl/tcl.h
-- Found Boost: /usr/local/lib/cmake/Boost-1.80.0/BoostConfig.cmake (found version "1.80.0") found components: serialization system thread 
-- Found Boost: /usr/local/lib/cmake/Boost-1.80.0/BoostConfig.cmake (found version "1.80.0")  
-- Found Eigen3: /usr/local/share/eigen3/cmake/Eigen3Config.cmake (found version "3.4.1") 
-- TCL readline enabled
-- Tcl Extended disabled
-- Python3 enabled
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/tmp.ghacZKCkuQ

To Reproduce

This is taken from the CI run https://jenkins.openroad.tools/job/OpenROAD-flow-scripts-Public/job/PR-2657-head/

Download the attachment below (it's not a video, I've changed the extension to bypass github upload limits), rename it to a .tar.gz archive, and unpack.

To reproduce the placer divergence, run

(source vars-bp_be-nangate45-base.sh; openroad scripts/global_place.tcl)
final_report_bp_be_nangate45_base_2025-01-06_13-57.mp4

Relevant log output



Screenshots


No response


Additional Context


No response

      		
    

      
  





        



            

              
            

        

      


      
    








      

    



      

  
            


    

    

      
    

    


      

          @maliberty
maliberty




          added
  the 

  gpl

  Global Placement
 label


      Jan 16, 2025

    

  








      

  
      



  

  @maliberty





  


    

      

  

    

        
    
    

  


      
          
  
      
            Copy link

  

      
    

  


  

      

  
  Member


      

  


  

    

  





      


        


  
    

      
          
I don't see a way to download the "video" as it just appears as a player.  Is the original file in Jenkins?  If not you can put it in gdrive and put a link here.

      		
    

  





        


            

            

              
            

        

      


      

            
  

    
  
    
    

  

  



    












      

  
      



  

  @povik





  


    

      

  

    

        
    
    

  


      
          
  
      
            Copy link

  

      
    

  


  

      

  
    Contributor


      

  Author


  


  

    

      

      
            povik
  

      

      

      commented


        Jan 16, 2025


      
    


  





      


        


  
    

      
          
I have a right click -> "save video as" option (Firefox), in any case here's a Jenkins link: https://jenkins.openroad.tools/job/OpenROAD-flow-scripts-Public/job/PR-2657-head/1/artifact/flow/final_report_bp_be_nangate45_base_2025-01-06_13-57.tar.gz


The reason I wanted to upload it into the issue is I am not sure about the persistence of Jenkins artifacts

      		
    

  





        


            

            

              
            

        

      


      

            
  

    
  
    
    

  

  



    












      

  
      



  

  @maliberty





  


    

      

  

    

        
    
    

  


      
          
  
      
            Copy link

  

      
    

  


  

      

  
  Member


      

  


  

    

  





      


        


  
    

      
          
As long as the PR is open the artifacts should persist.

      		
    

  





        


            

            

              
            

        

      


      

            
  

    
  
    
    

  

  



    












      

  
      



  

  @gudeh





  


    

      

  

    

        
    
    

  


      
          
  
      
            Copy link

  

      
    

  


  

      

  
    Contributor


      

  


  

    

      

      
            gudeh
  

      

      

      commented


        Jan 16, 2025


      
    


  





      


        


  
    

      
          
Never mind my previous message. I didn't noticed I used a modified branch for that run. Here is what we get the moment the divergence is detected. Timing-driven and routability-driven modes happen separately, as they should. I will investigate further.


Image

      		
    

  





        


            

            

              
            

        

      


      

            
  

    
  
    
    

  

  



    












      

  
      



  

  @gudeh





  


    

      

  

    

        
    
    

  


      
          
  
      
            Copy link

  

      
    

  


  

      

  
    Contributor


      

  


  

    

      

      
            gudeh
  

      

      

      commented


        Jan 17, 2025


      
  •

  

    
      

        
          edited
          
        
        
      

    
    
      
        
      Loading

      
    
  



    


  





      


        


  
    

      
          
Hi! Issue #6465 also has a similar divergence on megaboom. We thought of attempting a workaround to save a snapshot prior to the divergence and instead of throwing an error, finish with the saved snapshot.


The divergence is interesting because it usually goes on reducing overflow, although the HPWL increases too much. I saved the snapshot storing the min HPWL if the overflow is below 0.25, and finished with the stored snapshot when the divergence is detected.


For this BP_BE it worked nicely. It finished with the following placement, and successfully finished DRT.


Image


[NesterovSolve] Iter: 1460 overflow: 0.186 HPWL: 4451466576
[NesterovSolve] Iter: 1470 overflow: 0.204 HPWL: 5216452376
[NesterovSolve] Iter: 1480 overflow: 0.183 HPWL: 7694873000
[NesterovSolve] Iter: 1490 overflow: 0.180 HPWL: 9198722870
[NesterovSolve] Iter: 1500 overflow: 0.215 HPWL: 12663756677
[NesterovSolve] Iter: 1510 overflow: 0.232 HPWL: 15009139831
[NesterovSolve] Iter: 1520 overflow: 0.196 HPWL: 4578771851
[NesterovSolve] Iter: 1530 overflow: 0.206 HPWL: 5237016833
[NesterovSolve] Iter: 1540 overflow: 0.226 HPWL: 6824969538
[NesterovSolve] Iter: 1550 overflow: 0.199 HPWL: 8403828118
[NesterovSolve] Iter: 1560 overflow: 0.208 HPWL: 10136766760
[NesterovSolve] Iter: 1570 overflow: 0.202 HPWL: 11161608369
Divergence detected, reverting to snapshot with min hpwl.
Revert to iter: 1453 overflow: 0.201 HPWL: 4099702111



metadata:


[WARN] Overwriting Tag finish__runtime__total
[WARN] Overwriting Tag finish__cpu__total
[WARN] Overwriting Tag finish__mem__peak
[INFO] synth__design__instance__area__stdcell pass test: 223673.016 <= 271352.58
[INFO] constraints__clocks__count pass test: 1.0 == 1.0
[INFO] placeopt__design__instance__area pass test: 256329.0 <= 298541.0
[INFO] placeopt__design__instance__count__stdcell pass test: 51456.0 <= 66169.0
[INFO] detailedplace__design__violations pass test: 0.0 == 0.0
[INFO] cts__design__instance__count__setup_buffer pass test: 64.0 <= 5754.0
[INFO] cts__design__instance__count__hold_buffer pass test: 179.0 <= 5754.0
[INFO] globalroute__antenna_diodes_count pass test: 0.0 <= 0.0
[INFO] detailedroute__route__wirelength pass test: 2943939.0 <= 3494576.0
[INFO] detailedroute__route__drc_errors pass test: 0.0 <= 0.0
[INFO] detailedroute__antenna__violating__nets pass test: 0.0 <= 0.0
[INFO] detailedroute__antenna_diodes_count pass test: 0.0 <= 0.0
[INFO] finish__timing__setup__ws pass test: -0.311452 >= -0.99
[INFO] finish__design__instance__area pass test: 260548.0 <= 302367.0
[INFO] finish__timing__drv__setup_violation_count pass test: 110.0 <= 2877.0
[INFO] finish__timing__drv__hold_violation_count pass test: 56.0 <= 775.0
[INFO] finish__timing__wns_percent_delay pass test: -13.476098 >= -46.18
All metadata rules passed (17 rules)


      		
    

  





        


            

            

              
            

        

      


      

            
  

    
  
    
    

  

  



    












      

  
          

  

    
  


  

      

    @gudeh
gudeh



    mentioned this issue
    
      Jan 21, 2025
    





  


  

      
   Merged

  







  




  

  

    
  


  

      

    @openroad-ci
openroad-ci



    mentioned this issue
    
      Jan 21, 2025
    





  


  

      
   Closed

  







  









      

  
      



  

  @gudeh





  


    

      

  

    

        
    
    

  


      
          
  
      
            Copy link

  

      
    

  


  

      

  
    Contributor


      

  


  

    

      

      
            gudeh
  

      

      

      commented


        Jan 27, 2025


      
    


  





      


        


  
    

      
          
I tested @povik 's branch with latest master and the flow went successfully for the reported design, so I am closing the issue.

      		
    

  





        


            

            

              
            

        

      


      

            
  

    
  
    
    

  

  



    












      

  
            

    

      
    

    


      

          @gudeh
gudeh




            closed this as completed


      Jan 27, 2025

    

  


    







      

  
      



  

  @povik





  


    

      

  

    

        
    
    

  


      
          
  
      
            Copy link

  

      
    

  


  

      

  
    Contributor


      

  Author


  


  

    

      

      
            povik
  

      

      

      commented


        Jan 27, 2025


      
    


  





      


        


  
    

      
          
Thanks Augusto!

      		
    

  





        


            

            

              
            

        

      


      

            
  

    
  
    
    

  

  



    
















  
  

    

      

  





  




  
  

              

  
    Sign up for free
    to join this conversation on GitHub.
    Already have an account?
    Sign in to comment


  



  






  
                  




    

  


      
  

    Assignees
  



        

      

        

  
      @gudeh
  
    gudeh
    

      








      


  


  

    Labels
  



    

      

    gpl

  Global Placement








      

  

    

        

    Projects
  


        




    None yet




  



      


  

    
  

    Milestone
  


      No milestone





    
      
          
  

    

      

        
          

            
    

    Development
  




              
    
No branches or pull requests







        
      

    

  


      

    

  
      

  

    

      3 participants
    

    

        
          @povik 
        
          @maliberty 
        
          @gudeh