Improvements to Ablation Paths

[LabChameleon]

1. Minor bug fixes for using Ablation Paths when reading target and source configs from CSV files.
2. Minor improvements for parsing data frame to config.
3. Minor improvements when loading data from CSV files.
4. Ablation paths now support conditional search spaces with a two-level hierarchy and every hyperparameter having only at most one parent.

I think the above improvements are not controversial. More problematic are the following two changes:

1. I introduced a finish_run to the ablation path and the hypersweeper in general. This method allows for computing final statistics after a hypersweeper run is done (e.g. computing figures of the ablation paths or creating a CSV file with the configurations on the ablation path). This change made it necessary to add the finish_run method to every other sweeper as well. I am not sure if this aligns with your code philosophy of the project. What do you think?
2. The ablation path logic makes it necessary that the optimiser can give a termination signal before the number of trials is exhausted. Therefore, I added another optimizer_termination flag returned by the ask method to every sweeper. This allows a sweeper to early stop when it is done, as necessary by the ablation path. This fixes the work around with the error that was raised and catched in the previous implementation.

[TheEimer]

Thanks a lot 🙏

Comments on the "controversial" changes:

1. Additional "finish_run" method: this is not great as for most sweepers it does nothing... Maybe there's a better way of doing this: instead of always calling "finish_run", it could only be called if the method actually exists. I admit, a method check is not great either, but maybe it's just time to include an abstract adapter or something (not a bad idea anyway) that has a notion of callbacks, e.g. a class attribute "self.finish_run_callbacks" that is an empty list by default and would be set to "finish_run" in the ablation path sweeper. The nice thing is that an abstract structure will prevent us from having this discussion again for future changes. I can take a look at that.

2. How is this different from the other termination flag? I think that flag might just not be set correctly in the ablation path sweeper, no?

[LabChameleon]

How is this different from the other termination flag? I think that flag might just not be set correctly in the ablation path sweeper, no?

The hypersweeper currently has two nested loops:

Outer loop:

hypersweeper/hydra_plugins/hypersweeper/hypersweeper_sweeper.py

Lines 447 to 449 in 0515002

	self.optimizer.tell(info=info, value=value)
	while not done:
	opt_time_start = time.time()

Inner loop:

hypersweeper/hydra_plugins/hypersweeper/hypersweeper_sweeper.py

Lines 456 to 458 in 0515002

	terminate = False
	while t < self.max_parallel and not terminate and not trial_termination and not budget_termination:
	try:

Currently, the termination flag is only used to terminate the inner loop, but not the outer loop since the termination flag is not used in computing the done flag

hypersweeper/hydra_plugins/hypersweeper/hypersweeper_sweeper.py

Line 506 in 0515002

done = trial_termination or budget_termination

If this is intentional, we need a second termination flag to terminate the outer loop as well. Or is it a bug and we should simply include the termination flag in computing done?