You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Adding the issues I want to work on and some initial doubts:
issue
Doubt/line of thought
Add a HugginFace dataloader to apply IReRa to any multi-label classification task
- Add dataloader such that loader.py logic is changed to accomodate HuggingFace dataloader
Add a cli argument parsing class
- Make the argparse stuff into a class.
Create a config for the Optimizer class
- use dataclasses for defining config.
Make the Optimizer class agnostic of the implementation details of the program that gets optimized, so it can define a high-level strategy which gets compiled across many programs
- generalise methods and abstract away methods so that strategy logic can be implemented at a high level.
Track amount of student and teacher calls during optimization so system-cost can be compared.
- simply track calls to llm
Make src/programs/retriever.py more efficient
- work on logic to optimise retrieving of dataset
Track intermediate pipeline steps and log these traces for debugging
- track each step in IReRa and log it.
Use logging instead of print-statements, and dump logs to experiment file
- Define a logger in loader and experiments to log the states
Control seeds in LM calls and data loaders
- Add seed to optimiser.
I hope that i am thinking in the right direction, I am intrigued by the IReRa style and want to help the project get better.
The text was updated successfully, but these errors were encountered:
Adding the issues I want to work on and some initial doubts:
- Add dataloader such that loader.py logic is changed to accomodate HuggingFace dataloader
- Make the argparse stuff into a class.
- use dataclasses for defining config.
- generalise methods and abstract away methods so that strategy logic can be implemented at a high level.
- simply track calls to llm
src/programs/retriever.py
more efficient- work on logic to optimise retrieving of dataset
- track each step in IReRa and log it.
- Define a logger in loader and experiments to log the states
- Add seed to optimiser.
I hope that i am thinking in the right direction, I am intrigued by the IReRa style and want to help the project get better.
The text was updated successfully, but these errors were encountered: