Day24_Microsoft-Phi2-LLM-Fine-Tuning-QLORA.ipynb

{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":1192499,"sourceType":"datasetVersion","datasetId":622510}],"dockerImageVersionId":30648,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"## This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://static.wixstatic.com/media/3eee0b_4d2513facc0f4a1cb21b2b8f4e1df179~mv2.jpg/v1/fill/w_1080,h_720,al_c,q_85,enc_auto/3eee0b_4d2513facc0f4a1cb21b2b8f4e1df179~mv2.jpg)","metadata":{}},{"cell_type":"markdown","source":"Phi-2 is a Transformer with 2.7 billion parameters. It was trained using the same data sources as Phi-1.5, augmented with a new data source that consists of various NLP synthetic texts and filtered websites (for safety and educational value). When assessed against benchmarks testing common sense, language understanding, and logical reasoning, Phi-2 showcased a nearly state-of-the-art performance among models with less than 13 billion parameters.\n\n- Model Link - https://huggingface.co/microsoft/phi-2","metadata":{}},{"cell_type":"code","source":"!pip install -U trl accelerate peft bitsandbytes transformers trl huggingface_hub","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%pip install -U datasets","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Loading the Libraries","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport os\nfrom tqdm import tqdm\nimport bitsandbytes as bnb\nimport torch\nimport torch.nn as nn\nimport transformers\nfrom datasets import Dataset\nfrom peft import LoraConfig, PeftConfig\nfrom trl import SFTTrainer\nfrom transformers import (AutoModelForCausalLM, \n                          AutoTokenizer, \n                          BitsAndBytesConfig, \n                          TrainingArguments, \n                          pipeline, \n                          logging)\nfrom sklearn.metrics import (accuracy_score, \n                             classification_report, \n                             confusion_matrix)\nfrom sklearn.model_selection import train_test_split","metadata":{"execution":{"iopub.status.busy":"2024-02-19T09:31:59.263177Z","iopub.execute_input":"2024-02-19T09:31:59.263424Z","iopub.status.idle":"2024-02-19T09:32:19.917656Z","shell.execute_reply.started":"2024-02-19T09:31:59.263402Z","shell.execute_reply":"2024-02-19T09:32:19.916741Z"},"trusted":true},"execution_count":1,"outputs":[{"name":"stderr","text":"2024-02-19 09:32:09.036402: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n2024-02-19 09:32:09.036526: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n2024-02-19 09:32:09.158753: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n","output_type":"stream"}]},{"cell_type":"markdown","source":"- **accelerate** is a distributed training library for PyTorch by HuggingFace. It allows you to train your models on multiple GPUs or CPUs in parallel (distributed configurations), which can significantly speed up training in presence of multiple GPUs (we won't use it in our example).\n- **peft** is a Python library by HuggingFace for efficient adaptation of pre-trained language models (PLMs) to various downstream applications without fine-tuning all the model's parameters. PEFT methods only fine-tune a small number of (extra) model parameters, thereby greatly decreasing the computational and storage costs.\n- **bitsandbytes** by Tim Dettmers, is a lightweight wrapper around CUDA custom functions, in particular 8-bit optimizers, matrix multiplication (LLM.int8()), and quantization functions. It allows to run models stored in 4-bit precision: while 4-bit bitsandbytes stores weights in 4-bits, the computation still happens in 16 or 32-bit and here any combination can be chosen (float16, bfloat16, float32, and so on).\n- **transformers** is a Python library for natural language processing (NLP). It provides a number of pre-trained models for NLP tasks such as text classification, question answering, and machine translation.\n- **trl** is a full stack library by HuggingFace providing a set of tools to train transformer language models with Reinforcement Learning, from the Supervised Fine-tuning step (SFT), Reward Modeling step (RM) to the Proximal Policy Optimization (PPO) step.","metadata":{}},{"cell_type":"markdown","source":"#### Loading the Data\nDataset Link - https://www.kaggle.com/datasets/ankurzing/sentiment-analysis-for-financial-news/data\n\nThe FinancialPhraseBank dataset is a comprehensive collection that captures the sentiments of financial news headlines from the viewpoint of a retail investor. Comprising two key columns, namely \"Sentiment\" and \"News Headline,\" the dataset effectively classifies sentiments as either negative, neutral, or positive. ","metadata":{}},{"cell_type":"code","source":"filename = \"/kaggle/input/sentiment-analysis-for-financial-news/all-data.csv\"\n\ndf = pd.read_csv(filename, \n                 names=[\"sentiment\", \"text\"],\n                 encoding=\"utf-8\", encoding_errors=\"replace\")\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2024-02-19T09:32:19.919359Z","iopub.execute_input":"2024-02-19T09:32:19.919808Z","iopub.status.idle":"2024-02-19T09:32:19.961933Z","shell.execute_reply.started":"2024-02-19T09:32:19.919775Z","shell.execute_reply":"2024-02-19T09:32:19.961085Z"},"trusted":true},"execution_count":2,"outputs":[{"execution_count":2,"output_type":"execute_result","data":{"text/plain":"  sentiment                                               text\n0   neutral  According to Gran , the company has no plans t...\n1   neutral  Technopolis plans to develop in stages an area...\n2  negative  The international electronic industry company ...\n3  positive  With the new production plant the company woul...\n4  positive  According to the company 's updated strategy f...","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>sentiment</th>\n      <th>text</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>neutral</td>\n      <td>According to Gran , the company has no plans t...</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>neutral</td>\n      <td>Technopolis plans to develop in stages an area...</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>negative</td>\n      <td>The international electronic industry company ...</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>positive</td>\n      <td>With the new production plant the company woul...</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>positive</td>\n      <td>According to the company 's updated strategy f...</td>\n    </tr>\n  </tbody>\n</table>\n</div>"},"metadata":{}}]},{"cell_type":"markdown","source":"#### Data Pre-Processing","metadata":{}},{"cell_type":"markdown","source":"- Splits the dataset into training and test sets, with 300 samples in each set. The split is stratified by sentiment, so that each set contains a representative sample of positive, neutral, and negative sentiments.\n","metadata":{}},{"cell_type":"code","source":"X_train = list()\nX_test = list()\nfor sentiment in [\"positive\", \"neutral\", \"negative\"]:\n    train, test  = train_test_split(df[df.sentiment==sentiment], \n                                    train_size=300,\n                                    test_size=300, \n                                    random_state=42)\n    X_train.append(train)\n    X_test.append(test)\n\nX_train[:1]","metadata":{"execution":{"iopub.status.busy":"2024-02-19T09:32:42.733083Z","iopub.execute_input":"2024-02-19T09:32:42.733456Z","iopub.status.idle":"2024-02-19T09:32:42.757511Z","shell.execute_reply.started":"2024-02-19T09:32:42.733425Z","shell.execute_reply":"2024-02-19T09:32:42.756432Z"},"trusted":true},"execution_count":3,"outputs":[{"execution_count":3,"output_type":"execute_result","data":{"text/plain":"[     sentiment                                               text\n 228   positive  ( ADP News ) - Feb 12 , 2009 - Finnish IT solu...\n 835   positive  Finnish consulting and engineering group Poyry...\n 991   positive  These companies will be able to keep their mar...\n 2241  positive  The sale will lead to a pretax capital gain of...\n 13    positive  Finnish Talentum reports its operating profit ...\n ...        ...                                                ...\n 1761  positive  SysOpen Digia Plc , Press release , 7 February...\n 234   positive  Neste Oil Corp. has signed long-term procureme...\n 2294  positive  Outokumpu of Finland , stainless steel manufac...\n 55    positive  Shares of Nokia Corp. rose Thursday after the ...\n 745   positive  `` In terms of profitability and earnings 2007...\n \n [300 rows x 2 columns]]"},"metadata":{}}]},{"cell_type":"markdown","source":"- Shuffles the train data in a replicable order (random_state=10)","metadata":{}},{"cell_type":"code","source":"X_train = pd.concat(X_train).sample(frac=1, random_state=10)\nX_test = pd.concat(X_test)\nX_train[:1]","metadata":{"execution":{"iopub.status.busy":"2024-02-19T09:32:43.779318Z","iopub.execute_input":"2024-02-19T09:32:43.780197Z","iopub.status.idle":"2024-02-19T09:32:43.792773Z","shell.execute_reply.started":"2024-02-19T09:32:43.780164Z","shell.execute_reply":"2024-02-19T09:32:43.791686Z"},"trusted":true},"execution_count":4,"outputs":[{"execution_count":4,"output_type":"execute_result","data":{"text/plain":"     sentiment                                               text\n3683   neutral  Mr Jortikka is president of the base metal div...","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>sentiment</th>\n      <th>text</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>3683</th>\n      <td>neutral</td>\n      <td>Mr Jortikka is president of the base metal div...</td>\n    </tr>\n  </tbody>\n</table>\n</div>"},"metadata":{}}]},{"cell_type":"markdown","source":"### Evaluation or Validation Data","metadata":{}},{"cell_type":"code","source":"eval_idx = [idx for idx in df.index if idx not in list(train.index) + list(test.index)]\neval_idx[:5]","metadata":{"execution":{"iopub.status.busy":"2024-02-19T09:32:44.074001Z","iopub.execute_input":"2024-02-19T09:32:44.074600Z","iopub.status.idle":"2024-02-19T09:32:44.506079Z","shell.execute_reply.started":"2024-02-19T09:32:44.074551Z","shell.execute_reply":"2024-02-19T09:32:44.505152Z"},"trusted":true},"execution_count":5,"outputs":[{"execution_count":5,"output_type":"execute_result","data":{"text/plain":"[0, 1, 3, 4, 5]"},"metadata":{}}]},{"cell_type":"code","source":"X_eval = df[df.index.isin(eval_idx)]\nX_eval[:5]","metadata":{"execution":{"iopub.status.busy":"2024-02-19T09:32:45.570443Z","iopub.execute_input":"2024-02-19T09:32:45.570806Z","iopub.status.idle":"2024-02-19T09:32:45.587973Z","shell.execute_reply.started":"2024-02-19T09:32:45.570779Z","shell.execute_reply":"2024-02-19T09:32:45.586949Z"},"trusted":true},"execution_count":6,"outputs":[{"execution_count":6,"output_type":"execute_result","data":{"text/plain":"  sentiment                                               text\n0   neutral  According to Gran , the company has no plans t...\n1   neutral  Technopolis plans to develop in stages an area...\n3  positive  With the new production plant the company woul...\n4  positive  According to the company 's updated strategy f...\n5  positive  FINANCING OF ASPOCOMP 'S GROWTH Aspocomp is ag...","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>sentiment</th>\n      <th>text</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>neutral</td>\n      <td>According to Gran , the company has no plans t...</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>neutral</td>\n      <td>Technopolis plans to develop in stages an area...</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>positive</td>\n      <td>With the new production plant the company woul...</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>positive</td>\n      <td>According to the company 's updated strategy f...</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>positive</td>\n      <td>FINANCING OF ASPOCOMP 'S GROWTH Aspocomp is ag...</td>\n    </tr>\n  </tbody>\n</table>\n</div>"},"metadata":{}}]},{"cell_type":"code","source":"X_eval = (X_eval\n          .groupby('sentiment', group_keys=False)\n          .apply(lambda x: x.sample(n=50, random_state=10, replace=True)))\nX_train = X_train.reset_index(drop=True)","metadata":{"execution":{"iopub.status.busy":"2024-02-19T09:32:48.301740Z","iopub.execute_input":"2024-02-19T09:32:48.302145Z","iopub.status.idle":"2024-02-19T09:32:48.321646Z","shell.execute_reply.started":"2024-02-19T09:32:48.302117Z","shell.execute_reply":"2024-02-19T09:32:48.320450Z"},"trusted":true},"execution_count":7,"outputs":[{"name":"stderr","text":"/tmp/ipykernel_81/3968059888.py:3: DeprecationWarning: DataFrameGroupBy.apply operated on the grouping columns. This behavior is deprecated, and in a future version of pandas the grouping columns will be excluded from the operation. Either pass `include_groups=False` to exclude the groupings or explicitly select the grouping columns after groupby to silence this warning.\n  .apply(lambda x: x.sample(n=50, random_state=10, replace=True)))\n","output_type":"stream"}]},{"cell_type":"markdown","source":"### Transform Data into LLM Training Form using Dataset Library","metadata":{}},{"cell_type":"code","source":"def generate_prompt(data_point):\n    return f\"\"\"The sentiment of the following phrase: '{data_point[\"text\"]}' is \n            \\n\\n Positive\n            \\n Negative\n            \\n Neutral\n            \\n Cannot be determined\n            \\n\\nSolution: The correct option is {data_point[\"sentiment\"]}\"\"\".strip()\n\ndef generate_test_prompt(data_point):\n    return f\"\"\"The sentiment of the following phrase: '{data_point[\"text\"]}' is \n            \\n\\n Positive\n            \\n Negative\n            \\n Neutral\n            \\n Cannot be determined\n            \\n\\nSolution: The correct option is\"\"\".strip()","metadata":{"execution":{"iopub.status.busy":"2024-02-19T09:32:50.280720Z","iopub.execute_input":"2024-02-19T09:32:50.281080Z","iopub.status.idle":"2024-02-19T09:32:50.287019Z","shell.execute_reply.started":"2024-02-19T09:32:50.281055Z","shell.execute_reply":"2024-02-19T09:32:50.285765Z"},"trusted":true},"execution_count":8,"outputs":[]},{"cell_type":"code","source":"X_train = pd.DataFrame(X_train.apply(generate_prompt, axis=1), \n                       columns=[\"text\"])\nX_eval = pd.DataFrame(X_eval.apply(generate_prompt, axis=1), \n                      columns=[\"text\"])\n\nX_train.head()","metadata":{"execution":{"iopub.status.busy":"2024-02-19T09:32:51.167782Z","iopub.execute_input":"2024-02-19T09:32:51.168134Z","iopub.status.idle":"2024-02-19T09:32:51.195915Z","shell.execute_reply.started":"2024-02-19T09:32:51.168108Z","shell.execute_reply":"2024-02-19T09:32:51.194841Z"},"trusted":true},"execution_count":9,"outputs":[{"execution_count":9,"output_type":"execute_result","data":{"text/plain":"                                                text\n0  The sentiment of the following phrase: 'Mr Jor...\n1  The sentiment of the following phrase: 'Both o...\n2  The sentiment of the following phrase: 'Finnis...\n3  The sentiment of the following phrase: 'Renzo ...\n4  The sentiment of the following phrase: '`` We ...","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>text</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>The sentiment of the following phrase: 'Mr Jor...</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>The sentiment of the following phrase: 'Both o...</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>The sentiment of the following phrase: 'Finnis...</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>The sentiment of the following phrase: 'Renzo ...</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>The sentiment of the following phrase: '`` We ...</td>\n    </tr>\n  </tbody>\n</table>\n</div>"},"metadata":{}}]},{"cell_type":"code","source":"y_true = X_test.sentiment\nX_test = pd.DataFrame(X_test.apply(generate_test_prompt, axis=1), columns=[\"text\"])\n\ntrain_data = Dataset.from_pandas(X_train)\neval_data = Dataset.from_pandas(X_eval)","metadata":{"execution":{"iopub.status.busy":"2024-02-19T09:32:52.139690Z","iopub.execute_input":"2024-02-19T09:32:52.140563Z","iopub.status.idle":"2024-02-19T09:32:52.171489Z","shell.execute_reply.started":"2024-02-19T09:32:52.140529Z","shell.execute_reply":"2024-02-19T09:32:52.170708Z"},"trusted":true},"execution_count":10,"outputs":[]},{"cell_type":"code","source":"train_data['text'][1]","metadata":{"execution":{"iopub.status.busy":"2024-02-19T09:32:52.388156Z","iopub.execute_input":"2024-02-19T09:32:52.388504Z","iopub.status.idle":"2024-02-19T09:32:52.396227Z","shell.execute_reply.started":"2024-02-19T09:32:52.388477Z","shell.execute_reply":"2024-02-19T09:32:52.395303Z"},"trusted":true},"execution_count":11,"outputs":[{"execution_count":11,"output_type":"execute_result","data":{"text/plain":"\"The sentiment of the following phrase: 'Both operating profit and net sales for the 12-month period increased , respectively from EUR21 .5 m and EUR196 .1 m , as compared to 2005 .' is \\n            \\n\\n Positive\\n            \\n Negative\\n            \\n Neutral\\n            \\n Cannot be determined\\n            \\n\\nSolution: The correct option is positive\""},"metadata":{}}]},{"cell_type":"markdown","source":"Next we create a function to evaluate the results from our fine-tuned sentiment model. The function performs the following steps:\n\n - Maps the sentiment labels to a numerical representation, where 2 represents positive, 1 represents neutral, and 0 represents negative.\n- Calculates the accuracy of the model on the test data.\n- Generates an accuracy report for each sentiment label.\n- Generates a classification report for the model.\n- Generates a confusion matrix for the model.","metadata":{}},{"cell_type":"code","source":"def evaluate(y_true, y_pred):\n    labels = ['positive', 'neutral', 'negative']\n    mapping = {'positive': 2, 'neutral': 1, 'none':1, 'negative': 0}\n    def map_func(x):\n        return mapping.get(x, 1)\n    \n    y_true = np.vectorize(map_func)(y_true)\n    y_pred = np.vectorize(map_func)(y_pred)\n    \n    # Calculate accuracy\n    accuracy = accuracy_score(y_true=y_true, y_pred=y_pred)\n    print(f'Accuracy: {accuracy:.3f}')\n    \n    # Generate accuracy report\n    unique_labels = set(y_true)  # Get unique labels\n    \n    for label in unique_labels:\n        label_indices = [i for i in range(len(y_true)) \n                         if y_true[i] == label]\n        label_y_true = [y_true[i] for i in label_indices]\n        label_y_pred = [y_pred[i] for i in label_indices]\n        accuracy = accuracy_score(label_y_true, label_y_pred)\n        print(f'Accuracy for label {label}: {accuracy:.3f}')\n        \n    # Generate classification report\n    class_report = classification_report(y_true=y_true, y_pred=y_pred)\n    print('\\nClassification Report:')\n    print(class_report)\n    \n    # Generate confusion matrix\n    conf_matrix = confusion_matrix(y_true=y_true, y_pred=y_pred, labels=[0, 1, 2])\n    print('\\nConfusion Matrix:')\n    print(conf_matrix)","metadata":{"execution":{"iopub.status.busy":"2024-02-19T09:32:53.537980Z","iopub.execute_input":"2024-02-19T09:32:53.538343Z","iopub.status.idle":"2024-02-19T09:32:53.548079Z","shell.execute_reply.started":"2024-02-19T09:32:53.538316Z","shell.execute_reply":"2024-02-19T09:32:53.547110Z"},"trusted":true},"execution_count":12,"outputs":[]},{"cell_type":"markdown","source":"### Loading the Model\n\nModel loading and quantization:\n\n- First the code loads the Phi-2 language model from the Hugging Face Hub.\n- Then the code gets the float16 data type from the torch library. This is the data type that will be used for the computations.\n- Next, it creates a BitsAndBytesConfig object with the following settings:\n- load_in_4bit: Load the model weights in 4-bit format.\n- bnb_4bit_quant_type: Use the \"nf4\" quantization type. 4-bit NormalFloat (NF4), is a new data type that is information theoretically optimal for normally distributed weights.\n- bnb_4bit_compute_dtype: Use the float16 data type for computations.\n- bnb_4bit_use_double_quant: Do not use double quantization (reduces the average memory footprint by quantizing also the quantization constants and saves an additional 0.4 bits per parameter.).\n\nThen the code creates a AutoModelForCausalLM object from the pre-trained Phi-2 language model, using the BitsAndBytesConfig object for quantization.\n- After that, the code disables caching for the model.\n- Finally the code sets the pre-training token probability to 1.","metadata":{}},{"cell_type":"code","source":"model_name = \"microsoft/phi-2\"\n\ncompute_dtype = getattr(torch, \"float16\")\n\n# Model Configurations\nbnb_config = BitsAndBytesConfig(\n    load_in_4bit=True,\n    bnb_4bit_use_double_quant=False,\n    bnb_4bit_quant_type=\"nf4\",\n    bnb_4bit_compute_dtype=compute_dtype,\n)\n\n# Loading the Model\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_name,\n    trust_remote_code=True,\n    device_map=\"auto\",\n    quantization_config=bnb_config, \n)\n\nmodel.config.use_cache = False\nmodel.config.pretraining_tp = 1\n\n# Loading the Tokenizer\ntokenizer = AutoTokenizer.from_pretrained(model_name, \n                                          trust_remote_code=True,\n                                         )\ntokenizer.pad_token = tokenizer.eos_token","metadata":{"execution":{"iopub.status.busy":"2024-02-19T09:32:54.327730Z","iopub.execute_input":"2024-02-19T09:32:54.328069Z","iopub.status.idle":"2024-02-19T09:33:27.915330Z","shell.execute_reply.started":"2024-02-19T09:32:54.328041Z","shell.execute_reply":"2024-02-19T09:33:27.914553Z"},"trusted":true},"execution_count":13,"outputs":[{"output_type":"display_data","data":{"text/plain":"config.json:   0%|          | 0.00/863 [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"012d2e7a4cdd461eb333d2e3eb020208"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"configuration_phi.py:   0%|          | 0.00/9.26k [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"a4e6881360af446ca3c52d632b088f43"}},"metadata":{}},{"name":"stderr","text":"A new version of the following files was downloaded from https://huggingface.co/microsoft/phi-2:\n- configuration_phi.py\n. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.\n","output_type":"stream"},{"output_type":"display_data","data":{"text/plain":"modeling_phi.py:   0%|          | 0.00/62.7k [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"8b88e4d0fac84127b8162c59475a9cf8"}},"metadata":{}},{"name":"stderr","text":"A new version of the following files was downloaded from https://huggingface.co/microsoft/phi-2:\n- modeling_phi.py\n. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.\n","output_type":"stream"},{"output_type":"display_data","data":{"text/plain":"model.safetensors.index.json:   0%|          | 0.00/35.7k [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"63528e47704e4e9c8ae9cf392ff556ff"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Downloading shards:   0%|          | 0/2 [00:00<?, ?it/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"163dbc4b53b041debd30d56f8a560928"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"model-00001-of-00002.safetensors:   0%|          | 0.00/5.00G [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"3e23ecd806764970872d2c6364e8f477"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"model-00002-of-00002.safetensors:   0%|          | 0.00/564M [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"10064123cd9a4778863ef1cf065b2048"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"6198e8b2a4974a23befa557d5c546059"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"generation_config.json:   0%|          | 0.00/124 [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"3b4e806704f649dcb47b3c9a3176333e"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"tokenizer_config.json:   0%|          | 0.00/7.34k [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"85e1722f44ab4c68843161bedfbb3b3c"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"vocab.json:   0%|          | 0.00/798k [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"db258de0c0114c169c5015345ea30147"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"merges.txt:   0%|          | 0.00/456k [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"932ce5aec0064968b07dd417cc679229"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"tokenizer.json:   0%|          | 0.00/2.11M [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"dd7132eb30004fbf930047462540fdc3"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"added_tokens.json:   0%|          | 0.00/1.08k [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"60a8372b2c774f5999a286f23a9d22cd"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"special_tokens_map.json:   0%|          | 0.00/99.0 [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"31190ae6ff06422787b781c9c0bbaa4b"}},"metadata":{}},{"name":"stderr","text":"Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n","output_type":"stream"}]},{"cell_type":"markdown","source":"In the next cell, we set a function for predicting the sentiment of a news headline using the Phi-2 language model. The function takes three arguments:\n\n- test: A Pandas DataFrame containing the news headlines to be predicted. \n- model: The pre-trained Phi-2 language model. \n- tokenizer: The tokenizer for the Phi-2 language model.\n\nThe pipeline() function from the Hugging Face Transformers library is used to generate text from the language model. The task argument specifies that the task is text generation. The model and tokenizer arguments specify the pre-trained Phi-2 language model and the tokenizer for the language model. The max_new_tokens argument specifies the maximum number of new tokens to generate. The temperature argument controls the randomness of the generated text. A lower temperature will produce more predictable text, while a higher temperature will produce more creative and unexpected text.","metadata":{}},{"cell_type":"code","source":"def predict(X_test, model, tokenizer):\n    y_pred = []\n    for i in tqdm(range(len(X_test))):\n        prompt = X_test.iloc[i][\"text\"]\n        pipe = pipeline(task=\"text-generation\", \n                        model=model, \n                        tokenizer=tokenizer,\n                        max_new_tokens = 3, \n                        temperature = 0.0,\n                       )\n        result = pipe(prompt, pad_token_id=pipe.tokenizer.eos_token_id)\n        answer = result[0]['generated_text'].split(\"The correct option is\")[-1].lower()\n        if \"positive\" in answer:\n            y_pred.append(\"positive\")\n        elif \"negative\" in answer:\n            y_pred.append(\"negative\")\n        elif \"neutral\" in answer:\n            y_pred.append(\"neutral\")\n        else:\n            y_pred.append(\"none\")\n    return y_pred","metadata":{"execution":{"iopub.status.busy":"2024-02-19T09:33:27.916986Z","iopub.execute_input":"2024-02-19T09:33:27.917262Z","iopub.status.idle":"2024-02-19T09:33:27.924422Z","shell.execute_reply.started":"2024-02-19T09:33:27.917237Z","shell.execute_reply":"2024-02-19T09:33:27.923597Z"},"trusted":true},"execution_count":14,"outputs":[]},{"cell_type":"code","source":"y_pred = predict(X_test, model, tokenizer)","metadata":{"execution":{"iopub.status.busy":"2024-02-19T09:33:27.925559Z","iopub.execute_input":"2024-02-19T09:33:27.925850Z","iopub.status.idle":"2024-02-19T09:36:57.868218Z","shell.execute_reply.started":"2024-02-19T09:33:27.925825Z","shell.execute_reply":"2024-02-19T09:36:57.867324Z"},"trusted":true},"execution_count":15,"outputs":[{"name":"stderr","text":"  0%|          | 0/900 [00:00<?, ?it/s]/opt/conda/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:392: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.0` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`.\n  warnings.warn(\n100%|██████████| 900/900 [03:24<00:00,  4.40it/s]\n","output_type":"stream"}]},{"cell_type":"markdown","source":"#### Lets Evaluate","metadata":{}},{"cell_type":"code","source":"evaluate(y_true, y_pred)","metadata":{"execution":{"iopub.status.busy":"2024-02-19T09:36:57.870367Z","iopub.execute_input":"2024-02-19T09:36:57.870681Z","iopub.status.idle":"2024-02-19T09:36:57.893405Z","shell.execute_reply.started":"2024-02-19T09:36:57.870655Z","shell.execute_reply":"2024-02-19T09:36:57.892454Z"},"trusted":true},"execution_count":16,"outputs":[{"name":"stdout","text":"Accuracy: 0.348\nAccuracy for label 0: 0.050\nAccuracy for label 1: 0.920\nAccuracy for label 2: 0.073\n\nClassification Report:\n              precision    recall  f1-score   support\n\n           0       0.94      0.05      0.09       300\n           1       0.33      0.92      0.49       300\n           2       0.44      0.07      0.13       300\n\n    accuracy                           0.35       900\n   macro avg       0.57      0.35      0.24       900\nweighted avg       0.57      0.35      0.24       900\n\n\nConfusion Matrix:\n[[ 15 280   5]\n [  1 276  23]\n [  0 278  22]]\n","output_type":"stream"}]},{"cell_type":"markdown","source":"# Let's Fine Tune Now","metadata":{}},{"cell_type":"markdown","source":"- We configures and initializes a Simple Fine-tuning Trainer (SFTTrainer) for training a large language model using the Parameter-Efficient Fine-Tuning (PEFT) method.\n- Which should save time as it operates on a reduced number of parameters compared to the model's overall size. \n- The PEFT method focuses on refining a limited set of (additional) model parameters, while keeping the majority of the pre-trained LLM parameters fixed.\n- This significantly reduces both computational and storage expenses. Additionally, this strategy addresses the challenge of catastrophic forgetting, which often occurs during the complete fine-tuning of LLMs.\n\nPEFTConfig:\n\nThe peft_config object specifies the parameters for PEFT. The following are some of the most important parameters:\n\n- lora_alpha: The learning rate for the LoRA update matrices. (Scaling Factor)\n- lora_dropout: The dropout probability for the LoRA update matrices.\n- r: The rank of the LoRA update matrices. (Dimensions of Matrix)\n- bias: The type of bias to use. The possible values are none, additive, and learned.\n- task_type: The type of task that the model is being trained for. The possible values are CAUSAL_LM and MASKED_LM.","metadata":{}},{"cell_type":"markdown","source":"**TrainingArguments:**\n\nThe training_arguments object specifies the parameters for training the model. The following are some of the most important parameters:\n\n- output_dir: The directory where the training logs and checkpoints will be saved.\n- num_train_epochs: The number of epochs to train the model for.\n- per_device_train_batch_size: The number of samples in each batch on each device.\n- gradient_accumulation_steps: The number of batches to accumulate gradients before updating - the model parameters.\n- optim: The optimizer to use for training the model.\n- save_steps: The number of steps after which to save a checkpoint.\n- logging_steps: The number of steps after which to log the training metrics.\n- learning_rate: The learning rate for the optimizer.\n- weight_decay: The weight decay parameter for the optimizer.\n- fp16: Whether to use 16-bit floating-point precision.\n- bf16: Whether to use BFloat16 precision.\n- max_grad_norm: The maximum gradient norm.\n- max_steps: The maximum number of steps to train the model for.\n- warmup_ratio: The proportion of the training steps to use for warming up the learning rate.\n- group_by_length: Whether to group the training samples by length.\n- lr_scheduler_type: The type of learning rate scheduler to use.\n- report_to: The tools to report the training metrics to.\n- evaluation_strategy: The strategy for evaluating the model during training.\n\n**SFTTrainer:**\n\nThe SFTTrainer is a custom trainer class from the PEFT library. It is used to train large language models using the PEFT method.\n\nThe SFTTrainer object is initialized with the following arguments:\n\n- model: The model to be trained.\n- train_dataset: The training dataset.\n- eval_dataset: The evaluation dataset.\n- peft_config: The PEFT configuration.\n- dataset_text_field: The name of the text field in the dataset.\n- tokenizer: The tokenizer to use.\n- args: The training arguments.\n- packing: Whether to pack the training samples.\n- max_seq_length: The maximum sequence length.\n\nOnce the SFTTrainer object is initialized, it can be used to train the model by calling the train() method","metadata":{}},{"cell_type":"code","source":"import re\n\ndef get_num_layers(model):\n    numbers = set()\n    for name, _ in model.named_parameters():\n        for number in re.findall(r'\\d+', name):\n            numbers.add(int(number))\n    return max(numbers)\n\ndef get_last_layer_linears(model):\n    names = []\n    \n    num_layers = get_num_layers(model)\n    for name, module in model.named_modules():\n        if str(num_layers) in name and not \"encoder\" in name:\n            if isinstance(module, torch.nn.Linear):\n                names.append(name)\n    return names","metadata":{"execution":{"iopub.status.busy":"2024-02-19T09:36:57.894529Z","iopub.execute_input":"2024-02-19T09:36:57.894835Z","iopub.status.idle":"2024-02-19T09:36:57.901462Z","shell.execute_reply.started":"2024-02-19T09:36:57.894812Z","shell.execute_reply":"2024-02-19T09:36:57.900504Z"},"trusted":true},"execution_count":17,"outputs":[]},{"cell_type":"code","source":"peft_config = LoraConfig(\n    r=16,\n    lora_alpha=16,\n    target_modules=[\n    \"q_proj\",\n    \"up_proj\",\n    \"o_proj\",\n    \"k_proj\",\n    \"down_proj\",\n    \"gate_proj\",\n    \"v_proj\"],\n    lora_dropout=0.05,\n    bias=\"none\",\n    task_type=\"CAUSAL_LM\",\n)","metadata":{"execution":{"iopub.status.busy":"2024-02-19T09:36:57.902416Z","iopub.execute_input":"2024-02-19T09:36:57.902715Z","iopub.status.idle":"2024-02-19T09:36:57.909876Z","shell.execute_reply.started":"2024-02-19T09:36:57.902690Z","shell.execute_reply":"2024-02-19T09:36:57.908917Z"},"trusted":true},"execution_count":18,"outputs":[]},{"cell_type":"code","source":"training_arguments = TrainingArguments(\n    output_dir=\"logs\",\n    num_train_epochs=1,\n    per_device_train_batch_size=1,\n    gradient_accumulation_steps=8, # 4\n    optim=\"paged_adamw_32bit\",\n    save_steps=0,\n    logging_steps=25,\n    learning_rate=2e-4,\n    weight_decay=0.001,\n    fp16=True,\n    bf16=False,\n    max_grad_norm=0.3,\n    max_steps=-1,\n    warmup_ratio=0.03,\n    group_by_length=True,\n    lr_scheduler_type=\"cosine\",\n    report_to=\"tensorboard\",\n    evaluation_strategy=\"epoch\"\n)\n\ntrainer = SFTTrainer(\n    model=model,\n    train_dataset=train_data,\n    eval_dataset=eval_data,\n    peft_config=peft_config,\n    dataset_text_field=\"text\",\n    tokenizer=tokenizer,\n    args=training_arguments,\n    packing=False,\n    max_seq_length=512,\n)","metadata":{"execution":{"iopub.status.busy":"2024-02-19T09:36:57.910912Z","iopub.execute_input":"2024-02-19T09:36:57.911206Z","iopub.status.idle":"2024-02-19T09:36:58.413441Z","shell.execute_reply.started":"2024-02-19T09:36:57.911182Z","shell.execute_reply":"2024-02-19T09:36:58.412620Z"},"trusted":true},"execution_count":19,"outputs":[{"output_type":"display_data","data":{"text/plain":"Map:   0%|          | 0/900 [00:00<?, ? examples/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"18dda8c925f341ec8d1420ae3cd4ad77"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Map:   0%|          | 0/150 [00:00<?, ? examples/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"5a798cbb5af44a8c99aed7f23f9af6e0"}},"metadata":{}}]},{"cell_type":"code","source":"# Train model\ntrainer.train()\n\n# Save trained model\ntrainer.model.save_pretrained(\"micrsofot_phi2_sentiment_sim-model\")","metadata":{"execution":{"iopub.status.busy":"2024-02-19T09:36:58.414863Z","iopub.execute_input":"2024-02-19T09:36:58.415272Z","iopub.status.idle":"2024-02-19T09:40:54.576948Z","shell.execute_reply.started":"2024-02-19T09:36:58.415240Z","shell.execute_reply":"2024-02-19T09:40:54.575961Z"},"trusted":true},"execution_count":20,"outputs":[{"output_type":"display_data","data":{"text/plain":"<IPython.core.display.HTML object>","text/html":"\n    <div>\n      \n      <progress value='112' max='112' style='width:300px; height:20px; vertical-align: middle;'></progress>\n      [112/112 03:52, Epoch 0/1]\n    </div>\n    <table border=\"1\" class=\"dataframe\">\n  <thead>\n <tr style=\"text-align: left;\">\n      <th>Epoch</th>\n      <th>Training Loss</th>\n      <th>Validation Loss</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <td>0</td>\n      <td>1.420000</td>\n      <td>1.230299</td>\n    </tr>\n  </tbody>\n</table><p>"},"metadata":{}}]},{"cell_type":"code","source":"y_pred = predict(X_test, model, tokenizer)\nevaluate(y_true, y_pred)","metadata":{"execution":{"iopub.status.busy":"2024-02-19T09:40:54.578689Z","iopub.execute_input":"2024-02-19T09:40:54.579337Z","iopub.status.idle":"2024-02-19T09:45:24.690424Z","shell.execute_reply.started":"2024-02-19T09:40:54.579305Z","shell.execute_reply":"2024-02-19T09:45:24.689492Z"},"trusted":true},"execution_count":21,"outputs":[{"name":"stderr","text":"  0%|          | 0/900 [00:00<?, ?it/s]/opt/conda/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:392: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.0` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`.\n  warnings.warn(\n100%|██████████| 900/900 [04:30<00:00,  3.33it/s]","output_type":"stream"},{"name":"stdout","text":"Accuracy: 0.796\nAccuracy for label 0: 0.980\nAccuracy for label 1: 0.553\nAccuracy for label 2: 0.853\n\nClassification Report:\n              precision    recall  f1-score   support\n\n           0       0.91      0.98      0.94       300\n           1       0.81      0.55      0.66       300\n           2       0.69      0.85      0.76       300\n\n    accuracy                           0.80       900\n   macro avg       0.80      0.80      0.79       900\nweighted avg       0.80      0.80      0.79       900\n\n\nConfusion Matrix:\n[[294   4   2]\n [ 19 166 115]\n [ 10  34 256]]\n","output_type":"stream"},{"name":"stderr","text":"\n","output_type":"stream"}]},{"cell_type":"code","source":"evaluation = pd.DataFrame({'text': X_test[\"text\"], \n                           'y_true':y_true, \n                           'y_pred': y_pred},\n                         )\nevaluation.to_csv(\"test_predictions.csv\", index=False)","metadata":{"execution":{"iopub.status.busy":"2024-02-19T09:45:24.692780Z","iopub.execute_input":"2024-02-19T09:45:24.693068Z","iopub.status.idle":"2024-02-19T09:45:24.717323Z","shell.execute_reply.started":"2024-02-19T09:45:24.693044Z","shell.execute_reply":"2024-02-19T09:45:24.716582Z"},"trusted":true},"execution_count":22,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}