Unleashing the Power of OpenAI’s GPT-3: A Guide to Fine-Tuning Your Model in Python
OpenAI has streamlined the process of fine-tuning their GPT-3 models, making it a breeze to tailor them to our specific needs.
What is fine-tuning?
Fine-tuning a machine learning model involves taking a pre-trained model and adjusting its parameters to fit a specific task or dataset. This technique is commonly used in transfer learning, where a model trained on a large and diverse dataset is re-used for a different but related task.
Fine-tuning typically involves adding new layers on top of the pre-trained model and training them on the new data, while also adjusting the existing weights to improve performance on the new task. This allows for faster training and better results than training a model from scratch.
What is OpenAI GPT-3?
GPT-3 stands for Generative Pre-trained Transformer 3, which is an advanced machine learning model developed by OpenAI. It is a language processing model that uses a neural network to generate human-like text based on a given input.
For our project, we will be fine-tuning one of the base models (Davinci, Curie, Babbage, Ada).
Getting started
We will be generating fake data with prompts and outputs using OpenAI API and use them to fine-tune the model.
Prepare training data
You teach GPT-3 what to say by providing it with training data. The data you provide must be a JSONL document containing prompt-completion pairs for each training example. You can easily convert your data into this format using the OpenAI CLI data preparation tool.
In our case, we do not have data. Let’s create some example data using OpenAI Completion API.
import os
import openai
import pandas as pd
from dotenv import load_dotenv
load_dotenv()
openai.api_key = os.getenv("OPENAI_API_KEY")
l_age = ['18', '20', '30', '40', '50', '60', '90']
l_gender = ['man', 'woman']
l_power = ['invisibility', 'read in the thoughts', 'turning lead into gold', 'immortality', 'telepathy', 'teleport', 'flight']
f_prompt = "Imagine a complete and detailed description of a {age}-year-old {gender} fictional character who has the superpower of {power}. Write out the entire description in a maximum of 100 words in great detail:"
f_sub_prompt = "{age}, {gender}, {power}"
df = pd.DataFrame()
for age in l_age:
for gender in l_gender:
for power in l_power:
for i in range(3):
# 3 times each. Each time model will generate different response.
# This is due to temperature=1. See more: https://platform.openai.com/docs/api-reference/completions/create#completions/create-temperature
prompt = f_prompt.format(age=age, gender=gender, power=power)
sub_prompt = f_sub_prompt.format(age=age, gender=gender, power=power)
# Read more about this API: https://platform.openai.com/docs/api-reference/completions/create
response = openai.Completion.create(
model="text-davinci-003",
prompt=prompt,
temperature=1,
max_tokens=500,
top_p=1,
frequency_penalty=0,
presence_penalty=0
)
finish_reason = response['choices'][0]['finish_reason']
response_txt = response['choices'][0]['text']
new_row = {
'age':age,
'gender':gender,
'power':power,
'prompt':prompt,
'sub_prompt':sub_prompt,
'response_txt':response_txt,
'finish_reason':finish_reason}
# Add new row to the table.
new_row = pd.DataFrame([new_row])
df = pd.concat([df, new_row], axis=0, ignore_index=True)
# Data will be stored in out_openai_completion.csv file.
df.to_csv("out_openai_completion.csv")
# Then we convert out_openai_completion.csv to the format that corresponds to
# key-value pair (prompt -> completion).
prepared_data = df.loc[:,['sub_prompt','response_txt']]
prepared_data.rename(columns={'sub_prompt':'prompt', 'response_txt':'completion'}, inplace=True)
prepared_data.to_csv('prepared_data.csv',index=False)Fine-tune the model
Now we have prepared_data.csv to fine-tune our model. Run OpenAI tools and API to fine-tune your model!
## prepared_data.csv --> prepared_data_prepared.jsonl
subprocess.run('openai tools fine_tunes.prepare_data --file prepared_data.csv --quiet'.split())
subprocess.run('openai api fine_tunes.create --training_file prepared_data_prepared.jsonl --model davinci --suffix "SuperHero"'.split())
# The way openai implemented fine-tuning is that it puts your request in the queue.
# You have to wait in line until the request gets processed.
'''
[2023-03-28 15:07:42] Created fine-tune: ft-xxx
[2023-03-28 15:09:39] Fine-tune costs $5.18
[2023-03-28 15:09:40] Fine-tune enqueued
[2023-03-28 15:21:37] Fine-tune is in the queue. Queue number: 31
[2023-03-28 15:22:45] Fine-tune is in the queue. Queue number: 30
[2023-03-28 15:23:15] Fine-tune is in the queue. Queue number: 29
[2023-03-28 15:24:35] Fine-tune is in the queue. Queue number: 28
[2023-03-28 15:25:13] Fine-tune is in the queue. Queue number: 27
[2023-03-28 15:25:14] Fine-tune is in the queue. Queue number: 26
[2023-03-28 15:25:16] Fine-tune is in the queue. Queue number: 25
[2023-03-28 15:54:12] Fine-tune is in the queue. Queue number: 1
[2023-03-28 15:55:50] Fine-tune is in the queue. Queue number: 0
[2023-03-28 15:55:53] Fine-tune started
[2023-03-28 16:00:09] Completed epoch 1/4
[2023-03-28 16:01:50] Completed epoch 2/4
[2023-03-28 16:03:31] Completed epoch 3/4
[2023-03-28 16:05:12] Completed epoch 4/4
[2023-03-28 16:05:54] Uploaded model: davinci:ft-xxx-2023-03-28-xxxx
[2023-03-28 16:05:56] Uploaded result file: file-xxx
[2023-03-28 16:05:56] Fine-tune succeeded
Job complete! Status: succeeded 🎉
Try out your fine-tuned model:
...
'''
I had to wait about 30 minutes until the OpenAI server picks up my request and fine-tuned my model. You can check the progress with this command openai api fine_tunes.follow -i fine-tune-id .
Test your model
You can test your model via playground or run them via API. Unfortunately, you can’t export your model for self-hosting.
Here is how you can run Completion API to test your fine-tuned model.
openai.api_key = os.getenv("OPENAI_API_KEY")
response = openai.Completion.create(
model=os.getenv("MY_FINE_TUNED_MODEL"),
prompt="28, man, can squat 400 pounds ->",
temperature=0.7,
max_tokens=256,
top_p=1,
frequency_penalty=0,
presence_penalty=0,
stop=["END"]
)
'''
....
<OpenAIObject text_completion id=cmpl-xxx at xxx> JSON: {
"choices": [
{
"finish_reason": "stop",
"index": 0,
"logprobs": null,
"text": " \n\nHe stands 6 feet tall with a lean, muscular frame, broad shoulders, and a chiseled jawline. His deep brown eyes carry a certain confidence and maturity. His wavy, dark brown hair is cropped short and styled professionally. He wears a fitted, navy blue t-shirt, slim-fit jeans, and a pair of sneakers. He carries himself with the grace of an athlete - strong, balanced, and powerful. \n\nAs a 28-year-old man with a superpower, he has the ability to squat 400 pounds. He can easily lift four times his own body weight, never breaking a sweat. He can also use his power to help others - regularly donating to charity, volunteering at homeless shelters, and making donations to disaster relief funds. He is a true hero. "
}
],
"created": 1680020829,
"id": "cmpl-xxx",
"model": "davinci:ft-personal:xxx",
"object": "text_completion",
"usage": {
"completion_tokens": 165,
"prompt_tokens": 9,
"total_tokens": 174
}
}
'''