Posts
Introduction:
Recently I got access to OpenAI API beta. After a few simple experiments, I set on creating a simple test project. In this project, I will try to create good tinder bio for a specific person.
The abc of openai API playground:
In the OpenAI API playground, you get a prompt, and then you can write instructions or specific text to trigger a response from the gpt-3 models. There are also a number of preset templates which loads a specific kind of prompt and let's you generate pre-prepared results.
What are the models available?
There are 4 models which are stable. These are:
(1) curie (2) babbage (3) ada (4) da-vinci
da-vinci is the strongest of them all and can perform all downstream tasks which other models can do.
There are 2 other new models which openai introduced this year (2021) named da-vinci-instruct-beta and curie-instruct-beta.
These instruction models are specifically built for taking in instructions. As OpenAI blog explains and also you will see in our details observation while trying to tune our prompt, instruction models are even better than their normal counter parts to understand and follow instruction type prompts.
Now, if you are familiar with the prompt and zero shot performance concept, you can directly skip the next section. In this small next section, I will briefly explain what is prompt and zero shot learning or few shot learning in this context.
What is a prompt and how it relates with zero shot learning?
A zero shot learning means simply when a model can perform a downstream task without any training or learning for that specific task. In this case, we use the core gpt-3 models, and, we use them to perform zero shot learning(ZSL) tasks.
Where does the prompt comes you ask, the prompt is how the language model is told what to do in this task. The prompt is a specific string or piece of text which tells such a ZSL able model what to do. In the main project part, you will see plenty of examples how it actually works.
Also, in some cases, we give a more elaborate prompt. This kind of prompts will contain a description of the work, and then a number of examples too. This is a case of few shot learning, where the model will not be able to perform outright, but it will be able to do it just after a few simple examples.
The project:
In technical term, this project is an attempt to find out how to tune the parameters of the openai model, which model to choose and also how to exactly write a prompt to get a tinder bio generated from simple information as name, profession and gender.
Result:
At this moment, I have been successfully able to generate tinder biography from simple prompt, containing gender information, name ( partially successful) and also been able to generate various types of biography based on tuned temparature parameters.
Parameters to be tuned:
In OpenAI playground, there are a few parameters. Tuning them really changes the performance of your experiment. Let's review them quickly and in the next part we will manually tune them.
(1) model: among the 6 models, you have to choose one which gives the best performance. Also depending on your task, you would either choose a normal or an instruct task.
(2) temparature: this controls the randomness of the reply the model gives. The more the temparature (0-1), the more it becomes more random, and the lesser it becomes, the reply becomes that much more deterministic and repeatitive.
In generative tasks, it will be better to have a higher temparature to help the creativity; i.e. generative prompt should be able to generate different outputs based on same.
But in deterministic tasks such as classification or best answer from some question, you need to have a relatively lower temparature and generate same response from the same question/classification point.
(3) Top p: this controls the diversity using nucleus sampling. The lower it goes, the lesser number of options are considered. We can set it to 1 to have pretty diverse inclusion of words and results.
(4) Frequency and presence penalty: these are two separate parameters which controls how much the generated text is penalized for reusing same topic and words. While the frequency counts how many times a token has come up in the text so far, presence is a binary feature i.e. whether it has occurred in the text so far or not.
You need to tune these two above carefully. Increasing penalty may cause the outputs spirally out to new concepts. So I have used smaller values for my experiments so far, just to stop repeating verbatim lines. Maybe for a more multi-topic item like a essay or a story or ideas you may want to increase the penalty to introduce multiple new topics.
The experiment:
Now, we will post prompts from the beginning starting with default and all the changes, how they impact and responses recorded will be presented.
(1) first goal: getting a bio generated
First let's try a simple prompt.
prompt: tinder bio for a funny romantic person.
presence penalty: 0.25, response length: 364
Response: ( within the triple quote)
"""
I'm 24, 6'0, 180, athletic build. I have dark hair and blue eyes. I currently own a house and a motorcycle that I ride as often as possible. I'm not looking for a hookup but if it happens great! I'm just not looking to meet women through this app to take them back to my place that night.
I've been in the military for seven years now and am getting ready to commission into the United States Air Force in just two short weeks! It's been a long journey but it's almost over.
I enjoy watching football, playing xbox, going out with friends, trying new restaurants, going on road trips and taking my motorcycle out on the weekends.
"""
so basically, we are getting a combination of multiple bios. Also, almost all male centric response!
Let's try adding a bit of specification in hope of getting one specific bio.
prompt: tinder bio for a funny romantic person from India.
same presence penalty, response length of 100 tokens.
Response:
"""
But what is it about a good tinder bio for a funny romantic person that makes the app so addictive? If you’re not sure what we mean, let us explain.
First of all, the format of Tinder allows for some great one-liners. You don’t need to write paragraphs or describe specifically what you are looking for in a partner. Instead, you can craft a carefully crafted text that will stand out from the crowd and get people reading. This
"""
so when I add india, it turns into some guidance blog? Lets try the same thing with usa.
prompt: tinder bio for a funny romantic person from usa.
parameters same as last.
Response:
"""
i'm very independent and i like to have a lot in life. i like the idea of having my own little home with my family and just being happy.
I love to cook, travel, walk on the beach, read books and listen to music.
I think if I had to describe myself, it would be that I am an honest person that is looking for a man who is honest.
"""
so less male centric.
increasing the response length to 496 and asking directly to write it again creates blogs about the topic.
prompt: write a tinder bio for a funny romantic person from usa.
response length: 496
"""
The Perfect Tinder Bio For A Funny Person | POPSUGAR Love ...
11 Jan 2016 ... I'm looking for a girl who can write a tinder bio for a funny person. I'm not just looking for funny here, but also someone who has good sense of humor. Read more ...
How To Create The Perfect Tinder Bio | Coffee Meets Bagel Blog
7 Feb 2016 ... So you've got your profile photo sorted, your swiping technique perfected, and you're ready to start flirting with all the cute girls on Tinder; but have you...
Perfect Tinder Profile: How To Write The Best Bio | HuffPost
20 Jul 2015 ... All that said, the perfect profile is not easy to create. It goes without saying that the perfect profile is one that makes people want to swipe right--but ...
6 Tips on How to Make Your Online Dating Profile Stand Out and .... This is why it's important to have an eye-catching profile photo up top. But beyond this and your username, there's plenty of room for creativity in your profile text. Here are some tips on how to make your online dating profile stand out from the rest.
"""
same parameters:
prompt: write bumble bio for a funny romantic person from usa.
Response:
"""
man asks internet to write his online dating profile.
dating site bio examples.
6 ways to write your online dating profile.
how to write a great online dating profile examples included.
how to write a good online dating profile examples for men.
online dating tips for men how to write an awesome profile get the girl.
what is the best opening line for a dating site bio quora.
online dating profile examples the right way to portray yourself on.
"""
Upto this, model was da-vinci. It drastically improves performance when you use da-vinci-instruct-beta. starts to produce results I guess.
prompt: write bumble bio for a funny romantic person from usa.
response:
i'm a sarcastic, silly and sometimes serious person. i try not to take life too seriously, and just enjoy it. ive been told that i am blessed with a great sense of humor, so i try to use it as often as possible. ive been told that i am a very loyal friend and that people can always count on me. my favorite thing on earth is cheesecake, but my second favorite is box wine.
But just increasing the 'best of' parameter to 2 breaks it.
For example, creates this:
prompt: write bumble bio for a funny romantic person from usa.
Response:
"""
Her name is Bumble Bee. She is a very funny, romantic person. Her favorite ice cream flavor is chocolate and she loves to eat chips. Her favorite animals are dogs and cats. She likes to read books, watch tv and talk on the phone with her friends.
"""
so it mistakes as bumble bio as bio of a person named bumble. But still is a good bio-like creation.
Ok so this prompt format seems to be correct. And it regularly produces bio like content. Although the shape seems to be a problem.
For example,
Prompt: write a tinder bio for a funny, romantic person from india.
Response:
"""
I'm a funny, romantic person from India looking for something more than "Netflix and Chill."
"""
Important thing is that, here are two issues. it copies a chunk from the prompt. Need to see if it tries to replicate. Also, why the hell the length is so small.
Let's try to increase the bio length to near 120 words ( 500 characters) by increasing token number. Currently it is at 496 tokens; we will increase upto 2048.
So no improvement happens with the length.
A few samples:
"""
Cute, funny, romantic, and totally into you ;)
(I'm not from here but I will love you forever)
"""
"I'm a decent guy who enjoys long walks on the beach and romantic comedies. I can also be found at my favorite dosa place."
"""I love the smell of my own farts. """
"""Oof, I'm an Indian man and this is a really hard question."""
"""I have a good sense of humor and I'm always down for a good time. If you want to know more about me, I'll be happy to send it in a DM."""
"""I'm from India and I'm looking for someone who is down for a good time. """
So as we tried 2048, it didnt let us pass a prompt with that, so I guess you can't really use 2048; as your prompt token also counts. These above were from 1718 token limit; but simply produced a lot less. resetting token limit to 200 to see. also many are not bios, but similarly small sentences. So, the token chunk copy didn't happen in the new thing. Some bios are borderline offensive as well as the last one.
Lets set to 300 and see.
"""
Hi, I'm Akhil. I'm a 27-year-old male who is looking for someone to share a life with. I love traveling and experiencing new cultures. I enjoy spicy food, but don't like seafood.
"""
Comment: is it a generic real looking bio or actually a bio?
"""
I'm a time-traveler, social butterfly, and photographer. I prefer to be called "friend" because I'm not here to take your girl.
"""
Comment: so, it doesn't totally read like a bio actually.
"""
Dedicated to the love of my life.
I'm a Punjabi girl living in India and I love to laugh and be silly.
"""
Comment: atlast a female content.
Now, one thing we can do to reduce the occassional non-bio generations, we can reduce the randomness and check the results again. Let's get temparature from 0.7 down to 0.5; and see the results.
Responses:
1. I'm a software engineer who loves to travel. I'm also a very romantic person.
2.
Hi, I'm a funny, romantic person from India.
I love to laugh and make other people laugh.
I'm looking for someone who has a good sense of humor, loves to travel, and is looking for an adventure.
If you're that person, swipe right!
Comment: chunk repetition noticed. Maybe fluke.
3. I'm a witty, romantic person with a penchant for Indian food and bad puns.
Comment: probably non-indian bio as indians don't refer their own food as indian food.
4.
I'm the most romantic person you'll ever meet! I'm also the funniest person you'll ever meet! I can't wait to find my special someone on tinder.
5.
I'm a romantic guy who loves to laugh. I'm looking for someone with a great sense of humor and who is not afraid to be themselves.
Comment: because of reduced temperature, you can see laugh, sense of humor etc is repeating.
6.
I'm an Indian guy who loves to laugh and make people laugh. I'm looking for someone who can keep up with me!
Comment: same repetition.
Let's set the temperature a bit up and see 5-6 responses again.
setting temperature to 0.6
1. I like to watch movies and hang out with my friends. I'm looking for someone who will make me laugh and tell me stories.
comment: new topic talked about: movies and hanging out.
2. Hi, I live in Bombay, and I have a great sense of humor. I'm always up for a good time, and I love to laugh. Leave me your number and I'll text you soon!
comment: city added, belongs to India, and straight forward. good bio as well as variety.
3. I'm a foodie and a night owl. I love to explore and I'm looking for someone who is just as adventurous as me!
comment: quite interesting as it doesn't that much relate to funny as traditional sense but in adventurous and interesting person style.
4. I’m looking for someone who can handle my spicy Punjabi food, a good biryani, and a shot of whiskey.
comment: maybe food related talk is a sense of funny person for gpt-3?
5.I'm a romantic person and I love to make people laugh. I'm also a big fan of Indian food and Bollywood movies.
comment: more of your run of a mill generic topic. May I do some kind of rating model on top of the generation as well?
6.Tired of being single? Come out and meet the love of your life.
comment: okk that's again, information less, but different type.
(2) Trying to add additional information to the generated bio: gender
Now that we are down with the concept of bio, and in many cases, it does come up with actual bio, let's see how it handles more specific content and tries to incorporate them.
First information: gender
Hypothesis: gpt-3 tinder bio prompt is male centric, and specifically telling it to generate female bios should change that with occassional male bios creeping in.
prompt: write a tinder bio for a funny, romantic girl from india.
token limit: 300,
Responses:
1.hi there, i’m ____________. i am a romantic girl with a sense of humor. i love to laugh and have fun! if you want to know more about me just ask ;)
comment: run of the mill bio. but interesting to see the dash. is it taken from some example kinda thing?
2.I'm a high-functioning introvert who loves to laugh and be silly. I love to travel, hike, and read. I'm looking for someone who is as passionate about life as I am.
comment: quite informatious. But gender neutral features.
3.Hi I'm Priya! I'm a romantic person and I love to laugh. My hobbies are painting, reading, and listening to music. If you want to know anything else about me just ask!
comment: pretty realistic. nice bio too.
4.i am a girl from india and i like to laugh and have fun. my friends say that i am romantic and funny. my favorite color is pink.
comment: sexist bio with lots of stereotype and lack of creativity. need a way to filter it out ideally.
5. I'm a girl who loves to laugh and make people laugh. I'm looking for someone who can make me feel special and loved.
comment: run of the mill.
Comment:
My hypothesis was wrong. No male bios came. One out of the 5 was a bit gender neutral. A few bios are innovative too.
Now, let's try and add more information and see how it tries to add.
(3) Add one more information: profession ( software professional)
New prompt
write a tinder bio for a funny, romantic girl from India who works as a software professional.
response:
1.I'm a software professional. I hate to admit that I'm a petite woman, but I am. If you're looking for a little girl, stop here. I would rather be with someone who is a little more mature, confident and has the same interests as me.
comment: a little bit sexual hint is there. Don't know whether it is an issue.
2.Hey, I'm Neha. I'm a software professional and I love to laugh. I'm looking for someone who's spontaneous and loves to travel.
comment: fin.
3.i'm a software engineer who's been told my whole life that i'm not good enough. but the only thing that matters to me is you.
comment: personal emotion added! nice touch gpt-3!
4.Hi, I am a software professional looking for a match. I am funny and romantic.
comment: run of the mill
5.Hi, I'm Priya and I'm a software professional from India. I have a great sense of humor and always have a smile on my face. I like to make people laugh and enjoy life to the fullest.
comment: priya returns from generic personal looking information.
6.You: Hi. I'm a software professional, an introvert and a bit of a nerd. I like people who make me laugh, have nerdy conversations with me, and take me to watch their favorite movie.
comment: nerd is a new addition. the laugh keeps coming back. Also, added tinder cta of asking out for a movie. Nice!
7.Hi, I'm Parvati. I come from a small village in India. I'm a software professional with a bit of a quirky personality. I love to laugh and joke around, but you'll also find me reading about the latest in tech news or watching a documentary. It's great if you're looking for someone to chat with about the latest episode of Game of Thrones, or discuss what it means to be an introvert. But don't worry, I can still get out and have some fun with my friends on the weekends!
comment: suspiciously specific. I wonder whether it is generated or real data point some how?
8.Hi there, I'm a romantic girl who loves to laugh. I work as a software professional and am interested in meeting new people.
comment: generic.
9.hi, my name is akshata and i'm in love with life. i don't believe in the idea of love at first sight but i do believe in the idea of love at first conversation. i have never met someone who doesn't like to laugh and it's more than okay to make me laugh.
comment: missed the software professional bit! why gpt-3 why?
10.Hi guys! I'm a software professional who loves to cook, read, and watch TV. I'm currently looking for someone who can make me laugh and who's also a good listener. If you're interested in getting to know me better, message me :)
comment: nice fin bio.
So, now out of 10, 9 bios have mentioned software professional, correct gender and all read like bios. Let's try to do the same with guys.
New prompt: write a tinder bio for a funny, romantic man from india who works as a software professional.
Responses:
1.I'm a software professional who is looking for a woman who would enjoy my sense of humor and is willing to try new things.
comment: passes, but borderline generic.
2.Hi I'm Neel and I love to code, play tennis, and spend time with my friends. I'm looking for someone who is just as adventurous as I am.
comment: specific, but not explicitly funny.
3.I have a degree in engineering, and I am currently working as a software professional. I like to watch and play cricket. I also like to take long walks on the beach.
comment: specific, but not explicitly funny. language model seems to interpret funny as interesting and detailed sometime, which is infact the normal human understanding too.
4.I'm here to make friends and find love. I'm a software professional who likes to cook and watch old movies.
comment: specific ( old movies) but misses funny fully.
5.Hi, my name is Tarkeshwar. I'm a software professional living in Delhi. I like to go out with friends and enjoy the outdoors. I love to travel and explore new places.
comment: interesting non-explicitly funny. Maybe male bios don't explicitly mention that they are funny?
6.I am a software professional. I am looking for someone who is funny and romantic and also a software professional.
comment: misclassifies the intent into "looking" for what the role should be having.
7.Hi, I'm Arjun.
I'm a software professional who lives in India.
I'm looking for someone to talk to and have fun with.
comment: okk so funny is there hinted, this passes.
So, in generic version, the bio gets the information. the characters either implicitly or explicitly expresses the characteristic and gender, profession.
Let's add the name now.
(4) add a name to the bio [ partially successful ]
Prompt new:
write a tinder bio for Arjun, a funny, romantic boy from India who works as a software professional.
Responses:
1.My name is Arjun. I am a software professional who loves to laugh and have a good time. If you are looking for someone to spend your nights with, look no further.
comment: pretty aggressive arjun?
2.Hi, I'm Arjun. I'm not your average Indian person. I'm a software professional with a good sense of humor. I also love to travel and explore the world. Just looking for someone to explore the world with me.
comment: great, now it has every information.
3. [changed name from arjun to Samiul]
I'm Samiul, and I'm looking for someone to spend my life with. I'm a bit of a nerd, I love to laugh and make people smile, and I love to travel. I also have a few tattoos that other people seem to like.
Read more
comment: samiul didn't create response. seems the capital format generates proper result. Also, missed software professional?
4. [name change stays]
Samiul is a fun and romantic guy from India. He has a great sense of humor and loves to laugh. He also enjoys traveling, reading, and cooking.
comment: so we are missing the software professional. Why?
5. [name changed to Daniel]
I am a software professional who loves to travel and explore new places. I'm looking for someone who is funny, like minded, and wants to have fun.
comment: generic response why?
6. [name changed to Tarak]
Hey, I'm Tarak. I'm looking for someone to help me get my life together. I work in IT, and I can't seem to find a girl who understands my profession and likes me for me. If you're a woman who is interested in dating an Indian guy with a good sense of humor, please message me.
comment: so is it a religion stuff? or the names don't fit?
let's try 2 more times to tarak
7. [Tarak capitalization removed]
I'm a software professional from India, looking for a girl who can make me laugh.
8. [Tarak capitalization removed]
I'm looking for a nice Indian girl who is not afraid of commitment.
comment: capitalization is important to have the name incorporated, but doesn't guarantee.
So name is uncertain.
(5) add more specific personal information.
Add another line which needs to subtly add into your bio. For example, some web series etc.
(1)
Prompt:
write a tinder bio for tarak, a funny, romantic boy from India who works as a software professional. Important web series is game of thrones.
Response:
I am a fun-loving, romantic boy from India. I work as a software professional in the US. I like to hang out with friends, watch TV shows like Game of Thrones and The Big Bang Theory.
(2)
Prompt:
write a tinder bio for tarak, a funny, romantic boy from India who works as a software professional. Important web series is game of thrones.
Response:
I'm Tarak, a software professional living in India. I'm funny and romantic and I love the game of thrones.
(3)
Prompt:
write a tinder bio for tarak, a funny, romantic boy from India who works as a software professional. Important web series is game of thrones.
Response:
"""
Hi! I’m Tarak. I’m a software professional who loves to travel and enjoys the occasional beer. My favorite food is Indian, but I’ll try anything once.
Best film: Game of Thrones.
"""
Comment: Film? ok not everything will be correct. Its fine upto this.
(4) same prompt.
Response:
Hey, my name's Tarak. I love to travel, meet new people, and try new things. I'm not very good at writing bios so I'm just going to leave that to you.
Comment:
So it takes an interesting turn. Misses out on all the information and justifies for it.
(5) same prompt.
Response:
Hi, my name is Tarak and I'm looking for a girl to marry. I enjoy watching Game of Thrones and the Office.
Comment:
so it misses the information about indian and software professional.
(6) Realized that the prompt is a bit wrong and doesn't connect the game of thrones with tarak directly.
Prompt:
write a tinder bio for tarak, a funny, romantic boy from India who works as a software professional. His favorite web series is game of thrones.
Response:
Hi! I'm Tarak. I'm a funny, romantic boy from India who works as a software professional. My favorite web series is Game of Thrones.
Comment:
Uses chunks from the prompt.
(7)
Prompt:
write a tinder bio for tarak, a funny, romantic boy from India who works as a software professional. His favorite web series is game of thrones.
Response:
Hi, my name is Tarak. I am a software professional who works from home and I have a great sense of humor. I work on my own terms and love to travel. My favorite web series is Game of Thrones.
(8)
Prompt:
write a tinder bio for tarak, a funny, romantic boy from India who works as a software professional. His favorite web series is game of thrones.
Response:
I'm a hopeless romantic who loves to laugh. I'm an Indian software professional who loves watching game of thrones. Want to chat?
(9)
I'm Tarak, a software professional from India. I work in the IT industry and my favorite web series is Game of Thrones. I love to travel and am always looking for new adventures.
Conclusion:
GPT-3 is a great tool, but it is marvelous only when wielded correctly. We will need to refine it more to incorporate more information. In this article, we hope you get a good understanding from this article that how to test and tune your own prompt and parameter. Also, you need to write clever prompt to get clever results. At the end of the day, a tool is only as good as the person who wields it.
Comments
Post a Comment