OpenAI Python API: A Beginner's Guide with Code Examples

[00:00:00]hey everyone I'm sure and I'm back with the second video in the series on using large language models in practice in this video I'm going to talk all about open ai's python API so I'll start with an overview of what an API is and other key Concepts before diving into some example code and showing you how to make your very own chatbot so with that let's get into it like I just mentioned this video is going to be all about open ai's python API and the goal of this is to serve as both a complete and beginner-friendly guide so here's what we're going to talk about today I'm going to start with what's an API then I'm going to move on to specifically open ai's python API I'm going to talk about how to get started setting up an account and other things like that and then finally we'll get into some example code going over both the basics and showing you how to make a chat bot with the API so the million dollar question what is an API I feel like this is just thrown around in so many different contexts said people often be familiar with the term but not so familiar of what it actually means and so an API stands for application programming interface and here I'll Define it as a way to interact with a remote application programmatically so while that might sound very technical and very scary it's not let's just consider the following analogy suppose this is you and you've got a craving for some Salvadorian pupusas that you had during your Latin America trip from last summer however you're back home and you don't know where you can find pupusas in your hometown but lucky for you you have a super foodie friend that knows every single restaurant in town and so if you were trying to figure out where you can go find great pupusas you might send your friend a message hey any good pupusa spots in town and then you know maybe a few minutes later your super foodie friend would respond yes flavors of El Salvador has the best pupusas and so basically what just happened here is you sent your friend a request and then a little bit later your friend sent back a response and while this might sound completely irrelevant to programming in Python and what we're talking about here this is essentially how apis work what this can look like in the context of open ai's python API is instead of texting your friend your request you can send open AI a request using Python and then openai will send back a response which you can capture and use for whatever Downstream task you like for example let's say you want to send a API request to one of openai's language models and if you saw the previous video of the series you saw that a core task of language models is given a string of words to predict the very next word and so let's say we send open AI this string listen to your and we want to get back the very next word the response we may receive could be hard and so as we can see this is very similar to the chat GPT web interface but instead of going to their website typing in a prompt in the UI you're essentially doing the same thing but you're sending over your prompt using Python and this API as opposed to natural language and using their web interface and so you might think what's the upside of doing this way this just sounds like the same thing with more steps while there is a lot of overlap there are unique features in the API that are just not available in the easy user interface that is chat apt one example is a customizable system message this is essentially a message that you can use to set the tone for your chat bot basically set the context give it its motivation its task for chat GPT what this might look like is I am Chad GPT a large language model trained by open AI based on the GPT 3.5 architecture my knowledge is based on information available up until September 2021 today's date is July 13th 2020 23 and so you can imagine that you want to make your own chatbot and you don't necessarily want it to be called chachyvt and you maybe you wanted to do something more specific than this and so you can really set that with the system message some other things are you can adjust input parameters such as the maximum length of the response that is sent back from the language model also the number of responses that you get back so in chagy BT you'll type in a prompt and you'll just get one response but what if you want to get 10 responses and compare them programmatically this is something that would be a bit more time consuming using chat apt as opposed to using Python and then finally you can adjust the temperature which is like the randomness of the response generated by the language model but we'll get more into that later in the video also you can process other types of input not just text so images or other file types you can extract helpful word embeddings for Downstream tasks if you go to chat gbt you input text it'll spit text out but let's say you want a numerical representation of the input text so you can use that for some Downstream processing you can also pass an audio to the API for transcription or translation tasks and then finally there's some model fine-tuning functionality built into the API so even though it may seem like more work to use Python to interact with GPT 3.5 or gpt4 you get all these additional features using the API that just are not available with the chat GPT user interface and then it's not just GPT 3.5 or gbt4 that is available to you there are also a bunch of other models and so here's a quick snapshot of them feel free to pause the video if you actually want to read this this is also available in the blog associated with this video if you want to read more about these models additionally there is a full documentation on open ai's website linked here before getting into how to actually use the API I think one thing at the top of a lot of people's minds when it comes to a paid service is how much is this thing going to cost me and so in order to understand that we first need to understand the concept of a token which here I'll Define as a set of words and characters represented by a set of numbers so suppose we have the text the end with a period at the end while you and I as humans can read this and understand it unfortunately a language model or more specifically a neural network does not understand this text directly language models understand numbers so we need to translate this text into a numerical representation so that the language model can actually understand it and do something with it so let's say we translate the end into this list of integers so what we did is we took text and we translated it into numbers but what do these numbers actually mean and so let's say the relationship between the text and the integers here is shown by this mapping here so the word the is represented by 73 a space followed by the word end is represented by the number 102 and then the the period is represented by a six these three things that make up this text are tokens obviously these won't be the only tokens we have we'll actually have a huge set of these words and characters and their corresponding numerical representations and this will make up the set of all our tokens and the vocabulary that we're using and so the reason this is relevant to pricing is that pricing with openai's API is based on the number of tokens sent to the API and the number of tokens returned so that means bigger prompts and bigger responses will have larger costs additionally there are different costs per token associated with different models and you can read more about that on openai's website linked here okay so now let's start to see how we can actually use the API to do something but before we get into the example code we first have to do a few steps to get our account set up and get into a place where we can actually start making API calls and so there are four steps here first we need to make an account second we need to add a payment method third we can set usage limits and then finally we need to get our secret key to actually make the API calls and so for this I'm going to switch over to open ai's website which is here so the URL is platform.openai.com overview and the first thing we want to do is make an account the way to do that super simple go up to this top right corner click the sign up button and then you can create an account using your email address or continue with any of these continue with Google Microsoft or apple if you've used Chad gbt in the past odds are you already have an openai account and if that's the case you can just hit this log in button here and since I already have an open AI account I'll just go ahead and log into mine here we go now that you've made your account or that you're logged in now we can go to the next step which is to add a payment method how to do that is you can click on your profile in the top right corner and then you can click on manage account to add a payment method go to this billing tab here and then then there's this thing called payment methods so you can click on that and add a payment method pretty easily the next step is to set usage limits so this is something I recommend this will help you set hard and soft limits to how many API calls you can make this will just ensure that you don't spend any more money than you expect on API calls there are two options here there's a hard limit and a soft limit the hard limit basically once you hit that number so in my case I put it as five dollars once I hit five dollars any additional API call will get rejected so you will not be charged more than this number here and then there's also a soft limit and basically when you hit the soft limit openai will send you an email hey you've hit your soft limit threshold and so this kind of helps you make sure you're not going off the rails with API calls for a little context I spend a few days playing around with the API writing example code for this video and so on and I spent a whopping three cents so the API calls are pretty cheap unless you're deploying a product where you have many users making API calls on a specific account that's when this number will probably go up a lot but I would imagine for personal use you probably will not hit more than a dollar if you're not really using this API a whole lot and then finally you need to get your secret key and so for that click on this API Keys thing and you'll see a screen like this if you've never done this before you won't see any API Keys here so you need to create a new one and the way to do that is really easy you just click this you can say new secret key I'll give it whatever name you want then you can click this create secret key button and then it's going to create this secret key for you it's important that you copy this and save it somewhere safe and accessible to you because this is the only time you're going to get access to this key just showing that if I hit done the key I just made it just shows me the last four digits of it so I don't know anything else in between so if I didn't copy it then I wouldn't be able to use it so now that we've made our account added a payment method set usage limits and obtained a secret key and we can now finally start start coding so starting with the basics we need to import the open AI python Library if you haven't installed this super simple just pip install open Ai and it should download it automatically in your terminal this line here I'm importing my secret key so what I did is I created a python file called SK dot pi and in there I just Define one variable which is called my SK and I set that equal to my secret key we can see what that looks like here so I have this Jupiter notebook and then I have this SK dot Pi if we open that we have this python file with just one variable my SK and it's equal to a string and so here is where you will copy paste your string I copied it from earlier so I'll just throw it there so you you should never do this you shouldn't just make your secret key visible to people on a public medium like YouTube but I'm just going to revoke this key right after so you won't get the chance to use my five dollars of API credits so this is just like a clean way to do it you just import the key and then you can copy paste whatever key you want in here you can revoke access create a new one paste it here without ever having to change your code and then you can set what API key you want to use using the syntax here so openai.api key you just set it to my secret key alternatively if you don't want to go through all this I've created another file and import the secret key like this you can always just manually paste your secret key right here so we got that set up and now we can make our first API call the way to do this is really simple so we have this chat completion module and then there's this method called create so here we are just passing in two inputs we pass into the model we want to use and when you pass in the messages taking a closer look at this messages thing here messages is going to be a list of dictionaries here we have a list with only one element so this element is a dictionary and then the dictionary has two key value pairs so the first one is we define the role of whoever is saying this message so here here we say user alternatively you could put system to define the system message or you could put assistant to Define some text generated by the chatbot then you define the content so this is essentially what that role said and so here keeping with the listen to your example we're going to pass that in as content from the user essentially this is our request to the API it is all baked into this method here what we get returned back is actually the response from the API and so this will be in a Json format which is essentially like a dictionary so we can print it and we see that it has a bunch of different fields in it there's this ID object created model choices usage so on and so forth most of these are not important to what we're talking about here if you want to know what each of these mean I have a definition for each of these in the blog associated with this video published in towards data science but the only important field for the discussion here is this choices feel because that is what is going to hold the response from the language model in this case GPT 3.5 turbo if we take a closer look we can see that choices will in of itself have an index field and a message field and then the message field will have role and content fields in them basically we just want to extract the content and to do that we can use the syntax here chat completion is this dictionary like object here we can access the choices which is this first field here we can get the First Choice essentially so index 0 then we can access the message of this first choice so now we're here and then finally we can grab the content which is this line here it is very hierarchical but that kind of helps keep all this information very organized so that was our very first API call but this wasn't anything special this is equivalent to going to chat GPT and just typing in listen to your this doesn't make use of a lot of the other features of the API so let's see what else we can do so one is we can set the input parameters Max tokens which will adjust the maximum number of tokens that are allowed in the response from the language model let's say I just want a one word response so what I can do is have the same exact API request but I'll add this Max tokens parameter and set it equal to one if we do that and print it out we actually get the same exact response but notice there's no period so what that tells us is heart is actually a token in the vocabulary of GPT 3.5 turbo and the period is another token let's see what else we can do next we can set this n parameter which controls the number of responses sent back from the language model so in this case we have essentially the same API request as before now we're setting the max tokens equal to two and then we're sending n equal to 5 and so what this will do is return back five chat completion for us we can get those back and we can print them in this for Loop here and then we can see we get heart heart and heart again a new line character I'm guessing then heart comma a new line character Heart comma and so notice these aren't all the same thing and this could be a good thing or a bad thing depending on the use case for example if you're trying to create some AI tool that'll help you in some creative tasks you know maybe you do want these to be a little random and out there on the other side maybe you're automating some business process or standard process and you want it to be very deterministic you want the output to be very predictable and identical every single time it's called and so it turns out we can control this degree of Randomness or predictability using the temperature parameter temperature is this input parameter that essentially controls the randomness of the response and this will take values between 0 and 2. so a temperature value of zero will make the responses very deterministic so very predictable while on the Other Extreme a temperature equal to 2 will make the responses very random and out there to see what this looks like we have the same exact API call from the previous slide but now we're setting the temperature equal to zero so we're making this very predictable and deterministic and lo and behold we get the same exact thing for all five responses heart period heart period heart period basically this is the most probable two tokens that will follow the text listen to your but let's see what happens when we crank the temperature up so let's set temperature equal to two same exact API call and now we get some really interesting chat completions first we get listen to your judgment make sense listen to your advice okay that's good listen to your dot inner awareness interesting listen to your heart so notice we'd still got this most probable outcome so raising the temperature doesn't necessarily mean you get low probability and why old responses it just means that those low probability responses have a higher likelihood of actually showing up in your chat completion and then finally we have gingist which I don't know what that means so listen to your gangist okay so those were the basics of open ai's python API now let's see what a more advanced example could look like the code that we just went through and the code that we're about to go through is all available on the GitHub repository and so this is essentially a code we just ran through and here's the setup our first API call the max tokens the number of chat completions the temperature stuff and now into the demo let's say for whatever reason we wanted to create a lyric completionist system this could be some service where people know the first line of a song but don't remember the lyrics for the rest of the song so they'll just put in the first line and then this lyric completion assistant will fill it in or maybe they're writing their own music and they are kind of getting stumped on what the next line in the lyrics should be so they can use this lyric completion assistant to help them do that creative task with this chat completion API making a custom chatbot is like stupid easy basically all we have to do is Define these messages that will precede the conversation that the user will have with the assistant so there are a couple ways we can do this one we can use the system message to set the context for the chat bot or we can have some example back and forth between the user and the assistant so the chatbot gets an idea of how to respond to user requests or of course we can do both which is what we're doing in this example here we set the system message as I am Roxette lyric completion assistant when given a line from a song I will provide the next line in the song so that's the system message and then we have two example back and forth between the user and the assistant so the user will say I know there's something in the wake of Your Smile the assistant will say I get a notion from the look in your eyes yeah the user will then say you've built a love but that love falls apart and then the assistant says your little piece of heaven turns too dark then we'll will pass in one last message which is the same message we've used throughout this entire demo which is listen to your and then we can see what the chat bot spits out so this code is essentially doing what we did before you know we're making the API call using GPT 3.5 turbo instead of a list with just one dictionary we have this messages list which actually has five dictionaries in it here we set the number of responses to one and we set the temperature to zero so it becomes very predictable here's where a little magic is happening so we're going to print the response from the chatbot but then we're gonna take the response from the chatbot and we're gonna append it to the end of the messages list which will then get fed right back into the chat bot so essentially the chatbot will just loop on as long as we want going off its previous responses and so here's what the output of all that looks like so we passed and listened to your and then the chatbot said heart when he's calling for you listen to your heart there's is nothing else you can do I don't know where you're going and I don't know why but listen to your heart before I tell him goodbye these are powerful lyrics from this chat bot and they actually correspond exactly to the hit Roxette song listen to your heart and whose actual lyrics we can see here but here's a little bonus so let's see what happens when we crank the temperature you know what if The Roxette song went a little differently so here we put temperature equal to zero so we have listened to your I reach into the Shadows summon sweet Elaine pointing all steel values fails if friends remote empty reply new line character image existing long seconds confirm flesh pressed secretly remember Saint talk dying to unfamiliar pieces father blessed speech keeps passing shape raises you travel feeling Shadows thriven body swept Spirit consume so not entirely sure what that means so this could be some you know new genre of Art postmodern techno chat bot AI art or it could just be meaningless gibberish so you can't really tell with this kind of stuff so again example code is available on the GitHub repository shown here and linked down in the description below and so what's next in this video series you know one obvious downside of using open ai's API is that you have to pay for these API calls which if you're building some product or service it may not scale well because if you have thousands tens of thousands or more users these costs can start adding up pretty quickly so in these instances we can turn to open source Solutions where you don't have to pay for API calls you have some other way of Hosting these language models so this is where the hugging face Transformers Library can help hugging face is a AI community that is building the future this is just a screenshot from their website which is also listed here hugging face maintains this python Library called Transformers which essentially makes downloading and training powerful pre-trained language models really easy and so in the next video this series I'm going to be talking all about that and sharing example code and so if you enjoyed this video please consider liking subscribing and sharing with others if you have any questions about large language models or the openai API please feel free to drop those in the comment section below this whole series is part of my Learning Journey and process so I do find a lot of value in the questions that I receive because it prompts me to dive more deeply into the stuff which I really enjoy and as always thank you so much for your time and thanks for watching