How to Build a PyTorch Model for Image Classification

[00:00:00]in this video I'm going to show you how to train your very first P torch model if you're interested in deep learning you really have a few options pytorch or tensor flow but in recent years py torch has really exploded as being the front runner and the main reason is because of its flexibility but with that comes a lot of added complexity I've competed a lot on the website kaggle which hosts competitions for deep learning and machine learning problems and over the years I've train thousands if not tens of thousands of pytorch models now there are already a bunch of pie torch tutorials out there but I want this one to be different instead of focusing on the details of pytorch we're going to actually start by trying to solve a problem and create a model and we're going to focus on the three main paradigms that you'll need to know in order to train a pytorch model and this knowledge will transfer to whatever type of pytorch model you're looking to create training most deep learning models requires that you have a GPU so you can run these large computations very fast but everything I'm going to show you in this video I've done in a kaggle notebook the nice thing about that is you'll be able to go copy that notebook and run it yourself on a hosted virtual Cloud instance that has a GPU and everything you need already installed okay I hope you're excited I know I am let's train our very first pie torch model so here I I have a kaggle notebook that will be working in I'm going to link this notebook in the description so you should be able to load it up and like I said copy it and run it in the exact same environment but everything we're going to learn here you could also run on your own environment even a collab notebook if you'd like you just need to make sure that you'd have the GPU with Cuda installed and pytorch in the correct pytorch version in your python environment now a few things I want to show you in the Tagle notebook environment if I click over here to the right we can see some information about this environment and the main thing I want to show you is that we are running with a GPU enabled and the GPU is going to be critical in speeding up this training process and if I scroll up here I want to show you that I have a data set linked to this notebook and this is a image data set for classifying playing cards and we can see the way the data is set up there's a training folder and in each of these we have a folder for different cards in a playing card deck and if I click on each of these images you can see the playing card images show here at the bottom so this is the type of data that we're working with in this tutorial we're going to learn through doing and what we're trying to achieve is create a model that is capable of taking in images and then detecting which playing card is in that image and we're going to break this down into three parts and that's understanding the P torch data sets and data loaders and how that works learning how we set up the pytorch model module and then understanding the py torch training Loop now these three steps are critical to training almost any pytorch model it doesn't have to be an image classifier it could be for object detection audio classification they all mainly will follow this idea all right so the first thing we need to do is import all the packages we're going to be using for this tutorial so obviously we're going to be importing P torch which will import as torch we're going to import the NN module from torch this is all the specific functions related to neural networks we're also going to import the optim library which will Define our Optimizer later data sets and data loader will be the very first step in the process we need to import them and we're also going to import a few things from torch Vision which will make working with these image files a lot easier Tim is a great library for loading in architectures specific for image classifications we'll import that and then like always I usually import matplot lib for visualization and pandas and numpy in case we need to use those for data exploration I like printing all the versions out here so if you're running this later you you can compare it to what versions that you're running and there may be some slight changes we are running pytorch 2 and we're running python version 3.10 the first thing you're going to need to do to train a pytorch model is to set up the data set because that is how pytorch is going to be loading in the data as it trains the model and evaluates the model now the nice thing about pytorch data sets is they're extremely flexible you can really modify them to load the data in whatever way you want and then you can take that data set and just wrap it with a pytorch data loader to make it parallelized and load in the data in batches to the model and we create this data set by simply creating a python class in our class we're going to call playing card data set this will inherit from pytorch's Main data set which we imported above now if you're not that famili Amar with using classes in Python you might want to do a refresher but it's not as complicated as you might think each class takes an anit method which basically tells this class what to do when it's created and then in pytorch data sets we'll need two additional methods one is the length this is important because the data loader will need to know how many examples we have in a data set once we create it and then this method called get item and this method takes in an index location in our data set and we'll return one item so let's go and make this actually work for our data set as we have it situated so our init is actually going to take the data directory where our data is sitting then we'll also initialize with a transform this is what will apply to each item in the data set and we're just going to keep it super simple here and only provide it a transform that will resize all the images to the same size we're going to keep this as simple as possible and use from the torch Vision package this image folder class which will take our data directory and whatever transform we provide it the nice thing about this image folder class is it'll assume that any subfolders in that folder have the class name for the image and it'll handle creating all the labels for us we could do this manually if we wanted to but we'll do this for Simplicity now our length we just want this to return the length of our data and for git item we'll want to return the data item of whatever index that our git item is called with this image folder data set when called on an item will give the image and the class and I'm going to show you how that works now I want to add one additional method that will just make it easier for us to find the class names and that's going to be called classes and it'll return the data classes from this image folder and that's it now we have our playing card data set and we can create an instance of this data set now we do need to give it our data directory and in this kaggle environment that's going to be this location where we have the playing card data set loaded now we can test out our methods to make sure it works right we can run a length on data set and we can see here our data set is 7,600 and we can test the git item method by calling this data set on one of these index locations it should work anywhere between zero in the max length of this we could see when we call this at various indexes we're turned a tupple which contains the image and then the class which is a number between Z in 53 here in this data set cuz we have 53 different classes of the playing cards and then also a joker card we can also see that these classes are in order so if I go later in this data set 6,000 that will get this different class returned now we also could call it like this where we would pull out from this tupple the image which I can display here because it's a pil image and then the label which we could print 41 now our labels are represented by numbers and they need to in order for p torch to work with them but we don't know which number is associated with each label so that's why we added that class method don't worry too much about these lines of code but basically what we're doing here is creating a dictionary which allows us to link up each of these numbers with the correct label so running our example here above we see this is a 10 of clubs the number of the label is 41 and this dictionary just allows us to see that 41 is in fact the 10 of clubs in our label lookup now there's one more thing we need to do and that's ensure that our data set outputs all of these images in the same size because the model will expect the input to always be consistent and that's where we're going to use these transforms that we've imported from torch vision and we're keeping this transform as simple as possible and all we're doing is taking the image and making sure that they're always 128 by 128 in size and then we're converting it to a p torch tensor so now let's recreate our data set with this transform added and if we call one of the values in the index we can see we have our image and our label but now our image is a pie torch tensor and we can run a shape on this tensor to get an idea of the size of the 3D tensor and it's in 128 by 128 because we've resized it and the three here is the number of channels that's the red green and blue channels of the image now I do have a whole video on working with image data that you should check out if you haven't already where I explain this in more detail now one other thing to keep in mind about the data set is we can iterate over this data set and we'll just have it break out of this after the first time we run it but since the data set is in iterable we can call this Loop and run over each image and label in our data set and you can see the image here is same as before it's a tensor and the label is the number associated with each label so creating the pytorch data set is the main thing that you want to do and the nice part about the data sets is then we can take it and wrap it with a p torch data loader which will automatically handle the processing to parallelize reading in each of these images so creating our data loader is as simple as calling the P torches data loader and providing it our data set there are a bunch of other settings we can set here but the main ones we're going to focus on are batch size and the shuffle perameter so batch size tells us how many of these examples we should pull each time we iterate over the data loader and a nice one to start with we'll say is 32 and then Shuffle allows us to tell the data load loader that every time we load a new example from our data set if we want that to be pulled randomly from the data set or it to be pulled in order for reasons I won't go into in too much detail shuffling is typically done when training the data but you don't need to shuffle the data when you're running it on a test set or a validation set so we've created our data loader and now we can iterate over our data loader just like we could our data set but we'll notice one main thing has changed so let's iterate over this by calling for images labels in data loader and then we'll break now if we look at the images it's a tensor just like before but the shape of it is 32 by 3x 128 by 128 that's because our data set has been batched into 32 examples and our labels is a p torch tensor with one dimension that that has 32 different labels also note because we put shuffle here and we know that the labels were in order of the original data set if we just look at these labels now we can see that they're random and each time that we run this they should be a random order of the labels so just to recap we need to First create a pytorch data set before we train our model we create this by inheriting from the data set class and creating a few methods like the length in a git item method this creates an iterable object which we can Loop over then we can create a pytorch data loader which will handling batching these into and then we can wrap our P torch data set with the data loader class which will automatically batch this data when feeding it into our model main reason we want to do this is because the model trains much faster when it takes in batches of of examples instead of one example at a time now we're on to step two which is actually creating the pytorch model which we want to train on this data set pytorch models are extremely configurable and you could go in and build these models from the ground up but I want to keep it simple here so that we're really learning about these paradigms and pytorch and we're going to use a predefined architecture that exists and we're going to import that from the library Tim the library Tim has prepackaged a lot of the state-of-the-art architectures that are already created for image classification this allow us to get some really good results without having to Define much from scratch at all the one thing I'll say about creating a pie torch model is it's all about understanding the shapes in each layer of the model and we're going to do a little bit of that when we create our classifier so let's create our pytorch model we're going to create a class that's called simple card classifier and this is is going to import from the neural network module of pytorch the base module now when creating a pytorch model you can customize this a lot but there really are two main methods that we're going to create init method and the forward method I like to think about it like this where in a knit we Define all the different parts of the model the forward method we take in an example or a batch of examples and we connect all these parts which we've defined up here and return our output so to make this actually work let's start building the structure of our model when we initialize this we want to give it the number of classes that the model has which is 53 and we're going to use the efficient net b0 model architecture which is a really nice architecture and it trains fast the B 0 indicates the size of the model this is one of the smaller ones by setting pre-train to true we're saying the model weights within this model have already been trained on the image net data set now this depends on the model but the efficient net B 0 by default will output a feature size of 1280 and we're going to need to take this output size and resize it to our number of classes which is 53 so we're going to create a classifier which will simply be a linear layer from our efficient net output size to our number of classes because this base model actually has an additional layer that we want to remove we're just going to remove that last layer with this line of code don't be too confused about it just think about this as if we're cutting off the very end of this Tim model so that we can instead have it output at our classifier level now in our forward method we are going to take in an example or a batch of examples and we want it to return the output we're going to call this features on our input data and then we're going to call our classifier on the output of these features and let's just call this output so it's obvious what the last layer is and what we're returning I try to make this as simple as possible so you can see how the basic structure and Paradigm of creating pytorch models looks like almost forgot but we knew need to add this line of code to our init which will initialize this object with everything from the parent class and let's go ahead and create an example of this we could give it the number of classes now we have our model and if I print this it'll even show the structure of this model in detail with everything from the efficient net since that's really long I'll just curtail it in our print statement so that we can just see the beginning part of it now the first thing I like to do after creating the model just to check and make sure it works correctly if you remember before we tested out our data loader by iterating over it and testing to make sure it returned the images and labels we can test our model just to make sure the input and output or what is expected by providing it now those images from one example batch and we see this outputs a tensor and this is a good sign because it did run correctly which tells us that the structure of the model can accept the input data that we provided and if I call this example output and look at the shape we can see that the example output is 32 by 53 and this is our batch size by our number of classes so this is exactly how we would expect it for each of these examples we have probabilities for each class recapping the model that we've created we make sure that we import from P torch's neural network module we create an a nit in a forward method and nits where we've created the structure of this model and our forward method defines how these parts of the model are connected and will produce our output when called on some input data and we tested this forward method by calling the model on some example images from our data loader now we're on to step number three which is creating the training Loop that we'll use to train this model now we're going to write this training loop from scratch because will help us to really understand how the model is trained so essentially the idea here is we're going to feed this data into our model many times and then we're going to apply some sort of loss function to the outputs that we receive when we send in these training examples this is how the model learns we do this in batches that's why we create a p torch data loader and then we call one Epoch when we run through all the batches in one training data set and we're keeping this training loop as simple as possible but there are really two things that we'll need to select before we move forward and that's the optimizer we want to use and the loss function we want to use let's start with loss function we're going to call this the Criterion and pytorch has a bunch of built-in lost functions that you can use for common tasks we have a classification task here and cross entropy loss is a common loss function used for this type of case we also want to Define our Optimizer we'll pull this from pytorch's Optimum library and we're just going to use atom which is the best place to start when using an Optimizer it's works really well in most cases this Optimizer needs to take in our model parameters and then we need to give it a learning rate there's a lot more advanced techniques that we can learn about in future videos like learning rate schedulers but here we're just going to keep the learning rate constant as we train similar to how we tested out our model and our data sets before to make sure everything was working right let's test out our loss function we're going to give it the example output from the model that we ran before here and the labels for that data and you can see this worked correctly because it imputed our loss on that batch of data now the model hasn't learned anything yet so this loss is going to be pretty high but we're just checking to make sure this loss function does work with the input data that we have and the output from the model now we also want to create data sets like we did before but these are the actual data sets we'll use for training and evaluating our model our data set actually has a folder for training examples and a folder with validation examples that makes our job easy because we don't have to define the validation split I do have a whole video on how to properly create cross validation for your data set that you should check out if you haven't already so here I've set up our data sets since there is a test folder I'm including that here too but we don't necessarily need this for our training Loop but you can see we have this one transform which just resizes our images we've created our train data set and our validation data set based on these folders and then we've created data loaders for each with batch size of 32 and for our train loader we're shuffling the data as it's training but for a validation our test we are not having the data loader Shuffle Now that we've defined a loss function an Optimizer we have data sets and a model let's go ahead and create this training loop as I mentioned before it Epoch is one run through the entire training data set and we're going to set to train for five epochs then we also need to set up some lists which we will store our training and validation losses then we'll create the model and we'll start looping through five different epochs so for Epoch in range of the number of epochs and we're going to Loop through our training data loader to train the model but before we do that we need to set model to train and we'll set our running loss and then we're going to keep track of this loss as we train now we're going to Loop through our training data loader we're going to say four images labels in train loader we need to set our Optimizer to zero grad and then like before we'll create our outputs by calling the forward method on these images then we'll take that output and we'll calculate our loss for this batch using our Criterion which we defined above we'll provide this our outputs and the labels and this is the part where we run back propagation on the model which will update the model weight in each of these steps using loss.

[00:24:02]backward and then we need to run a step on our Optimizer and we're going to keep track of our running loss by just adding this loss like this once this inner loop has completed one Epoch is done and we want to store our training loss so let's first compute that by taking our running loss and divide it by the length of our TR train loader and mainly for tracking we're going to append this train losses list with our training loss and this is pretty much it for training but because we want to see how well the model is doing on the validation set as we train let's add a validation portion to this training Loop we need to make sure we change the model from being in training mode to evaluation mode and then we're going to track running loss but this time it's on our validation data set and we'll essentially calculate loss similar to how we did on the training set but this time on the validation set one thing we need to do just to make sure the model weights are not touched is to do this under torch no grad and then let's write our Loop so we've run through the entire validation phase of this model training Loop and we're just going to print Epoch stats each Epoch and this will allow us to track how well we're doing by showing the training and validation loss need to fix a few names here and if I run this cell it'll start running through our training stage now I purposely left something out and that's that our training is currently being done on this instance's CPU if I look here we can see that the instance's CPU is being used but the GPU is not being used at all if we were to train this way it's going to train very slowly and we want to use the GPU so let's go ahead and stop this and with some slight modifications we can modify this Loop to instead run on the GPU and to do this we're going to define the device that we want the model to run on now this could either be Cuda for a GPU or CPU and this oneliner will automatically use the GPU if it exists and we can see that now the device is set to Cuda and then let's go and modify our Loop slightly by making sure we move both the model and the data over to this GPU when we're running these loops and we do this by taking the model and telling it where we want it to go to which is our device and similarly with our images and labels we are just going to put these on the GPU images to device label to it'll give you an error if the device is not the same between tensors or the model when you're trying to run operations between them now while I'm running this cell I can click here and see that the GPU is actually being used as the model is training looks like I had this spelled wrong so let's run this again and now what we need to do is just wait and let the model train now if you're impatient like me you'd like to see a progress bar as the training's going on so let's go ahead and from tqdm notebook import tqdm by wrapping tqdm around these loaders it allow us to get a progress bar as we're training but as this runs we do have a progress bar as it runs each training Loop and validation Loop and we're printing out the losses train and validation as we train now what we'd hope to see here as we're training is the training loss would decrease and the validation loss will also decrease or else we'd see something known as overfitting to the training set but let's let this finish and see what our results look like our training Loop is done it's run five Epoch and we see that our training and validation loss have both decreased but let's go ahead and visualize but since we were tracking these losses as we trained let's just quickly visualize what the loss looked like while it trained we did did have a strange Spike here in the validation loss but overall our training and validation loss was going down which is a good sign and we're training for not that many Epoch here so typically you'd train for a little bit longer or maybe adjust the learning rate so just to summarize what we learned about the training Loop we had to pick a Criterion or a loss function we had to set an Optimizer at that point we create our data sets both training and validation then we wrote our training loop from scratch where we Loop over our train data loader back propagating our losses and then at the end of each Epoch calculating our validation loss now the whole point of this exercise was to get familiar with pytorch data sets creating models and creating the training Loop but let's go ahead and evaluate our results just to see how well it did visually I'm not going to spend too much time on this code explaining what it does but essentially what it will do is let us visualize what the predictions look like from the model when we feed it a test image so I'm going to run this code on one of the example test images for a five of diamonds we're going to see what the results look like so on the left here we see the image that was fed into the model we can see that is very confident in the fact that this is a five of diamonds so let's just print out 10 test examples randomly to see what they look like for this I'm going to take all of our test images which I found using glob and I'm going to use numpy to sample 10 of them and we'll run this code over these 10 examples so Four Diamonds it looks like it's right ace of clubs King of Clubs and you can see here most of these results it's predicting the correct class and we could just apply an ARG Max which is a post-processing technique to add identify the highest class predicted and that would be our ultimate prediction this maybe looks like the one where it was the most confused it still got the correct one right but you can see how the image is a little bit different than the others so the model struggled a little bit now if you've made it this far in the video I first just want to thank you for spending the time watching this make sure you like And subscribe I want to challenge you to do something with this results you should be able to follow the link in the description to this notebook and I want you to then here calculate the accuracy of our model on the test set and you should be able to do that given everything that I've provided you in this notebook uh by just finding out which of these predictions were correct thanks for watching put some comments below and let me know what you thought of this video and I can take a deeper dive into some of the more advanced features with pie toward in a future video If you guys like this one see you next time