OpenCV Python Tutorial: Computer Vision Projects for Beginners

[00:00:00]learn OpenCV in three hours hey everyone welcome to my channel in this video we are going to learn everything required to get you started with open CV invited when we'll be using Python since it is one of the most popular programming languages and it has opened up numerous job opportunities in various sectors we will start from the installation process right up to creating exciting projects such as detecting colors shapes humans and even vehicle number planes so if you are beginner don't worry this course is for you we will skip all the boring Theory stuff and focus on the practical implementation so you can get the computer vision skill set you have always wanted in your CV by the end of the course you will become familiar with the core principles of open CV and apply different techniques to solve real world problems using computer vision if you'd like to learn more about such projects don't forget to subscribe and hit that notification well as I upload videos on a weekly basis so without further ado let's get started [Music] before we begin let's look at the structure of the course we will start with introduction to images then perform the required installations then we will learn to read images videos and webcams along with some basic must-know OpenCV functions later we will learn to resize and crop images then we will draw different shapes and texts on images from there we will get into some more advanced topics such as word perspective joining images color detection contour detection and even face detection after we have understood the fundamentals we will create three different projects using these core principles these projects will include virtual paint paper scanner and the number plate detector so let's start with introduction to images so what are images made of let's say I wanted to display the number three I will take an array of boxes where each box could be filled or empty so to write the number three we would color a few boxes to create that shape now some boxes are white in color whereas some are black we can denote all the black boxes as zero and all the white boxes as one in this example we have ten by ten boxes if we wanted more detail we can increase the number of boxes in reality these boxes are pixels you would have heard of oag a HD Full HD and 4k all of these represent a fixed number of pixels for example VGA is 640 by 480 HD is 1280 by 720 this means VG has 640 boxes in the width and 480 boxes or pixels in the height currently the image with Drew has only two colors black and white this is known as a binary image in order to get more detail we can have an image with more levels this would mean instead of having only 0 and 1 we will have a range of values here we can see the difference between 2 6 and 16 levels or shades of gray but still the image is not very clear so we will be using an 8-bit value this will give us a resolution of 256 where 0 will be black and 255 will be white this means we now have 254 colors between white and black in other words we have 254 shades of gray this image is now known as a grayscale image for a colored image we have three grayscale images representing the intensities of red green and blue in short RGB adding these images together gives us a full color image this means a colored vga image is 640 by 480 by 3 so let's start with the installations first we will go to Python dot o R G and we will go to the downloads section here we are not going to download the latest version instead we are going to download three point seven point six as this works well with open CV so we will click on download and then we will head on to the Windows version now if you have Mac you can download the Mac version as well once the download is complete we will install make sure to add Python three point seven to the path so once the installation is done we will close the dialog and head on to PyCharm now PyCharm is the ide that we will be using to edit our code so it is a code editor that will allow us to write our code but it has a lot of functionalities which will help us in the coding process so we will head on to its download section and we will download the community version as it is free once the download is complete we will run the installation file here we will associate the dot PI files to pycharm and we will also add launchers directory to the path next we will restart the computer and run PyCharm so now we are going to create our new project here before we start the project we will look into the project interpreter now this has to be 3.7 so if it's automatically detected then we are good to go so let's write down open CV Python and hit create so now this is the pycharm environment over here we have the area where we will write the code and on the left hand side we have the project files and the project folders so if we open up our folder we only have the environment the virtual environment that was created now we need to install libraries or packages in these in this environment so we will go to file settings project project interpreter and here we will add and search now we will write open CV and here we have open CV Python so we will click on install package so once the installation is done we will close this it ok and now we will create our first python file so we will right click on our project we will go to new and we will create a python file here we will say chapter 1 now as you can see in our project we have the chapter 1 dot buy file and here is where we can write our code so let's import our OpenCV package so we will write import and then c v2 c v2 stands for computer vision then we will print package imported now if everything runs fine this should output package imported so we will right click on the chapter 1 and we will press run so now in the console we can see the package important message has been sent this means that our package is successfully imported and now we can proceed further so the first thing we will learn is how to read images videos and webcam so in order to read images we have a function by the name I am read so let's start by declaring the variable in which we will store the image so we will call it IMG and then we are going to use our package which is CV 2 and then inside that package we have the function I am read now this means we are reading an image now all we have to do is mention the path where this image is present so I have created a resources folder in which we have the image Linna dot PNG so we are going to define the path of this so it is in the resources and then it is Linna dot PNG so this imports our image from our resources folder but now we need to display it so to display we have the function known as I am sure so we will use our package cb2 dot I am show and in I am show we have to define two arguments the first one is the name of the window so we can say our output and then we have to define which image do we want to display so we will say IMG so let's run this now the image did appear but it went out immediately so in order to add a delay so that we can see we will write C v2 dot weight key and then we will add the delay if we put 0 it means infinite delay but if we add a value it means that many milliseconds so for example 1000 will mean one second so for now we will put zero and run so here is our output image so next we are going to learn how to import a video so we have a video in the resources folder by the name test video so let's import that so we are going to remove what we had before so now we are going to create a video capture object so let's call it cap C v2 dot video capture so now in this video capture again we just have to define the path where our video is so we will type in resources and then we will say we will mention the name of the video so which is test underscore video dot mp4 so this will import our video but now we need to display it and as you know video is just a sequence of images so we will need a wide loop to actually go through each frame one by one so we will add a while loop while true now we will capture our image so all this will do is it will save our image in this variable and then it will tell us whether it was done successfully or not so this variable will be a boolean which is true or false next we are going to show this result so we will use the imshow function as before and then we will type in our video for example and then we will say IMG now we are going to add something that will add the delay and wait for the keyboard cue press if we want to break out of the loop so you do not need to go into the detail of how this works all you need to know is that this adds a delay and it looks for the word the key press Q in order to break the loop so let's run this and see what happens so here we can see our video now the video is quite short so it played and it turned off so here you can see and if we press Q it will close so that's great now we have successfully imported a video next we are going to learn how to use a webcam now using a webcam is very similar to importing a video so instead of the file path we are just going to write the ID of our camera so if you just have one webcam or you have a laptop connected so you can press 0 and this will use the default webcam if you have more than one you can add the ID as you go along so here we will create this webcam object and then we will define some parameters for it so we want it to be of a specific size so we will define the width which is the ID number three as 640 and then we will define the height which is ID number four as 480 now the rest of the code pretty much remains the same so let's run this and see what happens so here you can see the live feed from my webcam so as you can see the brightness is not very pleasant so what we can do is we can change the brightness from the settings and the ID for that is ten so we can write for example a hundred so now you can see the brightness is much more pleasant so let's have a look some basic functions that will be required quite often when we are building OpenCV projects so let's start by importing an image we will use the I M read function and we will define the file name as Linna in the resources folder now the first thing we will do is we will convert it into grayscale so to do that we will define our gray image IMG gray which is equals to now we are going to use the functionality or the function from the CV to package known as color CVT now CVT color basically converts your image into different color spaces so you have to define your image the one you want to convert and then you have to define which color space do you want it to be convert it to so here we are going to convert it into grayscale now conventionally we use red green and blue which is RGB but in open CV the image Convention the channels are BG R so we will write BG r to gray so let's output that and see what happens so we will use CV to dot I am show and here we will write the file name or the window name let's say we will call it gray image and we will display our gray image IMG gray and again we need to add C v2 dot white key and we will put it as 0 so let's play this and see what happens so here we have our gray image now let's move on to the next one which is blur so we will declare our blur image now we are going to use the Ossian blur function to blur our image so we will write C v2 dot Hassan blur and we will use our we can use our original image which is the colored image or we can use our gray image to add a blur so let's use the gray image and now next if you see here you can see it says key size which is your kernel size so you need to define the kernel size so we will say for example this is 7 by 7 so it will add quite a bit of blur so it has to be odd numbers so it can be three by three five by five seven by seven so we will write that and then we will say that our Sigma X is zero so don't worry about too much of all the details we are just scratching the surface of how we can get up and running so we will copy this and we will write here blur and we will change it to blur as well so let's play that and here we can see the difference between these two images so here is the gray image and here is the image added with the blur next we are going to look at an edge detector now this particular edge detector is known as canny edge detector so in order to find the edges in our image we will use the image canny we will call it image canny and then we will use our canny function we will assign the image and now we will add the threshold values so we have two thresholds so for the sake of simplicity we will put it as 100 and 100 so now you can go higher or lower but again it depends on your particular situation so let's copy that and we will go right cannae and here we will so let's run this and there you go so here you have can you image the blur image and the gray image so here we can see we are getting a lot of curve a lot of edges so if we wanted to reduce that we will change the value of the thresholds for example we can put this to 200 and we can put this to 150 so let's run that again and here you can see the difference now the edges are quite low next we will look into dilation now sometimes we are detecting an edge but because there is a gap or because it's not joined properly it does not detect it as a proper line so what we can do is we can increase the thickness of our edge so in order to do that we will use image dilation and then we will write C v2 dot dilate now here we will use the image canny because we are talking about edges and then we have to add a kernel now a kernel is just a matrix that we have to define the size of and the value of so in this case we need a matrix which has all one values but we need to define the size of that matrix as well so as I've mentioned before there is a library or a package that helps us deal with matrices and that library is numpy but we did not install it it so we are going to go to file we are going to go to settings here we will go to our projects and we will add numpy and we will hit install once the installation is done we will go back and we will import numpy as NP so whenever we want to call a function we can write n P dot whatever the function is so in this case we are going to define a kernel so let's say our kernel is equals to numpy dot once which means we want all of the values to be 1 and we will define the size of the matrix which is 5 by 5 then we are just defining the type of the object which is unsigned integer of 8-bit which means the values can range from 0 to 255 so we go down and here we will add our kernel after that we need to define how many iterations we want the kernel to move around which means how much thickness do we actually need so iterations is equals to let's say 1 so we are going to copy that and we will write image dilation and here we will write dilation image by relation image so let's run that and here you can see this is the original canny image and this is the image with dilation so if we increase the iteration number let's say we do something dramatic and put five then we see a massive change in the thickness so the next function we are going to learn is the opposite of dilation which is erosion so we are going to make it thinner so for that we will define our image as image eroded is equals to C v2 dot erode function and again we need to define which image do we want to erode so we will take the dilation image and we will erode that and then we need to define the kernel again so we will keep the kernel same and again we need to define the number of iterations so in this case we are going to put for example 1 so let's put this back to 1 and we will add our image for erosion so here is our eroded image here is our dilation image and here is the original canny image let's have a look at the OpenCV convention in mathematics while plotting a graph the positive side of the x-axis is towards east and the positive side of the y-axis is towards the north in OpenCV the x-axis is the same but the positive y-axis is towards the south to understand it further let's look at an image given that this image is 640 by 480 the origin of the image would be at this point where as the maximum width and height would be at this point looking at a few more points we can further understand the convention first we are going to learn how to resize an image now to resize the image we need to know the current size of our image so here we have a simple example where we are importing the library and then we are importing image by the name lambo which is short for lamborghini and we are just displaying it so let's look at our image so this is our image now to find the size of our image we are going to write print and then we will check the shape of our image so IMG dot shape so if we run this again it gives us 462 by 623 by 3 now this first 462 is the height 623 is the width 3 is the number for your channels which is BGR now in order to resize we are going to use the resize function so we will write IMG resize is equals to C V 2 dots resize and here we are going to write which image we want to resize which is our main image and then we need to define the width and the height of our resize so here we have to define the width first and then the height so the width let's say we want to make it so currently we have 623 let's make it 300 and we can make the height as 200 so let's show this image so here is our original image and here is the image that we have resized so as you can see we have successfully resized the number of pixels so if you want to check the shape again you can write print and instead of image we can write image resize so if we print that again you can see the original image is the one above and the resized image is the one below so the same way you can actually increase the number of pixels as well but of course it will not increase the quality but it can increase the number of pixels so for example 1000 by 500 so here we have a stretched out image so yeah so you can do that too so next we are going to learn how to crop an image and cropping can be very useful when you want specific parts of an image now image itself is just matrix or an array of pixels so what we can do is we can deal it in terms of an array or a matrix so what we can do is we can write image cropped is equals to image so this is our main image that we want to crop but now we don't need and OpenCV function we can just use the matrix functionality so we can say we can define the starting point and the ending point for both our width and height so for example now this is a little bit tricky because the height comes first and then the width above in the opencv function the width came first and then the height so here we have the height first and then the width so we want to define how much height do we want to maintain so let's say we want so this is our maximum height 462 so let's say we are going to keep it from 0 200 0 to 200 so 0 till 200 and then the width we are going to keep let's say for example from 200 to 500 so 200 to 500 so we are going to display this now so let's run it and there you have it so this is your cropped image so we can hide our resized image and there you go this is your cropped image coming directly from your main image so if we can just put it back it will go somewhere here yes so what we have done is if you can yeah if we can do it like this we can see better the numbers so the height we cropped from 0 to 200 so this is your starting point 0 and till here is 200 so this area till here is 200 and then the width is from 200 so we started from this point 200 and we went till 500 which is this point so this is how you can crop an image you so now we are going to learn how to draw shapes on images we will learn how to draw lines rectangle circles and we will also learn how to put text on images so first we will create a matrix filled with zeros now 0 means black so we will use the numpy library to actually create our matrix so we will write image is equals to n P dot zeros now inside that we will define the size of our matrix which will be 512 by 512 and then we will just display our image so let's write it down here C v2 dot I am show and then we have the window which is image and then the image itself so if we run this and there you go so this is our black image now this is a grayscale image because it has only 512 by 512 pixels or boxes so we can we can confirm that by printing this out and we can write image dot shape this is how you check the dimensionality of an image or a matrix so this is 512 by 512 as we have mentioned here but now if we wanted to add the color functionality we have to give it three channels so we will write that this is 512 this is 512 by 512 by 3 this gives us the values from 0 to 255 now next we are going to look at how we can color this image so if you wanted to color the complete image we will just write IMG we will say that we want to do it for the whole image and then we will define the color so for example I want it to be blue so I will write 2 by five to fifty five and zero and zero so if we run this now you will see the whole image becomes blue now what does this : in the middle means so if you remember from the previous chapter when we did cropping we used the limits of width and height so this is the same the same concept so for example if I write here 200 and 300 then I will write another range the first range is the height and the second range is the width so let's say I write here 100 to 300 so if I run this now you will see the colored part is only the range that we have defined here so if we wanted to color the whole image we will just write a column this means that it is for the whole image itself or the whole matrix now moving on we will learn how to create let's let's just put this back to black and let's just comment this out as well now we will learn how to create lines so for line we have the C v2 dot line function so C v2 dot line and then we have to define which image are we talking about so we will say IMG which is our image then we have to define the starting point and the ending point of the line so the starting point let's say we will put 0 0 and the ending point let's say we will put 300 and 300 then we need to define the color so we will say 0 then 255 and let's say 255 this will give us green and then we can define the thickness it is not important but you can define the thickness so let's say we will put it as 3 so let's run that and there you go so our image is now green sorry our image has now a green line and it starts from zero zero and it goes up to 300 300 now if you wanted to bring it till the end we what we can do is we can write the width and height instead of giving it a number so we know that image width and height we can get from the shape now the shape matrix has three elements the the height the width and the channels so the first one is zero which is the height so in this annotation we have to define the width first so we will write here one and then we will write here image dot shape and then we will write here zero so this is the height this is the height and this is the width so if we run that we should get a diagonal line going all the way okay that's great so next we will move on to rectangle so we will write CV to not rectangle and it follows the same convention so you have to write the image and then you have to define the points let's say zero zero and then we have to define the ending points so that will be the corner point at the diagonal point and let's say that is 250 by 350 okay and then we will add the color so let's change the color this time so we will put 0 0 and 255 and then we have to define the thickness so thickness let's say 2 so let's run this and there you have your rectangle now the thickness you can increase but what if you wanted to fill this area so you can't just keep increasing the thickness until it gets you know in the middle oh you can't do that but it's not a good yeah cv2 actually has a function for that so you you can write C v2 dot filled and you have to write it in capital letters so you can use that to actually fill your rectangle so next we will move on to let's let's put it back first next we will move on to circles so we will write C v2 dot circle and then we will write our image we have to define at the center point of this in the circle so let's say 450 then we have to define the radius let's say 30 and then we need to define the color color we can put 255 255 I think it will be a shade of blue and let's put thickness as 5 let's see here you go ok so now you have the circle and you can see our center point is 450 which means we start from here till here 400 and from here till here is 15 and we have the light blue color okay so next we are going to learn how to put text on images so we have the function called C v2 dot put text and we will write again like before we will write our image and then we have to define what text you want to show so we can display here for example open CV it should be say open CV and then we will write the origin where we will start it so let's say we want to start it at 301 right and then now this one is a little bit different than before we have to define the font of our text so CV to already has a few fonts in its library but there's there's no not a lot of them but we have a few so you can write C v2 dot font see me - dot font and then a bunch of fonts will show up we don't need anything fancy so we you'll just select the first one and then we have our scale then we have our color let's put it 150 and then 0 and then we have our thickness so let's run this and see how it looks like so there you go we have it in green written open CV so let's put it down a little bit so we will put it down let's say 200 yeah so let me show you what thickness and scaled us to the text so if I increase it to 2 now you will see it's much bigger in fact it's going out of the image itself so let's put it back to 1 by the way you can put it at points decimal places as well so you can write 0.5 and it will it will become really small and then you can define the thickness as well so for example we'll put this as 1 and here you can see the difference so now we are going to learn how to use word perspective on an image to get its bird I will so what we have here is an image by the name cards in the resources folder and all we are doing is we are displaying it using the I am show function so if we run this we will get our image what we will do is we will try to get this King of Spades and we will try to get it as flat as possible so we will need the four points and using these four points we are going to apply the board perspective so the first thing we will do is we will define our four corner points of the card so I've already taken the numbers and we will just type them in so we will declare points as a numpy array of float and inside that we will have four different points so let's type in the values I have 1 1 1 then we have 2 1 9 so this is for this particular image and for that particular card that we were referring to before so this is 1 8 8 and 1 5 4 if you want to get these values you can open up paint on windows and it will when you move your cursor around at the bottom it will give you the values of the pixels so those are our points now for each one of them we need to define which corner are we referring to so is this origin has in the first point on the left-hand corner and is this the last one so you have to define all of these so in point 2 we are going to define it loads 32 and then again we have four points so first we have our 0 0 then we have width and 0 then we have 0 and height and then we have width and height now we did not define width and height that's why it's giving an error so let's define width and height so a playing card is normally 2.5 by 3.5 inches so we will keep the aspect ratio so we can put 250 by 350 so now we will get our matrix the transformation matrix that will be required for the perspective itself so the matrix we can get by CV 2 so we have a function get perspective transform so get press perspective transform there you go so you have to define the points so points 1 and point 2 now we can get our output image based on this matrix so we can say image output is equals to c b2 dot warp perspective warp perspective and then we need to define our source image and we have to define our matrix and then we have to define the width and the height that we defined earlier so width and the height so now if we output this image let's see CB 2 dot I'm sure and then we can say outputs and then we have our image up okay okay so let's run that and there you go so we are getting the word perspective of this image based on these points so now we are going to learn how to join images together now this can be useful if you have a lot of images and you are running it again and again so it's hard to manage all these windows together so we will put all the images together in one window so let's see how we can do that so we have an image here by the name Linna in the resources folder and we are going to stack it with itself so first we are going to use the horizontal stack function so horizontal here we will say numpy dot now these are numpy functions not OpenCV functions so we will use numpy horizontal stack and we will write our image first the first images image and then image again so let's display this I am sure so let's display this see v2 dot I am show our window name is horizontal and we will write our image actually it's better to write I am G horizontal and then I haven't G horizontal so let's run that and there you go so now we have the image stacked together with itself in the horizontal direction now let's do the vertical so image vertical is equal to NP dot for vertical stack and then we will define image and image so we will copy this paste it down image vertical and we'll call this vertical okay so now we have two images one horizontally stacked and one vertically stacked now there are few issues with this method one we cannot resize the image it will come as it is so if I want to stack two more images on on the right hand side it will take up the whole space or it might go out of the frame so the other issue is that if the images do not have the same number of channels which means they are not RGB both of them or maybe one of them is gray one of them is RGB then it will not work so both of them have to have the same number of channels because we are talking about matrices so what is the solution for that so for that I have created a small function that can be called and it can handle all these things so all you need to know is how to call that function so let's look at that function I will copy that here so I will just comment this out and at the top I will add the function so here we have our function let's bring this down so you do not need to worry about all the details of this function you just need to know that it stacks images together how to use it this is what you need to know so let me explain how it works so what you need to do is you need to create an image stacks for example you can say image stack is equals to now you will call the function stack images and then as it mentions you have to mention the scale so you can scale all the images down and you can scale them up as well so let's see we will put 0.5 as the scale and then you need to define the matrices of the images so let's say I have image image and image so this will give us a horizontal stack of weight we need to display it so let's copy that and we will write here amid stack and we will write here image so there you go so now it's scaled down and we have three images together so now if we wanted to add the vertical stack we will just add a comma and then we can add another row but again if if you have three columns in the first row then you have to have three columns in the second so it's quite intuitive anyway so here you can see you have easily you can stack all the images together and even if it's one of the images is not the same channels you can still stack them together so let me demonstrate that image gray is equals to see v2 dot CVT color and then we can put our image and see v2 dot color VG are two cray and we can put let's say in the middle here the gray image so let's run that and there you go so you have a gray image stacked with the other colored images so here we are importing our library and we are importing an image in the resources folder by the name Lambeau dot PNG so we are just displaying it using the imshow function and we are adding a delay so that it does not stop it does not disappear so this is our image so our task will be to detect the orange color in this image so first we are going to convert this into HSV space so we will say image HSV is equals to now as you remember we have been using the CVT color to convert it into grayscale so we will use the same function CV to color to convert it into HSV so we will say that we want to convert our image and we want to it to be CB 2 dot color color underscore BG our to HSV so this will convert the image into HSV we can copy this and then we can check out our new image so here we have our new image which is the SS way and I did not write a new name that's why it's overwriting HSV so this is the original and this is the HSV so now we need to define some color values some ranges in which we want our color to be so we will define the hue the saturation and the value limits and within that limit if the image region falls within that color range we will grab that so let's do that so but one thing to note is that we do not actually know what are the minimum and maximum values that we need for this particular orange color so what we are going to do we are going to introduce something known as track bars that will help us play around with the values in real so that we can find the optimum minimum and maximum values of our color so to introduce track bars we are going to create let's create it on the top here so we are going to create a new window by let's say by the name track bars so we will say C v2 dot named window we will call it track bars and then we are going to resize it so that it is not weird-looking and we will write here track bars now that this name should be the same so just keep in mind not to do any spelling mistakes here and then we need to define the size let's say 640 by 240 then we are going to create our first track bar cb2 dot create now we are using the create track bar function now keep in mind that the T here is capital now first we will define what value are we going to change using this track bar so this is just a name so we can write anything so the first value we will be changing will be the hue minimum and next we are going to define which window are we going to put this track bar on so we have already named our window as track bars so we are going to use that now we have to define the the current value so when the script runs what will be the initial value that it will run with so we will put it at 0 and what will be the maximum value of our hue now as you know you has a maximum value of 360 now but we do not have 360 here in Company B we have till 179 which is basically 180 values so we will put 179 and at the end we have to we have to call a function which will run every time something changes in the track so every time the user changes the track bar it will call this function but we are going to get the values in another way I will show how we will use that later on but for now we do have to define this function so but we can say this is an empty function and at the top we can define empty and we can say that just pass that's it so that will pretty much do nothing so that's how you create the track bar so all you need to do now is to run and see what happens whoops okay so once we run it we are getting this error I would say see v2 dot resize it's not actually resize it's resize window so we will resize a window and let's play that again we should have yeah we have the track bar here so we have the hue minimum and you can see that the value is basically going okay we are missing yeah we just need to put here something it will take in an argument and that's it we'll run that again and if we use the track bar again so now you can see that the hue value is moving around and the minimum is 0 and the maximum is 179 so how many values do we need we need six values because we will have humanum um then hue maximum then saturation minimum saturation maximum and value minimum and value maximum so we will copy this a couple of times and we will just change this to max and then we will change this to saturation saturation and again we will change this to max and minimum will stay the same this will be value this will be value and this will be max so now these values they range from 0 to 255 so we will right 2:55 and the initial values we will keep them the same but for the maximums we will keep them at maximum so here we will put one seven nine here we will put 255 and here we will put 255 so if we run that now we will have six track bars that we can move around so that is good yeah that's pretty much good so what we will do next is we are going to read these track bar values so we can apply on our image so here we are going to get our values using the kept track bar position function so we will say our H minimum basically is equals to say V 2 CB 2 dots gets track bar position there you go so then we will write which which value are we talking about now the spelling here have to be exactly the same so we will write this here and then we are going to say to which track bar window does it belong so our window name is track bars so I'm going to copy this and I will paste this here so to confirm we can just print H underscore minimum now in order to get the value what we need to do is we need to put it in a loop because we have to run it again and again to keep getting that value so instead of the image we will have to change it to a webcam or just add a loop so we can write here while true we want this to keep running and we will put this in as well so instead of adding complexity we can just put 1 here and that should do it so right now we can see the value is 0 and then if we change it you can see the value keeps changing right so next we are going to apply this to all of them and we will copy this so that we have all the values so you will do it again five times and this time around this will be max then minimum and Max and then minimum and Max then we will have the saturation okay then we have the saturation then the value and then the value so that is about that and then we will write here max and here will be saturation saturation and then max here will the minimum value and then value and then max so we can print all of them out just to see just to make sure that if it is correct we will just say H minimum then H maximum then we will say s minimum then s maximum then value value minimum and value maximum so let's see how that works out so there we have it 0 1 7 9 0 255 0 and 255 so if I change my values you can see the track bar actually changes these values in real time so now that we have these minimum and maximum ranges of the hue saturation and value we will use these values to filter out our image so that we get that particular image in that range particular color in that range so now we will create a mask we will say that the mask is equals to see v2 dot in range so we are creating a mask that is in the range of these colors so which image are we talking about we are talking about the HSV image and then we have to give it the minimum and the maximum range so we will say this is the lower limit and we will say this is the upper limit now we need to define this lower and upper limit so we will write here let's do the lower first lower is equals to we will create an umpire array so let me just add that here import numpy 10p now down here we are going to add an umpire array so NP dot array and then we are going to create the minimum array so which is H minimum and then we have s minimum and then we have the value minimum value minimum the same way we are going to do for the maximum so we will say upper is equals to numpy dot array and we will add our maximum limits which is H max then s Max and then okay so this will give us the mask so basically what it will do it will filter out and give us the filtered out image of that color so let's see how that looks like or you know need to copy that we can just base this here and we can say this is the mask and this will be mask so let's run that and there we have the mask here we have the original image and here we have the track bar so if I move this around you can see how the value how the image changes so we want to keep all the colors that we don't want as black and we want to keep the color that we need in white so if I was to detect the orange I would say that is pretty much good so I recommend keep changing the values try to keep it smoother and smoother and eventually you will get some good results so now that we have these these values what we can do is we can put them as our initial values so we have 0 19 110 240 153 and 255 so we can go back here we can still open up our track bar and we can go up here oh it's let me keep it on the side and I can just say what it is so this is 0 and then this is 19 this is 110 and this one is 240 then we have 153 and then 255 so now if I run this again by default I will get the mask right and next what we can do is we can get our result which will be our original image so instead of getting this black and white mask we can get the actual colored parts the orange color over here so how we can do that is by creating using this mask we will create a new image so we can say image results is equals to we are going to we are going to use the and operation so we have the bitwise hand operation which will add two images together to create a new image so it is basically checking both the images and wherever the pixels are both present it will take that as a year or as a one and it will store that in the new image so what it does is we are going to say that we have our original image that we want to use and our new image will be also like our original image but with a mask applied which is our mask which is the one we created before so let's look at the image results image result and we should change the name here let's run that and there you go so now we have the colored image so if you did not get that basically we are we are checking these two images the mask and the original image and we are checking wherever we have these white pixels we are getting from this image and creating a completely new image from it so that is what we are doing so one thing we can do here is to add our function from our previous chapter in which we joined the images so that we don't have to play with all these images again again so if we go back to our previous chapter we have the stacking function we can copy that and bring its hair let's bring it here and then now we can go down this is the by the way the tracking function so sorry the stacking function so it is stack images so we can go down and instead of showing all these images separate we can just show the stacking image so what we can say is that stacked or we can say I am G star I am game stack is equals to now we have to write our function which is stack images we have to define the scale the scale let's keep it at 0.6 and that we have to define the array of images so we are going to put here we are going to put here the images that we need so we can put two images and then two images okay so we will create IMG and then I am the HSV and then in the new row we can create we can add the mask and then the image results and then we just need to display the final image so we can stay say start images and we will write here IMG stock so let's remove all of these and we can play again and there you go so now we have a neat so now we have a neat image with all these images staggered up and showing us the values altogether so now we can have the track bar on one side and we can see the results directly applied in real-time so let's check what the results are and yeah so so that's how you can detect colors so now we are going to detect shapes in an image so we will detect their contours the corner points and based on those we will decide what is the shape of this object so what we have is our packages we have the image of shapes in the resources and what we are doing is we are just printing it out using the imshow function so this is our image it has triangles circles square and rectangles so we are going to detect each one of them and we will categorize which category it belongs to and we will show how many corner points it has plus the area of each of the shape so let's start so we are going to pre-process our image first so we are going to convert it into grayscale and then we will find the edges so that we can find their corner points so let's start by converting it into grayscale so we will use the CVT color function and here we will write our image that we want to change and we will define what color do we want the channels to be so we will say BG our to gray then we will add a little bit of blur so we will use the Gaussian blur function so we have our image gray and then we need to define the kernel so we will say let's say it is seven by seven and then we have the Sigma so the higher the value of the Sigma the more blur you will get so let's let's put it at 1 and we will see if we need more so then we are going to detect the edges actually before that let's print it out and see if everything is working well so we have the image gray image gray and then we have the image blur so we will write here gray and we will write here blur so if we run this we should get three images and yeah so we are getting the gray image and then we have some blur added to it actually at this point it will be a good idea to use our stacking function so that we don't have to play with that too much so in chapter 6 we yeah this is the function we used so let's copy dots and we will go back and we will paste it here at the top so let's use the stacking so we will say that our image stock is equals to tagged images we will give it a scale of zero point six and then we will define our image array now inside that let's say we are going to put our image and then image gray and then our image blur so we can remove all of these and then we can just say stock and then image stock so I put on this and there we go so we have our image all three images stacked together so next we are going to find the edges in our image so we will use the canny edge detector so we will write image canny is equals to C V 2 dots canny canny we will define the image will give it the blur image and then we have to define the threshold so we can just pick some threshold let's say 50 and 50 and see if it works well so we can add another another row and so we can add the image Connie image Connie and then the rest we can keep the same or we can define a block image so we can define our image blank so we can say image image block is equals to numpy dot zeroes like image and then we can put this here let's run this and there you go so now we have our edges and from these edges we are going to find our contours so let's go back and now we are going to create a function so earlier we were not creating functions but because this is a little bit of more code so now we are going to create our function so let's go up here and we are going to define a function by the name get contours so in this we are going to define our image and it will input an image and then from there we will find our contours so the first thing we will do is we will write contours and then we will write hierarchy hi are let's keep it small and then we are going to use our cv to dot find contour function find contours now in that we have to define our image the one we want to find the contours in and the second parameter is our retrieval method we have different types of retrieval methods the one we are going to use is the external method so retrieval we will write our e what is it our et he R yeah our et he R yeah external and what this does is it it retrieves the extreme outer contours there are other alternatives that will detect all the contours they will not be filtered out but this one is specifically good if you want to find the outer details or the outer corners so we will use this and then we have the approximation where you can request for all of the information or you can request for compressed values so it will reduce the points for you for this example we are going to get all the contours that we have found so we will say C v2 dot chain approximation as none so once we have our contours they are saved in the contours so what we will do is we will say that we want to loop through it so for C and T in contours on tours so for each contour we are going to find the area fest so we will say area is equals to C B two dots contour area and we will just mention the contour that we want to find the area for now the second thing we will do is we will draw this contour so okay let's before we do that let's just print it out so you can see what kind of areas are we getting so if you run this oh actually we have to call the function my bad so we need to go down and here we will say get contours and we will send the image scanning and there you go so these are all the areas we are finding for each of what we call the shapes that we have detected so next we are going to draw them so that we can see them clearly we are going to use the draw contours function so see v2 dot draw contours we will give it the image that we want to draw it on so we are going to use let's create I don't want to put it on the original image so what we can do is we can create a copy of our original image so we can say here that image contour is equals to image dot copy so we will copy the image so that we can put the drawings on this image the image contours so we are going to define where is it so we will say image contour we want to put it on that and then we have to give it the contour itself so that will be CNT and then we have the contour index so here we will say negative one because we want to draw all the contours and then we have to write 255 then let's say we are going to make it what color is that that is blue okay and then we will put it as thickness as three so so each time it will loop it will draw the complete control it finds in CNT so this is what this statement is doing so let's run that and we did not put the image contour helps image contour and there you go so now we have all the shapes and we have detected all the contours you can see there is a blue line around them and it seems to have detected fairly well so next what are we going to do so next we are going to check for the minimum area so we will give it a threshold normally if it's just an image like this you don't have to but it's a good idea to give a minimum threshold for the area so that it does not detect any noise so we can say if our area is greater than for example five hundred five hundred pixels then we are going to do the calculations the maps of it if you want to completely neglect it then we will just put it down here rather than up there and if we write that and we can see all of our shapes have areas greater than 500 so we should not have any issues so the next thing we are going to do is we will calculate the curve length so the curve length will help us approximate the corners of our edges corners of our shape so let's write that down so we are going to write here perimeter is equals to C v2 dot arc length and we want to find the arc length of our contour and we will say that it is closed so we will put it as true so we can we can print this out if you want to see its prints Perry and so so now we are getting the length of each of the contour the arc the arc length or the parameter recall or the control parameter so next we are going to approximate the the corner points how many corner points we have so we will write a procs is equals to C V 2 dots aprox Poli and we are going to give our contour and then we will give it a resolution so this you can play around with so we will multiply it with our length the arc length so here you can play around with this a little bit if you are not getting good results and then you can write here true again because it is closed we are expecting all our shapes to be closed so next we are going to print the value of our aprox now this will give us the points of the corner points so of each of them each of these shapes so if we see here for each of them we have certain number of points so here for example we have 3 points here we have more than 3 maybe 7 or 8 then we have 1 2 3 4 3 and then so on so what we can do is instead of printing this we can print the length of each one of them so that will give us the idea of what the shape is so here you can see we have 3 8 3 so we can say that 3 is triangle 4 is either a square or a rectangle that we will find out later as well and then we can say that anything above 4 is a circle so because we are only detecting 3 or 4 categories so we can assume that so next we are going to create our objects corners so object corner is equals to length of our approximation and now we are going to create a bounding box around our detected object so now we are going to get the values of our bounding box if we were to draw a bounding box around our object what will be the X&Y and the width and the height so let's get that X Y width and height so we will get that from CV two thought's bounding box bounding rectangle and then aprox okay let's put some spaces here okay so this will give us the X Y and the width and height of each of the objects or the shapes and let's let's draw them right away so that we get a clear picture of what we are doing cv to dot rectangle and we we want to draw it so where do we wanna draw it let's draw it on the image control and then we are going to give it the point number one which will be your x and y so it will be X and Y and your point number two will be your X plus width and X plus y plus height so it will be X plus width and y plus height then we are going to give it a color let's give it a different color so that we can tell it is a bounding rectangle and then we will give it a thickness so that should do it so let's rerun it and there you go so now we can see that we have bounding boxes around each of the shapes we are detecting so from these bounding boxes we can get information such as what is the total width and height of the object where is the center point of the object so these kind of things can be useful maybe in another project but for now what we will do is we will categorize these objects into whether they are circle whether they are triangle rectangle or square so the first one is let's start with the smallest one which will be our triangle so we will say that if our objects corners are three then we want our object type object type this is a variable to be triangle there's just a string so what we can do is we can print the string out in our image so we can say C v2 dot foot text we will define our image contour and then we are going to write what we want to print which will be our objects what did I write it as oh this is capital let's make it small so object type and then we are going to write where do we want to print it out so let's print it at the center of the object a little bit shifted from the center of the object so what is the center of the object so this will take some room so I will put it down here so the center of the objects will be X plus width divided by 2 and we will shift it a little bit so we will reduce 10 pixels and then we will write the center of our Y which will be y plus height divided by 2 and this will keep the same or should we let's let's put some what you call deviation in this as well so let's put minus 10 so it will be a little bit higher than the center point so what is the next thing yeah the next thing is the font so we will write C v2 dot fonts we don't want anything fancy civil rights they will use the first one and then we will write our scale scale let's put it zero point five and then we have our color so let's put the color as zero two five five and then two five five and then the font scale we will put it as two okay so where did I do the mistake because it's not showing it properly yes the comma okay so that should give us triangle everywhere there is a triangle hmm object type reference before object type oh okay so we did not define what happens if it's not three so for now we can say else object type is equals to is equals to none so there you go oh that's yellow in color which is kind of hard to read so we will put it in a different color let's put it black I think black will be easy to read and we can increase the size a little bit so it's easier to see and we can also increase this size because we still have some room so let's run that and there you go so now we can see that all the triangles they are detected correctly and the rest of them are none so next we are going to define the rest of them so that will be here if our object all of it should be Elsa else if our objects honors equals four now this one is a little bit tricky here we are going to check if we are getting a square or a rectangle now how can we decide that what we can do is we know that a square if we divide the width and height we should get one because the width and height of a square is the same so what we can do is we can see that if we divide them we will get the aspect ratio and if that aspect ratio is between a certain range we can say it is a square if it's not we will say it's a rectangle so the first thing we will do is we will get the aspect ratio so we will write aspect ratio is equals to it will be width divided by height since we are dealing with decimal numbers we have to define one of them as float load so it gives us float value so once we have that we are going to put another if statement if our aspect ratio is greater than zero point nine five and our aspect ratio is less than one point zero five which means we can have a deviation of five percent then we will say that our object type is equals to square if that's not the case then our object type is equals to rectangle so let's run that and there you go so here we are getting a square rectangle square square rectangle square so next we will write it for the circle so we will write else if objects corner is greater than four then we are going to write object type is equals to circles so let's run that and there you go so now we have the circle we have the rectangle and we have the squared triangle so all of them we are detecting properly except for this one it will take a little more complication to solve that but we can try that later on to detect faces we are going to use a method proposed by viola and Jones this was one of the earliest methods that allowed real-time object detection so if we were to detect faces we could collect lots of positives which will be the images of faces and we will also collect a lot of negatives which will be the images of anything but faces using these negatives and positives we will train and create a cascade file that will help us find faces in our case we are not going to train the model but instead we will use a pretent file for faces which is provided by open CV now open CV has provided some default Cascades that can detect different things such as number plates eyes full-body etc if you want to learn more about creating custom Cascades I have a separate video for that which will be available in the description so we are importing the linear image from the resources folder and we are displaying it using the I am show function so the first thing we will do is we will add our Cascade so we will write face cascade face gasps okay is equals to C V 2 dots Cascades classifier and then we have to write the name so resources of three sources slash then we will write our Cascade underscore frontal face underscore defaults dot XML so let me check the spellings again heart cross Kade and frontal face default XML so that looks fine so next we are going to convert our image into grayscale so image gray is equals to C v2 dot C V T color and then image and then we have C v2 dot color RGB to gray then we are going to find the faces in this image using our face cascade so we will say the faces is equals to face cascade dot detect we will use multiscale and we will write image gray and then we have to define the scale factor we will put it as 1 point 1 and then we have to define the minimum neighbors which we can put it as 4 so these parameters you can change based on the results you are getting so next we are going to create a bounding box around the faces that we have detected so we have to loop through all the faces that we have detected and put rectangles around them so we will write for we will directly get the parameters of our X Y and width and height so these are the four parameters that you require to actually create the bounding box so we will we will write X Y and then weight and height and then we will say in faces and then we will just draw the rectangle let's draw it on the original image and then we will give it the initial points so which are x and y and then we have to define the corner points the diagonal points which would be X plus width and y plus height then we need to define the color so let's put it as blue two five five and zero and zero and then we will define a thickness let's put it as two so yeah that's pretty much it so let's run it and see and there you go so now we have detected the face and we have created a bounding box around it so if you want to detect more objects there's a lot of Cascades available online that people have trained already or you can create your own custom cascade to detect cars to detect mobile phones TVs anything you can think of you can detect using these cascade method now this cascade method is not the most accurate one but it is fast so a lot of the cameras they also use these are cascade methods to find the faces still even though it's quite an old algorithm but still it works and it works well in certain circumstances you so for this project we need to find our colors and we need to find it using a webcam and then we can place different points wherever we find our colors to create the paint example so what we need first is the webcam so what we are going to do is we are going to take the code from our different projects and we will merge these together so that we can see how we can efficiently create different projects so we are going to go back to our chapter 1 and here we have the code for our webcam so we are going to copy this and we will paste it in our project so as we can see we are importing our library we are setting the width and height of our frame and these are ID number 3 and ID number 4 then we are also setting up the brightness let's make it 150 and then we have our now our device is not 0 it's supposed to be 1 and then we are getting into our while loop where we are getting our image and then we are displaying it using the I am show function and that's pretty much it so if I run this now it should show me the webcam and there you go so here we have our webcam and let's see if we can see yeah so that is clear so once we have this or the next step we are going to do is we are going to find our colors so to find our color what we need to do is we need to bring in the code of the color detection so that was in Chapter seven or six thousand seven so here we have the code so we don't need the stacking we're trying to make it simplified and so we don't need that so what we will do is we will copy the main code of our color detection which is basically the lower upper and the mask so this is the main thing that we will be needing and we will also need to convert it into this HSI space so the normal image into HSV so let's let's copy that first so we will copy it here and here we are going to define define a function to find our colors so we will say find color and then we are going to input an image at least for now we will just keep it an image and then we will take this image and we will convert it into the HSV space and from there we are going to use our upper and lower limits too so here we have our upper and lower limits and then what can we do then we can just show that if it's working properly or not so see v2 dot I am show and we can put for example image and we need to show our mask so this just for testing will remove it later on so we don't want to find just one color we want to find different colors so whenever we call upon the function find color we want to find all the different types of colors and whatever is present we want you see that as the output so what we can do is at the top here we can define something in the form of a list as our color minimums and maximum for the hue so what do I mean by that so let me write that down we will say my colors is equals to a list now this is a list immediate comment is annoying so my colors is basically a list of colors that we have that we want to detect so we need to give it the minimum and the maximum hue and saturation values for that I have written down a code by the name color picker so basically what it does it it helps us pick the right hue and saturation values using a webcam so it's pretty much the same code but added with the webcam that we have done in our color chapter so here we have our webcam and we have the color orange so let me find the values for that what you need to do is you need to keep the orange has white and the rest you have to remove so here you can see you have to go until yeah so I would say that is a good point so if I move it around it's not so bad but my hand color is also been detected so we need to remove that let's bring it down a little bit and there you go so it's much better now so this way what we need to do is we need to note down these values so it's 5 1 0 7 0 19 255 so what we will do is we will create a list over here and inside that list we are going to write down these values so let me write this down so we have 5 so we have 5 5 1 0 7 0 19 255 and then 255 so these are our values as you can see here so we will create a list of these values different types where where we will write down all the different types of colors that we want to detect so I have done it for blue not loop a purple and green so we will just copy that values here so this is one of them and then we have another one over here so these are our values so the first one is orange then purple and then green so these are the values that you can find from the color picker once we have done that now we can simply create our masks so here we can say not ok we need to add an umpire so import numpy as NP and the second thing we need to do is we need to put down our values so we know that this part here is basically the first three values here and then this part here is the other three values sorry the other three values over here so what we can do is we can simply write that our whole you call list of my colors is clean light colors and we can define let's say we will pick the first element which is zero and then we will write down not from zero from zero till three we need those as the first elements so we can define here that we need my color as the input and then we can do the same thing here and this will be the upper limits which will be from three to six so that should give us one of them which is the first one which is I think orange so let's run that this is the previous code we need to run project one so let's run that and I know we need to call the function as well so here after this we are going to call our function and my colors so let's run now and there you go we're getting the mask and if I bring my orange color and you can see that it is detected properly but that's the thing like we are only detecting orange we need to detect all the colors we have in this list so for that we need to add on a for loop so we can say for let's say color color in my colors for each of these colors we are going to create a mask so we don't need to write this down we can just replace this with colors color and then again color so if we want to display this out we cannot have a generic name we need to put a name that is changing let's use so the first element is different for each one of them so we can put that as a name so we can say string and then color the first elements and that should give us three different windows so if we run that and there you go so we are getting three different windows forget about the naming for now we're just testing it out so if I put the orange color I can see the orange one if I put the purple one I can see purple and then if I put green I can see the third one in green so if I wanted to add more colors I can simply just add it to the list and that should give me more colors for example if I remove this now let me remove that and it will only give me two of the masks so it will not show me green anymore so this way we have created something generic that will work for multiple colors so you can keep adding more and more now for each of the masks that we have detected let me comment this out for each of the masks that we have detected we need to find where is this object that we have found in our image now to find that we need to get our contours and we need to approximate the the bounding box around it so we can find our location of the object so for that we need to get into the code of our one of our previous chapters in which we learned how to find contours so if we go back this is the code for finding contours so what we will do is we will just get this function as it is and we don't need to differentiate between different shapes so we will get it till our bounding box we will copy this and in the project we will paste it here so this is our get contours function and now what we need to do is we need to change it a little bit so that we get the relevant information that we need so there's not a lot of things that will change so let's remove anything image contour okay what we can do is we can create a new image we can say let's say image result is equals to image copy so this will be the image that will have all the final information on it so all the drawings and everything will be on this image so we can copy image result and we can replace it here so it will draw the contours on this image and then what we don't need okay we can remove the parameters and we don't need the length or the objects and that's that's pretty much it so what we can do is now just to see if we are on the right track we just can see that if it's printing properly or not so from the fine color we are going to send this image of our mask to find the contours so we will send this we will say get contours and we will send the mask and because we are calling this function it should draw on this image so let's see if that works so what happened okay I mean yeah we remove that so it should not display any of the masks so let's try that out orange green purple so none of them are working great let's find out why so when we are sending the mask oh okay because we did not put here image results so we need to display the image result because that is the one that will have the the elements on it so let's run that there you go so we have orange we have green and we have purple perfect so now that we know that we are getting those and we are getting the bounding box around it all we need to do is we need to send these values now we can send the center point but we want to draw from the tip of the pen not from the center of the object detected so we will send the center rather than so we will send the tip rather than the center so what we will do is we will return so here we will return and we are going to return the value of x plus our width divided by two so it's in the center and then we are going to send them the value of y as it is so this will give us the top point of our tip points and the center of it as well and in case if it's not if it's not greater than 500 or if it's not detected we still need to return something so for that we need to declare these as zeros so we can say X Y X Y W and height just in case if we need it later on we can put all of them as zeros so we can get those so it will return this value so we need to get this value over here so that we can use it so we will put it here as x and y so once we have that value we can draw a circle around it so we can say see v2 dot circle and we need to draw it on the image results and our centre point will be X and Y and then we will define the radius let's say 10 then we have to define the color for now let's say 2 five five zero and zero and then we are going to write C v2 dot fill as we want it to fit so if we run that now let's see what happens so if I go here yeah so I'm getting the blue point now I can see not I'm getting the correct point now again this is this is the centre of the bounding box so if I turn it around the bounding box it will be the center of the bounding box not the centre of the contours to fix that it's a little bit tricky so we're not going to go into that much detail but for now it should be fine if we keep the pen straight now one thing we need to change is the colour so it should not be blue it should be the colour that we are detecting so we can remove the contour we don't need the contours anymore we know that we are detecting it properly but we need to change the colors of these objects so what we need to do is we need to define our color values so for example if this is detected what should be my colour on my drawing so we can say that we can call this my colour values and again we have to create it as a list and in that list we need to define all the colors that we want to display so we have three so we will define three now next here we are going to write our values so how can you find these values that so here we can see an example website where you can find the RGB values so if I go to orange so orange right now if I click on it you can see it's 255 153 and 51 so this is my value of the orange so what I can do is I can use these values in my body called code so so but again you have to make sure you are writing in the format of BGR so let me write that down here so it is VG are not RGB so we have to write so here we will have 51 then 153 and then 255 similarly we can find for purple and green so for that for purple it is 255 0 and 255 and for green we all know it is B G R so it should be 0 255 and 0 so these are our colors now and what we can do is we can draw the color of the circle based on these values so what we can say is we can say that we need my color value as the input as well so if we go down when we are sending the values we need my color values as well so we can say what can we say okay we need a counter to actually count how many times so we will put the count is equal to zero and every time it counts we need to know which color are we talking about so we can say here counts plus equals one so instead of our main blue color here we can say my color values and I need to get the value of my counter index so my count right now whatever my count is I need to get that value from this what we call list so if it's that count is equal to zero it will get this value count is 1 it will get this value so I can write here count so this should give us the correct color of our what you call marker so let's run that so if I have purple it's giving me the color purple if I have orange it's giving me orange it's a little hard to see but you get the gesture and then you have green so we are getting green so that is good we are getting the correct colors and the correct values now what we need to do is we need to draw these points so in order to draw it's actually fairly simple we are going to create a list of points and we are just going to display it every time and we are going to loop it around so at the bottom let's go at the top first and we are going to create a list called my points and inside this list we are going to have three things we are going to have the value of x we are going to have the value of y and we are going to have the value of the color or not the color itself let's put the index of the color so we can say if it's zero it will be this color if it's one it will be this color and so on so instead of the real thing we can put this so let me copy that and let me put it as a comment here so so what we can say is we can loop this my points and every time we can check the value of XY and we can draw the circle of this color at this point so it's fairly simple so in order to do that what we can do is we can create a new function called draw on canvas so we can name it draw on canvas and then what do we need we need I yeah we need the points and we will also need the color values my color values so what we will do is we will say that for each point in my points what we need to do is we need to iterate and we need to just draw a circle let me just copy it from here brilliant copy the circle and we can put it here that what we need to do is we need to draw on I am result our image result and we need to use the color of our count so this time around the values are inside the points so we will say that point at 0 is our x points at 1 is our Y and our ID is points at number 2 so this will give us the point at this color of this color now what we need to do is whenever we are finding the colors we need to send to draw okay so what we can say is that we can say that our new points points is equals to whatever we are getting from our colors and then we need to draw them so in order to draw them or should we do that first let's go up and let's return something over here so once we are finding the color we need to return our new points but we need to make sure that these points are not like they are detected properly these are not 0 0 so if it's 0 0 if the x and y if the value that we are returning is 0 0 then we don't need to do anything we don't need to plot it it will just add on the memory so what we can say is that before the count we can say that if X is not equals to 0 and if Y is not equals to zero then only we are going to append our points so we'll say new points dot append and we are going to say that we are going to write down the X Y and the count because that is our ID and we need to define the new points every time so every time the new points will change so it will start with an empty meet empty list and then it will append and it will return these values so here we need to write return not in the loop we need to write here return and then we are going to return our points so every time it adds on to these points it will send these new points over here now once we have our new points what we can do is we can check if the new points are actually there or not so we will say that if the length of our new points is not equal to 0 that we are getting something then equals to 0 then we need to iterate through our points so for new points new points we are going to append my points my point dots append and we are going to append our new point so why are we putting a for loop here because we are getting it as a list so we cannot put a list inside a list and then run our code of draw on canvas because it doesn't work that way we need all the points not a list inside a list so this is a list we need to break it down to points again so that's why we are putting a loop here so for new point in in our new points we are going to append each one of them in our points and at the end we all we need to do is we need draw these points if they are available so we can say that if the length of my points is not equals to zero then we need to draw them so draw draw on canvas and my points and use my color can values so this should draw all our colors in the right let's see so here we have our purple and there you go so you can see it's drawing purple then we have orange it's drawing orange and then we have our green so it's drawing green so there you go so this is very simple and it is a very kind of a generic code so that you can add on to it it's not hard-coded that this color equals this this color equals this we are not creating a lot of masks hard-coded and putting them together so if you wanted to add a new color it's very simple all you need to do is you need to go and add to my colors and my color value so for example if I wanted to do blue I have checked out the values for blue and again you can use the color picker code to find the values you can add that and oh we know already that our blue is basically BGR so it will be 255 0 and 0 so this will add the blue color and if we run it now and let me try the blue here and there you go so now it's detecting blue as well so this is how simple it is to add more colors so the good thing about this is that it will detect all the colors at the same time as well so if we look here we have these 4 colors and we can detect them at the same time you can see that looks nice you you so in this project we are going to start off with the webcam code which we did in the first chapter so let me go and copy this and let's change the parameters we will do it as one and this will be 50 so let's run it and there we have our webcam so we are going to detect this document and then we are going to use word perspective to get the flat out image of our documents which we will scan now let's move on so the first thing we are going to do is we are going to pre-process our image so we will apply some what do you call pre-processing techniques so we can detect the what I call the edges in our image properly so let's define a function pre-processing and here we are going to input an image so we'll say image and then we are going to convert this image first of all we will resize it so image is equals to say v2 dot resize or let's resize it over here so that the size remains the same for the whole process so resize and we are going to resize our original image with our width and height so width of our image and height of our image so this we can declare at the top let's create a new section variations 640 and this one here let's remove that so that been done okay so so the first thing we will do is we will convert it into grayscale so we will say image gray is equals to C V 2 dot C V T color and then our source is our main image and then we want to convert it into grayscale so CP 2 dots color PG hard to pray next we are going to apply some blur so image blur is equals to C V 2 dot plus in blur and we have our image gray to blur and then we will apply we will give the size of the kernel 5x5 and then the Sigma X let's say 1 and then we will find our edges using our canny edge detector so we will say image Connie is equals to C v2 dot canny and our image is blurred and then we have the threshold so we'll put let's say 200 by 200 so these values we can change as we move along but for now we are going to keep some random values and let's see how they go once we have done that sometimes the edges are very thin so what we can do is we can use the dilation function to make them thicker and then we can use the erosion function to make it a little bit thinner again so what we can do is we can apply two passes of dilation and one pass of body called erosion now this is not compulsory but it helps so it is better to do it dilation is equals to C v2 dot dilates lay yeah and then we are going to use our image Connie and we are going to define a kernel and then the iteration array raishin is equals to two so there is a mistake somewhere okay so kernel is equals to we are going to define an array of a matrix of a once so we need to define here you port numpy as NP and over here we are going to say that our kernel is numpy of once and the size is 5 by 5 then we are going to erode it so image road is equals to c v2 actually this is the final result so let's call it threshold image threshold is equals to C V 2 dots erode and then we are going to input our image dilation and then we are going to define our kernel and iterations has one so once we process that we can return our image return image threshold so we can say here image threshold is equals to pre-processing of our main image so we can display the image - and there we go so we are getting so if you wanted to see the what you call the output of each of these steps we can simply what you call commented out and return the one we want to see for example if we want to remove these two we can comment it out and return the Kami instead so if we look at that so here you can see our edges are quite thin and sometimes when there is some shadow or some reflections then it might not detect properly that's why we are adding this here so we need to return the threshold and there you go so the boundaries are much thicker and easier to detect what is happening in the image so next we are going to move on to our contours so we need to find the biggest contour in our image so this is what we will do now so in chapter 8 for the contours we use the function get contours so we are going to copy that till our bounding box and we'll make some changes so here we'll paste it out and now we have the CAD contour function so what we can do is we need to find the biggest one and that is available to us so here we are first getting our contours and then for each of them we are looping again and again so what we can say is that first of all we need to give a threshold for the area I think it's better to add another zero because our area will be quite big we can remove the print and to draw we can copy an image here so we can say image let's say contours that's right contours is equals to image copy actually we should do it after the resize cuz all of them or all of them should be same so let's go down and then we'll write it here and then here we'll paste here we will paste our image contour actually let's use this spelling it's better here it's code so we can display it out what is happening so then we will remove this parameter and we don't need to see the length do we need to see the net yes we do need to see the length and it should be four and then do we need the bounding box yes okay so what we are going to do is first when we are getting our area and then we are checking that if it is greater than five thousand if it's good then we will move on to the next step we will draw it on our image or do we need to draw we will see if we need to remove it later now we will get our parameter and then we are going to find our approximation of our corner points and then we have to say that if our area not sorry if our length of our approximation is equals to four in that case what we will do is we will say that the biggest is equals to aprox and but as it loops we need to find the biggest one so we have to put another condition here we will say that if our area is greater than the max area so we'll put an hand here to put the hand here and then here we will say that max area is equals to our current area so what this will do is it will keep looping again and again and whenever it finds a value that is bigger than what it has before it will replace the value of the current one so this will give us the biggest area and the biggest of our approximation which will be our four points so do we need this not really so we can remove this with the bounding box and then all we need to do is we need to return our approximation so we will return aprox oh sorry will return the biggest one biggest so this will give us our biggest contour okay so let's run that and see what happens now we need to call it so here we are going to say that get contours and we'll send the image threshold okay so one thing that we forgot to do is to define initially our variables so the first one is a list so the biggest will be an umpire list an umpire a number dot array and the second one is the max area so we will say max area is equal to zero so if we run that again and there we go so now we are getting our contours and hopefully it is returning us the biggest one which is this blue one so once we have our biggest contour what we need to do next is using its corner points we need to work our image and get the bird's-eye view so in order to do that what we will do is we will first of all get our approximation here so we can say that our biggest is equals to get contours image threshold and then after that what we need to do is we need to send it for word perspective so we will write a new function so we will say you get work and inside we are going to send our image and we are going to send what do we need to send we need to send our biggest contour point so here we can call this function you get work off our original image and we need based on the biggest points so actually we didn't print out let's see if we are getting it properly so let's just print out once we are getting the biggest one let's print it out so there we go we are getting four points so one two three and four so we are getting them properly so whether they are the correct one or not we will look into that furthermore or we can print it out right now should we print it out night now yeah actually we can what we can do is instead of printing them all we can copy this and we can comment this out and the last one we can do is we can print out the biggest one before it sends so we can say here biggest and there you go so if we look at it let me increase that size let's make it 20 so there you go so we are getting the correct points of our biggest contour so the next part will be to work so now we are going to work where it's not function so okay so in the work we are going to follow what we have done before in our previous examples that was in chapter 4 the warping so here we have for warping we need two points and then we create a matrix and then we get our word perspective so we will copy all of that and we will bring it into our current project so we need to find out that this is point one and then these points two and then we create the matrix and based on that we have the width and the height so this width and height are basically the width and height of our image so we will say what image and I'd image again width image and height image and then worked image and then high-damage we can remove the bass okay so the next step would be to put our points here so we are getting our biggest contour and all you need to do is we need to input the thickest contours over here so there are already four points so that way we can easily put it directly over here now the next thing is fine over here so what we need to do is we need to look at our image and there is a tricky part here and I will explain what that is but just let's just look at our output image so we can say here that return our image outputs and here we can call this function we can say get work and then we can say that our burped image worked image work is equals to that and let's show that so there you go so now we knew that our points are being detected properly but as you can see here the warping is not correct and if I move it around and it even gives us an error so the reason why the working is wrong is because we have an annotation we have a structure at which our point should be so the first point in our biggest contour should be 0 0 and then with 0 then 0 height and then width and height if for example in the biggest contour our width and height is in the first one and this 0 0 interest in the second one or it's mixed up then it means that our word perspective will not proper properly work so what we need to make sure of is that when we are sending the points to work all of our points are aligned like this so the smallest one should be the first one then the width and height the width and 0 then the 0 and height and then the width and the height so how can we make sure of that because our values can vary it can keep changing for so what I'm trying to tell is that these values can change based on the angle of the paper and based on the contour as well so what we have to do is we have to arrange these values these biggest points before we send it for working so we have to reorder them so we will create a new function we will say V Porter and inside that we will need our points so my points we need some points inside and that we are going to apply some methodology inside here that will allow us to create or reorder our points so if we have for example let me make it a little bit simpler if we run this now right pass you if we add these values up what we can do is we can add these values up and we can find the smallest and the biggest point so for example if we add 0 0 and we add width and height width and 0 0 and height and width and height the smallest point will be the 0 0 and the biggest point will be width and height so this means that whatever point we are given here if we add all of them up the smallest one we will get will be our 0 0 and the biggest one we will get will be our width and height so this way we can quickly sort out which one is the biggest and which one is the smallest so which one is the origin and which one is the diagonal point of our corner of our rectangle or the last point and then to find the width 0 and the 0 height we can subtract them and one of them will give us a positive value and one of them will give us a negative value and based on that we can tell which one comes first and which one comes second so what we need to do is we need to first let me show you something before we go ahead here so if we look at the shape of our body called array or the matrix of biggest let's have a look at that so it is 4 by 1 by 2 so we have four different points that we understand and for each point we have x and y that we understand this well but that one is redundant so what we will do is we will remove that one so when we are about to reorder the first thing we will do is we will say my points is equals to my points dot reshape and we are going to rights we want to reshape it as four by two and then we are going to create a new matrix that we will send outside that we will return from this function and that matrix should be the same as what we are receiving over here which is 4 by 1 by 2 okay so we will write my points noon is equals to numpy dot zeros zeros and then we can say 4 by 1 by 2 and then we can write its type so numpy dot integer 32 so the first thing we will do is we will add as we have discussed we will add all of these values so inside we have four points let me print this out so it's easier to see we will print biggest so the first step we are going to do is we are going to add each one of them so 185 plus 55 then 61 plus 279 and so on so in order to do that we have easi function we can say add is equals to my points dot sum and we are going to use axis one so this will give us let's print it out and we can say our ad is we are not sending it to reorder that's why it's not showing up so what we can do is we can right here or this should be inside the world so we can right here reorder and we can send the biggest points so if you stop that we can see that over here 183 plus 55 is 238 and then 61 plus 279 is 340 and then so on we are getting our values so the smallest one here is our first one so we are going to say that is our point zero and the biggest one here is 651 and we are going to say that is our width and height so this one is at the correct position it should be at 0-0 but this one is at the wrong position this should be the last one so we are going to rearrange our order in our new points so we will say here that my points new at number zero which will be the first one is equals to my points and we are going to get the index so we are going to first of all find the smallest one so we will say that numpy thoughts Arg minimum and we are going to say that find the smallest one the index of the smallest of this list or of this matrix so here we are going to find the smallest value and we are going to get its index once we have is index we are going to put it here and from that we are getting the values the actual x and y and then we are storing it in our new points the same thing we will do it for the biggest one which will be the last one as 0 1 2 3 and this time around we will write it as max so if we run that let me print that out friends new points and we can write my new points so if we print that so here we can see that our new points now the smallest one is at the first point and the biggest one is at the last point here so now we need to find the ones in the middle so next we are good we can remove or let's keep them for now so next we are going to find the difference between them so we can say the difference is equals to number dot difference and we are going to use my points and again we are going to define our axis as one actually you read me write it you can define it like this axis as one so then based on this we are going to say that my points the new ones hat number one is equals to this should be our width and zero so this one will be my points at numpy dot the minimum argument of our differentiation and the same way we write here the same way the second one will be the maximum of our argument so let's run that okay let me print this out but okay we run that and we stop it so now you can see that our points are rearranged so we are getting width and then 0 and then 0 and then height by 0 I mean it's a lower value so here our points are rearranged you can see our original points are like this and then our arrange points are like this so now we have our new points so we can send them back so should I delete okay let me put it as a comment so if you want to check you can check it and then we can write return my points new so this will give us the biggest points rearranged so we will say that biggest is equals to reorder biggest so if we run this now hopefully it will come up properly and there you go so our image is showing properly and we are getting our word perspective the image has worked now the problem is that one thing is that the width is too much we need to reduce it let's make it 360 okay the camera cannot be so 640 my camera has this issue so it has to be 640 by 480 not you can do there okay okay okay so this is supposed to be opposite so this should be 640 and this should be 360 let's make it 480 yeah much better so this is what we are getting and now the thing is that our original image is 640 by 480 so don't expect a lot of resolution over here but I have put an image in the resources folder so if you look at the resources folder we have a paper image and this image is really high resolution it is 4000 by 3000 so if you apply the same method on this one it will give you a really good scanned image at the end but because we are getting it from a live webcam and it is only 640 by 480 that's why the resolution is not that good but if you can bump up the values you will get much higher resolution so that is good for now but one thing we can see is that the work is good but we can see some lines at the end and we don't want that we what we can do is we can afford a little bit of a cut out a little bit of crop around the edges so that we get some better results so what we can do is we can after working it we can reduce some pixels so let's put it in the work so here we can say that our let's say image outputs or should we name it different let's name it different so image cropped is equals to we are going to crop our image so we are going to say that our original image is IMG output and then we are going to remove 20 pixels from each side so we will say 20 then image output dot shape at zero and then we will minus 20 from here as well and then we will say 20 again and then we will say that our image output dot shape which is at 1 minus 20 so this is the same concept what we have done before in our cropping example so if you don't remember that you can check that chapter again so once we have done that we need to resize the image so that it becomes as the same size as before because it will be easier to play around with it so cropped is equals to see v2 dot resize and then our image crop and then our width of our image and then height of our image and then we can output the cropped image instead so if we run that now we are getting a much cleaner image we have removed 20 pixels from each side and now it looks more like a scanned paper ok so next what we can do is if we want to look at what you call the workflow we can add our joining images code which is our stacking if you want it to do that so we can go back we can go to chapter 6 and here we have the function for stacking we can copy this and we can paste it in our current project and just to put all these images together so which of the images should we put together so this is just a formality to see what is going on so and there is another case that I will discuss now and you will see why that is important as well let's just stack the images first so we will say that our image array is equals to we are going to declare an array so here we'll put our original image then we can put our image contour and then we can put image biggest where that image threshold so let's put it down so we can put it here image threshold and then we can put image work so this should give us a good idea and then we will just say that stack the images so we will say stacked images is equals to stack images and we will define a scale so let's say zero point six and then we are going to define our arrays or image worked oh sorry imagery and then here we will copy this and paste it paste it here so there you go so now we can see the workflow we have our document then we are getting our body called the biggest contours and then actually this should be here they should be here this is the other way around so where is it so that is better I guess yeah so that is better so again I know we are rescaling it so that it because of that it's looking a little bit weird but if we print it out separately it should be fine what is it in which world so here we have our final result now one thing you will note now if I remove the document it should give us an error there you go now this error is because we did not define anything when it does not define the biggest it does not find the biggest contour so we need to write that down here so we will say that we are going to find the biggest contour let me see up if we want to put it up or air so yeah we are sending in the biggest I think it's better to put it down here so what we can do is we can say that if our biggest is not equals to zero or it's if it's not empty then we need to go forward so we can say that if biggest dot sorry dot size is not equals to zero then we are going to do this image work otherwise and this should be inside as well and this can be outside otherwise else we are going to display the images still but we will not display the contours because we didn't find them and we will not display the work we'll just replace it with the original image so let's see if that works so if I remove that yeah so when I remove it it doesn't show anything and then when I bring it back it's now showing me the live image you so in this project we are going to start off with our webcam and then we will detect number plates from different cars and we are going to do this in real-time and we will be using the Cascade method that we have learned before in our face detection chapter and so let's start with our chapter number one where we started off with the webcam so we will copy that code and again we will change this to one so let's run that and see if it works properly so here I have my webcam and it is connected and we can see we have a few images of our car and we have these three images and we are going to detect their number plates so once we have done that we will go into chapter 9 where we did our Cascade method and we are going to copy this and we will paste it in project 3 so here to get the Cascade we do not need to define it again and again even though we have to get the image the image we're getting from the webcam we can remove that this we have to bring it up in our parameters so here we can define how our parameters and then we are converting our image to gray that is fine and then we are detecting the the faces so in this case we can say that number plates and we can copy this and paste it here so instead of the face cascade we should say plate class gate for example or number of plates and it's drawing the rectangle around it now what we need to do is once we detect our object we need to first create a filters so we are only taking objects that are bigger than a certain area and then we need to define a way in which we can save our images once we are getting our number plate so let's do that so the first thing we will add our filter so in order to do that we need to find our area so area is equals to width into height and then we need to define our minimum area so we can say that if our area is greater than for example 500 so this is our minimum area so we'll say minimum area and we will define it here minimum area is equal to 500 if it's greater than 500 then we are going to detect it as our rectangle and the next thing we will do is we will label it so we will say CB 2 dot put text and we want to put it on our original image and then we are going to put a name so we can say number number blades and then we have our x and y so we can write x and y we want to place it a little bit higher so we will subtract let's say five and then we will write our font so see v2 dot font we can select any let's select a complex small then we'll put one as our scale and then we will put a color so we can define color and then we will put the thickness as two so the color we can go up and define here as 255 0 and 255 and then what we can do is we can extract our region of interest our number plate so once we have our rectangle X Y width and height we can use this information to get our original our number plate image so we can say that our image of our region of interest is image our original image cropped so as you know we are using crop so the first element will be our height so we are saying y til y plus h the total height and then we will say our width till our I'm sorry not o8x till our X plus world so this should give us the the region of our number plate so we need to place this here whenever the object is detected whenever the number plate is detected then only we are going to display our image and the last thing we have to do is to change our casket which over here is the frontal face in the resources we have the Russian plate number so we are going to use that so we will change this to Russian underscore plates underscore number so let's run that so there we go we are getting our number plate on the original image and then we are extracting our ROI as you can see here it's a little bit smaller it is harder to grab but you can see we are getting our image crack' but the color here is blue why is that blue yeah we need to change it to 255 so it's the same so there you go so this is what we are getting and next what we can do is we can save this image in our scanned folder so in the resources I've created a scanned folder so what we will do is we will write a code that will save this number plate and it will also give us feedback that we have saved it so to do that we will write cv2 so let's first change this from Q to s so whenever the S key is pressed so we will write C B to C we to dot weight C we - thought I am right and we will write the file name so we will say resources and then scan and inside that we are going to write number plate underscore and then we are going to add a string to it and in that strength we are going to put account so that we can save multiple images at the same time with different names so we can say dot jpg and then we will say that our image of our Y now this count is not defined so we have to define the count so each time we press this we have to add to our counter so we will not break we will say count is plus equals 1 and we need to define the count in the beginning count is equals to 0 so that will save it but what we need is a little bit of feedback that we are actually saving it properly so what we can do is we can create a rectangle and then we can display that with text on it so that we know it has been saved so we can write C v2 dot rectangle and then we will write our image here we will put it at 0 200 and then the final point will be 640 by let's say 300 and then we will put a color 0 255 0 and then CB 2.50 and then we are going to put some text on it see v2 dot put text and we will put our image and we are going to say scanned let's scan saved and then we will put the initial values so I'll put 150 and 265 and then we will give it a font see v2 dot font let's pick a random one and then let's put down let's put the font scale as two then we will put our color 0 0 and 255 and then we will put our thickness as two lastly we are going to show our image so I am sure and then we are going to write the same name results and we are going to show the same image and in order to see it properly we will add a delay CP 2 dot YT and we will put 500 milliseconds so if I run this now and there you go so if I press s it will say scan saved and then if I go to the next image I can save again I can go to the next image and I can save so all three of these number plates should have been saved if we go to our scanned folder here we can double click it and we can see we have three images this is the first one this is the second one and this is the third one you