Deploy FastAPI on Ubuntu: CRUD with PostgreSQL, Gunicorn, and Caddy

[00:00:01]Hey all Welcome to TutLinks. In this tutorial I  will be demonstrating you how to deploy a FastAPI application on ubuntu 18 operating system our  first api application will be performing create read update and delete operations on a  postgresql database we will be running our FastAPI application on Gunicorn web server  which is configured to run as a systemd service and we will be exposing our FastAPI application to  the outside public internet which is exposed as a reverse proxy running on a web server Caddy  2. so without further ado let's get started as we want to deploy our FastAPI  application on ubuntu operating system let's grab a virtual machine that has ubuntu so  for that I'm on microsoft azure cloud platform dashboard so I'll start off by creating a resource  that has ubuntu as a operating system in a virtual machine so here I am on the create virtual  machine wizard so I'll be creating a new resource group just to accommodate this virtual machine  the name of the resource group will be links open to resource group and the name of the virtual  mission I would like to give as we would do and I'll be choosing the region which  is very nearest to me I'm located in india so I'll be choosing central india and  the image this is an important step because we are going to create a ubuntu virtual  machine we have to choose ubuntu server I'll go on with the default and very minimal  size which is a b1s for the sake of demonstration purpose and that in authentication type I'll be  choosing ssh public key and the username will be insert and I'll be going to generate a  new keypair and the keypad name will be winter underscore key and because I want to deploy a FastAPI on ubuntu I need to access this virtual  machine in some form that some form is nothing but secure shell and for that I have to enable port  number 22. as I also mentioned that we will be exposing our FastAPI application to the public  internet so we have to enable port 80 and 443 so once you are done with the deployment you  are recommended to disable the port ssh 22 but I am not done with the requirement I leave this  enabled click on next to navigate to the disk tab the desktop will be going on with the  standard ssd for the demonstration purpose in the networking you see the public ib let  it be a new ip to be created and click on next to management tab disable the boot  diagnostics because I am not needing it for the demonstration purpose disable the auto  shot now I do not want to automatically shut down the virtual version and click on next to  navigate to advanced tab and then tags I do not have anything here click on review  on create ensure that the validation passes yeah the validation passed and click on create  to finally create the virtual machine and we have to download the private key  so that whenever we want to connect uh to this virtual machine via ssh we have  to provide this private key as an argument sit back and relax because the virtual machine  deployment is going to take at least three to four minutes and we'll come back once it is  done all right so our virtual mission has been successfully deployed on azure blog  platform so let's click on go to resource and here you can see the public ip address and if we want to connect to this virtual machine  uh through ssh click on connect and click on ssh it will show you the command to connect to this  virtual machine so here is the command so copy it because we have already downloaded the private  key I'll be connecting to this virtual machine using uh putty or command line terminal but I  will be using cmder so let me launch cmd here cmdr is an open source and free to have the  various shell capabilities like windows of system for linux bash powershell and the typical command  prompt so here I have uh opened the cmd here and I will be giving the command that I have copied over  here and the private key part I will be giving the path of the private key the pdf file that  I have downloaded it is located at cuses so hit enter and type in yes for the prompt  and you will be successfully connected to the google virtual machine so let's  look at the operating system details so here we have ubuntu and the version  is 1805 1804 and dot five lts beaver and we have successfully connected  to this ubuntu virtual machine so the next step we will be doing is  to install and configure postgresql 13 database because our first api application is  performing create read and update and delete operations on a postgresql database so for that  so all you have to do is type in the command sudo sh hyphen c and down to the list and  you have to hit enter what I'm doing is I'm adding the latest postgresql report  to the app and I'm downloading the keys and then I'll be updating the app and finally once done I will be installing  postgresql 13 on the ubuntu operating system so finally run the command sudo apt  install hyphen y postgresql and hit enter wait for the installation to complete all right the postcard sql section  has been successfully installed let's specify the version of psql we have  got on our generation so it's uh 13.1 all right the next step is we'll be creating a db named FastAPI db so for that all you  have to do is type in the command sudo su lightning space postgres  space hyphen c then create db mask the name of the db will be FastAPI  tv all right we have successfully created database with the name as kpdp  next we will be creating a user all right now we will be creating  a user named patch api user for that we have to type in the command  as shown on the screen and hit enter and as we have mentioned hyphen p flag here  so we are supposed to provide a password for this user so I will be giving the password  secret all lowercase s e c r e t and reconfirm yeah so we have created a user FastAPI user  with the password now I will be granting all the privileges for this sdk user on the hashtable db  so for that all I have to do is to run the command sudo hyphen new postgres psql the command  will be grant all privileges on database past the page db that we have just  created to the user that's jp user we just have created and hit enter  so the grant is provided successfully all right so we have ubuntu virtual machine  available and it is installed unconfigured with postgresql version 13 database our next  step is to pull a FastAPI repository onto our ubuntu operating system and we will be  cloning a publicly available hdpa repository and here is my past api repository and  the branch that we are going to deploy and we are going to do is dash apa postgres sql  Caddy to deploy so let me click on that branch so this will be the full source code that we  are going to deploy on ubuntu operating system so let's access the ssh back again  and we will be issuing a command to clone the repository that is available  here here you can see git clone and the repository will be cast api available on  my username and the branch as I mentioned will be the branch where our application code  lies that will be hashtag hyphen postgresql so hit enter to clone our repository now if you  see the contents we have a directory named FastAPI downloaded from the github let's cd to it and  list the contents of this directory alright so we have all the required files necessary to  spin the FastAPI application up and running as we have cloned the repository we will start  up by creating python virtual environment so for that all you have to  do is type in sudo apt install so we are going to install  this module because uh python so we're not going is basically  creation of our children we need this package to be installed  available in app so hit enter all right so now we will start off  by creating a virtual environment python hit enter all right the virtual environment is  created and let's confirm that you can see that environment and we will be activating the virtual  environment by giving the command source env slash bin slash activate all right so  you can see the virtual environment is activated and we are within the  virtual environment and let's start off by running the sanity so that our application  successfully runs by giving the command you will main is the module and app is a  variable to which hashtag is initialized okay but before that uh all we have  to do is our pastify application is dependent on certain modules so which  are present in the requirements.txt so here all here is the list of all the modules  that you can find that our application needs in order for it to run successfully so we have  to install all those dependencies within this virtual environment but before that let's look at  the version of pip we have pip is the python uh module installer and we have the very old version  of pip but we have to upgrade before we can start installing all the dependent modules that  are mentioned in requirements.txt so for that all you have to do is to upgrade the screen  first run the command with install f1 upgrade and what we are going to upgrade the pip  itself. So we are using pip to upgrade pip!

[00:15:49]Quite funny and interesting. Okay so we have  successfully upgraded the pip from version 9 to version 20. all right so the next thing is we will  be installing all the modules that are presented requirements.txt by providing the command pip  install iphone or requirements.txt all right we are good to go with running the sanity application  now so we will be giving the command uv con module main and the variable to which the past  api is initialized with this app and hit enter congratulations but wait we are not  going to finish our tutorial right here let's stop our application because we want to  run our application on a Gunicorn web server and we will be configuring our gm web server to  run as a system systemd service all right so far we have set up and configured all the underlying  infrastructure to run our FastAPI application now it's time for us to run the FastAPI  application with the help of Gunicorn.

[00:17:20]Gunicorn or green unicorn is a python  based WSGI http server for unix.

[00:17:27]WSGI is web server gateway interface our FastAPI  application runs on asgi which is asynchronous server gateway interface compatible  servers like uvicorn and hypercorn.

[00:17:45]Gunicorn being a wsgi web server facilitates  running FastAPI with the help of its worker class uvicorn.workers.UviconWorker. we will create  a systemd file by accessing the ssh terminal we will be creating our systemd file  using the nano editor at the location etc systemd system and the name of the  service that I'm going to create will be gunicorn.service and hit enter. I will be pasting  the contents of this unique file right from the repository so here I already have checked in the  service file so let me copy the contents of it and make sure that you replace uh the  directory that reflects the installation of uh the FastAPI repository according to your directory  structure uh in the execute start command so let me switch back to the ssh name  and paste the contents of this unit file all right I have successfully copied uh  the contents of the Gunicorn service file and ctrl x capital y and hit  enter to apply the changes and we are ensuring here the contents got listed  properly in each and every line of their own so we have created a system diminished unit  service with the name gunicorn.service this could be any name of your choice like fastapi.service.  I have given gunicorn.service because this unit file we have created can not  only be used for FastAPI but can also be used for Flask or any web framework that supports  the Gunicorn web server. The only difference lies in the way we spin up the application using the  command held by execute start directive of the unit file. For the unix beginners, unit files are  the configuration files managed by the systemd that hold the behavior of specific service that  we are dealing with. They get added as services and allows various capabilities to start  the service when a system reboot happens or retry restarting the service in case of  failures and offers various other capabilities.

[00:20:36]If you notice the path for exit start  directive is having env in its path which indicates the name of the virtual  environment. If you have given a different name for your virtual environment you can replace env  with the name of your python virtual environment the Gunicorn unit file uses the Gunicorn  binary available in the virtual environment installed for all asked api to run the version  of Gunicorn binary is based on the version of Gunicorn mentioned in the requirements.txt  file if you look at the xxtap directive of the gunicorn.service file we are providing  the Gunicorn binary with the location of the configuration that holds various arguments in  the form of literals enclosed in a python file named gunicorn.py that is also available in our  repository so let's take a look at the gunicorn.py file we have gunicorn.py file wherein  we have provided all the configuration so I'm importing the multiprocessing module here  just to build the number of worker processes uh here according to the standard formula provided  by Gunicorn configuration so it's nothing but the number of cpu count multiplied  by two and then incremented by one so at max it will be able to process this many  worker processes to be run by the Gunicorn master service so the worker class is  the asynchronous ubicon marker class and the number of worker connections and the  rest of all the arguments that are going to be passed to the Gunicorn and I'm loading the  default configuration values from the load and here is the environment file where I'm  setting various environment variables and that's about it so let's look at how many  worker processes that our version mission will support just for understanding so I'll be  entering into the python 3 terminal and I will import multi-processing now I  will be running this command to see so this virtual machine will be able to  accommodate the maximum three workout processes the Gunicorn dot pipe provided  as a config for Gunicorn binary in the unit file offers the flexible  configuration for workers argument the devops doesn't have to worry about changing  the number of workers every time they deploy it to a VM but it is calculated intelligently by  using this formula present in the gunicorn.py file now that we have set up the gunicorn.service file  now it's time for us to add it to the system ctl so let's refresh the system ctrl using  the command sudo system game ctrl reload so this will refresh all the service files  and it will identify the newly created gunicorn.service file when we enable it now  the next thing is we have to enable our new gna contact service for that all we have to do is type  in sudo systemctl enable the Gunicorn service so now that we have enabled the Gunicorn service  the above command what it does is it will add our Gunicorn service to resume running  when the vm reboots now we will start our Gunicorn service by running the  command sudo system ctl start Gunicorn service to check the status of our service  whether it's running or what state it is in just type in system ctl status and the name of our  service okay our Gunicorn service seems to work fine and which is indicated by  the status active and running so this is the very first step  and we have achieved it quite well and as our application is up and running let's  try to send some data by issuing the post command so here you can see curl xposed as our application  is configured to run on port 8000 and the endpoint is notes and the headers is accept application  json content type is application json and the data payload is a text and the status of completion get  groceries from the store and when I hit enter it has successfully imposed the data and we got  the response back even with the inserted id so the next thing is we are going to look at  the Gunicorn access and error log files so remember we have created a gunicorn.py file  wherein we have posted all the literals that are argument stored g unicorn binary so we have also  passed two such log files access log and error log that are supposed to be dumped in these locations  so if I look at the access log here copy it and look at the contents of it yes the first post that we made has been logged  here and the status is to not run which means and the note got created all right  that looks very good and let's look at the error log all right we have also a properly  working error log which can be visualized by this command and let's exit out of this now that we have properly working access and error  log files for Gunicorn service configuration now it's our responsibility to rotate these Gunicorn  access and error log files using log rotate log rotate is a module available by default in  debian systems we have to perform the next steps to rotate the access and error log file for that  what we have to do is to use an editor nano as a sudo user and modify the file that is present in  slash atc slash log rotate dot configuration file so at the very end all you have to do is  provide the path of these two files which is this one and space and the path of the error log  and we have to start an opening flower bracket in the next line we have to ensure  that to add all the properties as mentioned here and close the parenthesis so  this tells the log rotate not to fail if at all these files are missing and rotate  these files daily on a daily basis and the extension of these slides will be date  extension rotate for means pressure only four files and compress the files that were rotated so ctrl x capital y and hit enter so in  order for these changes to take effect all we have to do is to run a command the  sudo user log rotate and the configuration yeah if everything goes smooth  you won't be provided with any uh outcome but this works as expected okay  so far we have made a very decent progress in making our FastAPI application up and running  on Gunicorn now it's time for us to expose this application to the outside internet via  reverse proxy with the help of Caddy to web server Caddy 2 is an open source web server built  using go programming language it offers a range of impressive features such as automatic https  for custom domains by default from let's encrypt easy configuration from json or cadi file it  also supports http 1.1 http 2 and experimental support for http 3 protocols now let's install  open source carry to web server on ubuntu by running the following commands I have  already documented the list of commands and I'll be running each of these commands  line by line in the ubuntu virtual machine okay and the next thing is sudo apt update and finally once the app is updated  all I have to do is to install Caddy all right we have Caddy successfully installed to  ensure we have our Caddy space version to see the version of cali that we have got which is 2.2.1  all right so we have successfully installed Caddy and uh we have also enabled port 80 and  443 right so let's navigate to the ip this is a public ip address of our ubuntu virtual  machine we have been working for so long and if we provide this ip address and hit enter you  should be able to see the Caddy homepage this indicates that you have successfully installed and  configured Caddy on your ubuntu virtual machine you can also check the status of the Caddy service  that is running by default on your virtual machine by typing in the command systemctl status Caddy  and hit enter you can see the status of carry service is active and running if you are seeing  this that means you also have the Caddy that is installed and configured correctly and even on the  system liquids the Caddy will be up and running the next thing is we have to create a Caddy file  under the location using the nano editor etc Caddy Caddy file right and all you have  to do is just provide the port that our application first apa application  is running ctrl x and save because we wanted to expose our FastAPI application  as a reverse proxy via Caddy right so minimize this and access the port 8000 here  this is not working because we have updated the Caddy file so in order for these changes to  be effective you have to restart the Caddy service and let's look at the status one more time after restart if your Caddy service is up  and running properly that looks good and so it is being exposed on port 80 and on port  80 we are giving a reverse proxy to our FastAPI application that runs on port 8000 so  all we have to do is access port 80 on the public internet so it will become  now port 80 instead of port 8000 and let's see if that works so let's try uh accessing  one of the nodes endpoint that we have got and success FastAPI by default provides uh open  api json spec which is often termed as swagger spec and in old school way so that is accessed  via docs and let's see if that is accessible now that you have successfully deployed the  FastAPI application and you are able to access it via the ip address of the virtual machine it  is quite difficult to remember the ip address for the developers who consume this api in  their fronted applications such as react view or angularjs to mention a few if you  already own a custom domain let's say example.com then you can point your domains a record with  android pointing to the virtual machine's public ip address if you already own a domain on  another registrars like godaddy it's fine but if you want to register a new domain on azure  you can refer to this video available on this url so here I have registered  a domain named golangiq.com all I have to do as I mentioned is to add a record  set I have to direct the add rate to a record and the ip address is the ip address of the virtual  machine I have deployed our test api application and it has to be a naked ip address and click  on ok refresh the dns changes and you will see the a record is mapped to the ip address of the  virtual machine that we own now the next thing is we have to update our carry file such that we  mention that domain and update the contents of it with the Caddy file which is named as cadify2  but you don't have to necessarily rename the file but I just have checked it into the repository  for reference for people who own the custom domain copy the contents of it and ssh to the box  we have to edit the Caddy file located at etc and remove all the contents right click and  click on ok and ok again to face the contents and finally we have to update  example.com with the domain that we own and configured our dns jones a  record to point to this virtual machine's ip so I'll be providing the golang iq.com and 443 443  is https and reverse proxy as I have been running my FastAPI application on  port 8000 tls is gmail id so ensure that you provide proper email address  so that any updates to the ssl certificate are emailed to this email id save and in order for  Caddy to pick up these changes all you have to do is restart Caddy server and finally ensure that  the Caddy is still working by checking its status upon running with our new changes  reflected in it so let's go to golangiq and see excellent if you can see  we are able to access the secure communication to our api https slash volume.com  and if we try to access notes that we have inserted it's being accessed here and let's try to  look at the status quick everything is working as expected that's all about it to summarize what  we have done in this tutorial we have created a ubuntu 18 virtual machine on azure cloud and then  we have installed and configured a poster sql that in database on ubuntu operating system and we  have set up the deployment environment for FastAPI using python virtual environment and then install  all the requirements that are required for our FastAPI application to run we have deployed  our FastAPI CRUD postgresql repository on our ubuntu operating system so we cloned our FastAPI  repository from github and installed the virtual environment and all the dependencies from the  requirements.txt and later we ran FastAPI on Gunicorn as a systemd unit service and also have  configured automatic log rotation for Gunicorn access and error log files finally we installed  Caddy 2 web server on ubuntu 18 operating system and we have also updated kd file config to  run for custom domains with automatic https congratulations you have mastered how to  deploy a FastAPI application that performs credit on postgresql database you have also  understood how to build a Gunicorn systemd file and how to install Caddy2 and expose your FastAPI  application served on Gunicorn as a reverse proxy to the external world through gadify if you found  this video helpful please give it a like and please do subscribe to this channel and share it  with your friends and colleagues have a great day