Publishing a Website on IPFS: A Technical Tutorial

[00:00:00]so I found this blog posts the other day it's called why build this blog or anything on ipfs ipfs is the interplanetary file system and it is this really cool decentralized internet protocol I had heard of ipfs before I had kind of chalked it up as a interesting crypto experiment but it was cool to see his blog actively running on ipfs I've had the domain just in Pagliacci comm for a while and I've been meaning to spin up a blog but I just have never really had the motivation to do so trying to see if I could get a blog running on ipfs was a good opportunity for me to one have a project that I could actually create my blog finally and to test this ipfs and see if it's easy to use and if it's a legitimate alternative to the Internet so my goal here is to just get a very basic website running on ipfs hosted on justin Pagliacci comm and we'll go from there [Music] [Music] that initial blog post it talked about problems with the current internet the Internet today is built on technologies like HTTP DNS and it works by looking up IP addresses and forwarding the content from the IP address so I type in google.com in my browser it performs a DNS lookup and says google.com should go to this IP address and then it forwards traffic to that IP address and returns whatever is at that server to me and is displayed in the web browser this is called location based addressing the users trusting whatever is at that server to be the content that they want that's one issue is that there's nothing really stopping the content from changing without the user doing anything about it another issue at scale is that when there is only one source of the content it creates bottlenecks and it creates these centralized data pipes if you can imagine YouTube servers being the one place where all of the YouTube video is there's a lot of traffic going through those central pipes to get to that content contrast that with IP FS or a distributed internet which at its core uses content-based addressing so when I'm requesting a piece of data it actually just refers to the content itself and not the location so there could be millions of nodes that have this exact cat image that I want to fetch an IP FS will be smart and determine which node it wants to pull the content from so my friend sitting next to me could have that cat picture on their computer and I could go straight to them I don't have to go through some centralized server to fetch that data as a user I don't really care where the content comes from if there's a cat picture that I want to see I care about the catfish you're getting to my computer it doesn't really matter to me where that request gets routed this makes it so that it could be a lot more efficient to fetch data in general there are studies and there are experiments that show that for video and large files you could see up to 60% bandwidth savings I know Netflix is experimenting with ipfs today when you use content based addressing there's no chance that the content could change and be kind of hacked from without you knowing and this really makes the web a lot more robust and really bulletproof there was a day like two years ago when AWS East went offline and I swear half of the Internet services that I use didn't work and that was kind of eye-opening it was kind of scary to see that we're so reliant on these central companies to provide our Internet as it is today I started on ipfs IO that's their homepage found this ebook I guess it was it's called a get book called the decentralized web primer through a series of experiments and tutorials it taught you all of the basics that you needed to know so I would recommend this if you want to kind of follow along and do a similar thing so first downloaded the Go ipfs binary which included a command-line interface who did the CLI I played around with ipfs locally I would add a lot of files I would create text files add them see what ID is created and how the file is hashed the light bulb moment was when I echoed a string put it into a text file which said sample text version two and then I I P FS add the file and it gave me the ID that was created for the file and then I echoed the exact same text sample text version two and piped it directly into ipfs ad without adding it to a file and it created the exact same IP FS Content ID so this proved that ipfs hashes the content my content here was sample text version two and output the exact same ID for both contents so the ID that's created from ipfs is a mathematical hash of the content itself and this is kind of what powers the whole system is that the ID is the content and if the content changes at all whatsoever the ID changes I also verified that when I made any slight change to the text and then re added the file to ipfs the ID would be completely different next I added a file and it's encompassing directory so I learned that directories are a thing in ipfs they're different from a file but a directory contains any number of files in it a directory has its own ID as well and the directory ID is derived from the content that's in the directory and this goes Turtles all the way down if there are any changes to any of the files in the directory the directory ID would change as well so at this point I know that I can add a file to IP FS which hashes the content of the file and then I could add a directory which has any number of files or directories inside of it and it is a derivative hash of all of its contents I learned about pinning next I listed all of the content that I had pinned locally ipfs apparently pins files that you specifically add and pinning really it just kind of keeps files around if you haven't explicitly pinned the file then garbage collection could get rid of it so I learned that pinning is how you basically specify yes I would like to host this file on my node I keep saying the file I really mean content it's kind of hard to break out of that and up until this point I've just been running ipfs locally the next thing I learned was how to run the daemon I think it's pronounced daemon I da I pronounce it Damon I don't know if it's pronounced daemon I'm gonna call it Damon I ran the IP FS daemon which was cool because that kind of made me accessible to other nodes in the network and made other nodes in the network available to me running the daemon spits out you know some information about you as a peer it creates a web UI which was really cool it connects you to the entire network so that's kind of the magic command to like get you actually running ipfs locally I hope the web UI is pretty cool that it spins up it allows you to see you know the status of your connection and it allows you to see all your peers and where they are and allows you to go through and explore a lot of the files that are hosted on IP FS and this is all stuff that I could do through the command line as well I could in theory go and find the ID of all of the files that I need to hub to connect to or go find the peer IDs of all the hosts and spit out their content directly it's just a lot easier to navigate this through a web UI once I was running the daemon I could run ipfs cat with an ID on the actual network not something that I've added locally and it was able to go and find that content through my connected peers and return it to me which is cool to see I was curious how it knew that I had a peer connected with that content and how to kind of route my request but it did it very quickly so the systems is apparently very efficient so at this point I was running the ipfs daemon I was adding content locally and I was viewing content from the network from the command line I didn't really understand how to access content through a browser browsers don't know how to connect to other peers yet so you need an extension that will connect to your daemon that's running locally and be able to facilitate routing to the other ipfs peers so I installed the ipfs companion browser extension for Chrome I was able to access ipfs content while I had the daemon running the browser would actually redirect to my local host and show the content in the browser which was cool there's a couple different other ways to access content but the other really main one is through an IP FS gateway a gateway is essentially just a proxy that your request is routed through I can send a request to this gateway and the Gateway acts as an IP FS node and then we'll go and fetch the ipfs content and return it to me so I experimented with this a lot to kind of understand how it was working I would find a piece of content like this xkcd comic I would be running the browser extension and the ipfs daemon and I would load the content with the right URL and it would redirect locally when I'm not running the daemon or the browser extension I would have to go through one of those ipfs gateways to see it but it's really just the prefix of the URL that changes the content ID stays exactly the same IP FS that IO or gateway that ipfs that IO are the gateway URLs versus if it's running on localhost that means that you actually are the node and you're the connecting to the content so it became clear that if I wanted to host my actual website on it I'd have to redirect users through a gateway because I can't expect all the users of my website to have ipfs installed locally on their machines until decentralized protocols are more supported in browsers and browsers natively speak ipfs you have to use a gateway for you know 99% of your viewers to be able to view it in a normal browser it became obvious that my website was going to change fairly often and therefore the content identifier was going to change fairly often it's a bit cumbersome to have to update that ID every single time you make one small change to your entire website ipfs has this concept called IP NS interplanetary naming system you can publish content to an IP NS pointer the idea is you could have one ID that would stay the same and then you could update the reference to other content as your content changes so I created this super simple web page with an index.html and a picture of a cat on it i used i p FS name publish with the ID of the directory of my web page on it but it took a while it took a while to publish this I actually thought there was an issue with it at first I was finally able to publish and then I noticed that when I was looking up the content it took a while to look up as well but it was slow enough that a couple of times you know it took like 30 seconds to load it was not gonna work you know I did some googling and found some links to github issues this is something that the ipfs team is well aware of these issues and I think this is something they're working on but until then you know I just decided I'm gonna have to use straight ipfs Content IDs and just update my identifiers every time I update my content on my website I created the super simple version of my web page that I wanted which is literally just index.html and my Twitter picture I definitely plan on expanding this into my blog afterwards but you know I wanted the most simple website I could build to start with just to make sure you know ipfs is working as expected before I build on top of it I just needed a directory with some web content in it hosted on ipfs that I could point my website to so I needed a way to make sure that my content was hosted on ipfs and always available I guess in theory I could put this content on a computer that is always on and have the daemon always running for ipfs but I found in the original blog post beau had used this service called pinata in pinata is it is an IP FS spinning service so I signed up for pinata I got an API key and I tested it out in postman and was able to make a postman call to Penn specific content I added my website locally on my local IP FS node to make it available and then I basically told pinata pinned this ID and it was able to go and find the content from my local IP FS node pinned my website content and I keep it there I verified by stopping my local IP FS node and then going and visiting the Content ID and it worked so that proved that pinata was successfully hosting it on ipfs so my content is out there it's good to go I just need my website now to redirect to it I'm using AWS route 53 to host my website I'm such a networking web developer noob that I really like don't know what I'm doing here but I through exhaustive googling found out that I needed to update my alias record in route 53 to point to an IP FS gateway any of the gateways will work and then in addition you there's this thing called DNS link which I had to Google I still don't fully understand it I just know that if you create a txt record and you specifically named it underscore DNS link dot your domain and then the value you put the IP NS or the IP FS link okay so now when I type in Justin Poly atra calm in the browser it performs a DNS lookup to the IP address of the Gateway because that's what I put in my alias record redirects to the Gateway the Gateway IP FS will notice that the request is coming from Justin Patrick comm in one of the headers it will look for the DNS link entry and find that underscore DNS link with the reference to the IP FS content so the Gateway will then redirect to the content that I'm asking for return it to the Gateway return it to the browser and my website is hosted on IP FS so if you visit Justin Pagliacci calm right now it points to actual content that's actually hosted on IP FS which is super cool in the future I mean I I obviously want to make this a full blog so I'm going to do another series of this video on how I'm creating the blog I plan on using hugo to generate my site and then i plan on using github actions to publish updates this was a very fun experiment I'm glad I got it working and I'm glad I have a base understanding and I hope that by you following how I learned it it helped you learn it as well I don't know how to outro okay