ChromaDB Vector Database Guide: Embeddings, Semantic Search, RAG

[00:00:00]Good morning, everyone. How's it going today? Welcome back to the channel. In today's video, we're going to be covering vector databases. We're going to be going from the bottom up.

[00:00:09]We're going to take a look at what are embeddings, what are vector databases, what is ChromaDB, which is one of the most famous open source vector databases that you can use.

[00:00:18]And we're going to actually be implementing our own vector database right here in a very quick notebook that I have available for you. And it's, of course, available in the description.

[00:00:28]We're going to be taking a look at how to create your own vector database, how to add data to it, delete it, query it, etc. How to run crowd operations on top of it, how to create collections, and everything very clear and very spelled out right here in this notebook, which is what we're going to be covering today.

[00:00:45]So, without any further ado, let's get right into it.

[00:00:48]all right so first of all let's talk about vector databases and this is very straightforward vector databases are essentially just as a regular database but they are specialized at storing embeddings now what is an embedding embeddings are essentially just numerical representations of a text. So you take a piece of text, you pass it through a machine learning model, which is usually a transformer-based model like BERT or OpenAI's embeddings, and this is going to extract a lot of numbers from that text. And what those numbers actually represent are a bunch of dimensions in a very high dimensional field that represent the meaning of that text. In other Whereas suppose that you have, for example, a plane like this, you have two dimensions, and you want to position a point right here, so you're probably going to put it somewhere right here, and each axis or each dimension right here is going to represent something, some meaning associated to your point or to your text.

[00:01:59]Same thing if you have three dimensions, etc. Now, of course, in the embeddings models that we usually use, they do not have three dimensions.

[00:02:08]They usually have hundreds or thousands of dimensions.

[00:02:11]like, for example, the embeddings model from OpenAI, Text Embedding 3 Large, has up to 3072 dimensions, which is, of course, modular in that case.

[00:02:23]You can actually lower it.

[00:02:25]But you can see that it's 3072 numbers that represent the meaning of any text that you send to it.

[00:02:32]And those numbers are going to be stored in a vector database.

[00:02:35]Now, why is it important to store those numbers in a vector database?

[00:02:41]Well, the reason for it is because all of the numbers in that vector database are going to have the same number of dimensions.

[00:02:48]So all of the points right here are going to have the same number of dimensions, which means that they are going to be in the same dimensional plane, which means that you can see how close they are to each other.

[00:02:59]So in this example, we have three dimensions, but consider that this is actually thousands of dimensions.

[00:03:05]Let's suppose that you have this point right here, which is a text talking about the Mona Lisa, and you have this text, this point right here, which is a text talking about, say, the Italian Renaissance. Now, as you can see, they are going to be close together because the semantic field or the meaning behind them is very close together, the monolism and the Italian Renaissance, so they're going to be clustered right here together. And if you're going to be talking about something else, I don't know, like Asian history or something like that, it's probably going to be somewhere right here. If you want to talk about cooking recipes, it's going to be somewhere here. So things that are closer together, they're going to have similar meanings. And that's why vector databases are so important, because they allow us to search for text based on meaning.

[00:03:51]And this is incredibly useful for RAG, because we're going to be able to search a piece of information that we have ingested into our database, and search for it by meaning, and then use that text in our RAC system, which is something that we're going to see later, or you can probably check a video about it. It is very useful for semantic search, for recommendation search systems, and for question answering with language models, as I saw before, with RAC. So now that we have understood what is a vector database, now let's actually see it in action. We're going to embed our own text and we're going to position it into our own multi-dimensional space using Chroma.

[00:04:35]Let's take a look at that. All right so first things first let's talk about Chroma. So what we're going to be using right here in this notebook is ChromaDB which is an open source vector database that is of course free to use if you host it locally or in your own servers or you can also go to their cloud which is pretty good. So what we're going to be doing In this case, we're going to be using our own instance of Chroma.

[00:05:00]We're going to be hosting it in our own machine, which is in this notebook right here.

[00:05:06]So I invite you to take a look at the notebook that is in the description and follow along with it.

[00:05:11]So first thing that you going to want to do is you going to want to install ChromaDB And in order to do that in this case I going to be using pip So install chroma db and it going to take a little bit so let me just pause the video while it finishes and there we go it's finished installing so now the next thing that we're going to want to do is we're going to want to initialize our chroma db client like this so just going to do import chroma db and we're going to import initialize our client like this client equals chroma db.client and right away we're going to initialize our collection.

[00:05:43]Now, what is our collection?

[00:05:45]Our collection is essentially just kind of a table, you can think about it, compared to a regular database.

[00:05:50]It is its own collection of points because remember that we saw that each collection of points is going to live together in the same dimensional space.

[00:05:58]Well, each collection is going to be its own dimensional space in a sense that if you have another point in another collection, it is not going to be accessible.

[00:06:06]it's not going to be possible to compare it to other points in other collections.

[00:06:12]So very, very, you can think about it as different databases actually.

[00:06:17]So let's initialize our ChromaDB client, initialize our collection.

[00:06:21]In this case, I'm going to call it my collection.

[00:06:24]And next thing, we're going to add some points or some text to it.

[00:06:30]So in order to add some text, you just do collection, which is the collection that you initialized.

[00:06:34]In this case, I initialized it, by the way, using get or create collection, which is a method that either creates the collection if it doesn't exist or gets it if it already exists.

[00:06:47]You can also create one and without testing, you can just do create collection that also works.

[00:06:52]We're going to see that in just a moment.

[00:06:55]But in order to create it and add some text to it, I'm going to just do collection.add.

[00:07:01]And right here, you can pass in multiple things.

[00:07:03]The most important things are, of course, the documents and the IDs.

[00:07:07]And this ID is going to be associated to this document, this one to this other document, etc. And what's going to happen right here is that this document right here is going to be embedded.

[00:07:19]Because remember that we mentioned that a vector database does not store text.

[00:07:24]It stores the numerical representation of that text.

[00:07:28]So what Chroma is going to do is it's going to embed this text for you.

[00:07:32]It is going to use sentence transformers, by the way, which is an open-sourced embeddings model that will run locally in the machine that you're running it.

[00:07:41]And it will embed it and add it to your database.

[00:07:44]So let's add that right now.

[00:07:46]It's going to take probably a little bit to actually embed to run the embeddings model.

[00:07:51]And there you go.

[00:07:52]And then once that is done, you can actually get the documents from that collection.

[00:07:57]So in this example, we're following the artificial documentation from Chroma.

[00:08:00]This is a document about a pineapple and this one is about oranges.

[00:08:04]So let's query it like sending a document called this is a document about Hawaii.

[00:08:12]And what's going to happen is that it is going to figure out where this text is positioned in the vector specs.

[00:08:20]and it's going to retrieve the most similar points to that query, to this text right here that are already in the database.

[00:08:31]So in the database, you have these two documents.

[00:08:34]And now let's take a look at which one is the closest one to this one.

[00:08:37]And as you can see, it retrieved the two documents, which are quite close.

[00:08:41]It mentioned that this is an embedding about pineapple.

[00:08:44]This is probably the first one that is the closest one.

[00:08:46]And the second closest one is this document about oranges.

[00:08:49]So apparently pineapples are closer semantically to Hawaii than oranges.

[00:08:55]And you can actually see the actual distance between them.

[00:08:59]So you can see that the pineapple is 1.04 points away from Hawaii.

[00:09:08]And the orange is 1.24 points away from Hawaii.

[00:09:19]so there you go as you can see we have I we have effectively identified the closest points to this to this query point and now let's take a look at actual crowd operations alright so now let's take a look at how to create crowd operations on your on your collection so we have already initialized the collection up here using to get or create method that you saw here and we initialized it called my collection and now what we're going to do is we're going to add some points to it. First things first is that you can run this like this and now to add some points as I showed you before you can just pass in the documents their respective IDs and their metadatas. Metadata is of course optional you don't have to send this thing right here but it is very useful if you want to actually identify some specific information about the data points that you're adding to your information. So for example, to your data. So for example, what is very useful right here is for RAG applications. In the metadata, you may add the page where this particular information is located, the file where it is located and of course you will retrieve the metadata alongside the document so that your LLM will have that information when retrieving So here we have a few documents so the Eiffel Tower penguins, human bodies, and just random facts. So let's just add them to my collection.

[00:10:50]It probably can take a little bit of time depending on how long your texts are, because remember that this is actually running the embeddings model inside the chroma I mean inside your computer so this is of course all local and what one thing you can do after you have added your data points is you can also update them so if you run collection.update and you pass in an id that already exists, it will be able to update that one. So in this case, my ID number one was associated to the Eiffel Tower in Paris, and let's just update it to the cosim. Let's just run this now, and after that, that particular point will be updated. However, something to keep in mind is that the metadata's parameter right here only updates the specified keys right here, so this right here will still have the metadata source architecture and the year built which is probably wrong for the coliseum now that I think about it. You can also use upsert method which updates the data point if it already exists and if it does not exist it creates it. So in this case I'm going to be again updating the point with the ID number one and I'm also going to be upserting or creating the ID number six. We're going to be talking about Rome on the first one and the sixth one we're going to be talking about Paris. And then we pass in our documents. And once that is done, you can just get those documents, get those data points. So in order to get them, you can just run, you can get them by ID. So you can run collection.get and you can just execute this right here.

[00:12:38]And in this case, you can also add the include parameter, which tells you which properties the data point is going to include in the results.

[00:12:52]And the default is only metadatas and documents, but you can also add the embeddings if you want them.

[00:12:58]In this case, I have retrieved the embeddings right here.

[00:13:02]And here you have the three documents that I have searched for, which is the Colosseum, penguins, and humans.

[00:13:08]And of course, the most important part about this is not just getting the documents by ID, like a regular database.

[00:13:15]You want to be able to query your database to get the most similar documents to the query.

[00:13:22]So if I search for information about the capital of France and I use the dot query method in my collection, it is going to return first the number six, which is the Eiffel Tower.

[00:13:33]It is an iron tower, a very big tower in Paris, blah, blah.

[00:13:37]And then the Colosseum is apparently the second closest one to information about the capital of France.

[00:13:43]The Eiffel Tower is 1.25 points away from this query of mine.

[00:13:49]And the Colosseum is 1.62 points away.

[00:13:52]And as you can see, the results also come with the metadata.

[00:13:56]Very important.

[00:13:57]That is essentially how you use, in a nutshell, CRUD operations over data points.

[00:14:03]Now let's take a look at how to do the same thing over collections.

[00:14:09]All right.

[00:14:09]So now let's talk a little bit about collections.

[00:14:12]And collections are essentially their own.

[00:14:15]Each collection is its own vector space.

[00:14:17]So if you're going to put a lot of data points into one collection, they're not going to be available in another collection.

[00:14:24]So usually you want to use each collection for, for example, a different use case or one collection per user in your application.

[00:14:32]Because that's very important.

[00:14:33]you probably do not want data points from multiple users in the same vector space because that can be a pretty huge leak of information. So as we saw before you can get or create a collection by using this method in this case I'm going to be creating a new collection called my new collection right here and then in order to get all of the collections that are in your database you can just use the list collections method from chroma right here and as you can see I have two collections now I I have my new collection and the other one that I showed you before, which is my collection.

[00:15:07]And after that, let's take a look at how to actually get all of your collections.

[00:15:11]Let's suppose that you have more than 100 collections.

[00:15:15]Well, list collections only returns up to 100 collections.

[00:15:18]So if you have thousands of collections in your database, you're going to have to loop through them.

[00:15:23]Very straightforward, just loop through the entire thing using an offset window, and you should be good.

[00:15:29]in order to patch a collection or to update a collection it's also very straightforward you just use the modify method right here and in this case and I think I have initialized my new collection right here so yeah I have initialized my new collection and I'm going to rename it my newer collection like this and I am also going to update its description and this is going to update that collection right here so now if i go right here and i list my collections again you can see that my new collection is now renamed to my newer collection next thing you can also delete it of course i'm going to delete my newer collection of course and now that collection is gone and now if I run this again you will see that I only have my collection right here now I'm going to get my collection back right here so my collection which is the one that contains all of the information and all of the data points that I showed you before and what we're going to do is we're going to peek into it and that is just a very convenient method just collection dot peek and it basically returns a given number of data points from it, including its embeddings. So if I click, if I peek into this one right here and I set it limit to two, I will get these two data points with all of their embeddings. These are, of course, all of the numbers in the high dimensional field, as I told you before. It returned the embeddings, sorry, the data points number one and two. And in order to take a look at what they actually include, Let me take a look at it.

[00:17:07]So here they are.

[00:17:08]The documents are the Colosseum is a Roman amphitheater and penguins can swim.

[00:17:12]There you go.

[00:17:13]And then it has all of their metadata as well.

[00:17:16]A very convenient method.

[00:17:17]And of course, the most convenient method of all is the one that I already showed you, which is collection.query.

[00:17:23]Okay.

[00:17:24]So there you go.

[00:17:24]That is a crud on top of collections.

[00:17:27]Now let's take a look at how to persist your database.

[00:17:31]All right.

[00:17:31]So now let's take a look at how to persist your database.

[00:17:34]because so far we have initialized our database like this, but we have not specified a persistent directory, which means that this database actually was living in memory, which means that if I restart my application at some point, it is just going to delete everything that I had previously ingested.

[00:17:53]It's not going to store anything.

[00:17:55]Everything is going to be deleted because it was in memory.

[00:17:59]Now, what I want to do is I want to be able to persist it in a local directory.

[00:18:03]And in order to do that, all that I have to do is initialize my database like this.

[00:18:08]So instead of doing just chromaDB.client, I'm going to do client equals chromaDB.persistentClient.

[00:18:14]And this one right here takes the path as parameter.

[00:18:18]And the path in this case, I mean, this one is, by the way, this is the default path.

[00:18:22]But in this case, it's going to be chroma.

[00:18:25]And then I'm just going to initialize a new collection called rackDocuments.

[00:18:29]So as you can see right here, here is my file system.

[00:18:33]Now I'm going to execute this.

[00:18:35]And as you will see, it will create a new directory called Chroma right here, which is going to include my SQLite instance with my ChromaDB right there.

[00:18:45]And now if I want to, for example, add new documents to this instance, let's suppose that I just want to add these documents right here.

[00:18:55]You will see that they are added right here.

[00:18:57]And that is actually persisted.

[00:18:59]There you go.

[00:18:59]Here they are.

[00:19:01]And this is how this is persisted.

[00:19:02]Now, if I restart my application, I can just reinitialize my Chroma client targeting this path right here, and this one right here is going to remain persistent.

[00:19:12]I mean, in this example, probably not because this is a notebook, but if you were in a local environment or in an application, this is how you would persist your database.

[00:19:22]There are, of course, other ways to initialize this.

[00:19:25]You can also initialize it in the cloud client if you're running this on their tenant service.

[00:19:30]or you can also host a Chroma database like you see right here and just connect to it via a host and a port.

[00:19:39]That's also possible.

[00:19:41]That's beyond this crash course, but that's beyond the scope of this crash course.

[00:19:46]But at least now you know how to persist a vector database locally when you're using Chroma.

[00:19:53]So there you go.

[00:19:55]And now the next thing to do is to join the completely free AI engineering community that's right there in the description where you will be able to meet AI engineers you will be able to talk to me you will be able to ask me questions and I'll be there to answer any questions you have if you ever get stuck with this content with the videos or with the courses and of course right there you also have the opportunity to get the complete AI engineering boot camp of which this video is one of the lectures so feel free to take a look at that. It'll be a great pleasure to see you around there. So there you go. Today we have covered what is Chroma, what is a vector database, what is an embedding, and we have also covered pretty much all of the operations that you can do with a vector database like Chroma.

[00:20:43]So there you go. It has been a great pleasure as usual and an awesome honor to have you right here with me. Thank you very much and I will see you next time.

[00:20:55]Thank you.