Building Custom Crews with CrewAI: Sequential vs Hierarchical Processes

[00:00:00]Okay.

[00:00:00]So in the last video we went through and looked at the basics of actually making an agent and what it takes to make an agent.

[00:00:07]in this video, I'm going to go through making a custom agent.

[00:00:10]I'm going to show you one version using the sequential, API and sequential process.

[00:00:15]I'm gonna show you one using the hierarchical and we'll talk about the sort of advantages and disadvantages of those two.

[00:00:22]So basically I've just got a simple project here.

[00:00:25]We're going to make something that's going to do some research and then write an article on that.

[00:00:31]All right.

[00:00:31]So this is similar to the standard one, but I've customized it a lot.

[00:00:34]So you'll see, as we go through it.

[00:00:36]So in this video, I'm going to use OpenAI for this.

[00:00:39]But in the next video, I'll show you how to convert it to Claude.

[00:00:43]If you want it to use, Claude and use the Haiku model or the Opus model in there you can do that.

[00:00:49]All right.

[00:00:49]So first up, one of the key things that I want when I'm doing something like this is, I want to track actually what's going on.

[00:00:57]so this is built on LangChain.

[00:01:00]So I can probably do some way of sending this to LangSmith.

[00:01:05]Although I don't think that's easily implemented yet in CrewAI.

[00:01:09]so what I've gone for here is just a simple callback function.

[00:01:14]So each agent can actually do what's called a callback step.

[00:01:19]So at the end of each step, they can basically call back to a function.

[00:01:24]and then from that function, we can get a bunch of information out here.

[00:01:29]So this function that I've got here is basically just going to be like the print agent output.

[00:01:34]and actually what it's going to do is just log everything to a text file called crew called backlogs And I'm going to basically have the step number so that I know, okay, what's going on for each thing.

[00:01:47]And then I'm going to be able to know which agent is actually calling this.

[00:01:51]so we can look at what happened in order.

[00:01:54]And then from this, we can use it to sort of debug.

[00:01:57]Now you'll find that with the sequential process, this is not needed as much.

[00:02:03]but with the hierarchical process, it's often very useful to see where your agent gets into loops, where it gets into problems in here.

[00:02:11]this code is basically just taking in some of what the callbacks will send back.

[00:02:16]So sometimes they will send back, basically something like an agent finish from a LangGraph or they will send back a dict and then we'll want to know just exactly what's inside that in there.

[00:02:28]Alright next up, tools.

[00:02:29]So the tools that I've gone for here are just very simple.

[00:02:33]I've basically just gonna use a Duck Duck Go search.

[00:02:37]this is free search.

[00:02:38]You don't need an API key.

[00:02:40]You could certainly use one of the other search engines for this or the other sort of search things like Serper like Exa some of the things like that.

[00:02:49]And you quite possibly can get better results out of this.

[00:02:52]so the search I think, is going to be quite simple for this in here.

[00:02:56]The model that we're going to be using, is going to be the GPT 4 turbo model.

[00:03:01]So by default, I think CrewAI defaults to just the straight up GPT 4 model.

[00:03:07]That still gets very expensive, for doing a lot of your results So by instantiating the turbo model, we're going to specify this model for each of the agents individually.

[00:03:18]And for sort of the master LLM agent the master agent as we go through this, you'll see that.

[00:03:24]another tool that I'm just making on the fly here is just saving something to mark down.

[00:03:29]So this is just useful for saving content to a markdown file.

[00:03:33]we're going to put a date in there.

[00:03:35]We're going to put a random number so that when we look at these, we can see what the last run actually saved in there.

[00:03:42]Another thing in there too, is that I'm going to print out to the logs that this blog post is saved.

[00:03:49]But I'm also going to return that and tell it, please tell the user we have finished.

[00:03:53]So this is the way of telling the agent, okay, you've done your saving now sort of just end.

[00:03:59]And you'll see that I'm going to actually run into some problems with this in in the hierarchy version that we'll go through in a bit.

[00:04:05]All right.

[00:04:05]We basically just put out tools together.

[00:04:08]The other tool that I'm going to use as well is this load human tools.

[00:04:12]So this is basically just giving it the ability to ask us for some information.

[00:04:17]So the idea, the sort of flow that I'm going for is that I wanted to start off by asking me what I want it to do some research about.

[00:04:28]It's then going to go off and do some research about that.

[00:04:31]It's going to write a blog slash article about that.

[00:04:34]and then it's going to save it to a markdown file and tell us that it's finished in here.

[00:04:39]That's what we want to see happening.

[00:04:42]All right.

[00:04:42]So the agents that we're going to use.

[00:04:43]So we've got a research, an AI research specialist.

[00:04:47]Now, this is going to be, what's doing our searching in here.

[00:04:50]Now in this, you'll see that though, that I've got, you've got the ability to take us topics suggested by a human.

[00:04:56]and rewrite multiple searches.

[00:04:58]So this idea of suggested by a human is really key here.

[00:05:02]in this first one, it picks up that idea quite easily.

[00:05:06]it didn't for the hierarchical one.

[00:05:08]So I'll talk about, some things changes that I made for that one.

[00:05:12]but in this case, I've combined two agents into one.

[00:05:15]So we've got the searching and the human part in here.

[00:05:18]So the idea is that it's going to ask us for the topic.

[00:05:22]We're going to give it the topic.

[00:05:23]It's going to run the searches.

[00:05:24]It's going to get the information.

[00:05:26]And then it's going to pass that along to the writer.

[00:05:29]and our writer here is going to take this.

[00:05:32]it's got a whole bunch of, different instructions of how to, basically process it.

[00:05:39]how to use this, to actually, get what we want out.

[00:05:44]And you'll see that when we talk about the tasks, we're going to give it some sort of, more clear definitions of what it's going to get out.

[00:05:50]Now you'll notice that both these, here, both of these are using Open, AI GPT 4 turbo for the large language model.

[00:05:57]for the tools, I've got the search tool and the human tools that I mentioned there.

[00:06:02]And the other thing here is I've got the step callback.

[00:06:05]So the callback function I'm calling a Lambda function.

[00:06:08]So that's just going to be calling an anonymous function, which is going to call this print agent output.

[00:06:14]And we're going to pass in the name of the agent, in there.

[00:06:18]for this one and the same for here.

[00:06:20]So as this gets updated, it's going to know that, okay, what agent is actually needed here.

[00:06:26]Now you notice the here I'm not going to use any tools.

[00:06:28]Originally, I had searched, but I've taken that out.

[00:06:31]I've got the OpenAI GPT 4 turbo in there.

[00:06:34]In both of these, I've got memory equals true so that it, we'll be getting the full memory as it's going through.

[00:06:42]Now, I feel that's useful for any time that you're doing something where you're basically, having information gathering and then passing that information along to something that's going to process it.

[00:06:55]Another thing that we could have done in here is, we could confine these to basically be using JSON and passing JSON around.

[00:07:03]That's something that, perhaps we look at in one of the future ones in here.

[00:07:07]Anyway, I've also limited this to max iterations.

[00:07:10]we don't want to getting into a loop and just going on and on and on and on.

[00:07:13]so there's going to be max iterations equals five in here.

[00:07:16]The final agent is a very simple file archiver.

[00:07:20]So this is just going to take the information, write it to a markdown file.

[00:07:24]so this is using that tool, save content.

[00:07:27]And remember, that's just going to basically take a string in, it's going to save it as a markdown file.

[00:07:33]It's going to return back to the file name.

[00:07:35]and tell us that it's you know, that the whole thing is done now.

[00:07:39]All right.

[00:07:40]So next up we've got our tasks.

[00:07:42]So first task is get the source material.

[00:07:45]you'll notice in the task I've got things like ask the human for the area of interest or the topic of interest I could have put in here as well.

[00:07:54]then I'm going to put in the current date so that it can use that in searches as well for doing this.

[00:08:00]And then the output I can tell it that it's going to basically deliver those search results as a comprehensive full report on latest advancements in the specified human topic.

[00:08:10]Again, stressing this.

[00:08:12]You don't want it going off by itself and then sort of inventing topics to do research about.

[00:08:17]And these things will, if you don't confine them especially with the hierarchical sort of examples All right.

[00:08:23]The agent that we're using for this is the info getter agent.

[00:08:26]We've got our expected output there.

[00:08:28]We've got a description of what the task is.

[00:08:30]Next up is basically taking the source material from the research specialist's report.

[00:08:36]So really your start here should address what you're getting as the expected output here, if you're going sequentially here.

[00:08:45]and then we basically talking about, turning this into an article.

[00:08:49]we've got some things about that we want it to have at least three paragraphs similar to one of their examples.

[00:08:54]but we also got that I want to set a bullet points with the key facts at the end for a summary in there.

[00:09:00]And then the expected output is going to be those.

[00:09:03]So it's just doubly stressing that we want this sort of article to have a number of paragraphs and have these bullet points at the end there.

[00:09:10]Finally the task is to save it.

[00:09:13]now one of the things that I could have put in here is that this expects a string going in.

[00:09:18]So I could have actually put that in here.

[00:09:20]and again, I'll talk about the hierarchical version where it kinda messes this up.

[00:09:25]it would have perhaps helped with it there.

[00:09:27]All right.

[00:09:28]We've now got the whole thing together.

[00:09:30]we now want to build the crew.

[00:09:32]So you can see the crew is going to be out info, getter, a writer, archiver.

[00:09:36]And the tasks are going to be, get source material, write the content, saving the output in here and of course they have to match, you know what we put up here as we go through this.

[00:09:47]you can see here, we've set the process to be sequential.

[00:09:50]I'm going to put full output is true with verbose so we can see things coming out here.

[00:09:56]and then what I've tried to do is actually use the step call back here, but it seems it doesn't like it now.

[00:10:02]It used to be ok with one of the previous versions.

[00:10:05]anyway, that's going to be fine for this.

[00:10:07]we can then basically kick off, this.

[00:10:10]And we'll see that very quickly, it's going to basically decide that it needs human, right as the first action.

[00:10:18]And it's going to ask us what specific area of AI advancements are you interested in exploring for your article?

[00:10:24]Now I've put, some stuff about the new, Jamba model, which is a sort of a combination of a state space model and a transformer model from AI21 in here.

[00:10:36]And I basically just told it that, I've said, okay, what I want is Jamba.

[00:10:40]Now I could have actually written this out long form and actually specifically said, please do research on this that kind of thing.

[00:10:46]I've just chucked in the topic and it's able to work it out pretty well in here.

[00:10:50]We can see that, from this it's been able to work out that, okay, now I know that the topic is AI21's SSM transformer model.

[00:10:57]I can create a search query and sure enough, it then basically, does a search for this adding latest news 2024 in there.

[00:11:05]if I wanted to, I could actually get it to put in, this month's date and last month's date as we went through this, for this kind of thing.

[00:11:13]We can see sure enough then it's basically getting out, the information back and it starting to put together the article.

[00:11:21]finally, it goes through, it gets that information.

[00:11:25]it's then basically, taking that.

[00:11:27]And going to craft that into a final article response with our, bullet points at the end, that we can see there.

[00:11:35]And it's going to then, save that and sure enough, we can see here that it's basically working out that okay, I need to save it to a markdown file.

[00:11:45]giving my tools.

[00:11:46]And then the return back from there is that the blog post is saved.

[00:11:50]We've got this, day date and a random number at the end there.

[00:11:56]So we know that this has done and we can see that the final output is the blog post about Jamba's revolution in AI has been successfully saved, to the markdown file and we've got the mark down named there.

[00:12:06]All right.

[00:12:07]If I come in here, I can just look at that final output.

[00:12:11]and I can see, sure enough, there's the name of there.

[00:12:14]If I cat that out, I can see sure enough, that looks like it's saved.

[00:12:18]we will look at that in markdown in a second, judge, the article.

[00:12:21]but here, what I want you to see too, is that, it's also save this out.

[00:12:27]Now, it turns out that I forgot, I ran this a few times and I forgot that it was a pending.

[00:12:32]so you can see that actually it's also saved at one before this, but this is the latest run here where you can see that I put this in.

[00:12:40]I basically got out this and I can see the steps here.

[00:12:43]Step one, step two, step three.

[00:12:46]I can see that, okay, this comprehensive report at step three has got a lot of what probably makes it into the final thing.

[00:12:54]Now, if you wanted to, in this sort of, you could have a step in between where you ask very specific questions for this.

[00:13:03]So if I was going to make one, just for evaluating blog posts about new models that people launch.

[00:13:10]I might ask things like, how long is the context window?

[00:13:14]How long is, these kinds of things and get those back.

[00:13:17]Now, it does seem to have done pretty well here with unprecedented context window.

[00:13:22]but we can see that if we go through, we've got, step four, step five.

[00:13:26]we can see what we're getting out, step six.

[00:13:29]We're finally coming to this sort of agent finish out.

[00:13:33]And then finally we're getting to the saving part where it's been saved out and we get the final agent finish telling us that, okay, this is done.

[00:13:42]If we come down here, load up, the markdown and we can see sure enough, we've basically got a reasonably good article within markdown, with the, three main.

[00:13:54]parts to it.

[00:13:54]I guess we've got an intro part as well there, and we've got the sort of, bullet points, summary that we asked for at the end, where it's basically talking about this, Mamba combines SSM technology with traditional transformer architecture.

[00:14:08]Somewhere up here, I'm guessing the SSM we'll talk about state.

[00:14:12]So it does seem like it got that there was a structured a state space model, this SSM model, but that didn't make it into our final output out out here, by the looks Anyway, we can see that with this version, it's done a decent result.

[00:14:28]We could definitely fine tune the prompts a lot for the specific kinds of tasks that we wanted to do.

[00:14:34]and we could certainly constrain it more in regards to, the length and things like that, by using JSON, Asking it to make Each response X number of sentences long and stuff like that.

[00:14:47]I do find that the Claude models tend to pay attention to length when requested better than the OpenAI models.

[00:14:54]but we'll have a look at that going forward.

[00:14:56]the other version that I wanted to show you was the hierarchical version.

[00:14:59]I basically just wanted to take the same thing and, use, it with just changing the process to hierarchical.

[00:15:06]The problem was just using the same prompt with the hierarchical it didn't want to ask the human for the topic.

[00:15:13]It just seemed to be constantly thinking that it could just make up the topic itself.

[00:15:18]So in that case you'll often see was things like this.

[00:15:22]So you need to break it down and what you can do is break it down to multiple agents.

[00:15:26]So here, what I've done is basically now make a topic getter agent.

[00:15:31]I'm saying that this is a senior customer communicator.

[00:15:34]And so I'm saying consult with the human customer to get the topic of an areas of interest for doing research.

[00:15:41]Give it a small backstory, but basically this is getting it to, ask the, the customer.

[00:15:46]You can see the only tool I've given it here is human tools.

[00:15:50]and then I go into the info getter, like we had before.

[00:15:53]And this time I'm not giving it the human tools I'm giving just the search tool.

[00:15:58]And again, I'm rewriting it when we look at the tasks so that the first task is get human topic.

[00:16:05]and it's just about getting that topic.

[00:16:08]And passing that topic along in this.

[00:16:11]So clearly state, the topic that the human wants you to research.

[00:16:14]and then with the next one, I've fine-tuned it a bit as well to say, take the topic for research and search for it on the internet.

[00:16:21]So the idea is that when we've got the handoff from here to here, we can see what the expected output should be communicating this human art topic in here.

[00:16:32]The rest of it is reasonably similar.

[00:16:35]the problem here is I probably didn't have clear enough instructions here about, passing this in as a string.

[00:16:42]and I'll show you why in a second.

[00:16:44]So just setting up the crew, we've basically got, one more agent now and one more topic, because we want to get that human asking bit at the start.

[00:16:53]Of course we could just hard-code that and pass that into, the first agent, if we wanted to.

[00:16:58]Here, I want it to show you just getting it to actually do some sort of asking out.

[00:17:03]So you can see here that it worked out that, okay, ask, question to coworker.

[00:17:07]what is the current topic?

[00:17:08]And then it decides that, oh, okay, well we really need to ask the human the topic.

[00:17:13]So I've given the topic exactly the same as the previous sequential model.

[00:17:19]it passes it in.

[00:17:21]and then in this case it will often ask again because it's got the hierarchical thing, so it can repeat things.

[00:17:29]It's not just one step to another step.

[00:17:32]So It feels like it wants some clarification.

[00:17:34]So it comes back and says apologize for any confusion.

[00:17:37]Could you clarify this specific topic and area of interest you are looking for us to conduct research on?

[00:17:44]and so at this point I put in research about the new model Jamba and then gave it the, the exact same thing.

[00:17:50]Okay.

[00:17:50]At this point, it now understands that, oh, okay, I know the final answer.

[00:17:55]I've got this, I've got to research this new model.

[00:17:58]It goes off.

[00:17:59]does a pretty good job.

[00:18:00]I think at finding, the different, things.

[00:18:03]it also, is perhaps good in the doing some comparisons.

[00:18:06]So it's now doing reception, comparison with GPT 3.

[00:18:11]not sure how useful that is, with this, but you can see that it's got the, Because we're in doing the hierarchical thing.

[00:18:18]It's now going to be often doing multiple steps on the same thing.

[00:18:22]So for hierarchical models often what I will do is have another agent, which is a checker agent that checks, the outputs and tries to steer it back on track if it goes off, in here.

[00:18:35]All right.

[00:18:35]It does finally go through and gets, an article done.

[00:18:39]and it seems to be, on point in many ways.

[00:18:42]it then runs into problems saving that though.

[00:18:45]And you can see that it's trying multiple times in the tool.

[00:18:48]So I've got a print statement in there just to check it out.

[00:18:51]You can see the, it errors out, two times before it finally realizes, ah, okay.

[00:18:57]That we need to basically, take this.

[00:19:00]And it understands that it misunderstood the input for the tool.

[00:19:04]it then passes that in.

[00:19:06]and finally saves it out and we get to our ending.

[00:19:09]and if we look at this, we can see that sure enough, we can look at the call logs.

[00:19:14]Ah, I forgot to reset, the call number in there.

[00:19:17]So it starting at 28 in this case.

[00:19:20]but we can see that it's gone through and gotten the information done the search here.

[00:19:25]we can see that it's done, multiple searches in this case, which is nice, perhaps compared to the previous one.

[00:19:31]and this is where you could actually write it so that it uses JSON and send out a list of, responses for each search to the final sort of writing element in here.

[00:19:43]we go through, we can see that it's written the content stuff like that.

[00:19:47]this is where we ran into the error.

[00:19:50]that we've got, basically running into this error of, saving the output in there.

[00:19:54]And we can see that each time sort of, we can see, you know what actually went wrong there.

[00:19:59]so that we could then rewrite the prompt to make better use of that going forward in here.

[00:20:04]Finally, we come and have a look at the output.

[00:20:08]We can see that.

[00:20:08]Okay.

[00:20:09]it's done, okay.

[00:20:11]But it haven't gotten the bullet points at the end like I wanted.

[00:20:15]And I'd probably want to refine it so that anytime it sees an acronym like SSM, it's got to try and spell out exactly what that is.

[00:20:24]so you know, it really in a good article, I guess it should, in the first paragraph or The first chunk of text, it should be, talking that this is a state space.

[00:20:33]model, in there.

[00:20:35]anyway, I'll let you go through those and have a look at this as well.

[00:20:40]You can certainly play.

[00:20:41]if I was trying to make this for real, I would certainly play with this a lot to basically also return the URLs for this.

[00:20:50]So I'd probably use a different search where I was getting the specific URL and passing that back along with, content for it.

[00:20:59]and maybe even having, a nother tool that actually goes to that URL and downloads the full content of that.

[00:21:07]for being used in here as well.

[00:21:10]But this should give you an idea now of how to put together some of these, yourself.

[00:21:15]Like I said in the next video, we'll look at doing this with the Claude 3 models from Anthropic and you can judge the difference of how they go in that.

[00:21:23]Anyway, as always, if you've got any questions, put them in the comments below.

[00:21:27]If you found the video useful, please click like and subscribe.

[00:21:30]And I will talk to you in the next video.

[00:21:32]Bye for now.