Master C++ Actor Model Concurrency with Header-Only Library

[00:00:00]What's up everyone? My name is Joe Scorson. Channel's called Scorsone Enterprises. And in today's video, we're going to be going over something that I made in C++.

[00:00:09]Um, you guys love the C++ videos. I make specifically options in C++. This video is going to be more pure programming, but I think that it'll still be um pretty sick. I think that you guys are going to enjoy it. I think that you guys are going to like this tool. Um, a while ago, I made an actor model in Zigg called Inklist. And I pretty much just repurposed that zig code to be C++. And um yeah, C++ has a lot of things that make it way easier, way easier to write, but some of the portions could be harder to understand. Uh because we have lambdas and we have uh smart pointers. So we can share state really easily. we can um uh write a lambda and just capture everything that we need to send par specify parameters really easily. We have function um we have stood functional right we have stood function we have invoke we have a lot of stuff that zigg doesn't and so instead of we also have um stood async so we don't have to use something like a thread pool we could just stood async it we have futures we have promises we could just get to block everything until we are until an actor's task is done instead of um needing a wake group right so that is just better in my opinion but C++ is still a schizophrenic syntax X. I think I don't know. It's um I'm growing to love it a little bit more. Um when there's also in this because it was such a so much less to actually write. I was like screw it. Let's add some. I've never used Squite. I'm a Prime Vision fan. I've never used SQLite, Squite, SQL Light, right? So I was like figured might as well um add SQL light logging, SQL light logging to this actor model.

[00:01:55]and uh in it also is a little um Python script so that way you could actually use the DB file. But yeah, we're going to hop right into it. I got a few cool cool things to show you guys. This is my X account. Everything will be here first to get this to a,000 followers before the end of the year. Uh this is my algorithms made in collaboration with Luxa. Every trade I take, I take with this. Um if you like to actively trade, check this out. You get two indicators with it. You get these autoplotted levels and this directional prediction.

[00:02:23]And if you like to trade futures, check out Clear Edge Automation. Uh you can use my code in my link in the description. That will help me out and you can use the code first month 20 for 20% off. Uh you can trade across 20 brokers and set up future trading automation with your own um strategies.

[00:02:39]Very seamless integration with a lot of platforms. And the code from this video will be on GitHub. So check it out if you um want to use it, you want to contribute to it in any way. I'll be down to um look at this code. Right. So, let's hop into it. So it is um a header only library which just this HPP file which I wanted to I I figured it had to be like this just header only just nice and simple right I didn't want it to be anything crazy mainly because I didn't want to learn anything weird but um I think the cool the best way to do this would be to go through a main right here how I would want you guys to use this and then I should put this example on the GitHub as well but go through this example and then um actually uh go through the header library. So um C++ has runtime polymorphism with virtual functions. If you're new to C++, we'll get into that. Basically, we're just overriding, but we're actually going to get into how that uh works later on. And so that's really nice because we can take whatever class we want and then with uh this bit of inheritance right here, we could say that it um has everything that actor has too. So now receiving um whatever else this thing has. Just making sure the types are consistent, the logging, processing messages, the message loop to actually wait for stuff is all um all included in this class without me actually have to specify it each time, which is nice. So all this does is this is our first override right here. Um because this actual scope is this function. So we want to be able to call internal functions from processing messages. So we're going to override the original one from actor, right? Because it's a virtual function. And then we can go through and check if it is a function payload or if it is a uh just a simple string payload. What do we want to do there? Because we can build messages.

[00:04:50]How do we build messages from the um Oh, also I should probably talk about the name of it. Magnificent solitude. Um I don't know why, but I thought about Edward Snowden one day and Edward Snowden one day, like really recently, I thought about Edward Snowden and uh his swapper called Epic Shelter.

[00:05:10]And I think that Epic Shelter is a cooler name, but Magnificent Solitude is kind of like just synonyms for each. Um yeah, so that's why. But so here is a a vector of messages. But so in the messages magnificent solitude, we can make a custom payload with a sender ID and a just a simple string um um oops simple string command or we can make a function payload which actually tells it what function are we calling here. And we could also use um lambdas as well which is pretty cool. So then we could send the messages as a vector of messages. If you want to send one, then just do one, but it still has to be a vector. And then we could block with because it returns a future. So then we could block with future.get until then. Then a little bit of memory cleanup and we're good. So let's go through and talk about how each part of this works. Um the settings is something I didn't talk about because it's really just for the logging. It's really just for the logging because as we when we make the engine, we need to give it a config. So this is just to this is just um it's it own name space uh just magnificent solid settings and it's just what log level we want if we want logging and um the database path because it's SQL light um control B is my face in the way? Yeah. Uh one second what just happened? There we go. Um right here in this logs directory there's um one there's my database but obviously I didn't push that to VS code. Um there we go. But so we have to specify that path for it.

[00:06:50]Some setters just actually set these things and then return itself return an instance of itself which is what this keyword is. Um it's like in it's like in zig when we say uh con self gets at this pretty much like uh what that's doing right there. And this is just the type. It's a reference to itself with these parameters set here.

[00:07:13]Um it could have been a strruct because everything's public but leave me alone.

[00:07:18]Um I wanted a constructor type thing like that. But right now let's continue right here. Let's go into the engine first. Let's just go in order. So, the magnificent solitude engine. That is a sick title the more I say it. Um, in the constructor, all we're doing is making a shared pointer to our actual logger. So, we're going to see what a logger is and then keep going throughout that. So, what this logger is is just uh I keep on wanting to say squeal. I keep a I can't decide on one is uh just making something to write to a SQL light database. So on the um initialization we're going to we're going to um try to make a path to it. And this is our command that we have here to create a a table. So we're going to create some magnificent solitude logs if they don't exist already. We have all this stuff.

[00:08:15]We have the time stamp, the event type, actor ID, sender ID, details, and then created at and pretty pretty sweet logs if you ever need you. You would know every single um everything your engine's doing. So then we can just execute that command and there we go. Now we have all these other functions to well, we have this function to log the event and this one gets changed. Well, the way we use this function gets changed based on the literal thing. Like each one of these calls log event but just with um something different. So to log an event, we get a time stamp uh depending on Windows or Linux. Uh get that time stamp and then a parameter to it is the event type, the actor ID, the sender ID and the details of the um of the login.

[00:09:07]The details we'll get into it a little bit later as well. Um are we in the right spot? Yeah. So then we can just uh get that command and then prepare a message, bind it all together and then we could do um we could finalize that statement and actually write to our um database with the proper stuff from whatever event we want to log. The events that we can log or that we do log rather if you have all the settings is we log when actors are spawned. So the actor type so it is why can't I talk right now? We have the event type which is an actor spawn actor ID which is whatever actor ID we get back the sender ID which in this case is not applicable because um like that's something we tell the uh the thing to do not another actor. I suppose you could but um in this case and the uh details of it which is pretty much just like a human readable message like actor spawn type the actor type and then the ID and then the ID um destroying the actor same concept but just for destroying it logging the message sent message sent receiver ID sender ID the message type and then logging the message process here we get uh some pretty cool stuff too because then we get the um we get a kadich.

[00:10:36]We get a processing time with it. That's what I wanted to say. Oh my gosh. We get a processing time with it which is really nice because we get some sort of um performance logging with it as well.

[00:10:44]We get message processed the message and then the processing time which is that and that is really nice as well because you get logging and performance. So if you're everything's going great and then you know that that's happened to all of us. everything's going great, looking good, and then it just look feels like you hit a wall and you get through it, you can see exactly what took that much time to process. Um, if there's errors, if there's warnings, and if you have a custom, so my chair keeps on sinking down. So, I'm going to get back up into the middle of the frame. There we go.

[00:11:13]And then we're going to keep on going with um what this engine does that way now that we understand how our logger is um is logging stuff. uh but it was with that config and this logger is a shared pointer. So now to spawn an actor we need a a type. It's generic, right? And then we need to make sure that it is a base class of actor.

[00:11:38]This is really important. So now we're going to get our ID for the actor. Maybe we could use UU IDs, but whatever. I'm just incrementing a one every time. Now we need to make an actor. Make a shared pointer of type T. And that's our actor.

[00:11:52]So, let's um yeah, let's take a look at what the actor is first before we get into um any of that stuff, right? Because we inherit from actor. Well, we inherit from we inherit from actor base.

[00:12:16]Nope, excuse me. We we inherit from actor. actor bases in when we actually need to um store the stuff to the uh engine because we need type consistency.

[00:12:24]But right so we inherit from actor. So this is what everything has. We have that um process message function that we override in the beginning. Remember that from our main and then we set our logger and we have all these um properties. So we have a message q to store the messages that we get q mu text that way we that way we can read and write to it safely conditional variables and atomic um running. And then we actually get back the future that we're going to have to get. So in our message loop, we are going to uh lock our mutex um uh check uh oh my gosh, we're going to use our conditional variable to wait with this lock uh and our um with this lock and that and this scope.

[00:13:10]And then while the message Q is not empty and we're running, which basically means that um we have a message to process, we're going to get that message, pop it from the queue, and then we can unlock it here because we really um we don't have anything to worry about.

[00:13:27]We're not reading and writing from something that multiple threads are going to be reading and writing from anymore. So now we can get our start time of processing the message. So then we can actually process that message.

[00:13:37]And then once we get through that um right to our logging the processing time. Now process message is uh the thing that we override in our um in our actual actor right because this is the uh virtual function. We we also have this receive which just um locks the mutex pushes the message and uses our conditional variable to um notify.

[00:14:06]So now the shared pointer as well is the logger. It is protected. It is not private or anything like that. Which means that um um base classes like inheritance can modify this variable.

[00:14:21]So now let's continue forward into our um so we have this actor right? So now we can set these properties. So we have our running uh being true. Our actor ID is the ID. Um if we have a logger set all the logging and stuff like that.

[00:14:37]Next, we can actually um get the worker future and start our message loop. This is well, we're going to start the message loop on our future. This is pretty much what does everything that we need to do. Um now we actually have to store like this is that message loop that we were talking about before to actually um process everything and um process all the messages running. So now uh this handle is like a type agnostic is what I like to call it. It's like this is what um the engine actually uses even though it does have an instance of your actual actor. We have an actor handle which inherits an actor base. So the actor base has a receive function and then we override that because we need to have an actor pointer. We need to know what our actual base is. And then we have this we'll have a receive function. We'll have this same receive function right here. So now this is the type that actually gets stored in our engine because we we need to have consistency with that. We're making a shared pointer. This T is generic and it is an actor.

[00:15:47]That make sense?

[00:15:49]This type is what we want to make an actor. So now we can keep on going ahead. We have these handles and our actors. It's just an unordered map of actors and actor handles.

[00:16:03]So, let's continue here, right? We have that and we want to return our ID so that way we can actually do something. Now, we need to be able to send a message and get back a future to wait for. So, that's exactly what we're going to do. Um, and then we're going to get into how the messages actually get uh created.

[00:16:20]So, to send a message, we're going to need an ID and this vector of messages.

[00:16:24]So find the ID in the handles. Um if we have the ID right from the iterator then we need to we're going to use promises.

[00:16:33]So we need to make a promise get the future from it and then let's actually go through and um let's go through and do some stuff. We need to um log all these messages in here and then we need to wait for every message. So go through every message.

[00:16:52]Then when we're on the last one, uh set the completion, the completion promise to actually be the promise. Now then for each um for this specific actor handle, we can receive this message.

[00:17:05]But we do that on every single one, but on the last one, then we actually have our promise and we can return our future. So that way we can wait for that promise. Um this is if we actually have an ID. If not, then we need to um send a error back in our promise.

[00:17:23]because that way we can return the same type each time. It's still a future, but this future just has an error. This future has work.

[00:17:30]Destroying the actor is just getting rid of all of its properties. Making sure that we actually um properly erase everything. And then that's how the engine works. So now let's go through the messages. I didn't do that. So the messages um is annoying if you try to literally translate the zig code to this instead of using C++ stuff. It is so annoying.

[00:17:58]But it's okay though because in the structure of a function, we're pretty much just storing a function. Uh we have this stood function. This is the function that we're actually be calling from our um this is the function that we're actually going to be calling from the message itself. And then we have a constructor for it as well. And we can stood forward it. So now we need an instruction payload which is either going to be a string string command or this function.

[00:18:29]Only one of these are going to be true at the same time. It's a union enum but C++ has to be ridiculous. and using this um stood variant, right? Just give me a union. Why are we why are we getting I I didn't make no language. Um so we have this class of a message.

[00:18:52]We have an instruction payload, a sender ID, and the completion promise, which is what we actually need to um we need to actually get our work. So when we when we want to make a function payload, we need to um have a type have a type we're going to get our sender ID and the function to actually call here. And now here we have a little um lambda. So I have a note about how I want to um explain lambdas. So we have a capture, we have the parameters, we have a return type and a body.

[00:19:36]Technically in here, technically in here we just have a um a body. So we have this capture which is the variables from the surrounding scope. So this is the um function that we're receiving from calling the function payload. We have a parameter which is the actor pointer that we're going to give it later. And then we have a body which is going to do something.

[00:19:59]So it's going to call this function. Um we're going to invoke I hate how it doesn't bring everything up like go and python do but we're going to invoke this method and we have this actor pointer to invoke it with.

[00:20:14]And then we're going to um return that message and the um and we're going to move this function that we just put it that we just uh that we just created this lambda capture. We're going to move it into this bunk that we made before with this function right.

[00:20:32]Same kind of concept for um for everything else. Uh we also have it if we just want to give it a function just as is, right? just here's my function. Uh, and then we also have it for a custom string command.

[00:20:48]And yeah, that's how all that works. So when we actually want to run all this, we're going to make our vector messages, get back our future, and then we can do get our future. This blocks until all those messages are processed. So if you want to just run it, let's say you're in your own thing.

[00:21:05]Um, let's go through our CMake right here. All we do in the CMake is just the project. Um this is the interface of course because this is a library add uh add SQL light and then get an executable. So let's build an executable right now. So we're going to go into our build directories cmake do uh cmake- d build everything cool and then we can debug uh magnet solitude exe and we can see uh the messages that we get here. So now I'll even delete my log and then just run it again. Make sure that you guys can see that. So I deleted my log here from logs. Uh we're going to run it again.

[00:21:52]See it gets created right there. And we're going to cd CD into logs and we're going to Python uh I called it I called it um parse DB.

[00:22:04]We get back everything in the form of a data frame. So all that logging actor spawn type class counter ID one. That's awesome. That's good stuff right there.

[00:22:14]Um this parse database file is just um we're just getting an SQL like connection uh uh executing a query on that connection, closing it and printing the data frame. Nothing too crazy right here. Um yeah, this is my um this is Magnet Solitude though. Um let me know in the comments any thing that you guys are like, "Why' you do that?" Um, I'm still not the best at C++. I think that I write other languages um better than C++. I think I have a lot more confidence when I write other languages.

[00:22:50]Not even because of garbage collection versus not. Um I like writing Zigg a lot. I think I write pretty good ZIG.

[00:22:56]Um, it's a matter of like C++ changed so much from when I it was my first language. It was the first thing I ever learned in high school.

[00:23:09]And so at that point when like the most complicated thing that you learn about is a hashmap or an unordered map, whatever. Um and then you want to use a lambda capture you know it's like and you have stood um you have stood invoke stood function all these things right you have um um you have smart pointers you have futures and stuff like that you have promises and futures it's I think really easy to get lost in the sauce of just like I know I want to do this thing what is the idiomatic approach for this language because even though this is not the size 430 lines even though this is well granted no comments even though this is not the size of my um Zigg one I feel like I understand my Zigg one more even though it does have a few more features but it's because you could follow every single allocation every single function call every single where this parameter goes for its entire lifetime um with your eyes just in what the code is telling you but not in C++ um it's scope based it's a lot more acidic above the shoulders mustardy it's not real it's not the elemental charts right if you know what movie that's from um but it's what we got and it's a good language I think and things like stood async and stood future right um they I do think they're fundamentally better than something like Zigg because of how seasoned it is. I don't even cuz I think that just once like it's not like not even specifically the language the technology like using a thread pool versus using something like this with turning a future because now you have to synchronize a thread pool in a weight group and now who has what where is the reference to this to this thread pool?

[00:25:11]We don't want to spawn a bunch of different ones. um does each actor have a weight group or or does the engine itself have a weight group? Right? The design choice you have to choose in this it's like per thing you get back a future because even if you wanted to um if you wanted to do something like uh let's do this right here and get rid of that increment.

[00:25:40]Let's just do that now. So it'll just be one thing and then we could say that right there and then autocf example is engine send message counter id and then um I'm just going to make it an entirely new variable messages that and if I do that then I need to actually put this back. All right. Um because if you want to do that, no, that's not what I wanted to explain.

[00:26:27]This is if you wanted to do something like that.

[00:26:35]Um if you wanted to do something like that, the second message happening later on.

[00:26:59]That's not even what I wanted to explain. What I did end up showing though was that um even though we awaited them out of order, not that we waited them out of order, that we got them out of order, these messages were still sequential.

[00:27:18]That's what I wanted to um point out because we have increment two down here because these messages were done first even though, right? Because there's two separate futures. It's not like one synchronized weight group to the entire entirety of whatever object we have.

[00:27:38]Even if we wanted to say like No, I think that is uh we could even we could even bring it one step further.

[00:27:52]I wonder I actually wonder what this will do.

[00:27:59]Let's see what order this gets done in.

[00:28:02]So here, because we're It depends on what's which message you send first.

[00:28:12]You guys following what I'm trying to orchestrate here. It doesn't matter because you have more control in this, which is weird because it's fundamentally less, but you do have more control. It's just based on when you call this and when you get back your actual future because this future still has all of these messages.

[00:28:32]Right? If I were to just move this line just right here, even though the gets are out of order like we had before, I should have kept that there, but whatever.

[00:28:52]Now you have this one back at the bottom because it's a matter of when you actually um these pro these messages are processed.

[00:29:01]They're good. They're done. We can now do other stuff if we so please.

[00:29:09]Even with that, these were still done first. So even though we get this one first, there it is.

[00:29:33]Oh, because I'm tripping now. Um because one second now because I want to um I want to visualize this better. There we go. Um, the motion will carry, but I want to actually see. Oh. Oh, come on.

[00:29:56]This was supposed to be a really quick example. There we go.

[00:30:02]Good. Do you need to?

[00:30:07]Oh, for the love of God.

[00:30:09]Uh, please hold.

[00:30:17]Why am I Why am I tweaking right now?

[00:30:27]All right, let's do that. And then this.

[00:30:31]And this. Yeah. And then that.

[00:30:37]And then this that this same uh thing should happen. But now let's see how the actual state of the uh object gets modified. This is a half hour video, but I really don't care. Um here we go.

[00:31:04]So this f we send this message. This future does work right now. Even though we're this one is done second, even though we're getting this first, see, so that's sick because now all of the state of your actor model can be in a nice mental sequential model like that, even though you're doing a bunch of stuff at different times.

[00:31:37]So that's what I wanted to um and the state is consistent um with when you're sending the messages. So that's what I wanted to um orchestrate in that example. Um hope that made sense. Hope you guys enjoy this.

[00:31:55]Uh comment below what other stuff you guys want to see from me. Um, what other features you might want on this? Like um like a send every like a send every message every like an interval in time.

[00:32:08]That could be cool. Whatever else. Um, if you guys want more quant stuff, if you guys want stuff in different languages. Um, yeah. What other C++ stuff do you guys want to see? Because this is kind of just I like actor models. I think they're cool, but I still don't know what to actually make with them.

[00:32:26]So, let me know what you guys actually want to see in C++ and I'll do it. So, yeah, that's um that's all for me right now. I think we're do I think we're doing great. I think we're looking good.

[00:32:37]Check out in the description all the stuff and um you legends, see you guys in the next one. Have a great day.