Multithreading in C++20: A Beginner's Guide to Concurrency

[00:00:01]okay good afternoon everybody so I'm going to give you an introduction to multi threading with C++ 20 and as part of talking this through then I'm going to make two assumptions firstly that you're starting a new project so there's no pre-existing code that's going to limit your uses of um C++ 20s uh threading facilities and secondly that you do indeed have a C++ 20 compiler in library uh if either of those don't apply to you then you won't be able to use things quite as I've presented on slides but um you can usually know take the ideas and apply them to know code that's either restricted to older compilers or that you can help migrate existing code but it won't be able to use things quite as they are on the presented so we're going to look at a couple of things going to start with choosing your concurrency model and then we're going to look at actually if you're using explicit threads for your concurrency model then how do you manage those and then finally the well the tricky part which is synchronizing data so when you've got a problem that you want to use know multi-core Hardware to take it know to make your program faster in some fashion then you've got to choose quite how you approach that so you've got to think know what is the reason that you're know you're doing this why are you try what are you trying to gain from it and there's two fundamental reasons that you might do and the first is scalability you know that your problem has lots of independent parts and could in theory be divided up over however many processes your users can throw at it and you want it to seamlessly work across whatever Target platform it runs on if someone runs it on a ancient single core machine then it should run and if they want to run it on something which has got know a thread ripper in with know loads of cores then it should run and you want it to just work without problem and take advantage of those morees so you want scalability and the second reason why you might want to use concurrency is because you want to have a separation of concerns you've got things which are are part of your application and but they are potentially long running processes that you want to have running and they are fundamentally separate from each other they might communicate during their lifetime but you want them to be running all the time know you might want to have something that's doing spell cheing in your application or know putting the red wigglies in your in your codes to highlight the fact that you've got a typo and you want that to be running in the back background and so you and you don't want that to be interfering with other operations that your application might be doing so you have run things on separate threads in order to separate the concerns and so you don't have to have them all tightly integrated into the same event Loop and your choice of reason now why you are fundamentally trying to use concurrency and parallelism affects the choice of model that you use if your purpose is for scalability then amal's law replies the speed up is one divided by know the the serial fraction of your code which is one minus the parallel fraction and then the parallel fraction divided by the number of CES and so now as the number of Calles goes up then the number on the bottom will get smaller and so the scalability goes up as the fraction that can be parallelized goes up then the then likewise then the overall speed up goes up but you have to be careful about this because it's not necessarily as straightforward as you might think if you say well 90% of my program can be paralyzed that might sound like quite a high number high high fraction of your application but if we put in a th000 CES that still only gives us 9.9 time speed up which on a thousand cores doesn't sound as desirable as you might have hoped for now even on 100,000 cores you still don't make it to 10 times speed up if you only 90% of your application can be paralyzed so we really need to push up the percentage that can be paralyzed if you're looking for scalability if you can 99% of your scalability of your of your application can be can be parallelized then on 100,000 cores you can get almost a whopping 100 times speed up so now this is just something to be born in mind now obviously from a general perspective then know you might think well actually it's the data processing part of my application that I care about and the startup and the Finish know don't matter so you can trim off lots of those and that might incidentally be serial and then say well it's the data processing part that I'm going to apply this this law to but you'll still need to make sure that it really really can be paralyzed and because any of the any little bits of overhead that cause things to run on a single thread at a time sap your potential for scalability okay that said then if you are looking to do do things in parallel then the easiest thing to do if it works for you is to use parallel algorithms your compiler provides them for you you can generally make use of it just by sticking standard execution part as the first parameter to most of the standard Library algorithms if you have a bulk of data that can be parallelized then your standard liary runtime will do this for you as best it can and if this works then this is the most straightforward thing to do it doesn't work for all data sets and so and you need it to be know a large enough data set to be worth the overhead but again if it works this is the easiest thing you should do this if you can and I'll say that as a general thing for the techniques and approaches that I'm going to introduce in this talk is start at the beginning if you can use the ones I present earlier and they solve your problem do that the ones that I that I introduce later on in the talk you should be using less frequently and only if you need to one problem with parallel algorithms is that they return and no they they join the results in between so if you have two calls for transform and then a second to R to reduce there is a little serial portion in there where it has to gather the result results from the first from the transform then start the reduce and farm everything out again that's a Serial fraction that's going to reduce your scalability by amar's law so what you want to do is you want to combine these consecutive calls if you can now you want to do something with transform reduce as a combined operation if otherwise you would be writing transform and then reduce and there are lots of ways that you can combine the operations sometimes with a bit of creative thinking in the way that you wrap things into ranges and pass um function objects to things like transform reduce but you can in a large number of cases convert big bulk data processing into a single P algorithms schol that then will parallelize nicely but that doesn't always work now you don't always have big bulk data that you can run a single parallel algorithm across so the next thing that you could think about is splitting your work into independent tasks which you can then give to a threadpool to actually run those tasks sadly we don't yet have a standard threadpool so you're looking for a non-standard extension here but there are many around but the model is the same now you have your threadpool which manages actually the execution the thread pool determines the number of threads it determines how the tasks get divided among those threads that's all abstracted away from you you just pass the work to be done onto the threadpool and then it's taken care of and the and the implementation takes care of scaling your your work and making sure that it takes advant AG of the inherent Hardware parallelism available in your in your platform so what about if we're not looking for raw parallelism we're not trying to just get better throughput on on a big problem but we're trying to know separate the concerns of our application so raw straight line performance is not necessarily a priority we're potentially after things like responsiveness and um low latency in and things like that rather than literal raw throughput and so you tend to be thinking in terms of larger sequential tasks that each each of them run can happily run concurrently with the others maybe in the background and in which case then you're probably starting to think about dedicated threads and if you're trying to do this then standard J thread is what you should reach for when you pass the Constructor you give it a callable thing that you want it to run and it will start you a new thread and run that task on it so starting in managing threads no how do we do that Beyond just the the simple Declaration of a j thread well going to take a little Sidetrack from that and talk about Cooperative cancellation first so many applications have tasks that are running that they might need to cancel you might have a 3D render that you wish to cancel and you hit the nice cancel button in the guey you want it to actually do something and stop that 3D render uh you might be downloading a file likewise you want it to cancel and you obviously you don't need a guey to want to cancel things you might be running a command line application and hit contrl C or your application might detect that the Network's gone down and so it wants to cancel something internal or detect that you've typed some new characters into your editor so that background compilation that it was doing it wants to stop that so it can start a new one that deals with the current state of the code so you need some way of interrupting those background tasks now some most operating systems provide you with a nasty Sledgehammer way of stopping a thread but this is a really nasty Sledgehammer way that then leaves the state of the system in something know you might as well stop and start again because it's unpredictable quite where it stopped that back that thread so that's undesirable so we want something better more C++ like and in C++ 20 we've got standard stop token and standard stop source that provide this it gives us a Cooperative means of cancellation and the nature of this Co Cooperative means is that you raise a request for a background thread to stop and if that background thread doesn't check it's not going to do anything and you need to periodically check to see whether you've been canceled whether there's a stop request so how do we use it first you create a stop Source this is your source of stop requests and then you get a stop token from that stop Source you pass it around it's a is intended to be a lightweight copiable value so you can take many copies of it you can pass it to multiple places all of those places can then see the the stop signal when it arrives and then at some point you've got something that needs to stop so you call request stop and then periodically those things that you wish to interrupt have to actually make a check know has somebody asked me to stop so they call stop requested and then it's up to them what they do it's just stop requested is just a function that returns a bullan true yes somebody has asked to stop false nobody has yet and so it it's entirely up to you what you do do you stop the entire thread do you just stop the little bit of processing that it's doing right now do you say yeah someone want me to stop I'm going to print a message and then carry on anyway now it's entirely up to you you need to make sure that whatever you do makes sense for your application and just to reiterate if you do not check whether stop has been requested then nothing happens this is purely Cooperative there is no automatic Interruption so as an example now we have two functions one create takes in a stop source that it got from somewhere and it's going to keep looping until it's done and know do something and and then when it's finished it's going to call request stop and then on on some other thread we've got this stoppable function running that takes a stop token and it's going to keep doing its stuff until the stop has been requested and so then when when the stopper function finishes it's going to request stop and that will in turn cause the stoppable function to finish because because at some point it will finish its current iteration of do stuff it will then check the flag and say oh yes it's time to terminate my while loop and then return to the caller one very Nifty part of the stopped token facilities is this stop callback feature this allows you to use the stop token cancellation mechanism with anything that provides its own can ation mechanism any third party library or operating system library that provides a means of canceling an operation can now be integrated with stop tokens so you're doing an asyn Kyo operation you're reading from a file you've got file system path and now you've got a stop token you open the file and it gives you a handle the io Library tells you that if you call cancel io on that handle it will cancel any operations that are currently happening if there's none happening it's okay that's fine there nothing's canceled but if there is an operation it will cancel it so we register that with our stop callback passing in the stop token and this little Lambda that calls cancel IO given the file handle and then we do our read of the data if this was a blocking call with the potential for being cancelled then if on another thread somebody calls request stop and this stop token is now signaled then our little cancel iio function our little lra is called and can cancel iio is invoked and so therefore our read data operation is cancelled and then it does whatever the library says whether it returns partial data or throws an exception check what your your IO Library says but no the operation will be interrupted I see there is a question if you want to make go out to the microphone so people can hear in which thread is the Lambda being called the the Lambda is called in the thread that calls request stop so it is very important that you make sure that this is a minimal minimal thing because you are in effect injecting some code into another thread so you need to watch out for that but it does mean that therefore it is running it's on a thread that actually def know has has scope for running some code because the thread that you want to interrupt is blocked on on its AO I have another question can't be moved or just should it always be copied yes you can move a stop token but you but it's lightweight so it's easy to copy as well okay so so stop tokens are really powerful but there's a reason why I introduced them before we started about managing threads and that's because they do integrate with J thread so 99% of the time if you're trying to manage explicit threads in a C++ 20 code base you want to use J thread you can potentially use standard async if you do it right if you want a count a a result from the thread operation so you know you you spawned a thread in order to calculate some specific result that then you can you you can get back back into future via standard async but most of the time then if you're managing explicit threads that not what you're doing because if you were calculating results then you're probably that's a task based thing that you probably want to put on the thread pool now we still have standard thread it's still there as an option in C++ 20 code bases but you should only use it if you've got no other choice if know you've got some compatibility constraint which we were trying to rule out by by a new project with C++ 20 sort of stuff but you might have a third party library that you have to talk to so when you construct a j thread object it starts a new thread and it starts a new thread with a with the function and arguments that you provide to it the callable thing is the first argument and then any to your J Constructor and then subsequent arguments are in turn passed to the callable thing so that can be a raw function pointer it can be a Lambda it can be a No callable Object like an instance of standard function or whatever and then it invokes the function the callable thing you provided with the arguments you provided and an additional stop token as the first argument so this is tightly integrated the stop token facility is tightly integrated into your J thread stuff so that you know know that it's your thread knows when it's going to be asked to stop okay for backwards compatibility with existing code that wants to start threads and doesn't need stop tokens well or for new code that doesn't need need to be stopped because it's potentially short running or you've got some other mechanism of stopping it then if it can't pass a stop token and that know your your your callable is not callable with a stop token as the first parameter then it emits it and just passes the arguments you gave the same as standard thread did so the basic API we have a default Constructor we have the callable Constructor that starts the new thread which constructs a stop Source internal to the J thread and passes the token the destructor is one of the things that is know really nice about J thread whereas with standard threads you always have to decide to explicitly join or detach and the D structure will terminate if you haven't done one of those the J thread Destructor always joins but first it asks the thread to stop by signaling the stop token and this is really nice it now makes it a proper raai type if you use it in a scope at the end of the scope then it will ask the background the thread to terminate and then wait for it to do so you can always get the ID which you might be able to use in know for a table look up or just for writing out you know for debug information and then you can explicitly wait for a thread to finish by calling join and if you call join then that doesn't on its own request stop from the stop token you have to explicitly call that later J thread is a value type it is a movable value type not copiable because what would that mean to copy the handle to to an existing thread so ownership can be transferred around you can stick it in a standard vector or any other container um and in particular there's no need to use new I've seen too many people who are used to Java threads and C threads where they say new standard thread whatever don't do that for safety purposes then the callable itself and the arguments are copied into storage local to the thread which potentially know helps avoid dangling references and race conditions see if you really do want a reference or something you can use standard ref so that the thing that's copied is the reference wrapper alternatively you can use a Lambda with a reference capture but again be wary of dangling references so just an explicit bit of code demonstrating the the structure semantics we have our thread function that takes a stop token as well as some other arguments and it's going to do stuff in a loop no until somebody has requested that it stops then we have our f function that starts A J thread it's running our thread function and then it's going to do some stuff of its own and then at the end of the function the destructor will signal know the stop request and then join with the the J thread and wait for thread Funk to finish so know J thread and stop token are tightly integrated we implicitly create a stop Source you can uh get that stop token that is as the the first parameter to your thread function and we get the request stop implicitly when we destroy it but again you still need to check the stop token it's all very well having having one but if you don't check it it does nothing but there's also additional API around it we can get hold of that stop source so you can manually request stop yourself you can get a stop token that you can pass to something else that needs to be canceled at the same time and you can explicitly request stop know without having to first get the stop Source but it is equivalent to doing so okay so if we're running threads then most of the time we're probably going to want to have some communication between our threads and therefore there's going to need to be some synchronization on the shared data that they're communicating across if you have got shared state you need to watch out for a data race this is where you have unsynchronized access to a memory location and from more than one thread and one of those threads does a right it doesn't matter whether the other one's doing a read or a write it's if it's unsynchronized and somebody is writing then it is a data race and the problem with that is that data races are undefined behavior and we all know undef defined behavior is Thoroughly undesirable therefore we need to make sure that we do have appropriate synchronization and thankfully C++ provides us with a whole bunch of facilities to do so we have latches barriers F Futures mutexes semaphores and atomics this is a a nice nice list of things that allow us to get the synchronization that we need you should use them in the order listed if you if some if you can use a latch to do your synchronization do that and likewise if you if you can't use a latch use a barrier and so forth down the list so what the latches look like well standard latch is a class object it's a single-use counter that allows threads to wait for the count to reach zero and when it reaches zero it stays zero forever it latches you when you construct it you specify the non-zero count that you want it to have and then very points the threads that are involved with whatever it is you're counting down no will decrease the count and either those threads or other threads might wait for the latch to be signaled and then when the count finally hit zero then all those threads that are waiting are now woken up and any additional threads that try and wait afterwards immediately return because the latch is already signaled so the API is quite simple we have a Constructor that takes a count you can count down to decrease that count you can wait for the count to reach zero or you can arrive and wait as a single combined function which does both both of the above it call it counts down and then it waits so you might wait for some tasks to finish you have some function food that's got somehow it cles up a thread count of how many background tasks it's going to run and it creates a latch with that many t with that count and then it's got a vector of data which your background threads are going to populate so we have our Vector of threads each of them no is going to make some data to fill in the appropriate data element and then it's going to count down the latch to say is created it its section of the data and then it's then the threads are they're still running they're going to do more stuff that's appropriate to whatever your task is and then the main thread having started all these background threads is going wait for the latch and this waits for all the threads to have done their make data section they're still running potentially doing their extra stuff but they've all completed the make data line and so we know that our data Vector is now complete and so now we can call Process data on that Vector data and we have nice synchronization there and then obviously at the closing brace then the J threats know all destroyed and we will wait for them all to finish and note that in this case there's no stop tokens because we decided that know do more stuff is just going to progress linearly to the end and we're not going to try and interrupt it and one case where you might actually choose to do this is for tests if you're writing multi-threaded code you need to test it you want to set up a scenario where you've got know one thread doing one part that's accessing your data structure another thread doing something else that's accessing your data structure you want to make sure that but it works okay and a nice way to make sure that they're all set up nicely is to use a latch you set up the test data you create the latch you create the threads and the first thing that the threads do is arrive and wait on that latch and this avoids a very specific problem and that is that starting threads takes a long time comparatively there are many supposed multi-threaded tests that I've seen where the the test scenario has multiple threads running intended to interact with the data structure at the same time but the use the test code starts a thread to do one aspect of that and then starts a second thread to do another aspect and the first thread has finished its operation entirely before the code on the second thread has even started because starting that second thread is such an expensive operation by using a latch to do this then we can get past that expensive startup both threads will now started and only when both of them have reached the latch do they continue so they're both actively running at this point ready to be scheduled by the OS and if you've got enough CPU calls they probably just sat there waiting to go and this is a really good way of ensuring that they all start together it's not guaranteed of course because you're still subject to the vagaries of the OS scheduler but it's the best way that I know of ensuring that they start properly okay if a latch doesn't work because you need something that know is more than just that one time one time synchronization then maybe a barrier works for you cuz standard barrier is reusable it's intended for a loop because it and works in phases you can struct your barrier with with a with the count of how many threads are involved and a completion function that was going to run at the end of each each Loop and then your threads they arrive at the barrier and when they've all arrived then the completion function is run and then the barrier is signaled so any waiting threads are released and the count is reset and so you can do this in a loop and this works great for know operations which you're running in a loop where you've got either data which you can have which arrives in chunks and then multiple threads process each chunk as it arrives but then they all need to finish on processing the first chunk before again then you reset you get the second chunk and then they all process the second chunk or alternatively where you've got a pipeline where you have have a discrete no step of moving the data to the next next thread in the pipeline but they're all synchronized on in their steps like that the API again is quite simple it's a template and has a template parameter which is the task type you don't have to specify the template parameter and if you don't then there isn't you don't don't provide a task for it and you also provide the count which is the number of threads that are involved in the barrier a thread can arrive at the barrier and it gets back an arrival token which you can then use to wait alternatively you can combine the two operations which is what you're going to want to do most of the time which is to arrive and wait for your barrier and as another alternative you can arrive and drop and say I'm arriving at the barrier this time around the loop but next time I'm not going to not going to participate anymore so decrease the count permanently so the first time through they're five threads the second time somebody calls arrive and drop the second time through then the count is only four if they again someone calls a dve arrive and drop the count is only three this time and so forth they're designed for Loop synchronization as I mentioned and the completion function allows you to do the necessary serialization step between the loops know so maybe you've got you've been rendering a frame and you need to ship it out across the network maybe you've got some got some data to write to a file maybe you need to read the incoming data and and get it in a prepared in a state ready for the threads to uh to to work on and that completion function is run on one of the participating threads but you don't know which one and there are there is a standardization proposal currently before the committee to know alter the wording around quite where the completion function runs but for now you know it's you can it's safe to assume that it's one of the participating threads so what might it look like in code well here we have a barrier that's going to take a standard function which takes no parameters because the barrier is not going to give you any U and then we're starting we've got some number of threads and we've got a task that we're actually going to run in this case it's just a plain function which naturally fits into a standard function we have our worker thread each of those takes an index to say which one they are and and potentially a stop token and whilst whilst they've not been asked to stop they're going to do stuff that's specific to that particular worker index and then they're going to arrive at the barrier and wait for all the other threads before they then proceed around the loop if you don't have a loop synchronization issue then maybe barriers don't work maybe Futures work for you instead Futures are a mechanism of onot transfer of data so that might be because you've got an asynchronous task that creating a set of data from which standard async will deal with for you or alternatively then you can use standard promise which gives you a means of explicitly setting the data from one thread to be transferred to another or package task which allows you to wrap up some data in a way potentially you can then submit your package your package task which is now a callable thing you can potentially pass that into a threadpool and then it will make the data available from the return value of your task become available in that future that you created from it they all give you a future of a t that holds a result the default constructure of a future says I am a future with no State it's also what you get if you try and move if you move from a future it's reverted to an empty one and therefore we have this valid function that you can call to check whether it actually has any state you can wait for it to be ready either just as a straightforward blocking call or you can wait for a particular duration or wait until a particular point in time and then you can then get the data now if you don't haven't preed previously waited for the data to be ready then calling get will wait for you at that point and then it moves the data out from the result leaving the future empty and valid will now return false on that future there's a One-Shot transfer of data so now if you're waiting for some data then you have a future you have know this blocking function know is going to wait for the data and then call get to retrieve it and pass it on to do stuff to calculate it that weight can be emitted but you might want to explicitly put it there depending on what else is going on around maybe you want to wait for the data to be ready then do some pre-processing before you actually retrieve the value or maybe you want to pull maybe you want to wait to check is the is the data ready yet if you wait for 0 seconds then that's that's a poll and then it returns you a flag to say is it ready or was there a timeout and if it's ready then we can get the value and then process that if you're explicitly using a promise then we have this sort of API we have a default Constructor this time the default Constructor creates the sh state it is waiting for you to give it a value at which point it can be transferred to the associated future you can get that future with a with the get future member function which can only be called once per promise you can set the value or you can set an exception because the future shared data can store an exception too if it stores an exception then when you call get it will throw that exception rather than returning a value so we might have two threads that are talking to each other with a promise and a future we create our promise we get the future from it we then move that future into into the thread that's going to receive the value which then calls get to wa for it on the other thread then we take a reference to the promise we could alternatively move the promise in and then we call set value on the promise with some generated data and then that value that was created by the make data call will then propagate via the future and come out via into the G and return as the result which is then pass the do stuff on the other thread if on the other hand you didn't store a data value but you stored an exception then when you call get as I mentioned already that will actually throw the exception so this will throw an object of my exception which you hope that uh you can actually cancel if you actually wrote the code AS on the screen then thread one would terminate your application because there's nothing to catch that exception I said already that you can use standard async to start a thread that's going to return a value standard async has an optional first parameter that tells you what you want it to do with your task that should always be standard launch async if you which tells tells async that you want to start a new thread if you omit it then the library can can decide at a whim whether it starts a new thread for you or not and if it doesn't start your new thread then it doesn't run your function at all until you call get on the future or wait in which case then it actually runs a task on the thread that is waiting and because you don't know in advance whether or not that's going to happen then you don't want to do that if you have the strange scenario where you do always want it to run the task on the thread that called waight then you can say standard launch deferred instead but really if you wanted to do that then you can emit Ayn all together and just call the function where you want the result the Futures returned from asyn are slightly odd they own the thread they have an internal handle a bit like a j thread and the destructor of the future will wait for the async thread to finish I said already that it's one shot if you call get then you can't get the data again the future is no longer valid and therefore trying to call get a second time will throw if you really want to do that then you need shared future F which you can get by calling share on the on the original future the original Source future becomes invalid and then the shared future can now be used multiple times you can then copy it into two threads both of which call get both of which will then get the value but Futures are one shot you can only transfer one piece of data even if you can receive it on multiple thread with a shared future and so therefore sometimes you need to do something else and if you need to do something else then probably what you want to do is synchronize your data with a mutex but be warned mutex is short for Mutual exclusion it is a means of preventing concurrent execution this is going to reduce the scalability of your application if you start putting mutexes in because it adds a Serial section so we should use them as sparingly as possible C++ has six mutex types this is mostly five too many you should use standard mutex and only standard mutex unless you have a very very specific reason and have profiled your code and determined that that really really really is important that you use something else and likewise there are RI types for dealing with locking and unlocking you want to use standard scope lock it's very nice you create a scope lock object and you pass it a mutex in the Constructor and it locks the mutex it will unlock it in the destructor if you pass it two mutexes in the Constructor or three or four it will lock all of them in an order that manages to somehow magically not deadlock and then it will unlock all of them in the destructor and that is guar gued Behavior it guarantees not to deadlock if you pass it multiple mutexes even if you use it on a second have a second thread and you pass the same for mutex is in a different order on a different thread it's okay the runtime library takes care of you very occasionally you will need to use unique lock particularly if you're using condition variables and lock guard is there if you only want to lock one mutex and you need backwards compatibility with something C+ plus 17 or earlier where you don't have scope blck shared lock is for the sh shared mutexes which you're not using so you don't need to worry about that Anthony yes the um the guarantee to not deadlock unlocking multiple blocks yes does that work well in practice I I've stack Overflow tells me that it's a little it's very difficult to run that algorithm and it might be costly or what's here kind of got feeling on it yes it works and yes potentially it's slow no so those cases where you do need to lock the multiple mutexes you should use it rather than trying to do anything else um but if possible you should design your code so you don't need to lock multiple mutexes makes sense yeah thank you okay so yeah we have an example we have some data in this case an INT it could be anything and we have a mutex to protect it we create our scope lock object and give it a mutex and it will lock the mutex we do something to our data and then the at the endoscope we unlock our mutex without doing anything explicit the destructor takes care of it if we have two mutexes and we try and use lock them explicitly in some order you'll probably going to make a mistake there will be threads that lock things in the other order which will deadlock now in this case if you try and transfer between the same two two accounts in different orders then we lock the same two mxes in different orders which is a deadlock this is why you might need to lock multiple mxes together because then it is deadlock free if you're passing data between applications between threads in your application then you might want to wait for it to to get there you're waiting for some the other thread to do something useful provide you some data that you can then use so how do we do that well and assuming you can't use Futures which have a nice weight member function because there something you want to happen repeatedly well we could busy weight but we really really don't want to this is nasty no it's in terms of program operation it's possibly correct but no in terms of actually being useful it is not because it's going to occupy CPU time checking to see whether the data is available it's going to lock the mutex whilst it checks which prevents the background thread actually making the data available and generally we don't want to do this so it it consumes CPU time it wastes electricity it delays the notification don't do that standard condition variable is therefore the answer it is a mechanism for avoiding busy weights as an optimization of our weight Loop and this is where we need unique lock the the primary difference between unique lock and scope lock is that unique lock allows you to unlock the mutex during its lifetime without having to wait for the destructor because we want that from our condition we want to say whilst we're waiting unlock the mutex so that the the thread that's producing the data can actually take the mutex lock so we use our unique lock to lock the mutex we then do our condition variable weight call passing in that unique lock and passing in a predicate which is anything callable that takes no parameters and returns a bull that tells us whether or not we we are found what we're waiting for so in this case it's a Lambda it's going to check whether that optional has a value or not and the library will now take care of it when data is available and we get a notification then we will wait from the waight we will check the predicate if it returns true then we will return from the weight and process the data the library might wake up and check and recall your predicate multiple times you can't rely on it being called only once it will certainly call it immediately when you call invoke weight to check whether the data isn't already ready and then it will potentially call it multiple times later whilst it's waiting but it will call it with the mutex locked every time so you don't have to worry about locking the mutex manually and and possibility of data races and then but it won't return from weight until the data until your predicate returns true on the other thread you want to notify and so we have a call to notify one after we've made the change to the data Note that because we're not using condition passing to condition variable here we can just go back to using scope lock putting the little the scope around the lock and then the set to the data is a minor optimization does two things firstly it reduces the scope of the lock to only that which is required to the um to make it work it's always good to minimize the length of time that you you're holding a lock so calling notifi whilst holding the lock is unnecessary so that's know an unnecessary extension of the newex lock time and also it allows the other thread to wake up quicker because it might wake up immediately upon that call to notify if you haven't yet hit the curly brace then this thread that's doing the notify still holds the mutex and the other thread will wake up from from the weight try to lock the mutex in order to check the predicate and then say Oh I can't get the mutex because the other thread still got it which then adds adds an extra delay if we had a busy weight Loop then we could always cancel with with request with stop tokens but standard condition variable doesn't work with stop tokens for that you need standard condition variable any but other than the change of adding an underscore any on the end and the fact that you can now pass a stop token into your into your weight function it's exactly the same the weight function now has a return value which you can check which tells you whether or not it was being signaled whether your and the return value is the return of your predicate so if is if the weight is interrupted with a call to notify then it checks the predicate and returns true hopefully if it is interrupted with a with a St request for stop then likewise it checks the predicate and this time it probably going to return false but maybe somebody has set the data to True after the stop request was received in which case then it will return that the data is indeed available so in this case if we if it returns false there was no data so we're going to return because that means we were interrupted okay semaphores are the last level thing that you want to look at and they represent a number of available slots you can acquire a slot which decreases the count and then when you release a slot it puts the count back up if you try and acquire a slot and the count is already zero then you will block unless you use a try function in which case then it will fail and return false and you don't have to be the same thread that acquired a slot in order to release it if you want to know how to do things useful with semaphores follow the link in the slides to the little book of semaphor or Google it it's not so little there are a lot of things you can do with semaphores mostly you're better off using the high level structures in the C++ Library if in rather than trying to construct the same something the same from semaphores but it's fun to read through and try and work out how it all works a binding semaphor is so named because it has two states one slot free or no Slots free and you could use it instead of a mutex but standard mutex is probably better in C++ 20 the type is standard counting semaphore with a maximum count and St binary semor is no an alias for counting semor with a count of one we have blocking calls to acquire and non-blocking calls know which either return immediately or after a duration or after a Time point if they can't actually acquire the sem4 by the time so if we have a simple function that acquires a semor at the beginning and does some stuff and then releases it at the end if the sehore has five slots then that most five threads can be here now there are two number fives on this slide they are different the five in blue in the angle brackets is the maximum count of the semor and the slide in red the five in red in the parenthesis is the initial count of the semaphore so that could have been 1 2 0 1 2 3 or four as well as the five with the same five in the angle brackets but know having that would depend on exactly what you using your semaphore for obviously if you start with Slots of zero you have to call release before any thread can acquire acquire a slot okay atomics are the last thing that you should reach for only if you really really need to but we have them in 20 because sometimes they are indeed necessary they are spelled standard of atomic angle brackets T where T is the type of the data that you're storing it must be trivially copyable and it must be bitwise comparable so using it copying it around with me copy and comparing it with meum is a must work for it to be usable with standard atomic but other than that it can be anything it can be a struct could be a struct with 10 million chars in it that you can use that with atomics it probably won't work exactly like you intended because you know you using Atomic hoping for nice Lock Free CPU instructions and then the compilers likely say that's far too big I can't do that I'll use a spin lock instead so you need to watch out for that there is one case well two I suppose where that it's not required to meet these trivial requirements and that is where the type is one of the standard Library smart pointers shed pointer or weak pointer since C++ 11 there's been three functions that allow you to do atomic operations on Shar pointers but nobody used them right if they knew they existed at all and so we have Atomic of shared pointer which deals with the cases where you do want that just because you said standard atomic of T that doesn't mean it's Lock Free doesn't mean that it uses the atomic memory uh CPU instructions from your current platform there might be an internal mutex which is probably a spin lock there are three types which are guaranteed to be Lock Free Atomic flag Atomic signed Lock Free and atomic unsigned Lock Free Atomic flag is a very very simple Boolean which doesn't even have the full set of Boolean operations no you and the integer types don't tell you how large they are you have to actually know query that in in some other way but on most platforms standard atomic of an integral type is going to be Lock Free and so is standard atomic of a pointer but you can always query with the is lock is always lock free member variable a Conex were variable so in summary avoid managing your R threads if you can use the parallel algorithms use thread pools even if they're non-standard ones although hopefully it's know coming soon then we we'll have know senders and receivers and the new execution model which will provide us some standardized means of having a thread pool if you must manage a thread yourself use J thread with stop token for cancellation then if you need to synchronize data start with latch and barrier and future because those are the high level things that are easiest to manage but if they don't do what you want then most of the other time you're going to be using standard mutex with standard scope block and potentially condition variables for waiting and then in those very rare cases where you really do want lowlevel stuff or all your or the only data that you're protecting is something very simple like an integer then you might reach for standard atomic okay that is prepared material we have time for some questions question about stb okay there are two execution policy that we can pass into the Deford one and then the asynchronous one ideally when I write a library write a computation model I would want to get the same result uh with either uh but I'm free that if I use something Advanced like St barrier and latge like one may get stuck or they may result in different result do you have any comments on that yes my suggestion is that you always use standard launch async um so yes I if if you use deferred then your then your um callable is run in a different context than otherwise and so yes you might end up with different overall Behavior if you're using latches and barriers or anything else but generally don't want to do that you don't want the defer thing you want standard launch async and so just always use that that is my recommendation uh so oh over here Anthony over this side okay hello uh like in the example we saw this morning where it was like oh I need to if I want to have a long running thread long running process and I create a j thread and my main then I have to keep it running somehow is there a paradigm for that is there a you know wow Loop or your dot run on the J thread and it'll just keep it going for till the end of time until there's a cancellation through a signal or some other as long as the J thread object is alive then the background thread will keep running so you can either just leave it on the stack and then and then it'll run until you exit scope so if it's in main then until you exit for some other reason return from main or call exit or something um alternatively then have a have a vector of J threads that you can stick them in and have have that some class that manages those and there's there's lots of ways but as just keep the J thread around for a while and so if you did something like um created a new J thread kept it on the Heap exited your program it would still be alive and the program actually wouldn't exit it would just keep it alive forever if you if you exit from Main then it return then it closes your application down okay was a question on this side first uh it's a bit of an esoteric question maybe are you aware of any work that would uh maybe use like uh profile guided optimization to switch lock types like Pro like profile your code like and run it and then like it might take one of your locks and turn it into a spin lock or something like that as an optimization I have never encountered any anyone that does that no okay thank you yes you might want to do it yourself but nothing that will do it automatically question from uh Zoom uh Philip asked why is there no stop token support for existing threading utilities like mutex for future okay the short answer is because nobody proposed it um and every every change to the C++ standard Library requires that somebody has to have actually suggested it in a formal paper that can then be discussed but there are other reasons as well and So like um mutex is usually a very simple wrapper around a a um OS provided type and the OS libraries typically don't provide support for things like stop token and so that would require then more heavy lifting moving into into the C++ runtime rather than relying on the OS and so we don't do that um future is entirely owned by C++ libraries and so in theory could have added stock token support but as a General principle then know the things that we're working with future then we're wanting to move to the the new senders and receivers execution model and that provides a better um solution and that does integrate with stock tokens and so hopefully when we get there then that will solve most of the problems around know trying to integrate existing existing code with stop tokens okay question on this slide twofold question any reason why something equivalent to F is synchronized didn't make it in the standard and regardless of that there any reason though why it should be avoided because in my opinion it's a pretty useful tool for signalization that guarantees corrects okay so um Folly synchronized is a library type that wraps some data of some specified type and a mutex there is an active proposal that I wrote which is in the pipeline for standardization it hasn't yet made it to the to the standard because there's been other things that were more priority to higher priority to discuss but it's still in the pipeline and hopefully we will get it at some point in the next few years so 23 or 26 it won't be in 23 because the feature set for 23 is now finished so hopefully 26 but at this point I couldn't commit thank you I have another question on the stop token so if if the Lambda is ex Ed on the thread uh which request the stop does there have to be synchronization between the data that the Lambda accesses in the local stack of the other thread and if not why so yes it it depends on what data that you're thinking of um so in general yes there requires to be synchronization but the act of creating the stop callback means that any data that was visible at the point of ating the stock callback if it hasn't changed in any way by the time the um callback is invoked on the other thread then you're okay so if it's just like literally the handle value that is now on the example we had the file handle if that's not then changed then it's safe to use it in the Lambda um if it's something that is changeable then you need to put in something synchronization there thanks okay hi thank you one of the thing I was talking to a colleague the talk and I realized there's sort of two uses of threads in the community um one isort of what was described which is using threads to make an application responsive either to events or a or something um and the context we were usually coming from is is using threads to make an application fast in terms of like a high performance Computing filling the machine um and previously the sort of state-ofthe-art had been the Intel TBB library for that case and the conventional wisdom was never used the standard Library mutex or threadpool um I'm wondering if you have insight into sort of you know where implementations are headed of because you we would love to be using standard Library instead of a vendor suppli extension um do you feel like the the thread pooling and mutex are approaching a reasonable performance for that kind of high performance case or is it it's still it's intended only for the responsiveness if the like if the parel algorithm provide the processing that you want then use those and on on GCC they use TBB under the hood anyway so it is probably very similar to know what you're expecting but obviously your particular um use case might not map nicely onto those in which case then yes you need a threadpool TBB is one of those that I would currently recommend um there isn't yet a standard threadpool um things like um lib unifa provide an implementation of the senders and receivers stuff and so that might be worth a look to try out a to get used to the API and B to see whether the um the thread Pool Works know for your use case but yeah until we have a standardized one then I can't say what the implementations are doing with a standard threadpool because there isn't one thank you okay this is the last question okay so let's say we're working with uh you start a a thread and you get a promise and you pass that promise to the thread you you assume that that thread would set a value to that promise there's three parts that I want to ask about this one what happens if you destroy the future before the promise has been set to what happens if you destroy the promise uh without setting anything and then third um if you set an exception without ever retrieving that value before destroying the future what happens to that exception okay so first one is easy if you destroy the future then you set the value on the promise and it just goes nowhere um and that applies whether it's a value or an exception if you don't if if you store an exception and you don't call get then the exception never gets rethrown it's just stored um on the other hand if you have a promise and you destroy it before you set anything on it then the destructor will store an exception of um a system error with um empty promise or promise not set I can't remember the exact thing but you get an exception that says no you you destroyed the promise without setting a value and then when you call get on the future then it will throw that exception so if I understand correctly if you set an exception and you destroy the the future without getting the the promise that exception goes away yeah thank you okay if anyone has any further questions feel free to grab me in know in the corridor or something and I'll do my best to answer but that's it for today