How Insects Anchor Spatial Information | Central Place Foraging

[00:00:05]yay all right we are live welcome uh welcome everybody um who is joining us for season three of uh future foraging seminar series um my name is Hannah habakyan and I'm one of the co-organizers of this seminar and today I'm joined by uh Laura who has been here since the beginning um and Barbara Webb who will be our keynote speaker today and then as well as two of our four new co-organizers because we've been expanding the team so today we have Shivam shitness and my Morimoto joining us and also new on the team is um Emma Scully and IBO touch taking and then from previous years we still have Dennis Goldschmidt and Rory Bedford so um if you've joined us before um you know that in for the very first episode we have a keynote speaker who gives sort of a broader perspective on some aspect of foraging and today our keynote speakers uh Professor Barbara Webb um so Barbara studied psychology at the University of Sydney before pursuing a PhD in artificial intelligence from the University of Edinburgh and then she held lectureships at the universities of Edinburgh Nottingham and Sterling before returning to Edinburgh where she became a full professor of biorobotics and her lab our lab's work focuses on how insects with their relatively small and complex nervous systems solve sort of real world problems and she focuses on exploring sensory motor transformations in the context of both more reflexive behaviors like phonotaxis in cricket and more complex navigation like you see in in ants and what I think is really special about Barbara's work is that although there is a focus on building computational models of of brain function her lab combines this with Behavioral Studies and they're often carried out in the field in collaboration with experimental scientists and I think this sort of connection really helped to identify what are like the sort of real world problems inside my face and her lab also tests proposed models or algorithms by not only simulating not only simulation but also implementing them on robots and then testing them Outdoors um so I think this this approach has been really successful in many cases it has inspired neuroscientists like me um to follow up with looking for sort of neural implementations um of these proposed algorithms and today's talk by Barbara will be um central place foraging how insects anchor spatial information I'm very excited to hear uh hear this talk so it's going to be just briefly reformat the 45 minute roughly 45 minute talk and um if you have sort of short clarifying questions please um put them in this chat um because Barbara can see that and can respond to that if you have longer questions that are better suited for the panel discussion we'll have 30 minutes or so after the talk to address that and please put those in the ask a question box below all right so um I think we're ready to go well thank you very much for the invitation I'll share my screen and hopefully it will all go smoothly um and yeah as Hannah said I'm actually quite happy to to take clarifying questions as we go along because often it's talking online you feel like you're speaking to nobody for a long time so kind of nice be nice to know there's a bit of interaction um as we go so I'm quite happy to to take questions and I'll try and be watching the chat for those so um as as Hannah said um my group does insect Robotics and although I'm not going to talk a lot about robots today um I thought it might be useful to just kind of give the background that I'm coming from which is by a robotic background where what we mean by that is that we take a very mechanistic approach to trying to explain biological systems um so we to understand a biological system we really should try to build the mechanism and embody our hypotheses as actual machines and so when we say machines we don't just mean simulations we mean literal physical robots a lot of the time although we do do computer simulations as well and just um you know some of the things I've done is to to build models of cricket sound localization multimodal models of Escape Behavior models of walking behavior and even some soft robot models of maggot Behavior um but a lot of the work we did you know was really fairly simple sensory motor control um and where we've been going lately and and being really interested in in more recent years is more complex behaviors and in a sense behaviors that happen you know in the real world and behaviors obviously that are relevant to this particular seminar series which is um foraging behaviors and we're focused on Central place foraging insects and this is an example of an ad and this lovely slowed down video of an ant from Antoine wistrack shows a desert ant that's trying to find its way home in real field conditions and we know that they forage to bring food back to the nest and for these species events and for a number of other related ants species they don't use chemical trails to do this for each ant is navigating individually back to inform the nest and this navigation is absolutely crucial to their their own Survival because it's extremely hot in the environment that they're uh operating in um and it's also obviously important for the colony survival that they navigate efficiently they've been shown to do this over very large distances up to a kilometer away from their nest which is often a tiny hole in the ground and in complex cluttered terrainas in in this example and also as I'll be describing there's a couple of key mechanisms that they use which is path integration and visual memory so we do do um field studies of these ants and this is one from a while ago now but we've only recently really analyzed the results um so this was work done in again in collaboration with Anton wisdirect and also with Mike Mangan who is a postdoc in my group at the time where we were trying to follow ants through their entire foraging careers um with differential GPS and then also downward facing camera and so um I'm actually going to focus on the the camera data so this is effectively although this looks like quite a complex set of equipment in fact we can now replace this literally with just one camera and following the ant as it moves around where we don't constrain where it goes at all we just follow it um so what this looks like when we're doing the tracking in the in our Outfield in civil is a bit like this so we're following ants around we see when they we control when they come out of the nest we have the marked so we know which ant we're following and then we try to follow a particular ant from the very first time it comes out and then each subsequent time it comes out and to see what it does and um what we have been kind of developing um is this fantastic tracking system by Benjamin Risser so this is actually now this is the downward facing view of the high resolution camera um and you can see we're actually able to track the ant even though it's this cluttered background that the camera is moving um we can still and that you know sometimes the ant is even getting occluded on the things we can keep a very good record of where the ant is going um we also in this particular context were also um getting information about the environment so also taking lidar scans of the environment to try and be able to reconstruct the actual 3D world and the Ant is in and then we could try and create this kind of anti point of view where there's this low resolution like the amateuria of the ant and also this pattern in the sky which we know that the ants can see corresponding to polarization so just to go back to the the tracking system um so this is the the kind of global results and um Benjamin who's now at the University of Munster has been working with his PhD student Lars Hulk to actually not just recreate the tracks but to then map the tracks back onto the ground surface so this is actually the tracks of I think 14 different ants all the tracks of these 14 ants from there from the nest and each of them is going as you can see in a different direction they're each foraging individually and this is roughly an eight meter Circle but some of them are going beyond that most of the time we try to give them food at eight meters and let them come back again but otherwise we didn't constrain where they where they went and so this shows all all the ants mapped all their trails from following them with the camera um mapped onto the ground surface and then we can actually zoom in and we have super resolution detail of what the ants are doing so you can see the little Wiggles in their Trails for example we can see what they're going through what the ground surface is like and and really start to analyze in detail what's going on in some of these tracks okay so let me talk in slightly more Global terms but um about this work uh so as I said we know that ants can do path integration so basically this is a naive event that's coming out of the nest for the very first time it's taking an outward route and here the the kind of thickness of the um of the line shows the um the angular velocity effectively so when when it's thick it means it's stopping and turning around and looking um so it's exploring out and eventually it finds some food and when it gets some food what it does is it doesn't take the same route back that it took out but rather it takes this fairly direct route back to the nest and then indeed when once it comes out of the nest again it's able to it doesn't follow the route that it took before but it goes back towards where the food was and then comes comes home again so the other when when Once the art has actually done these two trips we actually then try and force it not to use the path integration memory um but to also use visual memory so here we now pick it up from the nest so it's done this twice so it's been out in the world twice in its life it's been found some food come home gone back to the food come home that's all its entire experience of the world so far um we pick it up and we put it back at the release point so back at the the food position when we've taken it from The Nest so it has no path integration and what we see is that it does a very quick exploration and then is able to follow the route back home the visual route back home and as I showed you before this is in this quite cluttered environment so I can't see the nest from where it starts but it is able it's actually already learned visually to follow the route because you can see for example that it goes um faster when it's on exactly the same route as it was on before it if it gets if it comes off that route it slows down and starts to look around and then as soon as it gets back onto this same visual route it follows it again um yeah so briefly you you can use any kind of camera we've actually used this tracking system with a mobile phone camera uh the main thing that helps a little bit is to have some kind of gimbal that stabilizes it parallel with the ground and you keep it roughly at the same height but even some height variation can be dealt with so so where we're aiming towards here is that it's really very very simple to collect the data and then we can apply this method and we have a paper coming out soon about about the tracker so do get in touch with me if you want to hear more about that um so what I'm going to talk about though from now on is really you know what more about these two mechanisms the visual memory and the path integration mechanisms and then hopefully I'm going to go fairly quickly over both of those um and then try and talk about how they how they get combined um so let's start with visual memory what's the problem um basically this is what the world looks like potentially to an ant so this is again we've taken kind of pictures of the ants from the anti-point of view along one of these routes and then we've kind of recreated what it might be seeing and then done it in low resolution as we think there and I should be so what the answer seeing is like the lower one of these videos and seen that twice and after that it's able to follow that route home again so it really seems quite surprising that its memory would allow it to follow this this route through the a bunch of grass after just one or two experiences um we think that this is being this visual memory is encoded in the mushroom body um partly because it has to involve some kind of pattern recognition and we know that the mushroom body in the insect is important for our Factory pattern recognition um and we we kind of exploit this idea that was first suggested by the Sussex group and badly in the Sussex group that what the ant needs to do is not necessarily recognize exactly where it is it just needs to recognize does the direction it's going in at the moment look familiar so this graph is trying to illustrate that if you imagine that the ant as it goes along the Route show that stores a little kind of snapshot of what it sees every few centimeters along the Route so that's kind of go from North to 800 centimeters here um and then when it's trying to go along the Route again if it if it is on the route and it looks to the left and to the right so that's the heading angle if it's facing in the same direction as it was when it took the picture then it should have as very small image difference so that's what's so shown on the z-axis here and as you go away from that as it as it gets further and further from the root it the diff image difference increases so essentially what the animal has to do is just follow this Valley of familiarity and basically you have to associate the views you experience as you progress towards the food or towards the nest um and then to do this Association we think it's happening as I said in the mushroom body because it's the area of the insect brain that's specialized for factory pattern learning and for ants and for other insects that do this kind of visual navigation there's extensive visual input and we think the circuitry can perform the function so this is a kind of a complex picture which I'm going to go over quickly but happy to come back to later on um if people are interested um but essentially what we suggest is that it's it's quite a simple memory mechanism in a way um so the input either originally uh this was so I'm trying to get my pointer to come but it keeps disappearing um so the input is projection neurons either from olfactory input or here from visual input this gets projected into this set of cells called Canyon cells and it's a high dimensional projection so we go from a lower Dimension to a higher Dimension because there's a large number of V cells and for sparse encoding and then essentially um what you do is to learn is to associate the sparse code for anything that you've seen gets associated with which direction you're going so here either you're going home or you're going towards the food and you make an association and then if you see another sparse code you associate that and so on so because it's a sparse code you can actually associate many different images in this way um so I think that's a good point about you know does it require features in the environment versus pattern matching so in this model it's completely pattern matching we don't do any kind of feature extraction at all we literally just take the grayscale values and map them to the projection neurons and map that into the Kenyan cells and then we just learn those so it's it's completely a global pattern matching and probably the insect is doing some more visual processing before things get to the mushroom body but we didn't want to make any assumptions about the visual processing so we just did it as this kind of really raw data and it turns out this works surprisingly well and allows you to make this decision as I said am I on the route or am I to the you know am I facing off the route which direction do I need to turn to be facing the way I was facing originally and that actually gets you along the Route quite well um and it also even if you're not on the route itself you can kind of extend this method to to do homing um if you learn things while you're facing towards the root so this is what I call view Memory as a kind of more General thing than just root memory so we save the same mushroom body model but now we associate we have learning walks or flights so I think you might have had um Paula Fleischmann talk to you about some of the learning walks in ants before so this is data from ants where they come out from their nest and as they leave their nest they keep turning around and looking back at their nest and the assumption is that they're actually learning the view of the nest from different directions okay and again work from the Sussex group this time from Alex Stewart showed that if you if you put this into the model if you basically say okay I've learned what the nest looks like from these four different places around the nest and now you try and get home from all these other places around the nest using this simple kind of turn and look and comparison you know where do I get the best match between what I see now and what I've seen before you actually get very good directed uh information about how to get home from from all sorts of directions around the nest that you haven't been before so this seems to be quite efficient and and good way to get home as I said it's not the only way that the ants get home they also get home by path integration and I wanted to just um highlight this particular paper by vika stuff and Chung which I think very nicely explains some things about path integration that people often forget so essentially what you need to do in path integration is literally is to integrate your velocity along a journey so if you do a complicated outward path you integrate your speed and direction all the way along that path and you can do that that will produce a home Vector that points back to your original position but people often kind of discuss or assume you know what what frame of reference or coordinate system that could be done in so it could be a geocentric coordinate system where you're you're doing it based on your your home location is the origin of your axes uh or it could be an egocentric system where you're you yourself are the origin of your axes and it could be Cartesian coordinates it could be polar coordinates and the kind of natural assumption initially when you think of path integration is that it should be um egocentric polar coordinates they're basically a home Factor kind of implies it's wherever I am now I know how far I need to turn which is the Theta of the Polar coordinates and I know how far I need to run which is the r of the Polar coordinates and that will take me home um but as they they really nicely point out in this paper there's good reason why that's not the best solution in fact um so roughly speaking if you have polar coordinates to encode your location then um when you're getting close to your location the change in theater goes to Infinity so basically you're having to update your your R and your Theta As you move around and these are the update equations but that change suddenly starts to get really fast and basically as you cross over your whole position your theater is changing at Infinity which is pretty hard to imagine how you can have something how you can encode that um also for egocentric Cartesian coordinates that doesn't work terribly well either because if you imagine if you rotate on the spot then your X and your you're keeping your track of where your home is relative to you as you rotate on the spot then your x-axis and your y-axis are changing very rapidly and very continuously so your measure of X and Y have to change very fast just as you rotate on the spot when in fact you know where you are in relation to home is not really changing just just your angle to it and then for both the polar coordinates and for both egocentric coordinates you actually have to use your the current state of the vector as part of the update equation whereas for the geocentric Cartesian you just need to know your own movement and then you can update the equations so they conclude from this paper and I think it's quite convincing argument and I really encourage people to to read it if you haven't seen the paper um that really any sensible way to encode your path integration should be geocentric and Cartesian okay and the only disadvantage there is that you have to maybe you have to transform the coordinates somehow to to control how you get home from there okay so this will become relevant um when I now talk about how we think this is encoded in the insect brain where we think path integration is in the encoded in the central complex of the insect brain and again I'm going to go over this model quite quickly I've talked about it in a lot of detail in a lot of other places but I want to try and get to the the kind of combination of the two things but the essential message is that we know that the right information goes into this neural network so basically it gets input from the sky compass and from optic flow so that's direction and speed and therefore it could integrate those two things to do path integration so it gets input from the sky polarization and I'm just again going through this quickly so you can measure you know responses to polarizers in this Central complex you can measure responses to optic flow in different directions and both those bits of information are there and come together in in this circuit so this is the the model that we've been working on and it's not even though it looks like a beautiful tidy model it's actually based on real topology from neuroanatomical connectivity there are some some details obviously that I've left out but the the essential structure is is actually the real structure of this circuit in the insect and what it encodes the inner circuit in a circle here is basically the current heading of the insect relative using its sky Compass the middle ring are these neurons that get input both from the compass and from speed neurons from optic flow as I've shown here and then that forms what they will do is form a memory As you move of where you are and then the outer ring does the steering and I think my next slide hopefully will explain a little bit how that is one way to explain how that works so you can imagine essentially As you move in different directions you're integrating your speed relative to the direction you're moving and that means what you end up with is a kind of uh um population code across this set of neurons that encodes so this is your actual home factor and it's encoded as the activity in this set of neurons um and this is actually a geocentric Cartesian encoding it's it's a redundant Cartesian encoding because you have more axes than you actually need but it still is a Cartesian encoding and it's geocentric and then what happens in this circuit is that um the connectivity between the middle ring and the outer ring means that it's it's as though you're kind of rotating this memory 145 degrees to the left or 45 degrees to the right because you have a columnar shift in the circuit and you're comparing that to your current heading and if you basically take the difference here of this your current heading and your desired heading your memory of where you where you where your home is um so you just subtract these vectors you can see this is what you get that you get much bigger there's a much bigger difference between this one than there is between sorry between this and this there's a bigger difference than between this and this so this tells you which way if you just then sum up the overall difference that tells you whether you should turn left or right to get a better match so you want to get a better match you want your current direction to match your home Direction and that you can use that directly to steer home so let me just show a little animation of this happening so this is this circuit in in operation we have a little simulated insect going out from its nest and its current heading is encoded in this middle ring as it goes further and further away it's summing up more and more in this outer ring in the opposite direction which forms the memory pointing back home and then when we allow it to use the steering circuit it steers itself back home by comparing the current heading and the memory and then when it gets near to home the memory the zero is out so then if it overshoots it will just keep coming back again and do a little search pattern just emerges around the nest okay so I know that was quite quick um but as I said I wanted to get on to to some of the kind of really interesting further implications of this particularly for foraging and for spatial representation um so we've just talked about path integration and then earlier I mentioned this Vector memory the fact that if the animal gets home after finding the food um it then goes straight back out to the food again so that's the blue path here so what we are we we actually found that we can make a very simple hypothetical addition to our circuit that I just showed you that will do this Vector memory so this is essentially what it looks like is we just say um when when you find the food you should just store the current path integration state so you here we're just storing it as kind of Weights um from this Vector memory weights onto these 16 values and then that basically acts as a kind of inhibition for the steering so what that means is that when you've got back to the nest so your path regression has got to zero you you apply this this inhibition and that means your circuit is now going to try and get you back to zero again but now it's trying to zero your vector memory relative to your sorry your path integration relative your vector memory so it's trying to make this difference zero so that means it will move you to the place where your vector memory is equal to your path integration or vice versa your path integration is equally effective memory in other words it will take you back to where you started so let me just show you that again maybe here and it'll make it clearer so here we imagine that the animal has gone out from home it's found food it's stored a vector memory at the food and then it's gone home and when it gets home it now inhibits it's Central complex with this Vector memory and that's as though kind of in its imagination although it's actually at home it's it's Central complex tells it it's over here so now it tries to go home again and it will that means it moves in such a way that what its path integration tells it balances against the vector memory and because it actually started at home that means it's actually moving back to this place but then it couldn't if it wants to go home again you can just remove this inhibition and then it's path integration will be correctly at this place and we'll get it home but more interestingly what if it gets to this food place and there is no food um and he doesn't want to go home it wants to find some other food well if it has some other Vector memory that it's stored before it could ins basically stop inhibiting with the first Vector memory and now inhibit with the second Vector memory so that means it's actually got to here it's now doing this inhibition and it's trying to do something that means that it's part integration and the inhibition cancel out which is equivalent to trying to go home so that's the path that would go but this as I said this is kind of its imagined position it's actually starting from the food position and so now what it's going to do in while it's trying to take this motion is that it actually is going to take this shortcut to the second food position so this can actually explain how the animal could do a novel shortcut by effectively vector addition and we we've obviously done simulations to show that this this does actually work nicely in practice and then we can actually extend this to the situation which is not seen in ants but is seen in bees um called trapped lining where bees are able if there's multiple food sources they'll actually go around several different food sources before they come back to the nest and they've been observed to gradually develop more and more efficient routes around them so initially as they gradually discover different food sources they'll add them into their Loop but as they find these food sources and use them more often they their Roots get more and more efficient so they either go round one way or or the other way um this has been compared to solving the traveling salesman problem it's not quite a solution to that but it's it's a good optimization of the of the route so again for using the circuit that I just showed you we just added in a fairly simple Edition so now again as you explore you store a vector memory for each food source you find and then when you're leaving a food source you cycle through all your existing Vector memories and try and pick whichever one is closest and you can establish that very easily by just basically taking the difference between your your path integration and your vector memory which is what's being calculated anyway so you just say whichever one is this either is the smallest amplitude or Falls below some threshold um that's the that's the vector memory I'm going to use and then this is our our simulation of this gradual discovery of each of their locations and then a gradually more efficient route and then eventually we find the most efficient route around and this is just sort of showing you the the kind of statistics of that that the the length of the root taken drops as as we acquire these different things so we can actually explain this this kind of efficient foraging Behavior as well using the same circuit and then what's very interesting nice recent thing so this was a completely hypothetical idea that this is a you know the vector memory could be stored in this way um more recently Stanley heinzer has suggested that there's actually a specific neuron that is found in the anatomy of bees and not in other insects not in flies for example um that seems to have exactly the the requirements to fit this that it basically it's a it's a neuron that has extends and has synapses across all the fan shaped body um making all the connections that we would want and then there's multiple of these neurons which could correspond to these multiple Vector memories that we've suggested okay I have a few minutes left so let me try and talk a little bit about how the vectors and the views interact um so so far nothing I've said suggested the animal needs to explicitly associate vectors with views so you could imagine that it could you know if it knows what the view looks like from a place and it has a path integration Vector to that place in a vector memory maybe it could associate those two things together but the evidence for that is actually not not so clear so far however they're definitely the vectors and Views definitely have to interact so for a start um you need to use the vector information to learn the views in the first place so this is now actually um B learning flights and the idea is you know if you're going to use this method of having views looking towards the nest as a as a kind of way to get home when your path integration fails then you need to know where the nest is when you're first learning them so you need to know where to look back towards the nest to be able to learn the views in the first place or similarly to do the root following the very first time you follow the route the only way you can do it is by path integration so you need the path integration to to learn the views to learn which is a good view or or not um a second idea is that the two things can maybe be combined at output stage so um this is basically the idea that if you have both path integration and a view how do you combine them okay and normally you know they should both point the same way and so that's fine but there are several interesting studies that have tried to put these two things in Conflict so this is an example where the ants had learned this this was the usual food position and that the home position so they'd learned to go this way and now they're being released from different places where they can kind of see the views so that would Orient them towards the necessary but they also have their path integration which should tell them to go that way and so what you can see is that they seem to take this kind of compromise position between the two um and we were interested just to actually see could they actually be doing this compromise between the two um in an optimal ways could they be doing optimal cue combination uh I.E waiting each of the queues relative to the uncertainty of the queue so that's kind of what's expressed here is you have the uncertainty of the two queues and you can have a relative waiting that means the more certain queue gets a higher rating so we actually did a study to try and um look at this and we could exploit here the fact that we actually can calculate how the uncertainty should increase with distance in path integration and again I'm not going to go through this in in detail but the basic idea is that the variance of your directional information from path integration actually decreases the further you go and that's basically you can there's you can work it out from the geometry that it's basically proportional to 1 over d so even though your positional uncertainty as you can see you know you accumulate uncertainty in your path integration but your directional uncertainty gets less the further you go um by by this proportional amount so that tells us how much ants should weight the path integration versus their view depending on how far they've gone and in this experiment we pick them up at different distances from the nests so their path integration was zero or one or three or seven meters and we put them down somewhere where they had a conflict between what their path Integrations would tell them to do and sorry put them down here sorry where the the path integration should tell them to run downwards and their familiar view should tell them to run this way towards the nest so there's about 120 degrees conflict between them and what we found uh or we or we put them somewhere unfamiliar where they they can only rely on their path integration and what we found indeed was um you know if if the view is unfamiliar if they have zero path integration they have no idea where to go but otherwise they follow their path integration but if they can see The View if they have no participation information they go to the view but as the path integration length increases they're waiting they go somewhere in between and the weighting of the path integration increases and indeed it turns out it increases very nicely with the prediction of of an optimal of optimality so this is kind of showing all the data and the red line shows what we would predict from optimal integration based on this estimate of how the uncertainty of path integration decreases with with distance okay uh so now I think this is my final little section that I'm going to talk about um there also appears to be directional information transfer between views and vectors and this was came from an observation that um ants I just told you that ants you know Follow The View by turning so that they they're matching they get a match in the image between what they see and what they do but ants often go backwards so this is an ant dragging a dead Cricket backwards and it's going straight towards its nest even though it's going backwards yeah um so what we wondered was okay how is it doing this is it actually um able to to match the view from any direction or is it somehow getting some Vector information on for example as you can see the these occasional times it seems to look around so this is a more controlled experiment that that we did to try and understand this um so essentially we trained ants to go around this little slalom um and then we basically wanted to see which way they went so so we gave them a big piece of cookie so they had to go backwards um and we would pick them up from let me see if I can try and get this clue right so we'd pick them up from here so their their home Vector would point in this direction and they would place them over here so their view familiarity would point in this direction this is the the route that they should follow and we found indeed if they if they're going forward and can look they'd follow that and if they're going backwards they would follow their home Vector but what some of the ants would do is drop their food and have a peak of which direction to go so this is an example so this is an app that's going following its Vector backwards and then it drops the food looks around and then it when it comes back to the food it takes it in the direction that the view would tell it it should take this is another example where they just need one quick look at the view and then they know which direction to go um I think I'll leave someone okay so this is the kind of summary that that basically the direction they were going before they took a peek in the direction they're going after they took a peek it looks like they can actually transfer the information so they can look lying align themselves with the The View and then somehow transform that into a vector that they can then a vector information that they can then follow relative to the sky and um yeah okay so in the interest of time I'll go to my conclusions um so what I've tried to show you here is that we're trying to connect these kind of insect navigation field studies to to neural mechanisms and obviously it's quite hard to do this by direct brain recording or manipulation it's hard to do that in the field and it's also hard to get animals to do real foraging tasks in the lab um so modeling here provides a link between what we know about the neural mechanisms and what we can see in the field studies and at the very least can demonstrate the plausibility of some of these algorithms to be implemented in the insect brain um and we also gain complementary insight into the into the circuit function um so we've kind of suggested that the mushroom body architecture is a kind of general purpose multimodal associative net and that the central complex architecture is actually a general circuit for steering by comparing a remembered goal to a current heading and I haven't really shown you this in in this talk um but we also I think robotics kind of provides the framework here both for for evaluating our work but also just for thinking about the problem so we always start when we look at these problems we always start by saying well how would I do that on a robot how could a robot solve this problem um and then we think about what in the insect brain set constraints on those Solutions and that's that's very much very important in how we kind of come up with some of these uh ideas and then I'll just think my group members um the dark ones are previous group members and the current group members and my collaborators and thank you for listening and I'm very happy to have further discussion thank you very much this is uh really wonderful talk um I'm going to invite the other [Music] um panel remember sorry I have to remember where to do this ah yes all right just one second and everybody else please you know ask questions um if you have something you'd like us to address in the next few minutes all right okay I can see there's questions there but I'll let you all right yeah we can probably um no problem sorry I just have to find in this list um [Music] all right okay let's see I've invited uh Rory as well I don't and trip up so anyway I hope the rest is going to join us in a second what can we just start asking some questions that's been posted already yeah exactly so maybe I think the um pheromone question is a good one to start so yeah okay um so I think I think for the ants that we're studying the desert ants I think it is safe to say that they don't use pheromones to Mark Trails so they do they do still use smell and people have shown that they will use if there's odors like if you put odors along the route that that becomes part of their memory so if you then change how the odors are distributed along the Route they they notice that something has changed but that's not a pheromone Trail and it's basically because they have to move really fast um in the desert and because the ground is really hot so pheromones burn off and also food is very sparse so in general they're not all going back to the same place although if there is one food source they will all start to go to it so they do have some mechanisms for for you know other maybe communicating where the food is although we're not sure what that is for the ants at least but I think it's safe to say that they're not using for our own markings for the behaviors that I showed and in particular we were showing behaviors where it's you know a single act the other thing to mention there is even ants that do use pheromone trails and people have done tests if they if they use the ephraimone trail to go to the same place multiple times at least some of those species will actually learn it visually and then if you put it in competition if you give the fermentary in competition with the visual surroundings they will use the visual surroundings because it's actually faster if you've remembered the visual surroundings so even ants do use pheromone Trails will will also use this visual memory mechanism thank you um I think we can sort of group a couple of questions here um maybe first another one that is basic and sort of on its own is from Rory I'll just ask it so it's the specific Central complex circuit architecture that you mentioned conserved across insects or do you think networks like these are likely to be more sophisticated in species within ecological home on which to base path integration computations yeah that's a a good question and and Stanley Hines is really the person who should answer that question um so I'm really just kind of paraphrasing what he might say so I think the short answer is that they are very it seems to be very conserved um really surprisingly conserved across all invertebrates even not just insects um so that suggests it's not just for for path integration and we think it's involved in in steering any kind of goal directed steering on the other hand there do seem to be some special adaptations and some differences um that we observe in insects that do do central place foraging and that require path integration having said that there's some evidence now that at least uh that drosophila will do path integration over shorter um smaller scales so they'll they'll path integrate back to the location of somewhere where they found food recently and I think we're going to find path integration actually exists in a lot more insects than we we appreciate there's some info there's one paper I think showing it in crickets there's papers now showing it in um some uh other arthropods and so yeah it's so I think it may be more widespread than we realize as well so can I ask a quick sort of follow-up um so I guess there is maybe some some requirements on the central architecture side but like the I mean even for the sort of more fancy um view based learning the the insects don't require actually a lot of specialization on the sensor side right like they all all of this can be done with relatively course vision and I mean I don't know if there's any evidence for using developing any specialization basically in the earlier visual system yeah so I don't I mean I think it we Yeah in our models we can do it with quite coarse Vision um the only thing I'd mention there that we've found is that I'm having UV Vision might help because it helps you separate the sky from the ground and to get a very distinct Skyline and we've shown that can actually be a very useful way to to learn these this kind of global um view um so that that's maybe one specialization but otherwise yeah I don't think and even the mushroom body I mean so what what is again different between insects there is the number of neurons in the mushroom body it goes from like around 2000 in flies to 200 000 in in bees and ants so again that probably implies that you know but for the actual structure structurally there's a lot of there's a lot more similar than there is difference I mean it depends who you ask some people some people emphasize the differences I like to emphasize the similarities I guess yeah it makes sense um do you want to I think Laura you had a sort of question that is sort of related do you want to ask sure but now I have other questions yeah yeah you can uh I think which questions right so um I mean I guess I was I was also I I mean okay I was I'm kind of interested in the idea of like how flexible the memory encoding is um and I'm wondering about uh sort of if the environment changes for example um but also actually now thinking about different temporal scales as well like I mean what kind of evidence is there for ants for example using the same Vector memory like across days um sorry I'm a little I'm not an insect person so I know a little less about these things uh but but yeah I'm just kind of because you mentioned different Vector memories for example um sort of the like uh having different influences or you know and I'm just kind of yeah how those things interact and how they're encoded over to different time scales uh that's great yeah so so improve I mean they seem to the memory seems to be quite long-lasting relative to the lifespan of these animals which is short okay for ants they usually you know they forage maybe for a week um so um but they seem to be able to that memory seems to last for that it's only lasts over days you know they'll come out the next day and they remember where the food is yeah um for if you for the question sort of about changing the the scenery and so forth so what people have done is is shown that yeah they do it needs to be a fairly major change so sometimes things we think oh we'll move this big landmark and and they don't seem to pay much attention to that but it turns out if you look from the insect's point of view um often those things are not as obvious as you think so if you're if you're using the whole scene then those things don't matter but if you do something like chop down all the vegetation around their nest then they come out and immediately they start to do these learning walks again so so it's as though they realize okay nothing looks familiar I need to I need to relearn yeah I'm not sure if that does that answer I mean there's probably lots of questions and then if we get into bees it all gets more complicated as well what they can do but I mean I guess also in in the encoding side and again as somebody who knows like of the mushroom body I know all these things but I'm not not in great detail um like exactly how you could even encode so many different potential like vectors for example if there are multiple like when you're saying for example in the in the sort of uh multiple Place uh like location foraging um and and how those things interfere if if one is sort of if they're a little noisy for example I like like to me I felt like the way it sort of seems is that it's a very sort of like the Fidelity of the representation like seems really high um and able to use this kind of very non-noisy information to be able to direct uh the the paths whereas I guess there must be I'm thinking noisiness internally and externally um painting environment but also in terms of exactly how how it's encoded as well right yeah um okay I'm trying to think how to answer that I mean you know we haven't we haven't looked systematically at noise I mean we do include noise in our simulations you know in some form just to to make sure it's not you know all perfectly Clean and Clear um I think so and I think in terms of actually like pinpointing locations I think you know the insects have multiple strategies there as well so I think you know it for getting to the nest you won't you you just need to get near enough to the nest you don't necessarily have to get right through the nest because then there are other cues like maybe chemical cues or or again you know other visual cues that you can use to to pinpoint it um so I don't think they're doing pinpoint accuracy with these with either of these these things yeah if that if that helps but it's it's good enough to to get them back in the right vicinity yeah I think um we have like I guess two questions that are sort of summed by um maybe dealing with uncertainty and also maybe like using past experience so um do you want to ask do you want to ask you a question sorry everyone is muted right now just to make sure everyone's aware yeah yeah I can ask you a question otherwise let me know I think it doesn't find the microphone because I can't unmute him either so there's some um okay well I will um I will be shave up for a second so he was asking um does that and I think that was in relation to the experiment where um you showed like views and homing Vector conflict meaning that insects would know that uncertainty of each estimate to combine them is there a plausible circuit mechanism to do this at least that was my guess of what it was about and is I think related to a question I had which was whether if you know something about uncertainty maybe also from experience like controls instrumentally could they um weigh these cues differently and essentially switch between strategies yeah so so what what has been observed is that they um the more they have do the route for example the more they start to rely on the root memories so the visual memory certainty seems to increase over time and they'll start to weigh it more over time um whether the you know how how the uncertainty is is kind of encoded as it were that's that's quite an interesting question and it's it's quite nice how um in a way in the in this Central complex circuit if you imagine you know this population code that I showed you you can to some extent you can kind of encode uncertainty in that in a very natural ways so basically you know it's the equivalent to the length of the vector it um gives you the uncertainty estimate so so you can actually have you know if you have two different sources of information about which direction you should go um you and but one of them has a much bigger difference between the the activities than the other one then that would just naturally be a stronger effect so effectively it's like adding a longer Vector to a shorter vector and it will have a stronger effect and you can kind of show that that's um you know for a circular or a circular problem that's actually the optimal way to to do the combination so it kind of all falls out quite nicely in that circuit I'm not sure if that answers the question okay I think now you can hear me right yeah yes so just a quick follow-up so in that case then the uncertainty of the visual memory would be equivalent to you know what you get from the mushroom body in that sense like how much the mushroom body modulates that Vector would just be the uncertainty of of the visual memory in some sense yeah that's I guess that's what we would is yeah exactly that if you have a if you're if your mushroom body is is more strongly signaling that this thing is familiar then how that gets integrated into the central complex for steering would would be a stronger effect and um I'm kind of waving my hands a bit at the moment but we are actually doing some models now on how these things you know what the neural system could be that does that combination uh that's kind of ongoing work in my group at the moment is to put these two things together you know in a in a more systematic way and following what we know about the the neural connections now between the mushroom body and the central complex my yeah yeah so I was going to follow up on that and ask about those peaking ants in the end because that was very uh sort of an adorable behavior um say what so yeah what do you think is going on to trigger that speaking uh Behavior so do you think that the uncertainty has reached a certain level and that yeah I mean I think that's I think that's that's really a good guess I mean at the moment we don't we don't really know but I think it's it's a very interesting question exactly what what drives them to decide you know that they're uncertain so one thing I think we found in that experiment and I I must admit I don't remember the details I was going to reread the paper before I talked and I didn't um but I think what we found is yeah the this again the the shorter their Vector the the more likely they are to to Peak basically so they would certainly if they had a very you know if there was Zero Vector ants then they would definitely Peak more quickly and whereas if they had a good long Vector then they wouldn't Peak as often but I don't think we did that systematically I think that's more just a kind of you know an observation from from the from the studies that we did um but yeah I I think it's a good a good question about yeah exactly could we actually find that more systematically that there's some something that triggers the uncertain you know the level of uncertainty gets to something some point I wish they decide they're going to pick I have a question going in this lady like more foraging I guess related thing um so um but kind of on the on the um point of like different memories Etc is there any work on associating a memory of like a food source with also the type of food source like the value and is there anything done on sort of selectively right selecting a memory based on your current needs I think there is some some specialization for sugar versus protein foraging and Alliance as well yes yeah so there definitely seems to be something like that going going on um more probably people have done more on that in bees than than ants um but you know so bees for example seem to remember for flowers that open at a different time of day they'll know you know this is when I should go in the forage on I should activate this Vector memory to forage in that direction rather than another one um there's some interesting stuff with ants where people have done like food of different value and then will they go further to get the food of a better value or not um so so I think that probably there probably is something you know it's not just the memory that there's food there but it's a memory of something to do with the value of that food and interestingly might be also that they might learn some negative things as well so they might have memories of somewhere that was bad and the evidence for that in bees comes from when so bees will dance on the honeycomb to tell another bee uh you know he is a vector to food and if a bee is dancing towards a food source where another bee has actually been punished has experienced like something tried to capture it or whatever the second bee will actually basically bump into the first bee that's dancing as though to say well stop dancing that's a bad place don't go there um so that's really interesting that they can both interpret you know where the other beers referring to and have some kind of negative value in this case associated with that place instead of her positive value yeah that's crazy and I think the same might happen for you if you learn views as well as positive views I guess yeah because there is I mean sort of trap lining is nice because it's in some ways very efficient but it could be that you were like that you can take a slightly longer route to avoid something bad right and oh oh we lost Barbara I think am I uh so you're okay just to let you know yeah yeah yeah yeah yeah I'm not sure what happened oh we'll see it says waiting to reconnect for me so we're just I'll see if I can um invite her back in ah accept it and connect it um we got too far through this episode without any real Tech problems I know this was weird what uh and then maybe we can have a like okay sorry about that it's just crowd costs seem to just yeah randomly did something funny there I'm sorry about that no no it is not your fault we just said it was very smooth so far so um talk to you soon uh sorry so was I in the middle of saying something about no I think no no I was just I was just rambling and thinking like how basically if you factor in other stuff that could happen during a foraging I can see how having these different strategies could also help you basically achieve multiple goals at once right um and the the view matching base like root following has some advantages basically um you could probably integrate aversive cues in there um there was a another question so I'll um from Ellison comrie um and you ask is there evidence in your models or neural studies for sequential neural activity iterating ah no it moved sorry iterating um through potential next or prior foraging sites in ants Orbeez curious to hear about how this might be instantiated as they take and refine shortcuts right okay so again for the long question yeah um and it feels like there's kind of two things there so so one is you know do that is sequence matter at all for example in the in the roots I showed you know the the model that I showed didn't really have any sequence to it it was just at any point in time you know you're just saying does does this look familiar or not without sequence we have actually done some some more recent models where we included some sequential information there so that we basically had a motion inputs and you're actually learning how the world is changing which kind of is like a short sequence um so in that sense and and that does work better so so having a bit of sequential information is better for for recognizing the route but I think where's if I understood the question anyway that it was going towards was a little bit more you know does the does the foraging bee that's doing a trap lining does it does it kind of you know think about where it's going to go next um and you almost and I guess in in our model effectively that's sort of what it's doing it's like running through its memories in its head and then saying okay this one you know which how far is that one how far is that one how far is that one that one's close enough I'm gonna go there um and and that's that's just how we implemented it and whether there's any evidence that the insects can do that I don't know um but they certainly seem to find the more efficient route and I I think it's hard to imagine how they could find that more efficient route unless they do some kind of internal comparison of you know here are the possible places I could go next and which one am I going to end up at next but there might be there might be some other mechanistic way to to solve that that doesn't require them to actually you know imagine first where they're going and then and then go there I hope that's getting answering the question and feel free to clarify if you if you doesn't quite get it I'm keeping it and maybe um we sort of end on something that you ended your talk on as well which is a question about the robotics this is from Rory um please could you expand a little on your experience of using robots to reproduce and test these algorithms and what you see is the benefits for using them over computational models okay um so we always learn something from when we when we built it onto the robot and test it in the real world and there's always something we realize we've kind of skimmed over in the simulation and and just made some assumption about you know what what is possible to sense or what might be the the type of noise so for example in the path integration um we've actually put that now on a on a flying robot and and we're now getting the noise from wind information um you know wind noise rather than some other kind of noise and it turned out to be quite robust to that which was nice but the other problem was was height and we hadn't really thought about height at all but height changes the optic flow quite substantially and it turns out it's it's robust to a little bit of height variation but not to a lot of height variation and so that means we have to kind of now try and solve the problem well how might height be estimated or how can it be compensated or is is it the case that bees are trying to fly at a constant height that's another another option but it's just every time we do it we put something on a robot we we find something that we've missed and so and after many years of doing it it's also kind of as I say it works a little bit the other way around as well that just whenever I start to simulate something I I'm still thinking about it in robotic terms so I'm trying to think well you know it's very tempting to for example to simulate a neural circuit way what you simulate is the phenomenon in the neural circuit so for example this bump in the central complex I mean it's amazing and it's really interesting there's lots of interesting models of it but in the end the bump doesn't matter what matters is the behavior of the animal you know it has to be good for something and you can't really evaluate whether you've reproduced the right properties of the bump unless you're using it to control the behavior you know because the properties you're reproducing might be completely pointless ones that the animal doesn't care about or they might be completely crucial ones that make the whole system work or not and I think unless you think about it through from you know what's the sensory input to what's the motor output unless you think about the problem in that way you can often get misled into what's important to to reproduce or to understand and so that's you know that's what I kind of think of as a robotic frame of mind or point of view is that if I'm thinking how could I put this on a robot then I try and think about all these things from the beginning rather than kind of plug them on later to to the to the model so I hope that answers the question yeah it can make it made a lot of sense to me but Rory can't show you uh very it here's a follow-up question I think very interesting so what kind of sensor information can these robots detect and how does the Acuity of these sensors compare to the ants okay so um so we've done we've used optic flow on the robots and that that works fine and we can use low resolution and we've done um the visual recognition process we've done that with low resolution vision and that works quite well we're just currently working on uh um polarized light sensor to get the sky Compass information um and try and estimate you know can we get good enough Compass information to to really do this navigation so so we are trying to reproduce the sensory systems yeah well thank you um we've had like a we've been going for roughly like 30 minutes uh the discussion I think I I really enjoyed the whole session so uh thank you and um unless like uh we have anyone else with a burning question I would say oh okay um well thanks again Barbara for for um joining us and for this presentation I think it was really uh really good very different view from what we've had um so far okay good yeah thank you thank you um I will an episode actually the next one is coming up in just a week yeah yeah so just a yeah just a heads up um it's going to be at the same wait it's gonna be an hour later the time change but it's hard for us it's an hour it depends where you are okay check the website yeah yes and it's going to be on the topic of um sort of a group foraging or Collective foraging yeah all right bye bye bye hmm