How Fly Brains Compute Allocentric Direction via Vector Math

[00:00:02]so hi everyone welcome to the uh this uh new session so today it's our great pleasure to have uh two speakers so the first speaker is Chang Liu um and so he he worked previously at the Rockefeller with uh uh maimon and Gabby mayman and Larry Abbott and you will present that work is now what uh at Stanford University and so uh you will uh he will talk about how the the the the the the fly's brain do math to compute the flight uh Direction but it will give of course much more detail about it and the second talk is from Logan lugan shariker who was previously who did this PhD and postdoc at current with a and you work with Bob shapley too and it's about network of eni neurons and to model uh visual cortex so today these are really uh very applied uh math because they very deal with real real applications and so it's my pleasure to to have them today for a bdd all right [Music] all right thank you Ben for the invitation uh my name is I'll talk about uh how Prince add factors today uh and more explicitly I'll try to talk about how our brains perform this uh three basic operations of vector addition uh like vector scaling rotation and addition uh although I study fly I always keep thinking about how the animal prints or especially harm or medium bricks our brains could do math so let's start with uh listening to the spike train of heart cell hyderation cell in a rat do you hear hear the sound the audio okay no we didn't hear the sound uh okay uh I think given a limited of times the you hear a lot of spectrums when the rest had it pointed to this upper left Direction uh I think trying to fix this uh may take a few minutes uh but uh I I don't have much uh videos in my in my talk um so here is the tuning curve of the cells uh we just recorded uh just shown and you can see the cell has a sharp tuning uh curve at this orientation which it corresponds to the upper left of the of the animals environment and so this High Direction style since it's discovered as many actually many additional features that kind of remain unclear or only have models for and this is just to show that they're supposed to put in many heter reconciles that could pile the entire angular space but occasionally or sometimes you can find cells from muscular raspberry that has a tuning curve to a to a Direction but with a much broader tuning curve and sometimes this tuning curve fit well with the sinusoid which is shown in this dashland here and so it's been curious what are the functions of the the shape of this hetera Consulting curve I created early this year has been published the paper uh by Ruben project this lab that where they record in the hand burning of librafish and uh it's also High Direction so like activity but you can see the stomas are specialized in our localized in a very nice way that you can actually see activity about of the of English cells this is uh when the and um when the animal is moving and you can see that the activity bump changes indicating the animal sinks uh who is changing directions and if you look at the individual cell from this population this is the Soma IT projects to another green structure called the interpreticulous nucleus but interestingly the dendrites and the axons are separated and quite large largely separated almost to the half the extent of the structure so what are the functional of this large separation between Tinder and axons of heterection cells and most lastly but not least it's been known that Heather can sell if you record it more often than not you will see a conjunctive tuning to other spatial information such as animals location and animal speed Etc so what are the functions of this pervasive conjunctive tuning in high Direction so so that's uh from The Mouse and the rats and uh so the purpose of studying flies also to add Insight onto this question marks although there are many models uh already trying to characterize the function of this but hopefully after today I can provide some experimental data from the Flies similar to this observations okay so Flats also have high Direction cells and it's first being shown by Johannes silica and varic German in 2015 almost eight years from now since the audio is now working and here is a virtual trajectory of the fly and now the it's also uh respond crazily when the flights highly towards the upper left Direction and The Tuning style is very similar from the one you see from the rat uh so this is one cell it's actually called uh and uh before moving on to the anatomy of this neurons uh the the realness is uh virtual like a cartoon like video is because we can't put so far we can't put uh electrode on to a freely moving flight so the way we do it I mentioned earlier is that we tethered the flight to a plate and put the fly either our cushion ball like here and he can she can walk around and we hooked up this bright bar as a in a close loop with the rotation of the bar so it's mimic to mimic a landmark at Infinity to the flag kind of to simulate the High direction of the fly in in reality so we can record neural activities new flies in Walking flies and also do similar recording in flying flight also tether and this is one of the video that we collected when we record from flying flies also we can add this video information or other stimulus to the flag so this is the stimulus I'll use quite a lot into this talk uh so back to the neurons uh the headerical cells are shown on the left are actually recording from one of these blue neurons called EPG so they tell the almost all the ellipse body this donut like structure and most of the personal Bridge is a bicycle Handler like a super instructure they are located in the center of the fly bridge so these are the population of this neurons but if you look at it once they only inherit one wedge of the ellipso body and goes up output to Vancouver Maryland of the Apostle Bridge so this neuron this goes to the left and if you record from this neural you will have a tunic curve shown previously but the neighboring Europe have unique anatomical pattern that will go to a Red Bridge and left right left right so through this alternating pattern if these neurons are active uh what you will see if your image all the neural activity of this EPT cells you'll see this activity bump similar to what was shown before in the zebrafish brain that you will see three bumps here one from the ellipso body and the two others from the same your neuronal populations but in a bridge and uh of course you if all the neurons take turns of being active like when the fly is turning around in space uh what you will see another microscope with this bump and rotate in a synchronous way all indicating the hiding Direction High direction of itself of the flying head or heading um so back to the uh to the title of this talk how do Brands add vectors this is one of the uh very the key components of the vector computation another key component is the traveling Direction uh this it's it's not trivial to distinguish hiding and traveling direction for example in humans when we walk we can walk in and look at a different direction when we walk and this is the and going home backwards uh we're shown by the study of Matt waiting girl that they sometimes when the food is super heavy they walk home backwards and there are many behavioral evidence indicating that throughout this process this and do you have a sense of their traveling Direction which is different that they're hiding which are is like a 180 degree apart in this case and in flying animals here is some blue mannequins the males are core shipping the females and you can see the case that the female during each dance and traveling to the side so being able to distinguish hiding and traveling directions are very important in spatial navigation but so far maybe not so far maybe up to two years ago or one year ago it only had directional cells has been shown and it has been shown in almost all the animals as you can see there are many different animal species have this head Direction cells but the traveling Direction so hasn't been sure until like a year ago and since the focus of this talk is how do bridge as vectors I'll quickly show you the evidence we think for this traffic direction and focus on how that brings out the factors uh in short this uh traveling Direction signals uh in the full flight they they locate in this friendship about this middle layer of the fatigue body in two groups of neurons at least Weiss College dot b one it's called pfrs so only talk about H.B is in this talk they also have a bump of lactic activity moving left and right along the financial body so this three structure are closely located and they all belong to the same uh structure in the flyer brain called Central complex um so uh intuitively should be very straightforward if we record this neural activity and also simultaneously record The Traveling Direction in for example a walking fly we should be able to correlate these two signals and show if we had a traveling Direction a signal and but unfortunately the trial induction of a fly when it's working it's changing so fast like when it's Jitters a little bit to the left in maybe a sub 100 milliseconds the traveling direction will change dramatically and gcamp indicator from that we use or exist to not respond in such a fast way so to get things working we switched to recording from flying flash but the problem in flying flights is that it's tethered they are now actually traveling to any places to any direction so the trick we use is to not record the actual movement of the fly but to to use VR to uh to show the fly which direction you are traveling kind of like the virtual reality where using now in humans so the way we do it is to give the fly this optic flow thing that's the steam does on the on the lower part of the Arena that's that's surrounding a fly so when we move the steam dots collectively in this manner it's simulating what you feel like when you are moving forward and when you're moving forward the height your high Direction and traveling direction are are land and this is what we observed in these two groups of neurons we simultaneously record activity from these two groups of neurons now the x-axis is time and between the two dash lines are the moments where uh this optic flow is going and the y-axis is the the unwrapped linearized uh this brain struct structure so the top row is EPG that simulates indicates hiding Direction so that now the flashing and hiding direction is to the to the edge of the y-axis and the traveling direction from H Delta B this bomb also goes to the edge and we can estimate the phase from these two patterns and they they show that when a stimulus is on both phase is aligned uh indicating the hiding electron interaction aligned and across a population of 13 flights this is uh also true so this is just One Direction Let's test more other directions uh another direction is to simulate the fly going backward and when we display this movie uh what we saw is that now the high Direction and it keeps changing but that's why we record this High Direction simultaneously so the high directory goes to the edge and the traveling direction locates to the center and they are on average 180 degree apart from each other and we also do this recording in other four directions so in total six directions and you can see the separation of these two bumps uh actually changes in a gradient Manner and on average they change it they from a very needed linear correlation with the singularity ecocentric curve interaction so egocentric here is a word means this travel interaction is with reference to the flat body like going front back and there's another word called uh adults interpreter Direction traveling Direction with reference to the to the words like going south north west to the east will need to distinguish these two in the rest of the time so to summarize this first part now there are two are these two bulbs in the fly brain one the blue bump track tracks are flats highly Direction the pink or red bump characters plus traveling Direction so when the Flies flying around if they move forward both stops a lot and when the fly is blown backwards the traveling pump switch 90 degree whereas the Blue Pump remains steel here's another example so I think this is true so we show that if the case in flying flies and we also have a lot of data showing let's see uh qualitatively or semi quantitatively true in Walking flies and for more if you want yeah and uh we hope we think so now let's also I think this this pink bump is to is to represent hopefully the the traveling the abstract traveling direction of the animal no matter it's flying or working so how do uh the fly brain build this allocentric traveling Direction signal because the exact is now accorded so it's the the southeast northwest as how how do the flowering build this house in your current Direction so I think this process is done can be summarized as a vector computation process the first let me go through the algorithm we think that's going on uh why let's say that uh a vector computation process so here are the two angles the same abstract the same angle but uh reference to different uh the one on the left is egocentric and the one on the right Centric so the difference between these two angles are the slide with the flat High interaction and so you add equal traveling with hiding you've got atom crafting okay so the what we want is aerocentric trial interaction but the egocentric traveling directly is seemingly more is easier to compute so there are actually many other recordings from your activities in insects uh previously from bumblebees also shown in this work that we have like neural activity whose activity correlates nicely with the lens of this egocentric traveling Vector projected onto around this 40 party 45 degree axis I know it's very arbitrary but in short is we have neural activity who's that that accorded with this list out which is a production of travel interaction and we have different neural activities from different populations that correlates with different lens out which is the projection of this egocentric travel interaction onto these four different axis so one way the flag can reconstruct this green arrow is to use those newer activities those great arrows and some stuff and then you can get the Green Arrow and this is establishing a vector addition and once you have this green arrow you just rotate it by the amount of H and then you got allocentric routing Direction and this is what we call I think it's a coordinate transformation now you can switch from egocentric to Velocity so this is done in two steps and what's actually done in the fly brain that they combined these two steps into one step so the reason this is a green arrow is the egocentric reference frame it's because it's four axis they used to compute vectors are referenced to their body so if this axis are directly referenced to the other Centric War then the vectors they got the sound Vector automatically becomes reference to the allocentric world so this is what we think is going on in the library and throughout the rest of the talk aisle only talk about data that we think is reflecting this process so first of all how do in your neurons like represent vectors there's actually I think a trick that's been used is called a feeder representation of this Vector so every two dimensional vectors have you can have this length and um angle and but and you can also represent it in a sinusoidal way where the lens the angle represents that equals the phase of the sinusoids yeah the length of the vector equals the amplitude the amplitude is very important the amplitude of the sine of fluid and if you have another vector if you want to add these two vectors together you just add this to send your soils together so since these two sinusoids are of the same frequency the added um the sum is also uh sinusoid of the same frequency so this uh is a mathematical uh mathematically true and this is actually a idea that actually have been put forward in many models theoretical models that you use to calculate to understand or to explain many spatial navigation tasks and the the one thing I forgot to mention is that in Europe it's actually much easier to represent the things on the right than the things on the left because to to just to represent one sinus with you can use one group of neurons say this one drawn on the bottom and two so each each subgroup of this neuron can represent each being of the sinusoids and the height of each being in this sinusoids can be represented by the activity of a subgroup of neurons so all together this group of neurons with their activity they can use to have a one-to-one mapping between this purple sinusoid and you can have different groups of neurons to represent different and add them together you can achieve bacterial computation in the fly brain so that's I think the take-home message if you don't understand anything I say from now on uh okay now the biological evidence for it so first we need more copies of hiding direction or more copies of the sinusoids so that's the theoretical and computational paper including one also have from proper weapons so and that hands that I also have experimental data in in peace okay so the more copies of the hiding of this box so one way to look at them is in the in this ep the output region of this epd cells which is the hiding Direction cell in a flat rate that we talked about earlier so in ellipso body is dangerous and in the uh Bridge it's output so they have two outputs one on the left one right there are all the same uh first order so if we look at the output of this EPG cells we found these two other groups of neurons that both inner with the left and right bridge and send their output to the substantially body and these two are called PF and dmpfmb cells and it's a lot of names but I always use it I always color code them to make things easier the one we simultaneously record newer activities from EPG cells and pfmb cells for example in a bridge we saw these two bumps of activity in the EPG cells and also two bonds of activity in the pfmb cells that are moving along the bridge synchronously and they also have overlapping bump of activity adjusting this different cells more or less it's receiving the EPT cells input okay so that's now we have four additional bumps that all track The Heading direction of the fly I'll skip the data from Kevin D cells but to compare if we look at the shape of this hiding Direction cell of the newer activities for example if we look at EPG cell they have a very narrow tuning curve this is consistent with the data we got from single EPG cells from electrophysiology recordings where when the fighting direction of the flight changed a little bit the nearer the height the APD cell will lose its activity very sharply tuned to their preferred Direction and here if we image them all together you can see that they are also the shape is very narrow and and but compared to this scenarily distributed shape here are the pfmb cells they have a much broader wider shape and if we try to feed this shape to a dinosaur they actually fit very quite nicely so this is at least one piece of evidence that this orange and it's the same it's true for this brown bumps as well so it's now we have this for wider bump that's a Chinese so I don't like shape and to narrow Bob in EPG cells they all all these six pumps as they track the flat any interaction when the flag moves around it'll work okay um now we actually met the two criteria for for Vector computation one is we have more components more hiding bumps the second is that these bumps are sinusoid of shape okay a third requirement for Vector computation uh is the following so there's four bumps because they all their faith all pointing to the hiding direction of the flag if you put them together the accurate point they have the kick point to the same direction but what we need is to uh this is what they look like in a using a failure diagram into this space they're all pointing to the same direction the vectors represent what we need are these vectors to point to different directions and this in return requires this box to have Peaks to have their face also separated and this is the third requirement that we need the phase fifth between these different potential bumps and this can be achieved when this through the anatomy of this neuron that they project from the from their dendritic area towards their axonal output area from the bridge to the financial body uh so let's focus on one of the cells uh there's two pairs there's two pfmb cells that they inhibit the center of each side of the bridge and but when they propagate to the functional body the the one the right Bridge uh shifted a little bit to the left and around the Left Spirit shifted a little bit to the right and uh if you say the whole layer is like 360 degree because the bulb sweeps through the whole layer when the fly change its traveling directions 360 degree then this script we can quantify the amount of this anatomical shift and it's roughly 1 8 the width of the whole layer and which corresponds to 45 degree after minus and I just want to give a highlight to this uh work from uh Tanya wolf and Jerry Rubin that in 2014 which is almost 13 years from now they use that microscopy to sparsely labeling these neurons and they actually have this quality they have this conclusion that the there's a PFA neurons when a project from bridge to the factory body you have some anatomical shift and more recently from this homie brain Electro across the data set and we can more accurately quantify this in fact I'll show the quantification results later but uh this is this is a Em picture a reconstructing a PF and B neurons you haven't been us yes from the bridge to the function but you can see the shift it a little bit to the right so let's say this is 45 degree plus or minus but that's now the end of the story so the this means the sinusoids The Hiding bump when they propagate from the break to the Friendship body through pfmb neuron they have this anatomical shift of 45 degree but they need to actually travel to the external output of the this extra business which is what where we see covering Direction in the axonal output of this is not opinions so to travel from the dendritic field to the axonal output of the granular there is another of layer switch which is a corresponds to 90 degree let's do it so this Daniel switch from the bridge to the tip of the H dot b neurons actually have a combined minus 135 degree shift so that from the Left Bridge of pfmv South and from the Red Bridge if we sell this shift is positive 175 degree and for PF and D cells things are very similar the product to the functioning body they have this 45 degree shift they connect to H Delta B neurons but they connect very weakly to the the rest of the SPF of this HD neuron of the red neurons and much more intensively that connects directly to the output of the estabiners which is a rarely observed in the orange PF and business because the if you connect to the output versus the dendras of the red neurons is like have the opposite the effect because they are 100 degree apart so it's kind of like Winner Takes all and we can for Simplicity we can just think about this axonal direction connection so in this case there's no additional 135 degree shift uh I thought the pfnds are from the Red Bridge have 45 degree shift in total and from the Left Bridge have minutes for the fabric degree shift in total yeah uh and the so this is a parallel work published with spiritual worlds and slab and where they also show similar uh ideas of how the anatomical shift plays a role in Vector competition I I refuse a genuine Loop okay so in some now this four sinuses that are first learned in the bridge all tracking the epd The Hiding direction of the fly as shown on the right when they propagate to the Friendship body to the tip of this H dot b neurons they each inherit a different uh shift the Left Bridge 11d cells have minus 45 degree shift I rotated the angle 45 degree uh the right Bridge party 45 degree and positive 135 and negative 135 so this is what we think now uh these neurons actually this phase shift and this is just to show the grid a year for people working in Flight where this em data is quite accurate that we just using the synapse synapses detected from this em data between different groups of neurons and we calculated the anatomical shift is turns out much very close to a perpendicular orthogonal axis so now we have four sinusoids that point into the right direction in the financial body and the last the remaining step the last step is to have this sinusoid have different to have the factors that they represented to have different lengths and not just changing lens but also changing lens in a very specific manner because when the flight changed her traveling Direction uh because it's a projection relationship and uh so this in turn requires the essential source to change their amplitude so that's the last thing we try to observe in experiments so that is that either is for sinusoid will change their amplitude in a sinusoidal way those are two sides words here uh so we image the activity of this neurons in a bridge that's where we can uh separate them so when we simulate the fly going backwards that Kevin D cells have very small amplitude of their bump whereas people resell have very large bump and the same groups of flies when we change the traveling Direction similarly to the traveling direction to the front you can see the PFD bump increase their sinusoid sorry increaser amplitude and FMV decrease their amplitudes and this is true for different traveling directions simulated and it's about to become asymmetric uh it's a magically um uh have a different attitudes so we can quantify the amplitude of each sinusoid in this picture and what we got is for example the left bridge cap and D cells what we got is this mean tuning the tuning curve that's close to us fitted sinusoid with the peak of this fitting of the threatens towards pointing to the upper left so this up left is actually their maximum responding Direction and there's another up left for this pfnd left bridge in this cells where she is there anatomical projection shift so the fact that these two angles they match each other means that the PF and D cell from the Left Bridge if they are advocating for this left up left Direction it will respond must run into this upper left Direction this is also a prediction problem and the same is true for the Red Bridge we haven't D cells and uh left and B cell where you can see their maximum response to the front to the backlabs and back right just either control this EPT cells that uh signals The Hiding direction of the fly to now change their amplitude only change the hiding direct traveling direction of the supply okay uh so uh here here we are we have all the components at the let me summarize this in a video with no sound the video has no sound so here's a fly covering in space so it for Simplicity we just in this cartoon we show The Flash had it's always pointing to the app so the hiding direction of the flat leverage hit but it constantly is constantly changing her traveling Direction represented by this red bump scraping through the functional body so how does this wrap up calculated uh first of all the same traveling Direction it's not projected onto this front back left right Axis which we human use to they actually projected onto this four other axis also as having access so the reason I say this is because some data I didn't show in this talk for the sake of time that we have this four different groups of neurons whose activity correlates with the length of projection onto each axis and actually they are anti-correlated so I think it may show some quite interesting feature why this regulation needs to be hit like to to be done in a disinhibitive inhibitory manner but nevertheless we uh this is the traveling direction of the fly projected onto these four different axes so now this act this neurons only signal uh not a vector but a one-dimensional variable so the the place the steps that they become Vector is where the combined with EPG is a hiding bump and combine it in with the help of other neurons which I didn't talk about that they become sinusoidal shaped bump in its pfm Piaf and styles and there are four bumps of this PF and V pyramid cells and the conjunctively uh they represent the signal the flat hiding Direction by the phase of the bump and sigma flash um speed onto one of these four axis by the amplitude of the sinusoid and this is the step where Vector scaling is happening is achieved in the fly brain and now this bump all pointing to the same direction the the step where they achieve different shifts is through this anatomical projection from the bridge to the Friendship body and this is the step where is the vector rotation is implemented so here in the financial body it is four senior soils they all stamped they are all summed by the same group of neurons which is describe neural college that would be neurons and the sum is also a sign of soils and with the peak corresponds to this red dot which is The Flash traveling Direction and this is the step where Vector summation is achieved and so at the last slide let me go back to this data from the vertebrates so now we have similar activity of neurons compared to this data from vertebrates so now what are their functions this is now saying that these are the function of this features but just to give some insight so this broad tuning curve of the hiding Direction itself could be used to explicitly represent 2D vectors and this anatomical shift between genres and axons thrown in the vertebrate 90 degree as an example and actually in slice there are many different levels of shift and we think the function of this could be to service to rotate the 2D vector and lastly this conjunctive tuning of honey in Direction with other thickness could be used to uh as the coordinate transformation to transform the other signal into the reference where the hiding Direction relaxed so with that I oh and my talk I select my PhD inventory Gallery commandment and my collaborator and also my commenter Larry Abbott from Colombia and uh I think I don't think everyone from minimum lab where we work together to track this problem and here are the slides we received uh uh for for this project and here are the phonics thank you that's it