Lightkurve and Rust for Astronomical Time-Series Analysis

[00:00:00]and I great and I think we can start to show them people will will join us as time goes on uh welcome everyone to our April last seminar we're delighted to have two speakers today who are going to talk to us about uh light curve packages as well as uh some of their experience developing them before we move to those just a few announcements next month we're having a joint session with the psychodes group and we're going to hear about uh the software development process things like citations uh software publishing and so forth Alice is going to be lining up a great lineup of speakers for us and so everyone please come back for that and hear more about uh software infrastructure in June our proposal is to have talks from graduate students who might have been doing projects writing code uh in this past semester or past Academic Year uh Catherine is going to be organizing that and we're going to do it in the format of lightning talks and so if you are a grad or if you know a grad who has some cool project that wants to spend five minutes talking about it have them propose a talk on our GitHub page we'll post a link in in a little bit uh and finally July we're going to focus on teaching computation to students again in the format of lightning talks and we're going to gather up a bunch of people who have experience doing this and talking about what works what doesn't work sharing resources and and so forth I'll be organizing that and likewise if you know someone or if you have some experience and want to share propose on our GitHub and we'll do some work in preparing for all these over the next month um with that we're happy to move on to our speakers today we're going to start with Rebecca Hansel Rebecca is at Nasa Goddard space flight center and University of Maryland uh Baltimore college or Baltimore County Baltimore County County okay um and she's going to tell us about her light curve package and its applications to astronomical observations so Rebecca you can go ahead and share looks great just need to unmute myself and then uh I'll go ahead and play and just to clarify it's not my light curve package um it is something that we over at the test and German investigator office shoes but I will explain more of that in my talk okay so um could everyone see my screen okay Yep looks great wonderful thank you again so thank you so much for having me here today my name is Rebecca Hansel I am a support scientist in the transiting exoplanet survey satellite General investigator office over at Nasa Goddard and I'm here today to talk to you about our main Workhorse it's um a python package that we use a lot to analyze our data which is called light Kev and here just to put down that is a friendly python package for making discoveries with Kepler and test data that's it what was the original purpose of light curve well this was like I was originally developed um to set like help with um exoplanet Discovery and finding certain signals within data from the Kepler Mission which ran up until 2018 and then with the test mission which found from 2018 until well it's still going now we're actually in have entered um the fifth year so it's actually it was actually Tess's fifth birthday um yesterday I believe so he gave the test um and so light cups we need to developed to help the user take the data from these facilities and analyze it to find exoplanets and make those discoveries um now just a quick uh demonstration here of the tests how we are able to detect exoplanets so we use something called the transit method so essentially as you see here when a planet passes in front of its host star we have this dip in the light output as you see in the plot down here and then this dip is actually um relative to the size of the hostar versus the planet so from from that dip we can kind of determine um some lots of important information so Tess looks for this signal as it observes almost the entire night sky and um that's the data then gets um collected and then the user Community looks at it and finds these Transit signals so just a brief discussion here about the code functionality light curve itself is completely open source and Community Driven there's lots of documentation and resources which I have put here so there is uh just wonder if I click on this whether it will show up or not oh it did not and it just undid my talk okay um so yes light curve we have an API and documentation as shown here um it is all publicly available it's all up on GitHub so you can go and investigate for yourselves um it is PIP installable so very easily easy to install if you don't want to install Leica so you don't want to have it or you want to have other packages on your computer you can actually access it easily via um Google collab or notebooks or via tags so type is their services up on that and you can actually just create an account there you don't need to install anything and just simply run your code and look at your data through those Services the requirements for this are very common python packages for numerical Computing plusing and astronomy in general and additionally one thing that I love about light curve is because it's on GitHub and the way that we work with the community is you can suggest updates and modifications you can tell us if there are issues and just showing briefly in the next slide so this is just a little video I took off the um the GitHub just showing you know discussion about the package the documentation how you can easily get started with it and how you can contribute there are many pull requests so people essentially post lots of issues and then we address each one and then different pull requests get made and then there's been a lot of activity even just over the past month or so of like you know different issues that the community has reported and then um the German investigator office which has now taken over the running of light curve trying to respond and working with the community to fix and adjust those issues so it really is a very much a community um package where you know if you have problems if you have concerns if you have ideas you can post about them and you know we can work with you on those issues and fix them and make different pull requests Okay so talking about some just some examples of what like curve does its functionality well let's go back to talking about the science so um LM 9859b this is a terrestrial exoplanet that orbits an m-type star um it's masses point three Earths and it takes 2.3 days to complete more orbit around its hostile and so just for an example what I'm going to do is talk through how one might obtain information about this using light curve and then reduce that data to put out the transit signal that was discussed earlier so searching for the data so how everyone can see there's my screen here I'm searching through the data so um test has um so light Cav has a variety of ways in which to search for the different data products so I'm just going to pause here for a second so we have full frame images so these are set of four all science classroom pixels across the sky so Tess operates by viewing sectors of the sky in these 24 by 96 degree fields and it looks at that region of the sky for about 27 days and then it moves into the next one so a full framed image is a set of all science and collateral pixels across all cdcds of a given camera there are four cameras and they used to have a Cadence of about 30 minutes in years one and two and then 10 minutes in years for him four and 200 seconds in year five onwards so you can see the difference if you look here the different times exposure times so you can kind of um search for these if I'm gonna go back on this one by using these different functions so we have um a test search cut um and then which will basically show you whether your object is in a given Forefront image and I'm just going to show you the result table of that search function and it gives you in the table this link to another database which tells you how to use test cut so that would be like how to cut out a little region around your object of interest from a Forefront image the other day stated products are Target pixel files so there's a poster stamp files focused on select targets of interest and each stamp has a case of two minutes or 20 seconds and this is what I'm showing here where these data are reduced by the spot pipeline so they have further data reduction applied to them like removal of noise and things like that so the user can get a better grip on the data faster and then we have like curve products so these are the light curves that are produced from the target pixel files for your objects of Interest again we have the Spock one which is the um the the typical science one but there are also other higher level science products here which if you click on each link as I'm showing you in the table takes you to a different page and that explains what the high level science product is and again I'm clicking here on the Quick Clip products and that takes you and shows you a discussion about what a quick product is and so what I'm saying is that the light curve functionality when you search for these objects gives you a lot of information in that table just as is Eleanor um but if I take you back to the beginning hopefully if I can do that here um yeah one of these so yeah so whether you're searching for the full-framed image information the target pixel file or the light curve this gives this table typically gives you you know which sector they're observed in this is a sector it gives you the year that the year that data was collected and then I said these hyperlinks to give you more information and then the exposure times and then your object okay so okay you can search for that data but how do you go about downloading it well it's pretty simple essentially um I've noticed that there seems to be a delay in my video here is everyone seeing that I'm just looking at it I don't know do you see the downloading data page I do okay great sorry um in my video it seems to be having a delay so I'm glad that you see the downloading data page wonderful thank you um okay so if you want to download your data how do you go about doing that so it's pretty simple you just put you pick them which element of the array you wanted beforehand download and then you can specify um different quality bit masks so that is basically just identifies the um Quality flag that should be used to mask out bad cadences or not typically you have default or you can have hard or you can have soft and then you might get some more data that isn't has been affected by other things I I typically use default or hard just because I don't want my data to be affected by various systematics that I'll be talking about later so I'm just gonna show you a quick movie of doing that here so you can download in this instance I'm downloading a Target pixel file so a little cutout of Anna region and you can actually have different ways of plotting this information to see what it looks like so you have the pixel number versus the flux there's also a nice fun little thing called interact here and this shows you essentially um It's usually the aperture since the pixels are selected here and also shows you the light curve that's created as part of the aperture and you can move across this bar here to see how things change with time so you can see in your image here you can also manipulate the um the um the contrast essentially um but yes you can also watch it with time and you can adjust the pixels and that make up your aperture as well so you can reselect um if you want there and then you can also save that light curve and you can save the aperture as well um so this kind of offers a very nice way to interact the data and also gives you information about the surrounding region another way to get information about the surrounding region is to use this um function called interact sky so if you have a look here this shows you your object of interest and it also shows you um the surrounding region around it so I'm just adjusting the contrast as you can see here and then if you can see these all these red dots and if you and click on those that gives you information so they can give you the test magnitude also gives you some Gaia information array and deck and you can do that with any one of the red dots here to see what is kind of in your surrounding region which can be quite useful if you're trying to understand you know potentially contamination or you know anything else that's going on in that region as well so I particularly like these kind of interactive functions that a user might use yeah okay so that was looking at a Target pixel Farm but you can also download the specific light curve data itself that's been produced by the um the science operation centers and so this is just showing you a quick video of that so that's essentially you know you downloading your light covers before and this provides you with a table of information so your time versus your flux and your Cadence and a lot of other useful information now in the Target in the light curve files there are two kinds of flux there is the simple aperture photometry so this is a light curve is calculated by summing together the brightness of the pixels at forward in the aperture step by Tessie so that before when I was selecting the pixels and then there's a pre-search data conditioning sap flux and this is flux from which long-term trends have been removed using co-trending basis vectors and this is usually cleaner and this is the the information that's produced by the science operation sensor um and so what I'm showing you here is just some of the information that you can find within a light curve file so you can plot things like the time and the flux they're typically stored um as you would expect with a master pi and then um you can plot this up so we have the time the um um the date time and date shown here and then you can plot the sap flux versus the PDC sap flux as well so you can just get an illustration of how they look compared to each other and therefore try and get an understanding of how you know you might need to process your data if you only have the sap flux but yes there's lots of different ways that you can interact with your like curve data as well and I'm going to give a few examples of how you might manipulate and interact with your light curve data using light curve with a k um the light curve software so this is just showing you know a couple of different functions here we have the flatten so you can remove long-term trends um using the flatten function um you can fold the light curve and this is really useful if you want to fold the light curve on the period of the planet potentially and how to recover that Transit signal which you can see here and then you might want to also Bing your data especially you know if we're using 200 second Cadence data or two minute or 20 seconds data bidding can be very useful to pull out some of those signals as I'm going to be showing here so this is then we're covering the um Transit from the planet and this example was just shown on the PDC flux you could do it on the sap as well but then be aware of other systematics that might be shown okay so there's another function which is very useful in like curve which is um creating periodograms so in this example I look at kic 1024 42o2 an eclipse and binary that was detected by Kepler and two periodicam literally just creates this periodogram from the like curve object data and it within that you can again search the API of light curve and it gives you all the options for you know the methods by which you want to do this so I'm just going to go through this video here just showing this so this is for this is the um scattered data that is recovered from the light curve this is the PDC sap flux and this is just showing if you wanted to normalize the data and then apply the periodogram and then plot it and showing the frequency power function and then if you want to adjust that and make that log scale that's also an easy option and then you can obviously look at the properties of that program so you've had periodic on object and then you can look at the properties of that to derive certain things so such as you know the frequency at max power which is 1.9 days the period of max power all those sorts of things now this can be really useful in trying to you know get the period of the system and again you can use um the period that you derive to fold your light curve to show that now that doesn't look great so you might want to manipulate that and play around with that a bit as shown here so this is a very quick run through of how you know the period function Works in like curve additionally we have a function which you can look at um to um obtain the average frequency spacing and the frequency of Maximum oscillation for um solar like oscillators as well so this is just a very quick rundown and they're actually on light curve there are tutorials which run through each of these examples of using the periodogram and using the seismology function here as well so these metrics what I'm talking about here can be used to estimate the mass and radius and surface gravity of a star and there's very functions that are inbuilt in like curve to help you do that when you obtained your data so um it's essentially you know you create a periodogram you create a smooth version of the power Spectrum you remove the background noise for the periodogram object and then you essentially run through and use these things called um two Source two seismology to help you to diagnose and obtain things like the frequency spacing and the frequency of Maximum oscillation aptitude we're just running through here so it's pretty pretty easy to recover these values from your data obviously you need to dig around into the API the documentation more for setting off specific parameters and the best case for your objects of interest but um it's pretty easy to derive these things foreign and then again you can go on to obtain okay difference power spectrums Etc I'm just giving you a breakdown of this very quickly and so yes you can use these parameters to then calculate the following such as Stellar Mass Stellar radius and surface gravity and there's all these sorts of diagnose methods as well to help you get a better understanding of what your data is doing to make sure that you know what you're pulling out is correct so this is one of the diagnosed methods it's being shown here and then there's finally estimating your seller parameters foreign I mentioned that there were different kinds of flux that could be um obtained from your light curves and your target pixel files the PDC sap and the sap flux and um what I'm going to go over now is just how if you do not have a light curve or Target pixel file that has been processed more um by the Spock how you would trying to remove those systematics your cells using some of the inbuilt functions of light Kev so I'm going to start this with a k2199 um and just talk about this so there's actually again that all these tutorials exist online within the like curve um on the light curve page and I'm just going to give you an example of how you can use a the pixel level decoration function in light curve to do this so as I said Tess has Institute instrumental systematic Trends which can make um detection of various signals very difficult and like have the excess both methods to remove these so co-trending basis vectors regression correctors pixel level declaration or self flat fielding and I'm just going to show this video here which is I'm using this um object I discussed earlier and showing you um what the target pixel file looks like and then essentially just importing from the like curve corrector class um this pld corrector so pld is a method used to remove systematic Trends introduced by small spacecraft motions during observation and identifies these set of Trends in the surrounding pixels of the target star and then performs this linear regression to create um a combination of these Trends and then um we use it to remove the noise and so this is essentially just going through calling that corrector and applying it to the data and you can see how the data looked previously in the red and then once it's been corrected as shown in the black here and then it just shows you the dramatic effect in the uncertainties and then again there's this diagnose function so you can see the uncorrected data the corrected data and then what um exactly was modified and done to correct that data and then any outliers that you might see as well yeah okay so this is just my final remark here um so test test even though it has it was originally designed to um you know find in the text exoplanet to the transit method because it essentially observes you know almost entire night sky I think by the end of the second extension it has now observed 93 of the sky and some areas several times it really is amazing an amazing facility for looking at all sorts of things it's not just exoplanets like Stellar variables explosive transients solar system objects as well and so while the as a community that uses test is growing and using it in different ways we're developing you know new techniques or thinking of new ways in which to you know light curve needs to be um adjusted or you know new ways to show the community how to use test data on things that was not originally designed for and so that's really um comes in with you know having this GitHub page where people can report issues or problems and that we as a community can work together to getting the best data from tests and with that I'd like to thank you for your time fantastic first off these are very nice slides great I'm glad so I mean I was like really impressed but uh it's also an impressive amount of work in developing this package um for a long time yes yeah we have some virtual applauses if if you look in the participants uh we have time for a question and otherwise we're going to move on and then we'll take all the common questions at the end if anybody has a quick question you can unmute or raise your hand Rebecca have you seen any Novi go off yes so the amazing thing of test is because it is the stand and stair operation if a Nova happens to go off in that region and it's bright enough to be detected by test so typically test can tests can get down to 21st magnitude if it's not too in a crowded region but typically needs to be a bit brighter than that um but yes yes there are Novi that have been observed in test data and then like I said the nice thing about it is if it goes off then you tip you can get actually the nice rise and the pre-maximum halt as well so Tess is amazing for getting that really early data at a very high Cadence as well the Cadence now as I said in the second extension with the full framed images which is what typically you'd have you know you can't select objects of interest if you don't know about them yet um typically would be 200 seconds now so yes you can get beautiful light curves of movies and tests okay I'll continue this offline wonderful awesome and we'll have time for more questions uh to understand but we're going to move on to our next talk uh let's see I had to do the after three years I still have to check to see if I'm unmuted on on Zoom I'm still paranoid about this uh we're gonna hear from uh Constantine malinchev who is going to tell us about light hyphen curve um but also about his experience using rust uh for developing Astro programs so let's give our attention Sebastian uh hello everyone uh thank you Rebecca for very uh beautiful talk and uh for great package which we most of us probably enjoying views and if somebody is not enjoying they just haven't used capital or test detail yet I think uh so uh and hello everyone Chef I'm postdoc at Euro city of Illinois Urbana-Champaign uh moving this summer to Pittsburgh to join a link Frameworks uh project as a research scientist at same here and I would like to present you uh my experience on developing light curve sorry very lazy naming I know light curve uh packages uh for Russian python for time series analysis basically for uh liteco feature extraction Okay so I will try to spend more time talking about rust itself because I think my package is quite Niche and quite straightforward doing straightforward things very simple and straightforward tasks but very fast uh and the community would be more interesting to think about rust and how we could use and should we use it in astronomy so let me start with this simple question um why go and rust so thrust is a quite a modern language it started something like 10 years ago even more probably in uh Mozilla as a experimental project uh to develop a new compatible language which would be Memory safe fast and convenient for difficult different types of problems but mostly for quite low level coding and of course first of all it was used for Firefox and then it was released in public reached version one and now it is stable language development which is really you know High very uh High very high velocity of development of ecosystem compiler and language itself so the first aims of rust are to be a fast uh paralyzable and safe and by safe it is you know quite uh safety is a big topic and to in Rust when people say safe they usually mean memory safe uh soon no undefined Behavior or almost no and uh safe in terms of parallelization safe in terms of uh access to the memory in the motor threading so uh it has a compile time memory management model so no garbage collector uh it has a resource initialization like in C plus plus so you could create a vector for example like here and when you access cook your vector is automatically delegated and the object is destroyed uh so still uh since it doesn't have garbage collector it has no virtual machine or other runtime requirement which is great for using it uh with other languages with other Technologies uh you don't need to bring anything with you after you build your binary or your library so uh another uh specific thing about trust is ownership model which makes it memory safe uh you could think about it if you use C plus plus uh more or less modern uh you can think about it as a C plus plus move semantics so almost everything is in Rust is moved not copied by default only you know small and easy to copy objects references numbers other small stuff could be copied and you could create your own copyable tabs but generally everything is moved and since the object is moved you couldn't use it anymore not because of you know UB or some standards but because compiler wouldn't compile the code after you trying to use moved value like it shown here and so we're creating a string we move it to a function function takes it and the the string is destroyed after the function regulation so we cannot use it it's delicated is destroyed we cannot use it anymore here to print uh the stroke itself so we could fix it in two ways we could uh just not trying to use it after moving or we can pass reference so another specific thing I wouldn't talk about it too much because it would take probably too much time is lifetimes it's another thing to make uh in other parts of this memory safety model and this brief example showed this lifetime generics you see this Apostrophe a or quad a uh generic variable which shows um lifetime which is kind of Lifetime annotation which actually uh probably uh will be appear in C plus plus uh in in near future in 27 or something I'm ready uh not followings it was possible to and you could annotate uh your references to help Kampala to understand that uh outputs reference will not outlive any of input references here so rust is not object oriented language at least not in in the same way as a C plus plus or Java for example or python so it has encapsulation you could think about it just like structures in C but has no Constructors and no polymorphism so so uh still it has a structures and it has Associated methods but it doesn't make it really you object oriented so this new method is not a kind of static method is not really a Constructor uh because it it's finally just creating a structure you know in a sea like way but we have this syntax sugars so we could use attributes and we could call methods uh but instead you could generally more or less do the same job as you do for example in C plus or Java with other tools rust provides namely generics trades uh enums which are rich enough and syntax level macros in contrast to CC plus plus you know textual level macros so I don't have time to show all the things right now uh but I really enjoy rust enums and this is a small example so inamin rust is not just you know integer uh having some titles having some names it is algebraic uh type so it's a uh it's a a union of several types and you have very convenient match syntax to select a specific variant you have inside your enum uh and so you see you could pass the detail site and use it for example to create string return it and print something in our uh main function uh and Russ is designed to have easy interrupt with a c using surname unsafe code so there is again short example where we create C string and getting uh pointer and call function from C standard library inside this unsafe blocks so uh rust memory safety requirements are too strict to you know uh design a real life language with interrupt with other languages which do not have these restrictions and to develop for example based line Collections and other low-level data data structures so it has these unsafe regions where you could bypass some not all but some memory safety uh rules and you should be very accurate using these unsafe code and uh test it to be memorizing yourself so rust is not really a magic tool which makes memory save everything and uh about sinterop it has a really nice tools to generate for example um uh generate uh libraries libraries to call uh C functions just from you know C header files so you think uh all of them which is a um engine of rust compiler of rust C okay rust has really great ecosystem of tools uh so the main tool you probably will use it if you starting building crust it will be cargo it is a package manager build system and much more unit test Runner so there is kind of built-in uh unit simple unit test framework interest uh rust has centralized package repository you could think about it like PayPal for python of course you could use your private repositories if you want and while you produce your code to create IO it automatically pushes to docs RS and the documentation is being built from uh dog strings you have in your code very convenient so uh since Ross uses llvm it can compile to many llvm targets like wasn't for running in browser or even no OS binary is with you know uh no Center no approaches it's in standard library with no rust centered library with no allocator probably but you could still run it on bare metal if you want rust has a great uh async Frameworks for web development mainly but also for other kinds of input output like Tokyo and Ross standard 2 Chain is really great and large and huge so it includes compiler itself code formata think about it like black or in a python or uh ceiling I forgotten its name so silink fmt or tools like these four C plus plus so it has a linked which is called clippy it has a docs generator so you could generate HTTP Docs on your computer before posting them to docsrs it has language server for IDs by the way it has great support in C line in jetbrains ID for CNC plus plus and it even has an experimental interpreter uh which is used to finding uh undefined behavior in these unsafe blocks if you have some and it has other great tools like critering for benchmarks uh debugging tools interrupt with other languages including C plus which is quite limited of course because you couldn't uh you uh uh couldn't connect all the C plus plus features to rust features but it's still very convenient and code coverage tubes uh rust is already highly adopted by industry so uh the greatest thing happening in open so in very large open source projects I already told you that Mozilla initially developed it for Firefox I believe that CSS engine in Firefox is now written in Rust and it's already used in chromium I say Google Chrome it's not open source but I'm in from here it used in Android and it is the second official language of Linux which is great so rust chain is already in uh release kernel uh versions there is still no real rust code running in your with your Linux kernel but there are a few drivers so rust is aimed to use it as a uh for drivers in Linux for now and it has already used in in development GPU drivers and drivers for SSD so there are many Cool Tools written in Russell uh for example uh about python what it is a rough linter which is very fast and easy to use it is polar data frame Library it is kind of of pandas but it's more about you know it's going deeper into sql-like processing of your details so if you run the whole pipeline from Reading uh your file to getting the final numbers from it and write it all in colors it would run really fast and they advertised that even after pandas version 2 boilers is still the fastest way to read CSV file and create pandas data frame then pandas itself so there are some cool awesome projects using uh rust for example our beloved allo dim light version three if I remember right uh uses rust uh so you probably already use rust uh right in your browser and uh great adoption by large companies to all these you know largest complaints like Microsoft Google Amazon or Facebook uh Apple they use uh rust internally and sometimes for their open source projects okay um so what about python uh there are different attempts how you could use uh rust with python but the the most development uh developed and probably greater thing to use it is with pi or three uh create which provides bi-directional intervalability you could create you could call python from rust and you could create a python binary extensions using uh these create is just you know name for packages in in Rust world so here an example where in Rust you just call C python apis but translated to rust here so we import something getting attribute search some locals for our code and running running the code and getting the result back in Rust or there is a very cool inline python create which uh allows you just running a python code very conveniently from rust and even using QC rust variables inside python cubed like five minutes oh my God it's terrible okay thank you I will speed up okay so uh there is great interpretability with numpy I will skip it uh let's create but still good and probability with arrow and there is a great packaging facilities uh for for for python binary extensions like measuring um and you could use it to build uh your wheels and publish them to PayPal okay okay uh drawbacks very fast so of course it is Young language young ecosystems so it's not major in many many ways uh it has really steep learning curve because you need to get uh this ownership uh think uh to start to use it uh so it's right now it's not so great for science because not a lot of uh science packages uh exist and not a lot of uh buildings to science Popular Science packages exist but some of them are they are for example blasts G cell F of TW and so on okay and tap I probably skip the rest here I'm going to light code because I have just few minutes okay so the motivation of the project is creating uh like Refuge extraction package uh for regular 10 series for light curves we usually have uh when we use ground-based facilities not a great uh cellphone facilities like coupler tests and there are many options which we could go through but features are still fast preferable easy to deal with and uh they work for varic heterogeneous uh types of objects uh so okay I'll probably just keep more on that the history of go very fast through it so we started with uh in a snap project which is dedicated to light curve anomaly detection we first started with a light cover approximation for supernovae but found that it is not scalable for skills of chance in the light curves of uh for example ztf lead releases or Gaia or pan stars or future lsst so we just moved to feature extraction okay I'll skip probably this part and initial version of feature extraction very initial version 0 was written in Python of course but we found that it is really slow for many reasons interrupt with our databases slow features fraction slow everything is really slow and we wrote it in Rust and then created python being linked back to interrupt with um okay skip these slides as well and these with results so uh the package itself contains uh something like 30 different feature extractions for pretty simple features like you know magnitude statistics like mean median momentum and quartile based shape based statistics and more sophistication itself was like Fast flops cargo periodogram which utilizes fftw or mkl on Intel which makes it pretty fast for example faster than astrophy implementation by factor of 6 or 10 and we have a few parametric lighter feeds so to feed light curves like these like Supernova are other transients for which we also use some CRC plus libraries like service or GSL and some other features they probably skip them for now so okay this is small example how you run it quite straightforward uh you create this feature structure uh objects from python classes and then we extract features and you have uh you have them for your single light curve for for many light curves uh you could run it in parallel utilizing multi-processing written in Rust so it uh it it doesn't affected by Geo in Python so some benchmarks here which are a comparison between uh rust which is rust implementation which is green blue which is ourselves python implementation which we made for unit testing and from benchmarking and feeds is quite popular feature extraction package written in in Python so you see that this green is much faster usually than on Temptations in Python and we also have pretty low interrupt cost or I I have no time I know uh okay I I I'll almost finishing uh even which is even lower than interrupt cost for numpy functions uh but we did it to be the same a small trick we do not support lists or other you know array like objects we support only numpy race but that makes it fast really really fast uh even if you have a very low number of observations and all the time is going to interrupt between Python and rust okay I'll skip dmdt Maps which is cool about seal and going to my last slide uh so we have dozens of light coefficients which run really really fast and you could provide them uh in multiple threads uh very fast which not affected like metal processing called multi-threading uh things affected by Jill and python it's used by the TF also seller brackets to analyze alerts real life uh of photometry coming from ztf and we will be used to analyze lsst allows the future uh it is good called by test it has binary wheels for all popular and non-popular platforms so you could deep install it right now thank you and sorry uh that I use more tablet I should thank you for attention that's great question uh thank you for for sharing with us your package and also all your enthusiasm about rest I know there are some rest users in the participants Channel that I can see here and so uh they might have some questions at this point we're going to open things up uh to any questions about either package if you just use the raise your hand or drop it in the chat and I can read it out um otherwise maybe I'm gonna get things off to a start uh Rebecca you talked about running things in the cloud you mentioned Google Cloud but you also mentioned something called type and I'm not familiar with that yeah type is actually if you maybe I can find it on the mask um so it is um something that maths is doing to try and okay I'm not familiar with what Mast is either so ah okay well then I'm gonna I'm gonna start from the beginning um okay so maths stands for and I'm going to pronounce this wrong so please forgive me it is um the um Space Telescope archive portal so it's the um um archive I'm gonna actually share my screen so you can see what I'm doing yep that might be more beneficial um okay let's do share okay all right I'll put this down here okay so Mast is um is this it's the um archive the Space Telescope is where all the test data gets stored so you can go to the test portal and so it's to the portal and um you can type in an object of Interest so Katie for example and then that kind of basically shows you um where it comes up with a list of all the data whether it be on Swift or test I click test and it will find the data being just test for me but they've also created this kind of Jupiter Hub area so type stands for the time series integrated knowledge engine and you have to have an account with space telescopes it'll be like sign in with mass um and that will take you to essentially like uh almost like a jupyter notebook configuration work base where you can create new notebooks and then you can specifically choose the one that's tailored for um light curve and using light covered things and therefore you can like have it has light curve already inbuilt into it and you can run it like you would run a Jupiter notebook essentially and so some of the data is on Amazon web services like actually on the cloud and then the type can also interact with that data as well or it's another way of again using light curve but not installing the package on your computer yeah great yeah haven't seen that before in Jupiter hubs are very popular now so it's nice to see that that you're integrating it directly with your data yeah and named after our uh off after the firmware Congress yep um okay we have a question from just uh do you want to unmute and say your question um sure I can do that yep um so my question basically is from a from a very high level point of view should you think of rust as a um memory safer um replacement for C or C plus uh for performance sensitive binaries within our python modules or are there any differences in in how we would use that yes it definitely is so uh the uh rust is a great replacement for uh C plus plus over C or Satan or whatever you use to create a performant code for python memory safety really cool thing because it saves a lot of time and removes a lot of nasty bugs you could have and you could you know Miss uh for a while yes and which could affect your scientific results because of undefined Behavior also you could use uh rust if you would like to use some for example C code or a code having this interface but you need to add a bit upon over it here so use it instead of sighton in this way yes so you uh Bend some in the C library or fortune or C plus plus something with this interface and you add a bit of code for numpy for provision for something else you need and you create it uh binary extension for python you could use so and from other side of course uh Ross is not uh major for science so there are not a lot of uh rust packages you could use for silence yes so it's basically for things you'd like to write from scratch or for which you could easily get uh see or other type of buildings great other questions for either Rebecca or customer thank you uh we have a handout Abdul do you want to unmute and yeah um this is a question for uh Rebecca I and is um how do you deal you know how do you guys at the goth deal with at the guest Observer facility deal with you know leading or I don't I don't understand what's the role of the golf in light curve and being at Nasa and dealing with NASA regulations and contributing to uh open source software and things like that licenses and things like that can you see just you know a little bit about that stuff so um we recently took over right curve um so it used to not be under us um we recently took over the you know address working on on the code itself um with regards to how that fits in with working at Nasa um so I am actually a contractor so well I work through the Cooperative agreement so I work at UMBC but I work at Nasa so UMBC um with regards to software and like Kevin things like that I would have to say that's above my pay grade for specifically addressing that question but you can um send those types of questions to my boss Christina Hedges who is the director of the general investigator program at Nasa Goddard and she would be in a better position to answer those questions for you she was online for a while but yeah she's not now okay all right thank you thank you all right and as I mentioned I only recently learned rust users are called restations um for those of you who are using rust um why did you choose it or why did you and and how did you learn why did you pick that over you know what any of the other new languages like Julia and other things that are are hot well it's it seems just from the slides uh Costa was showing that speed is definitely a factor that that so maybe I'll ask how did you learn it then okay yes so uh from so it's speed and safety because I tried uh different things for CNC plus plus before and just for new project uh it it wasn't uh historically it wasn't the way you know where I had some patent code and I would like to speed it up it's it would be a great idea to do it but initially I had some very very simple uh features and I just couldn't make my python code work fast with the database we used so it was very slow to pass the database output and it took much more time than recording the features and and I couldn't paralyze it and I was just interesting to go for some new language so uh before it I tried Julia which is uh great uh but just doesn't fit my uh needs sometimes because it has a huge runtime you should install somehow to users computers and to rust is good because it is general purpose language so it was for example super easy to create uh web API for features which we use in snap viewer and in other projects uh where we don't have so where we don't have opportunity to use Python but we have ability to go through HTTP to request uh these features and so I guess for those of us who are interested is the is the online rust tutorial in their docs is that a good place to start is that it yeah so rust official Rust book is a great uh thing to start with great definitely yeah all right I think we have time for one more question if there are any lingering questions while people pick them up I'll remind everyone that next uh next month we're doing a joint meeting with the psychodes group following that is going to be grad lightning thoughts and following that is going to be teaching computational last trip any final questions and I find pets any final pets yeah if you have an animal this is this is your opportunity to show us mine's snoring here um all right if there's um oh it had been of code problems indeed that's a great great way to force yourself on a new language all right if not let's thank both of our speakers again for wonderful talks thank you for taking the time to share with us today thank you thank you all very much thank you everybody thank you and I'm going to end recording