Neural Networks for Recommender Systems (PyData 2017)

[00:00:03][Music] last year I spoke about Python recommend doing recommendations in Python and especially hybrid recommendations where you not only use collaborative filtering signals but also also metadata information as I was double sort of my talk last year and I guess every other talk at this conference and sort of other conferences this year next year and probably last year as well is sort of on on neural networks and deep learning so I thought oh I'll capitalize on this and see how to do recommendations using good learning so here goes and throughout the talk if you have any questions just just let me know and happy to answer them okay motivation so the rise of deep learning has given us new and wonderful techniques and your wonderful toolkits to do very fun stuff and it's been incredibly successful and image recognition has been incredibly successful machine translation that's amazing results in reinforcement learning but what really is the use of recommendations like the probably the best recommender system you can deploy straight away in your application is just recommend the most popular item for everyone it will do amazingly well and I'll give you what get you 60% of the way and the other two the workhorse of the recommender systems literature are the matrix factorization models which are extremely simple and again work very well so really what's the what's the draw of deep learning was the draw near Network toolkits for doing recommendations so my position on this is follows there are a bunch of a bunch of huge advantages firstly if you're using a neural network toolkit you get speed of iteration you can make very you can make changes to model very very quickly try them out very quickly you don't have to implement sophisticated SGD models by hand you just change your loss function or you change your representation and the neural network that you're using it takes care of the backpropagation for you so that's really useful another advantage is flexibility so if you're using existing packages for accommodations like cards spark mostly there'll be it's a fairly fixed functional form and only certain things that you can do so in most applications this will be collaborative filtering you just have information on what users did and what items they interacted with and that's what you put into your model and that's what comes out with new electric toolkits and would be learning you can do much more fun stuff quite quickly so let's say for your application the metadata of the items or the metadata of the users is very important you can put that in straightaway and the final advantage is powerful representation techniques so let's say any application you know the visual side of things is very important so let's say if your Etsy a large part of how people interact with your items is what they look like so something they might want to do is put features of items which are visual into your model so you take your classic factorization model or collaborative filtering model and their new train is CNN on your images and you put that in and hopefully they'll make that recommendations and this is something again that deep learning gives you in your network is giving so in recommendations these are the things that you can do you can add site information again images there's a paper by Etsy where they look oh can we can we take the images of our products can we run and run the CNN on them can we take those representations and put them into our recommender system can we make better recommendations second advantage is flexible loss functions so mostly what people do recommendations is well you know if they user user interact with an item that's a1 if they didn't that's a zero that's a classification problem that's what you do maybe that's not the best approach maybe that's not the best approach in general maybe that's not the best approach for a particular use case maybe you have both interactions like the Netflix case of the museum's case you have both interactions did they watch the movie but also have ratings like after watching the movie did they like it or do they think it sucked so maybe you want a more flexible loss function to take take advantage of the additional information and finally sequence based representations so arguably for a lot of recommendation cases you have a user that performs a sequence of actions on your website or in your product and you want to take advantage of that sequence information to predict what they're going to do next right this is what's interesting they bought these items in the past was the next thing that's going to buy and you know they may have bought books in the past but they just bought a camera maybe batteries is what they want or maybe a memory cannot okay so the plan for this talk is sort of straightforward the first is how do you take a classic matrix factorization model and express it to a neural network toolkit once you've done that well you know the promises past iteration how do you experiment with a number of ranking losses and finally well lift sequence prediction is what you want how do you switch out the factorization representation for a sequence based one so instead of just taking users the whole representation of everything that I've done in the past maybe you want to take the sequence of most recent events into that count more prominently so as they go through this this is going to be FLE this my talk is going to they're fairly practical bumps I'm going to show implementations so that sublimity the complexity but also hopefully will help to explain and all the examples are available on github and let me know if your questions happy to explain and everything will be done the pie talk which is probably what people don't have that much experience with but I found it to be extremely accessible expressive and quite pleasant to use okay so I'm going to start with a classic model the factorization model who here knows what the factorization model is for accommodations they affect quite a lot of people but I'm going to explain anyway so in recommendations in collaborative filtering what you do is you take information on what people interact with so we have a set of users and a set of items and you express it in this big sparse matrix we've got users as rows and items as columns and then if I use interacted with an item that's the one in your matrix and if they didn't that's a zero so that's quite big and that's quite fast and really what you want to do is factorize this matrix and to latent representations of users and latent representations of items so these are two the dense matrices that when you take their product they will hopefully reproduce the original matrix that you have accurately busts through discovering the late instruction of the problem show you the items that given user has not interacted with yet but would be interested to see so this model was famously introduced during the Netflix prize contest and is still a workhorse in huge amount of applications and originally it was sort of in the form of ratings people had ratings in the matrix and you wanted to predict rating for announcing movie but the more modern form of it is well actually the more important piece of information there is what people watch in the first place so the implicit fall out of that problem is something that's used more often now so a way of expressing this in neural network form is well you just take a to embedding layers and embedding for users and embedding for items and then you take that when you have a user and you have an item pair you take you take the user embedding from the user embedding layer and you take the item embedding from the item bedding layer take the dot products password let's say for a sigmoid function that's your prediction so again terminal a is for users and items connected by a dot products and in this case fit with logistic loss so from that matrix that you have you take it one sim you take your zeros and you index your embedding matrices by those indices take your products and ducts let's go so in pi torch you might do something like this so the way you define a network an pi torch as you define this class which inherits from n and module and then you define the DES layers that are in your model and after that's finished you then put it to an optimizer into back propagation so in this case we just define the embedding dimension which is the latent dimensions of the vectors they use then you define two to embedding layers in this case the scaled embeddings we want to make sure that we start the initialization has the right scale and then maybe you want to have some biases as well so if an item is very popular than the bias with high if a user watches a lot of movies that that they will have a high bias to sort of soak up the effect of that and user so once you've done that you designed a forward pass so given those layers what actually happens in your network so in this case you take the you make you can express your matrix as basically pairs pairs of items so if user one watch two movie one that's a pair one one and that's what you pass into your neural network so in this case let's say this is user IDs is one and item IDs is one notice of course in mini-batch index your embeddings by those by those IDs and then transform them so reckon you can actually do the multiplication you know the multiplication you add the you are the biases and that's your top the output of the network so that's that's effective your prediction you can pass it through some sort of non-linearity like a sigmoid perhaps your prediction so how do you once you have the network how do you actually fit it well commonly what you're doing these sort of problems is you have a an output vector in this case the output vector would have for a given user would have a dimensionality let's say of all the items that are in your corpus with ones where they interact with an item of zeros everywhere else now for recommender systems this is not an amazing approach the reason for this is these problems are very high dimensional so even in simple cases you will have tens of thousands of items in bigger systems like Pinterest or Etsy you'll have millions of items so you can't really have this loss function well in every evaluation you evaluate millions of the products and then try to compute a loss from this so what we will do in recommendations instead is they've use a sampling approach so you take your positive pair of user an item and then you sample randomly a negative item from all the other items with the implicit assumption being well this user didn't interact with this item therefore they're less likely to like it so we should treat it as a negative so that's a sample growth function and we can express it in part in titles like this yeah the other display is too far away I'm just going to walk this so you you have your you have you users so these are basically IDS of the users who has who have positive interactions in data set then you have the items which aired the IDS of the things that interacted with and then you sample negatives so in this case I'm just going to say well for every user that I have in my mini batch I'm just going to sample a random item that they haven't looked at and that's going to be negative so then the objective of the of the network is to predict one for all the things that they interacted with but 0 for the things that it didn't so I have this composite loss what I want to predict one on the positives and 0 on the negatives and InTouch compute compute the losses can catenate this into one vector this is for a mini buff so I take a mean and then I can back propagate in this so we can actually run it and again all the code isn't get help so I'm going to use the movie length 100k dataset which is which is a very famous research data sense but I guess at this point is sort of a unit test for a recommender system so if you recommend the system doesn't work all the movie lens 100k dataset you have a problem and should fix it the nice thing about this is well-known it's easy to get and it runs very quickly because it's just too small so in this case we're going to define a model and we use the point wise loss but they just described oh yes I should probably say what the actual dataset is it says it's a data set of a hundred thousand interactions between users and movies and basically you have you have a 1 in this case if a user watched a movie in a 0 if they didn't so we want to we want to fit a model trainer model get the embeddings and then at the end when we evaluated we're going to construct the ranked list of movies for every item and hopefully the movies that they have actually seen in the test set will be at the top of the rank list and we're going to measure this with a with the metric called the mini reciprocal rank which is which is very simple basically for every for every ranking that you construct you take the rank of the true positive items and take 1 over that so with the UV ranked a true positive vitamins 1 it would be 1 over 1 so that's one if you rank a true positive item 100 it would run over 100 so the closer to one the better if it's more that's about model yes so let's let's run them on though just defined we have the point wise loss we put the data through it with fit and we produce a skull which in this case is zero point zero six which is okay but now that we have this neural network toolkit model can we actually do better than how we did can we do it so one way of doing better is to use ranking losses instead of classification losses so what we just run is a classification loss so we have the representations with the embeddings and then we have a classification was predictive one on the positive items and is zero on the negatives now a ranking loss is slightly different what ranking loss does is it says we don't actually care that we predict one on the positives and of 0 and negatives all we care about is that we predict the positives the scores every predict the positives should be higher than the score they predict for magazines and the way you do it is you construct Triplets of a user a positive and a negative item and then say that a positive item should always be ranked higher than the negative item so one way of doing this as this loss right here which is 1 1 minus the sigmoid of the difference between the positive predicted scoring the negative predicted score so the higher positive predicted score is the smaller the loss the higher your negative predicted score is the larger the loss so minimizing the loss ensures that you never flip the ranking between the positives and negatives in data sense so how do we do it with the model well we use a Siamese Network so we use the same embedding representation from the front before we compute the score for the positive and negative item using the shared embedding layers and we feed it into the pairwise loss so graphically you can think of this sort of like this you have you have your representation model for the non positive and you have the representation model for the sample negative and you take your IDs you pass them to the embedding layers to get the embeddings out you make you take the job products and then passengers loss it's called a Siamese Network cause even though you sort of have two branches of the model and in principle we could have different parameters here and here in this case the parameters are shared so the actual number of parameters in the model is exactly the same as we had before and by the way these sort of networks often use for metric learning and some stuff so what might implementation look like again very similar to the previous implementation you sample some negatives but instead of just concatenating to two lost X's you feed it into the PP a loss or the loss for the 1 1 minus Sigma of the difference between a positive and a negative Scott so do we actually get a better result when you run it we do we run the model with the DP Alice and we get a slightly higher MRI of 0.08 so 0.06 so that sort of illustrates how nice and easy it is to expand with different losses ordinarily if you have to do this yourself before the advent of neural network toolkits you just have to code up the losses yourself and make sure that the SGD flows nicely and so on so forth but as here you just define a different function and the toolkit takes so of the rest but given that given that this ranking loss for experts I can we do even better what we can and it's called an adaptive ranking goals so what we do is we have the positives and with sample set of negatives but then we sample a couple of couple more negatives for every positive and then we pick the one that's closest to violating the ranking because remember we have a positive and we have a negative and we want to make sure the positive ranked hide in the negatives but if we just sample the negatives randomly from the entire data set we may find something that's actually not very discriminative so if we sample as a negative and a movie which is just terrible that's not going to help us model tear apart what the user likes more because it doesn't like so what we can do instead is sample a couple of negatives and then for the gradient step pick the one that's closest to violating the ranking that we want to induce that gives the model a better information about what the user liked that's an excellent question and that's possible that's possible so what you probably wouldn't do is to exhaust it sampling if you do it if you do it if you don't do it you get the result that we just had if you sample everything and then pick the highest ranked one that is not known to be positive that is going to give you a terrible model because you're going to be just defeating yourself as you fit the model but if you do it's the hyper parameter if you do it just the right amount it gives you a better result that's true that's yes there is attention when it comes to this sure sure and I mean that's that's definitely a problem and a lot of people have thought about this I don't know that there's a good solution for this yet but it's definitely definitely a thing there's a there's a extension to this is if you show if you show people items and then say what we showed this item to you and you didn't like it we'll take that as a negative so then you know that they have seen it but you take it as a negative in the fitting of your model that's not necessarily the best approach because you may be interested in you may still be interested in the item you just didn't have the attention span maybe to interact with it and then if we take it as negative we're just going to be giving us recommendations later on but yes that's that's particular so in fact if we if we look at the implementation this is very simple and this is why pie talk is so nice when we actually when we sample the negatives you just do a loop sample a couple of negatives and compute the scores for all of them and then pick those ones which have the highest predicted score but I just I have a hyper parameter here which is hard-coded which is the rule 5 so we're just sample 5 negatives for every user and pick the one that that has the highest prediction score but coming back to your question is if we make this higher the performance of the model actually Falls so I think this this would be this will be an example of where you you have a self-defeating model yes so again this is a slight variation in the previous law such as sample some negatives pick the one that's got the highest predict it's gone so does it does it help yes it does help if you pick your hyper parameters right so in this case we've got em er our of 0.11 instead of 0.08 so as we had before on 0.06 so the banking losses did really do help so a summary is you have the same representation but you can very quickly iterate on the different ranking losses and I just looked at ranking losses but maybe you want to include the rating in your loss as well and then that's super simple again and then you can see if if that improves your model all of this is incredibly simple to implement and you can you can easily write a loss function that suits your unique model and that's something that used to be very hard now that deep learning has come into its own it's sort of it's like a windfall profit to to us who don't maybe need extremely deep purpose representation walls so the second thing I want to talk about is not here but in the next example I will mention this yeah that's a good question so the second thing I wanted to talk about is recommendations of sequence direction so again maybe that's the right way of thinking about this we want to we have a user and all we know about them is what they've done in the past this is how we represent them and maybe we want to encode this sequential representation or model and treat the recommendation problem as a problem of predicting the next element in the sequence of user actions so there is a number of way of doing so but in all consists in representing the user as a function of the items they've bought in the past so the user presentation can be a pulling representation so we can just take the users an average or maybe the sum of all the of the representations of the items that they interacted with in the past we we can use a recon general Network where we actually trying to capture the sequence structure so they did something a long time ago and there's something else and did something else and what are they going to do now and maybe you want to use a CNN maybe we think the sequence structure isn't that important that maybe if we supply the CNN window over what they did maybe there are some things that occur together that carry more information than just individual events yes so the advantages capture sequential information there may be less pressing need to retrain like in the model in the representation model that I described before every time you get a new action from a user you need to go back to training of your model and add the other action into it and take a gradient steps and update the parameters in this case the input to your model is just the sequence of events so you can just train the model and then every time user carries out another action you just increase you just expand the input vector into the model you don't have to do retrain disadvantages well if you have a complicated model it may be slower to evaluate if your applications perform a sensitive Tami numpy that may not be feasible another thing is well is it actually worth paying the cost if data has no clear sequential structure then there's really no use of doing and I guess that's why we're using the movie lens dataset we'll see if the sequential structure there is important and again here we're changing the user representation all the loss functions that I talked about before you can take them as they were and apply them to this model in the same way so you can sort of the choice because the flexibility of the formalization is it's so great you can you can choose your representation dependently what you love functions examples so - to prepare your data for this sort of model you need to construct sequences so one way of doing this is your input matrix could look something like this so if you have events ABCD you want to predict from zero ABC - ABCD where from zero predicts a and from B you predict from zero a you predict B and zero a view project zero zero is the sort of the the null token in your prediction and this is actually what I'm going to do here because if we if we run a return you'll network over this we can get the outputs of the network and every time step and then use the output of that network as the user representation and use that for predicting the next item that's actually quite nice one alternative way of doing this is which i think is more common because it's more flexible but it has worse performance is to split your sequences so if you have a 4 element sequence you split it into four rows and on the first row you predict from completely empty sequence you predict the first element and then you power you left part and you have the first element predict the second and so on so forth and this is more flexible but then if you're using a recurrent neural network for every of these rows you run your recurring neural network from left to right and then sort of computing the synced in States over and over again for this so it's not quite as it's not as deformed okay so given this values I'm going to try a couple of representations so I'm going to try the popularity representation or the lack of representation a pulling one we're just averages users previous interactions and a recurrent one and see how fares from the movie lands datasets so they're pulling representation again part of it is very similar to what we did before we have a B we have the embedding layer a note that we don't have an embedding layer for the user anymore because the user representation is in this case just an average or the sum of the representations of all the items they interacted with in the past so we don't have that anymore we only have the items and then the forward passes well you have some item IDs and item sequences where this is what the user has interacted with in the past and this is what you're trying to predict you get the embeddings and then you you create a rolling sum in this case across the sequence so the first element of your sequence we'll just well the first element that you did and then the second one will be the sum of the two preceding ones and so on so forth and that would be at every time step the representation for your user and then you take the item biases and you do the dot product so at every step you would take again like before a dot product between the representation of the user and the representation of the item and that's and then take the sum in that little product and that's your touch your prediction the recurring representation was very similar again you have the embeddings you have the biases but this this time you also have the lsdm layer this is this is based on a paper that uses gr use but I'm going to use an LSD M here so we have the item beddings item biases and LSD m and then in the forward pass you take the embeddings of the items the user has interacted with in the past and simply run an LSD game over them and then at every time steps the representation of the user is the output of the y lsdm and the last time step and to get the prediction you take a dot product of that with the representation of the item that you trying to predict and that's your outfit okay so let's try this on the movie lens dataset we we import the data in a slightly different format this time so that we get the sequences so this is what R this is what training rows look like so from this sequence you're trying to predict this sequence you'll note those just shifted by one and will stand with the popularity baseline to know that we're actually doing anything so the evaluation here is again m RR where at every step we want to predict the next item in the sequence so with the popularity baseline we get 0.12 if we move on to pooling we get 0.14 on the testers and we if we move to LA CMS which is the best and the most sophisticated representation here we get zero point eleven which is which is worse than the popularity one so there are a couple of possibilities here did we set band hyper parameters maybe the sequence information is not important or maybe which is probably most likely I got the model architecture wrong and doing something completely different than I thought it will be doing so to get at this and try to disambiguate between those we can run a very simple experiment where we just have something with very clear sequence structure and just use number sequences so the we use we use numbers of the IDS of the items and then one always follow zero and two always fours 1 and so on so forth and run them all along this so if we do this and we use the popularity baseline and the terrible performance is 0.07 if we is pulling it slightly better but if we use lsdm to perform really really well so at least the model is fine which is reassuring and it sort of goes back to the point that well lsdm is are great if your problem has a sequential structure but if it doesn't maybe it's just overhead maybe it's just cause you don't want to pay okay so in summary what deep learning has given recommender systems is amazing flexibility like if you want to set your model up differently you can if you want to use a different like it different lofts you can if you want to add pieces of side information it's extremely easy to talk it will take care of everything for you second you have powerful building blocks if you want to use images well you can use convolutional representations maybe you have enough data to be able to fit representations of your own maybe you can just use a pre trainer unit and fit this into your model and will work better and the final thing is speed of iteration whereas previously you have to do it all by hand now it's really really easy really straightforward and you can you can think in terms of your normal numpy operations and mathematics and a new model will be good for you is everything perfect no recommender systems are inherently very sparse problems you have lots of users and you have lots of items and neural network toolkits and neural network approaches work best with highly dense problems you'll see nn2 highly dense problems and recommender systems implemented neural network toolkits can be quite slow the reason for this is well if you have this enormous embedding matrix for your items which is maybe you know 10 million long and 128 floats wide and if you update that entire matrix on every time step on every gradient updates and you know when you're fitting your model that's a lot of overhead that's going to take a long time a lot more time than if you implement it the same thing by hand and use this fast as you give the problem to your advantage so there are a couple of solutions to this I mean you can in certain frameworks you can use path gradients so when you're updating the embedding of a single item you don't have to update all the parameters embedded layer you just update that one item but that can still be quite slow one alternative is well maybe you can run it on GPU with huge mini-batches so in other problems maybe your mini batches are 64 128 large in this case well maybe you want to run a mini batch of 8,000 examples at one time maybe maybe that's the right way recruiting sparsity because your parameters s passed so maybe does enough so if you run a huge mini batches and that actually works quite well yes so I guess to take away pointers they're very good they're very highly flexible you can accomplish amazing things with them but the performance may be a problem [Applause]