Code for Analyses and Models?

[ljchang]

Hi Caroline, I hope you're doing well. Congrats on the Neuron paper! Was curious if you're planning on uploading the code for the analyses and models either in this repository or on the OSF repository.

I didn't quite follow all of the details for a couple aspects of your models in the paper and was hoping to be able to look at the code to better understand what you did.

For example:

1. in Equation 9 how were "partner's action consistency with g being valuable" determined?
2. in Equation 14, what's the difference between the values for \omega and \delta? Wouldn't they be proportional to each other?
3. in Equation 10, I have a few questions about the entropy calculation. Why use the estimated min/max from practice data rather than just a local memory of past trials?

Thanks!

[ccharpen]

Hi Luke,

Thanks for your message and for creating some social accountability - I've been meaning to add as much code as I can here - I just wanted to spend time cleaning it up and commenting it properly before I do, and things have been a bit crazy lately!

In case that helps for now I am attaching a folder with data for an example subject that can be directly passed through the function to generate choice/likelihood/model accuracy, so you can see the code we use in the model.

arbitration_choice_function.zip

To answer your questions specifically:

1- This was either 0 or 1 depending on the situation; basically either the partner's action is consistent with a given token being valuable or it's not. In the code, this is pre-defined l.33-41 of the function and used in l.80. If the partner's choice is ambiguous (i.e. suggests that either green or blue could be valuable), then both probabilities will be 1, but the scaling over token values performed in l.108-112 will basically reduce those to 0.5.

2- Delta is a free parameter that's estimated for each subject (params(3) in the model, see l.163), whereas omega varies trial by trial depending on the reliability of emulation. They are related indeed, especially if delta is very high or very low, because then omega will pretty much be equal to 1 or 0 (respectively) and won't vary much with the reliability of emulation.

3- Our assumption was that people will have roughly experienced the range of entropy values throughout the practice trials and will therefore be able to normalize the current level of entropy according to this range (and update the range in case they experience a more extreme value at some point during the task - so it's actually not only the min/max from practice). We had at some point tried to fit a model in which entropy is learned over time, but the learning rate parameter was not recoverable at all, and fixing it to a given value felt a bit arbitrary and gave us pretty similar results to the min/max normalization. But it's a good question. I've never tried to use just a local memory of past trials, but the assumption was that in order to best estimate reliability of a learning strategy it makes more sense to take into account the entire knowledge/history of the task and maximize the information available to compute a reference point (here the range of possible entropy values) to compare the current trial to and determine whether it's high or low. Min/max normalization seemed like a computationally cheap way to do this. At least in this task, I think that comparing entropy on the current trial to a local memory of the last few trials would be pretty misleading, but maybe it's what some participants do and could be worth checking.

I am hoping to upload more code in the next week or two - in the meantime, let me know if you have other questions!

[ljchang]

Wow, thanks for your fast response and detailed answers to those questions! The min/max entropy idea is really interesting. From a psychological perspective, I could imagine some type of normalization happening online, but as you note, it's also possible that you have some prior beliefs about the entropy from quick samples in the beginning - even from the practice trials. This could shape your experience and learning. Lots of interesting things to follow up.

Sorry, I've only skimmed through the paper and haven't read it super carefully yet, but one other detail I had a question about was the normalization that was done in Equation 3. You cite Kenway Louie's paper, did use a similar type of normalization? or was is it something more straightforward, like a softmax?

Thanks for the zip file!

[ccharpen]

What we did was very straightforward - it's just that before passing the token values through equation 3 (i.e. the linear combination of token probabilities & token values to compute the value of a given slot machine), the token values were normalized so that they sum up to 1
So for each token j: Vj,norm = Vj/sum(Vj).

We simply cited Kenway Louie's paper to justify that it makes sense to compute the value of a given token relative to the value of other tokens, so more as a support for the general idea of normalization than for doing exactly the same thing as he did.