MatrixNormal and MatrixNormalWishart added to ExponentialFamily

[wmkouw]

I'm adding the MatrixNormal(Wishart) distributions to ExponentialFamily. The natural parameters are vectorized when converting to an ExponentialFamily distribution but that means I can no longer infer the support of the distribution (here n by p real matrices).

My proposed solution is to add a dimensions attribute to ExponentialFamilyDistributionAttributes. Anyone opposed to this?

[wouterwln]

Is this something that could be stored in conditioner? I am not familiar with the details but this is usually used to store parameters like dimensionality and such

[wmkouw]

I think that's misuse of the conditioner. But alright.

[ismailsenoz]

We reserved conditioner to store parameters that allows some distributions to be in exponential family. For example Laplace distribution is not in EF if we consider it as a distribution over 2 parameters. But if we fix the location parameter then it is in exponential family. The dimensions are required when mapping from flattened vectors. So we could view it as some sort of a conditioner fixing the dimensions of the two covariances of a matrix normal (not ideal or intended usage but can work). I think distribution attributes could be a good place to add it, as well (better from a semantic perspective as the dimensions do not decide whether matrix normal belongs to ef or not.). Trying the two options and choosing the one that yields least amount of code/effort to create the distribution/convert/etc. can be kept. That being said, I am inclining towards putting it to attributes.

[bvdmitri]

Putting this into attributes might be a bit more work than you think. We probably should have implemented it differently, but the current assumption is that the attributes are either all nothing (so the entire attributes is set to nothing) or you must set all the fields of the attributes, which means it will ignore all the functions like getlogpartition(::Type{MatrixNormal}, ...) and instead look them at the attributes, e.g. attributes.logpartition. This is not a great design choice in retrospective. Still, saving them at attributes is probably better, I can help you to adjust the code and make it work

[wmkouw]

I think I have a working solution now, involving attributes and dispatching on Tuple-type dimensions. I'm going through unit tests now, and will hopefully push a PR this weekend or Monday.