TM.anomaly: fix score for likelihood anomaly modes

[breznak]

we used to return likelihood (of the score), but for anomaly we must return 1 - likelihood

Fixes #665

TODO:

• optionally integrate sine wave tests for anomaly modes

[breznak]

I'd like to include the sine wave experiments before this is merged.

[breznak]

Please review, @Thanh-Binh does this resolve the issue in your tests?
I have a test-case comming in a separate PR.

[dkeeney]

I see no problem.
Thanks for the expanded explanations.

[breznak]

Thanks for reviewing!
I'm waiting a bit for @Thanh-Binh if he has time to confirm the seen issue is resolved, otherwise I'll just merge this soon.

[Thanh-Binh]

@breznak for log-likelihood, I think your proposal

      const Real like = tmAnomaly_.anomalyLikelihood_.anomalyProbability(raw);
      const Real log  = tmAnomaly_.anomalyLikelihood_.computeLogLikelihood(like);
      tmAnomaly_.anomaly_ = 1.0f - log;

is not correct.

I believe it should be :

      const Real like = tmAnomaly_.anomalyLikelihood_.anomalyProbability(raw);
      const Real log  = tmAnomaly_.anomalyLikelihood_.computeLogLikelihood(1.0 - like);
      tmAnomaly_.anomaly_ = log;

[breznak]

believe it should be :

this is one thing I'm considering: if we want to log anomaly (= 1-like) or the likelihoods (?)

does

const Real log = tmAnomaly_.anomalyLikelihood_.computeLogLikelihood(1.0 - like);
behave "correctly" like the plain likelihood (but anomaly) mode?

[breznak]

I'm calling hold-off on this PR, until this is properly explained and tested.

[breznak]

actually, I'm not sure this is correct at all.
Revisitiing anomaly likelihood in original Numenta's py code:

def anomalyProbability(self, value, anomalyScore, timestamp=None):
"""
Compute the probability that the current value plus anomaly score represents
an anomaly given the historical distribution of anomaly scores. The closer
the number is to 1, the higher the chance it is an anomaly.
...

And both codes (Numenta's py, and ours cpp) internally toggle likelihoods to anomaly before returning
likelihood = 1.0 - likelihoods[0], so doing another 1-anomalyProbability() switches back to likelihoods[0]

So, according to docs and logic of the code, the TM.anomaly should represent anomalyProbability() in AnomalyLikelihood.

@Thanh-Binh can you describe in detail your experiment and where you see errors? We must replicate it here, ideally even simpler, synthetic test for likelihood: anomaly scores high 1.0, 0.9, 0.98, 1.0, ... -> anomalyProbability() should drop. Same for low score: 0.0, 0.1, 0.05, 0.0,.. -> anProb() drops to 0.
For a sudden change (high anomalies -> low, and vice versa) the anProb() should be high.