strict synchronization check on multivariate operation ema_cov

[bournejt]

I noticed that ema_cov will fail when you delay one of the variables (cpp link). I created a self-contained example to reproduce this issue. Is this something we can relax, or we can have a workaround? I think this use case can be very meaningful, and should work theoretically. I have created a sync function on my side to hack through this for now, but it's quite ugly.

#!/usr/bin/env python3
"""
CSP Multivariate Statistics Synchronization Issue Demo

This demonstrates why csp.delay breaks multivariate statistics.
"""

import csp
import numpy as np
from datetime import datetime, timedelta

@csp.graph
def test_normal_ema_cov():
    # Create test data - 10 data points for each time series
    x_data = [
        (timedelta(seconds=i), np.array([i * 0.1, i * 0.2, i * 0.3]))
        for i in range(10)
    ]
    
    y_data = [
        (timedelta(seconds=i), np.array([i * 0.05, i * 0.15, i * 0.25]))
        for i in range(10)
    ]
    
    x = csp.curve(typ=np.ndarray, data=x_data)
    y = csp.curve(typ=np.ndarray, data=y_data)
    
    # Normal ema_cov (works fine)
    normal_cov = csp.stats.ema_cov(x, y, alpha=0.1)
    csp.print("Normal ema_cov:", normal_cov)

@csp.graph
def test_delayed_ema_cov():
    # Create test data - 10 data points for each time series
    x_data = [
        (timedelta(seconds=i), np.array([i * 0.1, i * 0.2, i * 0.3]))
        for i in range(10)
    ]
    
    y_data = [
        (timedelta(seconds=i), np.array([i * 0.05, i * 0.15, i * 0.25]))
        for i in range(10)
    ]
    
    x = csp.curve(typ=np.ndarray, data=x_data)
    y = csp.curve(typ=np.ndarray, data=y_data)
    
    # ema_cov with delay (fails)
    y_delayed = csp.delay(y, delay=1)
    delayed_cov = csp.stats.ema_cov(x, y_delayed, alpha=0.1)
    csp.print("Delayed ema_cov:", delayed_cov)

def main():
    start_time = datetime(2023, 1, 1)
    end_time = start_time + timedelta(seconds=10)
    
    # Test 1: Normal ema_cov
    print("Test 1: Normal ema_cov")
    csp.run(test_normal_ema_cov, starttime=start_time, endtime=end_time)
    
    # Test 2: ema_cov with delay
    print("\nTest 2: ema_cov with delay")
    try:
        csp.run(test_delayed_ema_cov, starttime=start_time, endtime=end_time)
    except ValueError as e:
        if "must tick in sequence" in str(e):
            print(f"ERROR: {e}")
        else:
            raise

if __name__ == "__main__":
    main() 

[AdamGlustein]

An assumption for any multivariate statistic in CSP is that all variables must tick together. I could see an argument for an option to these calculations that allows (and discards) out-of-sync ticks, but it would have to be set explicitly by the user. If out-of-sync data is dropped silently, the statistic may be using a much smaller sample size than you think it is, which is dangerous - so users would have to opt-in to this behaviour knowing its expected, like in your case where one time series is delayed.

A general work around is to sample the first argument with the delayed series, which I think will be cleaner than creating your own sync.

# ema_cov with delay (fails)
y_delayed = csp.delay(y, delay=1)
delayed_cov = csp.stats.ema_cov(csp.sample(y_delayed, x), y_delayed, alpha=0.1) # this is the only change
csp.print("Delayed ema_cov:", delayed_cov)

[bournejt]

Thanks for the direction Adam. That really worked. I think dropping non-overlapping time index out of covariance calculation is the default behavior of pandas? In the current design, the code simply cannot run if there's a mismatch in ticked timestamp. Adding a kwarg to allow this seems to be a good solution. It's nice to embed data integrity check into the computation tool, but I personally think that's the duty of user.

[AdamGlustein]

Interesting, I tested out a simple pandas script and it will drop any non-overlapping data points between the two series like you mentioned. On one hand, I think that in all cases where x and y tick out of sync, the user can always choose the series they want to sample on (x or y) and sync the two arguments using the above approach. However, this gets complicated if you don't know whether x or y will tick first. It's also less intuitive than having a keyword argument to the csp.stats function.

I think it makes sense to add an allow_non_overlapping kwarg to these multivariate nodes which will be opt-in. Implementation wise it will be very simple, I am just always hesitant to add even more arguments to these stats functions since most already have a complex API. Can you go ahead and create an Issue to track this? Thanks for bringing up the discussion here.

[bournejt]

Yeah let me take a look at the issue writing practice here and create one for this. Glad to be able to contribute a little to this magnificent project:)

[bournejt]

New issue created here: #546