Skip to content

[Code scan] Do not require save_path for in-memory target scaling #39

Description

@njzjz

This issue is a result of a Codex global repository scan.

Summary

TargetScaler.fit always removes, creates, and writes target_scaler.ss under dump_dir after fitting. DataHub accepts save_path=None, so direct regression or multilabel regression use with target_normalize enabled can fail even though the fitted scaler could be kept in memory.

Code references

try:
os.remove(os.path.join(dump_dir, 'target_scaler.ss'))
except:
pass
os.makedirs(dump_dir, exist_ok=True)
joblib.dump(self.scaler, os.path.join(dump_dir, 'target_scaler.ss'))

self.data['target_scaler'] = TargetScaler(
self.ss_method, self.task, self.save_path
)
if self.task == 'regression':
target = np.array(self.data['raw_target']).reshape(-1, 1).astype(np.float32)
if self.is_train:
self.data['target_scaler'].fit(target, self.save_path)
self.data['target'] = self.data['target_scaler'].transform(target)

Impact

Library users who construct DataHub directly, or future call paths that do not provide save_path, can crash during scaler fitting despite not needing scaler persistence on disk.

Suggested fix

Only persist the scaler when dump_dir is not None. If dump_dir is None, keep self.scaler in memory and skip file IO.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions