Skip to content

Difficulty Reproducing ETTh1 Performance - Parameter Verification Request #2

Description

@Yoontae6719

Hi,

I'm experiencing a significant discrepancy between the performance reported in the paper and my own experimental results.

I noticed that if I run the sh scripts/etth1.sh script provided in the GitHub repository, the parameters used are quite different from those described in the paper, which leads to a major difference in performance.

Therefore, I tried to set the parameters exactly as specified in the paper for the ETTh1 dataset. Here is the configuration I am using:

if [ ! -d "./logs" ]; then
    mkdir ./logs
fi

if [ ! -d "./logs/LongForecasting" ]; then
    mkdir ./logs/LongForecasting
fi

model_name=EntroPE
root_path_name=./dataset/
entropy_model_checkpoint_dir=./entropy_model_checkpoints/
data_path_name=ETTh1.csv
model_id_name=ETTh1
data_name=ETTh1
enc_in=7
seq_len=96

quant_range=4
dim=8
multiple_of=128
heads=2
layers=1
batch_size=64
learning_rate=0.001
dropout=0.05
monotonicity=1
patching_threshold=3
patching_threshold_add=0.25
max_patch_length=24
train_epochs=20
patience=7


for random_seed in 2025 
do
    for pred_len in 192 336 720
    do
        python -u run_longExp_gpu1.py \
        --random_seed $random_seed \
        --is_training 1 \
        --root_path $root_path_name \
        --entropy_model_checkpoint_dir $entropy_model_checkpoint_dir \
        --data_path $data_path_name \
        --model_id $model_id_name_$seq_len'_'$pred_len \
        --model_id_name $model_id_name \
        --model $model_name \
        --data $data_name \
        --features M \
        --seq_len $seq_len \
        --pred_len $pred_len \
        --enc_in $enc_in \
        --vocab_size 256 \
        --quant_range $quant_range \
        --n_layers_local_encoder $layers \
        --n_layers_local_decoder $layers \
        --n_layers_global $layers \
        --dim_global $dim \
        --dim_local_encoder $dim \
        --dim_local_decoder $dim \
        --n_heads_local_encoder $heads \
        --n_heads_local_decoder $heads \
        --n_heads_global $heads \
        --cross_attn_nheads $heads \
        --dropout $dropout \
        --multiple_of $multiple_of\
        --max_patch_length $max_patch_length\
        --patching_threshold $patching_threshold \
        --patching_threshold_add $patching_threshold_add \
        --monotonicity $monotonicity \
        --des 'Exp' \
        --train_epochs $train_epochs \
        --patience $patience \
        --lradj 'TST'\
        --pct_start 0.3\
        --batch_size $batch_size \
        --patching_batch_size $((batch_size * enc_in)) \
        --learning_rate $learning_rate \
        >logs/LongForecasting/$model_name'_'$model_id_name'_'$seq_len'_'$pred_len.log 
    done
done

Could you please confirm if these are the correct settings? If there have been any updates or changes, I would greatly appreciate it if you could provide the exact parameter configuration required to reproduce the results reported in the paper.

Thank you

Image Image Image

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions