Remove all global variables in Paddle

[reyoung]

This issue used to record which variables are removed and will be removed. Related issue #849

• FLAGS_cudnn_dir in hl_dso_loader.cc @gangliao
• FLAGS_cuda_dir in hl_dso_loader.cc @gangliao
• FLAGS_warpctc_dir in hl_dso_loader.cc @gangliao
• FLAGS_memory_threshold_on_load_data
  • ProtoDataProvider.cpp, Should we just removed ProtoDataProvider? because it is not useful. Or just give a deprecated mark.
• FLAGS_allow_only_one_model_on_one_gpu in MultiGradientMachine.cpp maybe it should be in ModelConfig
• diy_beam_search_prob_so in RecurrentGradientMachine.cpp
• log_error_clipping in Layer.cpp
• rnn_use_batch in RecurrentLayer.cpp
• allow_inefficient_sparse_update in SparseRowMatrix.cpp remove allow_inefficient_sparse_update since it's only defined but never used. #1238
• pool_limit_size in Storage.cpp
• log_clipping in FirstOrderOptimizer.cpp
• enable_grad_share in Parameter.cpp
• grad_share_block_num in Parameter.cpp
• small_message in LightNetwork.cpp
• sock_send_buf_size in LightNetwork.cpp
• sock_recv_buf_size in LightNetwork.cpp
• pservers in ParameterClient2.cpp
• parallel_thread_num in ParameterClient2.cpp
• pserver_num_threads in ParameterServer2.cpp
• async_lagged_ratio_min in ParameterServer2.cpp
• async_lagged_ratio_default in ParameterServer2.cpp
• Flags in SparseParameterDistribution
  • It seems that they are not very useful
• Flags.h

DECLARE_bool(parallel_nn);
DECLARE_int32(async_count);
DECLARE_int32(port);
DECLARE_int32(data_server_port);
DECLARE_bool(use_gpu);
DECLARE_int32(gpu_id);
DECLARE_int32(trainer_count);
DECLARE_int32(ports_num);
DECLARE_int32(ports_num_for_sparse);
DECLARE_string(nics);
DECLARE_string(rdma_tcp);
DECLARE_int32(trainer_id);
DECLARE_int32(num_gradient_servers);
DECLARE_string(comment);
DECLARE_string(load_missing_parameter_strategy);
DECLARE_int32(log_period);
DECLARE_int32(log_period_server);
DECLARE_double(checkgrad_eps);
DECLARE_int32(enable_parallel_vector);
DECLARE_bool(loadsave_parameters_in_pserver);
DECLARE_int32(beam_size);
DECLARE_bool(show_layer_stat);
DECLARE_string(predict_file);
DECLARE_bool(prev_batch_state);
DECLARE_string(init_model_path);

I just go through Paddle code quickly, it seems the Flags.h should be our first thing to removed.

Or maybe we should think a strategy about how to remove them.

• 我们是不是应该在我们的Protobuf里面增加一些字段，这些字段就是这些flags，然后在main里面把这些flags直接set到protobuf里面。
• 亦或是，我们先一个一个删掉一些flags？

[jacquesqiao]

There are two kind of global variable in Paddle:
1. gflags in paddle cpp code to set config value.
2. global var in python code to construct neural network.
Both of them should be removed, so is this issue focus on the global variable in Paddle cpp src code or both?

[wangkuiyi]

I see @reyoung added an item FLAGS_trainer_count in above list, so I guess @reyoung 's proposal might be that this issue works as a list of subtasks in both C++ and Python codebase? If so, we should create sub-issues and file PRs to fix them?

[reyoung]

I see @reyoung added an item FLAGS_trainer_count in above list, so I guess @reyoung 's proposal might be that this issue works as a list of subtasks in both C++ and Python codebase? If so, we should create sub-issues and file PRs to fix them?

Yes. This issue is a list of subtasks. I am finding all global variables in Paddle and trying to give a list on this issue.

Python code's global variables are currently harmless, we could focus in Paddle Cpp Part now.

[jacquesqiao]

i have already removed some flags in parameter server in #1051 , change some of them to proto config, some of them are in the list above. But for Compatibility i keep the old way to set flag. you can see if this way is properly.

[reyoung]

We are refactoring Paddle, and do not allow global variables expect registries.