Skip to content

BERT Language Models: Training model metas no longer supported with TF2.x #4

@jthestness

Description

@jthestness

When migrating to TF2.x support, we're no longer able to load BERT training model metas. Caused by ops that have been removed from TF:

$ python catamount/tests/full/tf_bert_language_models.py --model_name uncased_L-3_H-768_A-12.training
WARN: Cannot import tensorflow.contrib.mpi_collectives... not built?
WARN: Cannot import tensorswift... not installed?
Traceback (most recent call last):
  File "catamount/tests/full/tf_bert_language_models.py", line 752, in <module>
    run_tf_bert_lm(model_name=args.model_name)
  File "catamount/tests/full/tf_bert_language_models.py", line 42, in run_tf_bert_lm
    graph = catamount.frameworks.tensorflow.import_graph(graph_meta)
  File "build/bdist.linux-x86_64/egg/catamount/frameworks/tensorflow.py", line 287, in import_graph
  File "build/bdist.linux-x86_64/egg/catamount/frameworks/tensorflow.py", line 276, in load_tf_session
  File "/net/joel-dev/srv/nfs/joel-data/ws/local/envs/py3.7.4_tf2.2.2/lib/python3.7/site-packages/tensorflow/python/training/saver.py", line 1462, in import_meta_graph
    **kwargs)[0]
  File "/net/joel-dev/srv/nfs/joel-data/ws/local/envs/py3.7.4_tf2.2.2/lib/python3.7/site-packages/tensorflow/python/training/saver.py", line 1486, in _import_meta_graph_with_return_elements
    **kwargs))
  File "/net/joel-dev/srv/nfs/joel-data/ws/local/envs/py3.7.4_tf2.2.2/lib/python3.7/site-packages/tensorflow/python/framework/meta_graph.py", line 799, in import_scoped_meta_graph_with_return_elements
    return_elements=return_elements)
  File "/net/joel-dev/srv/nfs/joel-data/ws/local/envs/py3.7.4_tf2.2.2/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/net/joel-dev/srv/nfs/joel-data/ws/local/envs/py3.7.4_tf2.2.2/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 405, in import_graph_def
    producer_op_list=producer_op_list)
  File "/net/joel-dev/srv/nfs/joel-data/ws/local/envs/py3.7.4_tf2.2.2/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 497, in _import_graph_def_internal
    graph._c_graph, serialized, options)  # pylint: disable=protected-access
tensorflow.python.framework.errors_impl.NotFoundError: Op type not registered 'MapAndBatchDatasetV2' in binary running on joel-dev. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) `tf.contrib.resampler` should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed.

Decide whether to update, deprecate, or otherwise fix.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions