Skip to content

Tf2 #155

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rnpatien wants to merge 24 commits into
base: master
Choose a base branch
from
Open

Tf2 #155

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
c9e3254
Change for NNCad v1
rnpatien Feb 4, 2021
f4348bc
Yaml ignore
rnpatien Feb 8, 2021
44deacd
hrnet4t6 hrnet resblock
rnpatien Feb 11, 2021
37aef0d
Merge branch 'master' of https://github.com/rnpatien/O-CNN
rnpatien Feb 11, 2021
5232876
Merge remote-tracking branch 'upstream/master'
rnpatien Feb 22, 2021
3ce0ea1
lv6 testing
rnpatien Mar 1, 2021
30e4954
Merge branch 'master' of https://github.com/rnpatien/O-CNN-rp
rnpatien Mar 1, 2021
4e0a6d4
unet fix
rnpatien Mar 13, 2021
02c11d9
TestInfo tidy up
rnpatien Mar 13, 2021
25f9e2d
ignore tidy
rnpatien Mar 14, 2021
65c70f8
unet for lv 6 7
rnpatien Mar 19, 2021
c85b31f
Merge branch 'master' of https://github.com/rnpatien/O-CNN-rp
rnpatien Mar 19, 2021
7613f67
aenet fixes
rnpatien Mar 19, 2021
986fe09
Merge branch 'master' of https://github.com/rnpatien/O-CNN-rp
rnpatien Mar 19, 2021
b370137
LR to 8, infoGain added
rnpatien Apr 1, 2021
0ed2494
Merge branch 'master' of https://github.com/rnpatien/O-CNN-rp
rnpatien Apr 1, 2021
1fe5137
hrnet dropout failed
rnpatien Apr 23, 2021
3712f8f
First ae for deep fakes
May 1, 2021
0160d94
auto 20
rnpatien Jul 10, 2021
201d509
work TF2
rnpatien Jul 21, 2021
a27d183
get_variable tf2 type check
rnpatien Jul 23, 2021
474403d
ignore nncadTF2
rnpatien Sep 4, 2021
c2f0186
Update build.py
rnpatien Sep 15, 2021
3887125
test update
rnpatien Jun 23, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,7 @@ __pycache__
*.png
*.fig
log
ae_output
ae_output
*.bak
*.yaml
nncadTF2/
52 changes: 52 additions & 0 deletions tensorflow/data_report.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
TensorFlow 2.0 Upgrade Script
-----------------------------
Converted 5 files
Detected 0 issues that require attention
--------------------------------------------------------------------------------
================================================================================
Detailed log follows:

================================================================================
================================================================================
Input tree: 'data_14/'
================================================================================
--------------------------------------------------------------------------------
Processing file 'data_14/cls_modelnet.py'
outputting to 'data/cls_modelnet.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'data_14/completion.py'
outputting to 'data/completion.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'data_14/midnet_data.py'
outputting to 'data/midnet_data.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'data_14/seg_partnet.py'
outputting to 'data/seg_partnet.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'data_14/seg_shapenet.py'
outputting to 'data/seg_shapenet.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

120 changes: 60 additions & 60 deletions tensorflow/libs/__init__.py
View file
Open in desktop

Large diffs are not rendered by default.

8 changes: 5 additions & 3 deletions tensorflow/libs/build.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,14 @@
import tensorflow as tf
import subprocess
import argparse
print(os.getcwd())
#os.chdir("libs")

parser = argparse.ArgumentParser()
parser.add_argument("--octree", type=str, required=False,
default='../../octree')
parser.add_argument("--cuda", type=str, required=False,
default='/usr/local/cuda-10.1')
default='/usr/local/cuda-11.4')
parser.add_argument('--key64', type=str, required=False,
default='false')
parser.add_argument('--cc', type=str, required=False,
Expand Down Expand Up @@ -156,11 +158,11 @@
# make
if os.path.exists("object"):
os.system("rm -r object")
os.mkdir("object")
os.mkdir("object")
os.system("make -j all")

# test
os.chdir(LIBS_DIR + "/../test")
# env variable
cmd = 'export OCTREE_KEY=' + ('64' if KEY64 == '-DKEY64' else '32')
os.system(cmd + " && python test_all.py")
os.system(cmd + " && python test_all.py")
2 changes: 1 addition & 1 deletion tensorflow/libs/octree_batch_op.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ class OctreeBatchOp : public OpKernel {
void Compute(OpKernelContext* context) override {
// input octrees
const Tensor& data_in = context->input(0);
auto octree_buffer = data_in.flat<string>();
auto octree_buffer = data_in.flat<tstring>();
// int batch_size = data_in.shape().dim_size(0);
int batch_size = data_in.shape().num_elements();
vector<const char*> octrees_in;
Expand Down
10 changes: 7 additions & 3 deletions tensorflow/libs/octree_samples.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,16 @@ class OctreeSamplesOp : public OpKernel {
OP_REQUIRES_OK(context, context->allocate_output(0, names.shape(), &octrees));

for (int i = 0; i < num; ++i) {
string name = names.flat<string>()(i);
string& oct = octrees->flat<string>()(i);
string name = names.flat<tstring>()(i);

size_t size = 0;
const char* str = (const char*)octree::get_one_octree(name.c_str(), &size);
oct.assign(str, str + size);

// string& oct = (string&)octrees->flat<tstring>()(i);
// oct.assign(str, str + size);

tstring& oct = octrees->flat<tstring>()(i);
oct.assign(str, size); //.assign(str, str + size);
}
}
};
Expand Down
6 changes: 3 additions & 3 deletions tensorflow/libs/points2octree_op.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ class PointsToOctreeOp : public OpKernel {

// init the points
Points point_cloud_;
point_cloud_.set(data_in.flat<string>()(0).data());
point_cloud_.set(data_in.flat<tstring>()(0).data());

// check the points
string msg;
Expand All @@ -75,8 +75,8 @@ class PointsToOctreeOp : public OpKernel {
// output
Tensor* out_data = nullptr;
OP_REQUIRES_OK(context, context->allocate_output(0, data_in.shape(), &out_data));
string& out_str = out_data->flat<string>()(0);
out_str.assign(octree_buf.begin(), octree_buf.end());
tstring& out_str = out_data->flat<tstring>()(0);
out_str.assign(&octree_buf[0], octree_buf.size());
}

private:
Expand Down
2 changes: 1 addition & 1 deletion tensorflow/libs/points_property_op.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ class PointsPropertyOp : public OpKernel {
void Compute(OpKernelContext* context) override {
// input points
const Tensor& data_in = context->input(0);
auto points_buffer = data_in.flat<string>();
auto points_buffer = data_in.flat<tstring>();
int batch_size = data_in.NumElements();
vector<PointsParser> points_in(batch_size);
for (int i = 0; i < batch_size; ++i) {
Expand Down
10 changes: 5 additions & 5 deletions tensorflow/libs/points_set_property_op.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,9 +57,9 @@ class PointsNewOp : public OpKernel {
// output
Tensor* tsr = nullptr;
OP_REQUIRES_OK(context, context->allocate_output(0, TensorShape{1}, &tsr));
string& out_str = tsr->flat<string>()(0);
tstring& out_str = tsr->flat<tstring>()(0);
const vector<char>& points_buf = point_cloud.get_buffer();
out_str.assign(points_buf.begin(), points_buf.end());
out_str.assign(&points_buf[0], points_buf.size());
}
};

Expand All @@ -80,7 +80,7 @@ class PointsSetPropertyOp : public OpKernel {
int channel = data.dim_size(1);

// copy the data out of the input tensor
auto points_array = data_in.flat<string>();
auto points_array = data_in.flat<tstring>();
vector<char> points_buf(points_array(0).begin(), points_array(0).end());

// init the points
Expand Down Expand Up @@ -117,8 +117,8 @@ class PointsSetPropertyOp : public OpKernel {
Tensor* out_data = nullptr;
const TensorShape& shape = data_in.shape();
OP_REQUIRES_OK(context, context->allocate_output(0, shape, &out_data));
string& out_str = out_data->flat<string>()(0);
out_str.assign(points_buf.begin(), points_buf.end());
tstring& out_str = out_data->flat<tstring>()(0);
out_str.assign(&points_buf[0], points_buf.size());
}

private:
Expand Down
16 changes: 8 additions & 8 deletions tensorflow/libs/transform_octree_op.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ class OctreeDropOp : public OpKernel {
void Compute(OpKernelContext* context) override {
// input
const Tensor& data_in = context->input(0);
const string& octree_in = data_in.flat<string>()(0);
const string& octree_in = data_in.flat<tstring>()(0);
int depth = context->input(1).flat<int>()(0);
float ratio = context->input(2).flat<float>()(0);

Expand All @@ -55,8 +55,8 @@ class OctreeDropOp : public OpKernel {
Tensor* data_out = nullptr;
OP_REQUIRES_OK(context,
context->allocate_output(0, data_in.shape(), &data_out));
string& str_out = data_out->flat<string>()(0);
str_out.assign(octree_out.begin(), octree_out.end());
tstring& str_out = data_out->flat<tstring>()(0);
str_out.assign(&octree_out[0], octree_out.size());
}
};

Expand All @@ -70,7 +70,7 @@ class OctreeScanOp : public OpKernel {
// input
OctreeParser octree_in;
const Tensor& data_in = context->input(0);
octree_in.set_cpu(data_in.flat<string>()(0).data());
octree_in.set_cpu(data_in.flat<tstring>()(0).data());

const Tensor& axis_in = context->input(1);
auto ptr_in = axis_in.flat<float>().data();
Expand All @@ -84,8 +84,8 @@ class OctreeScanOp : public OpKernel {
Tensor* data_out = nullptr;
OP_REQUIRES_OK(context,
context->allocate_output(0, data_in.shape(), &data_out));
string& str_out = data_out->flat<string>()(0);
str_out.assign(octree_out.begin(), octree_out.end());
tstring& str_out = data_out->flat<tstring>()(0);
str_out.assign(&octree_out[0], octree_out.size());
}

protected:
Expand All @@ -104,8 +104,8 @@ class OctreeCastOp : public OpKernel {
// output
Tensor* data_out = nullptr;
OP_REQUIRES_OK(context, context->allocate_output(0, {1}, &data_out));
string& str_out = data_out->flat<string>()(0);
str_out.assign(ptr_in, ptr_in + data_in.NumElements());
tstring& str_out = data_out->flat<tstring>()(0);
str_out.assign(ptr_in, data_in.NumElements());
}
};

Expand Down
14 changes: 7 additions & 7 deletions tensorflow/libs/transform_points_op.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ class TransformPointsOp : public OpKernel {
}

// copy the data out of the input tensor
auto points_array = data_in.flat<string>();
auto points_array = data_in.flat<tstring>();
vector<char> points_buf(points_array(0).begin(), points_array(0).end());

// init the points
Expand Down Expand Up @@ -178,8 +178,8 @@ class TransformPointsOp : public OpKernel {
Tensor* out_data = nullptr;
const TensorShape& shape = data_in.shape();
OP_REQUIRES_OK(context, context->allocate_output(0, shape, &out_data));
string& out_str = out_data->flat<string>()(0);
out_str.assign(points_buf.begin(), points_buf.end());
tstring& out_str = out_data->flat<tstring>()(0);
out_str.assign(&points_buf[0], points_buf.size());
}

private:
Expand All @@ -195,7 +195,7 @@ class NormalizePointsOp : public OpKernel {

void Compute(OpKernelContext* context) override {
// input
const string& data_in = context->input(0).flat<string>()(0);
const string& data_in = context->input(0).flat<tstring>()(0);
const float radius = context->input(1).flat<float>()(0);
const float* center = context->input(2).flat<float>().data();

Expand Down Expand Up @@ -234,8 +234,8 @@ class NormalizePointsOp : public OpKernel {
Tensor* out_data = nullptr;
const TensorShape& shape = context->input(0).shape();
OP_REQUIRES_OK(context, context->allocate_output(0, shape, &out_data));
string& out_str = out_data->flat<string>()(0);
out_str.assign(points_buf.begin(), points_buf.end());
tstring& out_str = out_data->flat<tstring>()(0);
out_str.assign(&points_buf[0], points_buf.size());
}
};

Expand All @@ -251,7 +251,7 @@ class BoundingSphereOp : public OpKernel {

// init the points
Points pts;
pts.set(data_in.flat<string>()(0).data());
pts.set(data_in.flat<tstring>()(0).data());

// check the points
string msg;
Expand Down
113 changes: 113 additions & 0 deletions tensorflow/libs_report.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
TensorFlow 2.0 Upgrade Script
-----------------------------
Converted 2 files
Detected 4 issues that require attention
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
File: libs_14/__init__.py
--------------------------------------------------------------------------------
libs_14/__init__.py:202:13: WARNING: tf.get_variable requires manual check. tf.get_variable returns ResourceVariables by default in 2.0, which have well-defined semantics and are stricter about shapes. You can disable this behavior by passing use_resource=False, or by calling tf.compat.v1.disable_resource_variables().
libs_14/__init__.py:220:13: WARNING: tf.get_variable requires manual check. tf.get_variable returns ResourceVariables by default in 2.0, which have well-defined semantics and are stricter about shapes. You can disable this behavior by passing use_resource=False, or by calling tf.compat.v1.disable_resource_variables().
libs_14/__init__.py:236:13: WARNING: tf.get_variable requires manual check. tf.get_variable returns ResourceVariables by default in 2.0, which have well-defined semantics and are stricter about shapes. You can disable this behavior by passing use_resource=False, or by calling tf.compat.v1.disable_resource_variables().
libs_14/__init__.py:256:13: WARNING: tf.get_variable requires manual check. tf.get_variable returns ResourceVariables by default in 2.0, which have well-defined semantics and are stricter about shapes. You can disable this behavior by passing use_resource=False, or by calling tf.compat.v1.disable_resource_variables().
================================================================================
Detailed log follows:

================================================================================
================================================================================
Input tree: 'libs_14/'
================================================================================
--------------------------------------------------------------------------------
Processing file 'libs_14/build.py'
outputting to 'libs/build.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'libs_14/__init__.py'
outputting to 'libs/__init__.py'
--------------------------------------------------------------------------------

167:10: INFO: Added keywords to args of function 'tf.shape'
173:7: INFO: Renamed 'tf.variable_scope' to 'tf.compat.v1.variable_scope'
180:7: INFO: Renamed 'tf.variable_scope' to 'tf.compat.v1.variable_scope'
187:7: INFO: Renamed 'tf.variable_scope' to 'tf.compat.v1.variable_scope'
189:11: INFO: Added keywords to args of function 'tf.reduce_mean'
200:7: INFO: Renamed 'tf.variable_scope' to 'tf.compat.v1.variable_scope'
202:13: WARNING: tf.get_variable requires manual check. tf.get_variable returns ResourceVariables by default in 2.0, which have well-defined semantics and are stricter about shapes. You can disable this behavior by passing use_resource=False, or by calling tf.compat.v1.disable_resource_variables().
202:13: INFO: Renamed 'tf.get_variable' to 'tf.compat.v1.get_variable'
203:41: INFO: Changing tf.contrib.layers xavier initializer to a tf.compat.v1.keras.initializers.VarianceScaling and converting arguments.

218:7: INFO: Renamed 'tf.variable_scope' to 'tf.compat.v1.variable_scope'
220:13: WARNING: tf.get_variable requires manual check. tf.get_variable returns ResourceVariables by default in 2.0, which have well-defined semantics and are stricter about shapes. You can disable this behavior by passing use_resource=False, or by calling tf.compat.v1.disable_resource_variables().
220:13: INFO: Renamed 'tf.get_variable' to 'tf.compat.v1.get_variable'
221:41: INFO: Changing tf.contrib.layers xavier initializer to a tf.compat.v1.keras.initializers.VarianceScaling and converting arguments.

233:7: INFO: Renamed 'tf.variable_scope' to 'tf.compat.v1.variable_scope'
236:13: WARNING: tf.get_variable requires manual check. tf.get_variable returns ResourceVariables by default in 2.0, which have well-defined semantics and are stricter about shapes. You can disable this behavior by passing use_resource=False, or by calling tf.compat.v1.disable_resource_variables().
236:13: INFO: Renamed 'tf.get_variable' to 'tf.compat.v1.get_variable'
237:41: INFO: Changing tf.contrib.layers xavier initializer to a tf.compat.v1.keras.initializers.VarianceScaling and converting arguments.

253:7: INFO: Renamed 'tf.variable_scope' to 'tf.compat.v1.variable_scope'
256:13: WARNING: tf.get_variable requires manual check. tf.get_variable returns ResourceVariables by default in 2.0, which have well-defined semantics and are stricter about shapes. You can disable this behavior by passing use_resource=False, or by calling tf.compat.v1.disable_resource_variables().
256:13: INFO: Renamed 'tf.get_variable' to 'tf.compat.v1.get_variable'
257:41: INFO: Changing tf.contrib.layers xavier initializer to a tf.compat.v1.keras.initializers.VarianceScaling and converting arguments.

267:7: INFO: Renamed 'tf.variable_scope' to 'tf.compat.v1.variable_scope'
269:11: INFO: Added keywords to args of function 'tf.transpose'
274:7: INFO: Renamed 'tf.variable_scope' to 'tf.compat.v1.variable_scope'
283:7: INFO: Renamed 'tf.variable_scope' to 'tf.compat.v1.variable_scope'
288:11: INFO: Added keywords to args of function 'tf.transpose'
294:7: INFO: Renamed 'tf.variable_scope' to 'tf.compat.v1.variable_scope'
299:11: INFO: Added keywords to args of function 'tf.transpose'
300:23: INFO: Added keywords to args of function 'tf.shape'
300:43: INFO: Added keywords to args of function 'tf.shape'
301:23: INFO: Added keywords to args of function 'tf.shape'
304:22: INFO: Added keywords to args of function 'tf.reduce_prod'
307:13: INFO: Renamed 'tf.div' to 'tf.compat.v1.div'
308:43: INFO: Added keywords to args of function 'tf.transpose'
315:7: INFO: Renamed 'tf.variable_scope' to 'tf.compat.v1.variable_scope'
325:11: INFO: Added keywords to args of function 'tf.transpose'
326:23: INFO: Added keywords to args of function 'tf.shape'
326:42: INFO: Added keywords to args of function 'tf.shape'
327:23: INFO: Added keywords to args of function 'tf.shape'
336:22: INFO: Added keywords to args of function 'tf.reduce_prod'
339:13: INFO: Renamed 'tf.div' to 'tf.compat.v1.div'
341:43: INFO: Added keywords to args of function 'tf.transpose'
342:41: INFO: Added keywords to args of function 'tf.transpose'
349:7: INFO: Renamed 'tf.variable_scope' to 'tf.compat.v1.variable_scope'
359:26: INFO: Added keywords to args of function 'tf.shape'
359:45: INFO: Added keywords to args of function 'tf.shape'
360:23: INFO: Added keywords to args of function 'tf.shape'
370:22: INFO: Added keywords to args of function 'tf.reduce_prod'
374:13: INFO: Renamed 'tf.div' to 'tf.compat.v1.div'
382:7: INFO: Renamed 'tf.variable_scope' to 'tf.compat.v1.variable_scope'
409:10: INFO: Added keywords to args of function 'tf.boolean_mask'
411:10: INFO: Added keywords to args of function 'tf.shape'
414:10: INFO: Added keywords to args of function 'tf.boolean_mask'
417:31: INFO: Added keywords to args of function 'tf.reduce_prod'
418:13: INFO: Added keywords to args of function 'tf.boolean_mask'
423:8: INFO: Added keywords to args of function 'tf.shape'
446:13: INFO: Renamed 'tf.div' to 'tf.compat.v1.div'
447:43: INFO: Added keywords to args of function 'tf.transpose'
452:7: INFO: `name` passed to `name_scope`. Because you may be re-entering an existing scope, it is not safe to convert automatically, the v2 name_scope does not support re-entering scopes by name.

452:7: INFO: Renamed 'tf.name_scope' to 'tf.compat.v1.name_scope'
457:12: INFO: Added keywords to args of function 'tf.boolean_mask'
472:7: INFO: Renamed 'tf.variable_scope' to 'tf.compat.v1.variable_scope'
491:7: INFO: `name` passed to `name_scope`. Because you may be re-entering an existing scope, it is not safe to convert automatically, the v2 name_scope does not support re-entering scopes by name.

491:7: INFO: Renamed 'tf.name_scope' to 'tf.compat.v1.name_scope'
499:7: INFO: `name` passed to `name_scope`. Because you may be re-entering an existing scope, it is not safe to convert automatically, the v2 name_scope does not support re-entering scopes by name.

499:7: INFO: Renamed 'tf.name_scope' to 'tf.compat.v1.name_scope'
509:7: INFO: `name` passed to `name_scope`. Because you may be re-entering an existing scope, it is not safe to convert automatically, the v2 name_scope does not support re-entering scopes by name.

509:7: INFO: Renamed 'tf.name_scope' to 'tf.compat.v1.name_scope'
517:7: INFO: `name` passed to `name_scope`. Because you may be re-entering an existing scope, it is not safe to convert automatically, the v2 name_scope does not support re-entering scopes by name.

517:7: INFO: Renamed 'tf.name_scope' to 'tf.compat.v1.name_scope'
--------------------------------------------------------------------------------

Loading