Sparktf - example replication

[lucabarbaglia]

I am trying to replicate the example for the sparktf package proposed on the associated GitHub page:
https://github.com/rstudio/sparktf

When fitting the model, I obtain the following error:

history <- model %>% fit(dataset, epochs = 100, verbose = 0)
2019-05-24 17:19:09.373716: W tensorflow/core/framework/op_kernel.cc:1401] OP_REQUIRES failed at strided_slice_op.cc:106 : Invalid argument: Index out of range using input dim 2; input has only 2 dims

@kevinykuo do you have a clue on what's causing this issue?
Thanks

[kevinykuo]

Are you using the latest versions of tfdatasets?

@dfalbel does this error ring a bell?

[lucabarbaglia]

@kevinykuo Yes, I have the tfdatasets 1.13.1 version, which is the latest one available

[dfalbel]

@kevinykuo Nothing comes to mind..

@lucabarbaglia what's your TensorFlow version? You can get that with tensorflow::tf_config(). It could also be the Keras version. Can you try installing Keras from Github?

[lucabarbaglia]

@dfalbel I'm running TensorFlow v1.13.1

tensorflow::tf_config()
TensorFlow v1.13.1
Python v3.7

I have installed Keras (version 2.2.4.1) from GitHub using devtools::install_github("rstudio/keras"), but I still get the same error

[dfalbel]

I can reproduce the issue here, but adding the steps_per_epoch = 10 argument to fit will fix the problem. @kevinykuo Do you think this is expected?

[kevinykuo]

@dfalbel I'll update the README since it looks like there's been a change to tf.data. Does fit() have no visibility into when a dataset runs out of records for each pass through the data? I wonder how it worked before.