H5Ldelete error

[jeremymiller]

Error in H5Ldelete(h5loc = loc$H5Identifier, name = name) : Specified link doesn't exist.

This error shows up constantly and prevents some functions from working when "overwrite = TRUE". Usually re-running the function after getting this error solves the problem, but I've run into a situation where I cannot update a variable at all once it is written without completely deleting the tome and restarting. If that comes up again, I'll update this issue with more specifics.

[hypercompetent]

Hi @jeremymiller , I think I have this fixed in #20 on the Dev branch. Was this mostly an issue with updating annotations, or did you run into this using a different write_tome function?

[jeremymiller]

@hypercompetent : I will rerun my code and let you know how this (and the speed of data subset loading) goes.

[jeremymiller]

@hypercompetent : UPDATE

1. If I overwrite the anno file with write_tome_anno(anno,tome) it works properly, except that the file size continues to grow. My best guess is it is saving multiple copies of anno rather than overwriting?

2. I've solved the error below. The problem is that if there are any NA values in geneInfo then write_tome_gene_meta throws an error. Here is how I solved it:

geneInfo <- geneInfo[match(read_tome_gene_names(tome),geneInfo$group_name)

# Need to convert factors to characters first (function for this?)
for (i in 1:dim(geneInfo)[2]) if (is.factor(geneInfo[,i]))
  geneInfo[,i] <- as.character(geneInfo[,i])

# Replace NAs with other values to avoid write_tome_gene_meta crashing.
whichNA <- which(is.na(geneInfo$gene_symbol))
for(i in 1:dim(geneInfo)[2]) 
  geneInfo[whichNA,i] = 0
geneInfo$gene_symbol <- read_tome_gene_names(tome)

This should probably be included into scrattch.io somehow.

---

It appears that this error is corrected in all cases except 1. When I run "write_tome_gene_meta" I get a bunch of errors (including the one above) and it doesn't work. All the intermediate errors appear to be new since a few weeks ago.

[1] "Removing existing /gene_meta/genes"
[1] "/gene_meta already exists. Skipping."
[1] "Creating Group /gene_meta/genes"
[1] "Writing /gene_meta/genes"
Error in H5Dcreate(loc$H5Identifier, dataset, tid, sid, dcpl = dcpl) : 
  HDF5. Dataset. Unable to initialize object.
Error in H5Dopen(h5loc, name) : HDF5. Dataset. Object not found.
Error in h5writeDatasetHelper(obj = obj, h5dataset = h5dataset, index = index,  : 
  object 'h5dataset' not found
restarting interrupted promise evaluationError in h5writeDatasetHelper(obj = obj, h5dataset = h5dataset, index = index,  : 
  object 'h5dataset' not found
Error in H5Dclose(h5dataset) : object 'h5dataset' not found
Error in H5Dcreate(loc$H5Identifier, dataset, tid, sid, dcpl = dcpl) : 
  HDF5. Dataset. Unable to initialize object.
Error in H5Dopen(h5loc, name) : HDF5. Dataset. Object not found.
Error in h5writeDatasetHelper(obj = obj, h5dataset = h5dataset, index = index,  : 
  object 'h5dataset' not found
restarting interrupted promise evaluationError in h5writeDatasetHelper(obj = obj, h5dataset = h5dataset, index = index,  : 
  object 'h5dataset' not found
Error in H5Dclose(h5dataset) : object 'h5dataset' not found
Error in H5Dcreate(loc$H5Identifier, dataset, tid, sid, dcpl = dcpl) : 
  HDF5. Dataset. Unable to initialize object.
Error in H5Dopen(h5loc, name) : HDF5. Dataset. Object not found.
Error in h5writeDatasetHelper(obj = obj, h5dataset = h5dataset, index = index,  : 
  object 'h5dataset' not found
restarting interrupted promise evaluationError in h5writeDatasetHelper(obj = obj, h5dataset = h5dataset, index = index,  : 
  object 'h5dataset' not found
Error in H5Dclose(h5dataset) : object 'h5dataset' not found

[jeremymiller]

@hypercompetent : Sorry, one more gene-related read/write error.

Now when I run this, I get an error. This worked last week.

write_tome_exon_lengths(exon_length, tome)
# [1] "Writing /data/exon_lengths"
# Error in sub("(/.+/).+", "\\1", target_path) : object 'target_path' not found

write_tome_intron_lengths(intron_length, tome) 
# [1] "Writing /data/intron_lengths"
# Error in sub("(/.+/).+", "\\1", target_path) : object 'target_path' not found

[SarojaSomu]

@hypercompetent
The following error comes up when I try to write anno.feather to tome. I am using the dev branch of scrattch.io

write_tome_anno(anno_file,tome,overwrite = TRUE)
[1] "Creating Group /sample_meta"
[1] "Creating Group /sample_meta/anno"
[1] "Writing /sample_meta/anno"
Error in H5Dcreate(loc$H5Identifier, dataset, tid, sid, dcpl = dcpl) :
HDF5. Dataset. Unable to initialize object.
Error in H5Dopen(h5loc, name) : HDF5. Dataset. Can't open object.
Error in h5writeDatasetHelper(obj = obj, h5dataset = h5dataset, index = index, :
object 'h5dataset' not found
Error in h5writeDatasetHelper(obj = obj, h5dataset = h5dataset, index = index, :
object 'h5dataset' not found
Error in H5Dclose(h5dataset) : object 'h5dataset' not found
Error in H5Dcreate(loc$H5Identifier, dataset, tid, sid, dcpl = dcpl) :
HDF5. Dataset. Unable to initialize object.
Error in H5Dopen(h5loc, name) : HDF5. Dataset. Can't open object.
Error in h5writeDatasetHelper(obj = obj, h5dataset = h5dataset, index = index, :
object 'h5dataset' not found
Error in h5writeDatasetHelper(obj = obj, h5dataset = h5dataset, index = index, :
object 'h5dataset' not found
Error in H5Dclose(h5dataset) : object 'h5dataset' not found
Error in H5Dcreate(loc$H5Identifier, dataset, tid, sid, dcpl = dcpl) :
HDF5. Dataset. Unable to initialize object.
Error in H5Dopen(h5loc, name) : HDF5. Dataset. Can't open object.
Error in h5writeDatasetHelper(obj = obj, h5dataset = h5dataset, index = index, :
object 'h5dataset' not found
Error in h5writeDatasetHelper(obj = obj, h5dataset = h5dataset, index = index, :
object 'h5dataset' not found
Error in H5Dclose(h5dataset) : object 'h5dataset' not found