API to close h5 file handler of circuit?

[asanin-epfl]

I have a situation where I need to edit SONATA circuit nodes and edges. In order to read and edit them I use:

    circuit = Circuit(str(circuit_config_file))
    node = circuit.nodes[node_population_name].get(node_id)
    # adjust node and its edges
    ....
    with h5py.File(nodes_file)

The last line gives me

OSError: Unable to open file (unable to lock file, errno = 35, error message = 'Resource temporarily unavailable')
h5py/h5f.pyx:88: OSError

Can we provide a method to close all h5 files opened by the instance of Circuit?

[tomdele]

That s strange. I cannot reproduce this behaviour.
Did this:

from bluepysnap import Circuit
import h5py

circuit = Circuit("circuit_config.json")
node = circuit.nodes['population']
with h5py.File("nodes.sonata", "r") as h5:
    print(list(h5["/nodes/"]))

print(node)

no problem...

[asanin-epfl]

Of course, because you are opening it for reading only. Try r+

from bluepysnap import Circuit
import h5py

circuit = Circuit("tests/data/circuit_config.json")
node = circuit.nodes['default']
with h5py.File("tests/data/nodes.h5", "r+") as h5:
    print(list(h5["/nodes/"]))

print(node)

[tomdele]

ah ok this is different then ! To me:

with h5py.File(nodes_file)

was:

with h5py.File(nodes_file, "r")

Checking again then.

[tomdele]

we can add something like this to the Circuit class:

    def clean(self):
        if self.nodes:
            for name, population in self.nodes.items():
                del population._node_storage
            del self.__dict__['nodes']
        if self.edges:
            for name, population in self.edges.items():
                del population._edge_storage
            del self.__dict__['edges']

then this works:

from bluepysnap import Circuit
import h5py


circuit = Circuit("circuit_config_test.json")
nodes = circuit.nodes["population"].ids()
circuit.clean()

with h5py.File("nodes.sonata", "r+") as h5:
    print(list(h5["/nodes/"]))

[tomdele]

using:

from bluepysnap import Circuit
import h5py


circuit = Circuit("circuit_config_test.json")
nodes = circuit.nodes["population"].ids()
del circuit

with h5py.File("nodes.sonata", "r+") as h5:
    print(list(h5["/nodes/"]))

it seems that the Node/EdgeStorage are not deleted correctly.

[tomdele]

Ok I found the culprit !
I @cached_property the sonata storage. No need to create a clean function.
I will do a PR.

[asanin-epfl]

That is brilliant! Thank you!

[tomdele]

I need to do some tests first and maybe remove a bit more @cached_property.
I let you know.