Any documentation on how to use its capabilities/features ?

[ramukima]

Any documentation providing how to use the hot deployment of YANG modules in a running instance will be helpful. Documentation in general can be helpful here.

[ramukima]

Thank you Peter. This is great. May be by hot-plugin I meant something different. Here is what I am looking for -

1. I start my PetStore application (as your example) and get those endpoints published.
2. While my PetStore application is running, I decide to onboard another "YangPack" (An archive with Yang modules, Binding code etc) and copy under a specified location.
3. I somehow instruct the running PetStore application to load the new YangPack by invoking an existing REST interface (specify location of my YangPack archive)
4. The running server creates additional endpoints as necessary by loading the YANG modules and bindings from the YangPack archive.
5. I have additional endpoints become accessible to me via all existing interfaces exposed by yang-express (restconf, websocket etc).

This will allow extending the running application on demand by onboarding YangPacks. I believe, you already have most of it covered, its just about exposing the API endpoints from yang-express to be able to do such thing and coming up with an specification for YangPack (not much different from npm node modules). Does that make sense ?

[sekur]

Hi Amit, yes I'm clear on your query but the hotplug pattern I had in mind is a bit different:

1. A yang-express instance is started as a blank slate (no models) except for the newly introduced "yangapi" feature plugin and other system plugins such as "restjson"
2. A remote system communicates via "yangapi" to the yang-express instance to hotplug a new "YANGPACK"
   • POST a new YANG data model schema
   • PATCH the model with control logic bindings
3. A remote system communicates via "restjson" or "websocket" to perform data state synchronization (pushing data)

No need for back-channel archive transfer/download, background install, etc.

I've implemented the yangapi POST operation already so if you performed a POST /some-new-module.yang and sent the schema it will autogen the model, link it into the running system and be serviced automatically by restjson, websocket, openapi, yangapi, etc. There's no notion of bundling a set of YANG schema so you'll need to POST the import/include dependencies first and then POST the final schema.

I'm also thinking we'll need to consider a separate control protocol to perform the "linking" after the POST since you may not want every YANG module to be dynamically routed - at least not necessarily as part of the POST.

I have yet to support the yangapi PUT/PATCH (see #3) operation but expect it to be ready fairly soon.

The key open question at the moment is how to deal with event subscription driven SB integration into data persistence layer. I need a way to bind state change callbacks (which is also what you've asked before regarding Yang.bind on configurable data entities). The tricky part is how that control logic hook may interact with a resource asset only available inside the runtime instance (such as data layer adapter module). Ideally the event bindings can also be handled via the Yang.bind method so that we can send both the controller logic AND event listener hooks via the yangapi PATCH method.

[ramukima]

I like having no need for onboarding an archive. However, the way I understand is "the control logic" is some kind of instrumentation code (at this time its coffee or javascript). How do you load such "logic" in a running system.

I like your opinion about hotplug in many ways. My only concern is onboarding "control logic". If you believe thats already dealt with or can be dealt with easily, thats perfect.

Regarding state persistence, would it make sense to "extend the yang-express" YANG model to allow for a custom annotation ? This annotation may be applicable at "container/list/module/leaf" levels to allow for a callback into control logic ? The convention could be "yang-express:dp name" where the name represents a callback function name from "Control Logic" ? If there is one, that logic is called for "get/set/get-next" of relevant fields in the YANG model.

Alternatively, have the "Control Logic" provide implementations for "critical state change functions" just like they can bind event hooks to a given YANG path.

Just my 2 cents, may not be the perfect solution, but just a thought.

[sekur]

I've experimented extensively trying to enable archive-free (file system touch-less) loading of control logic bindings - and the best way this can be achieved is to do it exactly how a web browser fetch/loads arbitrary JS files into runtime.

We can basically grab the binding object structure, run it through browserify and send that along to the other side, which will then "load" that bundle into its runtime, extract out the object and apply it into the Yang schema instance via bind().

I'm not sure I fully understand the custom annotation approach? Perhaps an example will help illustrate the transaction workflow.

[ramukima]

For example :

list pet {
    yang-express:dp func_data_provider_for_pets_list;
    leaf id {
        type string;
    }
    leaf name {
        yang-express:dp func_provider_for_pet_name;
        type string;
    }
}

Where func_data_provider_for_pets_list and func_provider_for_pet_name are binding callback modules that must be provided during binding in order for this YANG module to be enabled. These modules must define "type related functions". i.e. for a simple string type node, they must provide implementations for get/set. For list type nodes, they must provide implementations for get_key, iterator, next_element etc. The appropriate functions are invoked when there is a request for GET/PUT/POST requests.

I am not sure what you have in mind for data provider APIs, this may not be aligned to what you have been thinking. Just one thing that came to my mind at the moment, so I shared.