Wavefront, Influx, OpenTSDB DCC and packages for wavefront,influx,opentsdb. #5
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| # -*- coding: utf-8 -*- | ||
| # ----------------------------------------------------------------------------# | ||
| # Copyright © 2015-2016 VMware, Inc. All Rights Reserved. # | ||
| # # | ||
| # Licensed under the BSD 2-Clause License (the “License”); you may not use # | ||
| # this file except in compliance with the License. # | ||
| # # | ||
| # The BSD 2-Clause License # | ||
| # # | ||
| # Redistribution and use in source and binary forms, with or without # | ||
| # modification, are permitted provided that the following conditions are met:# | ||
| # # | ||
| # - Redistributions of source code must retain the above copyright notice, # | ||
| # this list of conditions and the following disclaimer. # | ||
| # # | ||
| # - Redistributions in binary form must reproduce the above copyright # | ||
| # notice, this list of conditions and the following disclaimer in the # | ||
| # documentation and/or other materials provided with the distribution. # | ||
| # # | ||
| # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"# | ||
| # AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # | ||
| # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # | ||
| # ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE # | ||
| # LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR # | ||
| # CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF # | ||
| # SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS # | ||
| # INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN # | ||
| # CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) # | ||
| # ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF # | ||
| # THE POSSIBILITY OF SUCH DAMAGE. # | ||
| # ----------------------------------------------------------------------------# | ||
|
|
||
| import logging | ||
| from liota.dccs.dcc import DataCenterComponent | ||
| from liota.entities.metrics.registered_metric import RegisteredMetric | ||
| from liota.entities.metrics.metric import Metric | ||
| from liota.entities.registered_entity import RegisteredEntity | ||
|
|
||
| log = logging.getLogger(__name__) | ||
|
|
||
| class influx(DataCenterComponent): | ||
| def __init__(self, comms): | ||
| super(influx, self).__init__( | ||
| comms=comms | ||
| ) | ||
| self.comms = comms | ||
|
|
||
| def register(self, entity_obj): | ||
| log.info("Registering resource with influx DCC {0}".format(entity_obj.name)) | ||
| if isinstance(entity_obj, Metric): | ||
| return RegisteredMetric(entity_obj, self, None) | ||
| else: | ||
| return RegisteredEntity(entity_obj, self, None) | ||
|
|
||
| def create_relationship(self, reg_entity_parent, reg_entity_child): | ||
| reg_entity_child.parent = reg_entity_parent | ||
|
|
||
| def _format_data(self, reg_metric): | ||
| met_cnt = reg_metric.values.qsize() | ||
| message = '' | ||
| host = '' | ||
| device_name = '' | ||
| metric_name = '' | ||
| if met_cnt == 0: | ||
| return | ||
| for _ in range(met_cnt): | ||
| v = reg_metric.values.get(block=True) | ||
| if v is not None: | ||
| device_name = (reg_metric.parent).ref_entity.name | ||
| metric_name = reg_metric.ref_entity.name | ||
| if (reg_metric.parent).parent: | ||
| host = (reg_metric.parent).parent.ref_entity.entity_id+"."+(reg_metric.parent).ref_entity.entity_id | ||
| else: | ||
| host = (reg_metric.parent).ref_entity.entity_id #if device is not available, only gateway uuid | ||
|
|
||
| metric_unit = str(reg_metric.ref_entity.unit) | ||
| metric_unit = ''.join(metric_unit.split()) | ||
| message += '{0},unit={5},host={1} {2}={3} {4}'.format(device_name,host,metric_name,v[1], | ||
| v[0]*1000000,metric_unit) | ||
|
|
||
| if message == '': | ||
| return | ||
| log.info ("Publishing values to influx DCC") | ||
| log.debug("Device name: {0}".format(device_name)) | ||
| log.debug("Metric name: {0}".format(metric_name)) | ||
| log.debug("Host name: {0}".format(host)) | ||
| log.debug("Formatted message: {0}".format(message)) | ||
| return message | ||
|
|
||
| def set_properties(self, reg_entity, properties): | ||
| raise NotImplementedError | ||
|
|
||
| def unregister(self, entity_obj): | ||
| raise NotImplementedError | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| # -*- coding: utf-8 -*- | ||
| # ----------------------------------------------------------------------------# | ||
| # Copyright © 2015-2016 VMware, Inc. All Rights Reserved. # | ||
| # # | ||
| # Licensed under the BSD 2-Clause License (the “License”); you may not use # | ||
| # this file except in compliance with the License. # | ||
| # # | ||
| # The BSD 2-Clause License # | ||
| # # | ||
| # Redistribution and use in source and binary forms, with or without # | ||
| # modification, are permitted provided that the following conditions are met:# | ||
| # # | ||
| # - Redistributions of source code must retain the above copyright notice, # | ||
| # this list of conditions and the following disclaimer. # | ||
| # # | ||
| # - Redistributions in binary form must reproduce the above copyright # | ||
| # notice, this list of conditions and the following disclaimer in the # | ||
| # documentation and/or other materials provided with the distribution. # | ||
| # # | ||
| # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"# | ||
| # AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # | ||
| # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # | ||
| # ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE # | ||
| # LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR # | ||
| # CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF # | ||
| # SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS # | ||
| # INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN # | ||
| # CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) # | ||
| # ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF # | ||
| # THE POSSIBILITY OF SUCH DAMAGE. # | ||
| # ----------------------------------------------------------------------------# | ||
|
|
||
| import logging | ||
| from liota.dccs.dcc import DataCenterComponent | ||
| from liota.entities.metrics.registered_metric import RegisteredMetric | ||
| from liota.entities.metrics.metric import Metric | ||
| from liota.entities.registered_entity import RegisteredEntity | ||
| from datetime import datetime | ||
|
|
||
| log = logging.getLogger(__name__) | ||
|
|
||
| class opentsdb(DataCenterComponent): | ||
| def __init__(self, comms): | ||
| super(opentsdb, self).__init__( | ||
| comms=comms | ||
| ) | ||
|
|
||
| def register(self, entity_obj): | ||
| log.info("Registering resource with opentsdb DCC {0}".format(entity_obj.name)) | ||
| if isinstance(entity_obj, Metric): | ||
| return RegisteredMetric(entity_obj, self, None) | ||
| else: | ||
| return RegisteredEntity(entity_obj, self, None) | ||
|
|
||
| def create_relationship(self, reg_entity_parent, reg_entity_child): | ||
| reg_entity_child.parent = reg_entity_parent | ||
|
|
||
| def _format_data(self, reg_metric): | ||
| met_cnt = reg_metric.values.qsize() | ||
| message = '' | ||
| host = '' | ||
| device_name = '' | ||
| metric_name = '' | ||
| if met_cnt == 0: | ||
| return | ||
| for _ in range(met_cnt): | ||
| v = reg_metric.values.get(block=True) | ||
| if v is not None: | ||
| device_name = (reg_metric.parent).ref_entity.name | ||
| metric_name = reg_metric.ref_entity.name | ||
| host = (reg_metric.parent).ref_entity.entity_id | ||
|
|
||
| message += 'put {0} {1} {2} host={3}\n'.format(metric_name,v[0],v[1],host) | ||
| if message == '': | ||
| return | ||
| log.info ("Publishing values to opentsdb DCC") | ||
| log.debug("Device name: {0}".format(device_name)) | ||
| log.debug("Metric name: {0}".format(metric_name)) | ||
| log.debug("Host name: {0}".format(host)) | ||
| log.debug("Formatted message: {0}".format(message)) | ||
| return message | ||
|
|
||
| def set_properties(self, reg_entity, properties): | ||
| raise NotImplementedError | ||
|
|
||
| def unregister(self, entity_obj): | ||
| raise NotImplementedError |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| # -*- coding: utf-8 -*- | ||
| # ----------------------------------------------------------------------------# | ||
| # Copyright © 2015-2016 VMware, Inc. All Rights Reserved. # | ||
| # # | ||
| # Licensed under the BSD 2-Clause License (the “License”); you may not use # | ||
| # this file except in compliance with the License. # | ||
| # # | ||
| # The BSD 2-Clause License # | ||
| # # | ||
| # Redistribution and use in source and binary forms, with or without # | ||
| # modification, are permitted provided that the following conditions are met:# | ||
| # # | ||
| # - Redistributions of source code must retain the above copyright notice, # | ||
| # this list of conditions and the following disclaimer. # | ||
| # # | ||
| # - Redistributions in binary form must reproduce the above copyright # | ||
| # notice, this list of conditions and the following disclaimer in the # | ||
| # documentation and/or other materials provided with the distribution. # | ||
| # # | ||
| # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"# | ||
| # AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # | ||
| # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # | ||
| # ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE # | ||
| # LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR # | ||
| # CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF # | ||
| # SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS # | ||
| # INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN # | ||
| # CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) # | ||
| # ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF # | ||
| # THE POSSIBILITY OF SUCH DAMAGE. # | ||
| # ----------------------------------------------------------------------------# | ||
|
|
||
| import logging | ||
| from liota.dccs.dcc import DataCenterComponent | ||
| from liota.entities.metrics.registered_metric import RegisteredMetric | ||
| from liota.entities.metrics.metric import Metric | ||
| from liota.entities.registered_entity import RegisteredEntity | ||
|
|
||
| log = logging.getLogger(__name__) | ||
|
|
||
| class Wavefront(DataCenterComponent): | ||
| def __init__(self, comms): | ||
| super(Wavefront, self).__init__( | ||
| comms=comms | ||
| ) | ||
| self.comms = comms | ||
|
|
||
| def register(self, entity_obj): | ||
| log.info("Registering resource with Wavefront DCC {0}".format(entity_obj.name)) | ||
| if isinstance(entity_obj, Metric): | ||
| return RegisteredMetric(entity_obj, self, None) | ||
| else: | ||
| return RegisteredEntity(entity_obj, self, None) | ||
|
|
||
| def create_relationship(self, reg_entity_parent, reg_entity_child): | ||
| reg_entity_child.parent = reg_entity_parent | ||
|
|
||
| def _format_data(self, reg_metric): | ||
| met_cnt = reg_metric.values.qsize() | ||
| message = '' | ||
| host = '' | ||
| device_name = '' | ||
| metric_name = '' | ||
| if met_cnt == 0: | ||
| return | ||
| for _ in range(met_cnt): | ||
| v = reg_metric.values.get(block=True) | ||
| if v is not None: | ||
| device_name = (reg_metric.parent).ref_entity.name | ||
| metric_name = reg_metric.ref_entity.name | ||
| if (reg_metric.parent).parent: | ||
| host = (reg_metric.parent).parent.ref_entity.entity_id+"."+(reg_metric.parent).ref_entity.entity_id | ||
| else: | ||
| host = (reg_metric.parent).ref_entity.entity_id #if device is not available, only gateway uuid | ||
|
|
||
| metric_unit = str(reg_metric.ref_entity.unit) | ||
| metric_unit = ''.join(metric_unit.split()) | ||
| message += '{0},unit={5},host={1} {2}={3} {4}'.format(device_name,host,metric_name,v[1], | ||
| v[0]*1000000,metric_unit) | ||
| if message == '': | ||
| return | ||
| log.info ("Publishing values to Wavefront DCC") | ||
| log.debug("Device name: {0}".format(device_name)) | ||
| log.debug("Metric name: {0}".format(metric_name)) | ||
| log.debug("Host name: {0}".format(host)) | ||
| log.debug("Formatted message: {0}".format(message)) | ||
| return message | ||
|
|
||
| def set_properties(self, reg_entity, properties): | ||
| raise NotImplementedError | ||
|
|
||
| def unregister(self, entity_obj): | ||
| raise NotImplementedError |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,2 +1,13 @@ | ||
| Here we put example user packages for developers to refer to when they develop | ||
| their own packages. | ||
|
|
||
| # OpenTSDB | ||
| OpenTSDB is a scalable time series database built on top of Hadoop and HBase. It simplifies the process of storing and analyzing large amounts of time-series data generated by endpoints like sensors or servers. | ||
|
|
||
| # Requirements | ||
| - OpenTSDB - One can install OpenTSDB from [here](http://opentsdb.net/docs/build/html/installation.html) or use the docker image provided [here.](http://opentsdb.net/docs/build/html/resources.html#docker-images) | ||
|
|
||
| # Examples | ||
| Above OpenTSDB examples are sending metrics data to the opentsdb. You can specify the OpenTSDB-IP and port in the sampleProp.conf file provided in the packages folder. | ||
|
|
||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| # Influx | ||
| InfluxDB is a time series database built from the ground up to handle high write and query loads. It is the second piece of the TICK stack. InfluxDB is meant to be used as a backing store for any use case involving large amounts of timestamped data, including DevOps monitoring, application metrics, IoT sensor data, and real-time analytics. | ||
| # Requirements | ||
| - Mqtt Broker: Setup a mqtt broker which will allow liota to send metrics to the collector agent. | ||
|
|
||
| - Telegraf: Telegraf is a plugin-driven server agent for collecting & reporting metrics. You can setup telegraf from [here.](https://docs.influxdata.com/telegraf/v1.4/) | ||
|
|
||
| - InfluxDB: You can install and configure InfluxDb from [here.](https://docs.influxdata.com/influxdb/v1.3/introduction/) | ||
|
|
||
| # Examples | ||
| Above examples are sending metrics data to the mqtt broker, from where you can consume data using telegraf agent which will then pass data to InfluxDB. | ||
|
|
||
| You can also view data being sent to InfluxDB using Chronograf. You can setup chronograf from [here.](https://docs.influxdata.com/chronograf/v1.3/) | ||
|
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
explain this piece
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For replacing message as None, https://stackoverflow.com/a/1398628/4134429 as per this stack answer we should go with None, Should I go ahead and change this to None?