Merging changes for timestamps and API rate limits#40
Conversation
Added macros (genesys_index and genesys_status_index) for genesys cloud indexes. Indexes need to be added elsewhere Added all sourcetypes for all inputs Savedsearches.conf Created 2 savedsearches to populate lookups for genesys status API data Transforms.conf Added transforms to 1. create lookups for input check points and 2. support status API lookups Added automatic lookup (LOOKUP-Genesys API Status) for new field called function (aka name) to status API data Lookups Added empty CSV files for lookups
…id API rate limiting on initial loads. Added debug/info logging to track API call rates plus other useful info.
updated "Genesys Cloud Status Name Lookup"
added _time to Genesys Cloud Status Lookup
globalConfig.json for new version
edro15
left a comment
edro15
left a comment
There was a problem hiding this comment.
Hi @am-napier! I added some comments, requested a couple of changes and clarifications. Looking forward to discussing this later. Thanks for your input and efforts so far! ❤️
| from datetime import datetime, timezone | ||
| from genesyscloud_client import GenesysCloudClient | ||
|
|
||
| # Permissions required : [403] Forbidden - Unable to perform the requested action. You must have at least one of the following permissions assigned: [integrations:action:view, bridge:actions:view] |
There was a problem hiding this comment.
Yes this is mentioned in documentation. Would you need the comment here as well?
| from dateutil.relativedelta import relativedelta | ||
| from genesyscloud_client import GenesysCloudClient | ||
|
|
||
| # Permissions issues: [403] Forbidden - Unable to perform the requested action. You are missing the following permission(s): [audits:audit:view] |
There was a problem hiding this comment.
Indeed, as mentioned in documentation... Shall we add these errors in the Troubleshooting section of docs?
| "group": null, | ||
| "name": "genesys_cloud_ta", | ||
| "version": "0.3.0" | ||
| "version": "0.0.33" |
There was a problem hiding this comment.
Please do not change version. This will be changed only at release time. If you do, please keep it local.
| "version": "0.0.33" | |
| "version": "0.3.0" |
| import sys, os | ||
| sys.path.append(os.path.join(os.environ['SPLUNK_HOME'],'etc','apps','SA-VSCode','bin')) | ||
| import splunk_debug as dbg | ||
| dbg.enable_debugging(timeout=25) |
There was a problem hiding this comment.
Please remove this part. It's linked to local resources that won't be found elsewhere and it's a reason for tests to fail.
| import sys, os | |
| sys.path.append(os.path.join(os.environ['SPLUNK_HOME'],'etc','apps','SA-VSCode','bin')) | |
| import splunk_debug as dbg | |
| dbg.enable_debugging(timeout=25) |
| client = GenesysCloudClient(logger, client_id, client_secret, account_region) | ||
|
|
||
| checkpointer_key_name = normalized_input_name | ||
| # An 2025-10-14: Changed default start date to 5 minutes ago to reduce data volume on first run |
There was a problem hiding this comment.
| # An 2025-10-14: Changed default start date to 5 minutes ago to reduce data volume on first run | |
| # if we don't have any checkpoint, we default it to 5 minutes ago to reduce data volume on first run |
| start_time = now - relativedelta(days=int(history)) | ||
| else: | ||
| start_time = now - relativedelta(mins=5) |
There was a problem hiding this comment.
Please use timedelta instead of relativedelta as for the other inputs. Adjust imported libraries accordingly.
| @@ -11,6 +11,7 @@ | |||
| from dateutil.relativedelta import relativedelta | |||
There was a problem hiding this comment.
Please use timedelta instead of relativedelta to keep code consistency and reduce dependencies. See other comments.
There was a problem hiding this comment.
Will ask for some clarifications here..
Co-authored-by: Erica Pescio <epescio@splunk.com>
Co-authored-by: Erica Pescio <epescio@splunk.com>
removed splunk_dbg code from imports
2 participants
I've updated a few things on most inputs to ensure good defaults are used when an input first runs and the input doesn't over use the API and cause accounts to be rate limited. In my case the splunk user was consuming so much of the API it was at risk of stopping the business users from access API as genesys cloud rate limits this at the account not user level. In general all inputs have been changed to make the default starting run collect just the last 5 mins of data. Some where collecting 4 years, some were using epoch 0 for 1970. I changed conversation details from start_date to days_history as it makes a more intuitive experience.
I've added some logging to record every API action (posts and gets) to help monitor call volumes.
I've added collections to enable SPL to view and manage the checkpoint collections ie
| outputlookup genesys_cloud_conversation_details_chkptwill truncate the collection enabling it to restart as if it were fresh. Use with caution during setup and development.I've added some saved search to build status lookups to make the consumption of the status page data more efficient. The searches are not enabled by default. Note to use this efficiently the status input should write to a different index low volume long term index, not the same as the other inputs as its extracting timestamps correctly.
Added explicit sourcetypes for all inputs.