-
Notifications
You must be signed in to change notification settings - Fork 0
Phase 1: Foundation Layer - Centralized Config, Exceptions, and Tests #26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
37 commits
Select commit
Hold shift + click to select a range
1ee26f1
Implement Phase 1: Foundation Layer refactoring
SBFRF ad984b9
Add support for new wave and current gauges from THREDDS server
SBFRF bbedac5
Fix NCML indexing bug and satellite imagery bounds calculation
SBFRF adf7215
Merge branch 'main' into refactor/phase1-foundation
SBFRF 16925ed
Initial plan
Copilot 6c1db0d
Initial plan
Copilot d2c1cc2
Update murgtools/getdata/getDataFRF.py
SBFRF 4672489
Update tests/test_ncml_indexing_regression.py
SBFRF 6b991c0
Initial plan
Copilot 24190b9
Update murgtools/getdata/getDataFRF.py
SBFRF 6f74454
Move pyproj and re imports to top of file
Copilot 9eb688e
Initial plan
Copilot 337e9ce
Move imports to top of file and organize per PEP 8
Copilot dc1c169
Merge branch 'refactor/phase1-foundation' into fix/ncml-indexing-and-…
SBFRF ca31039
Update murgtools/getdata/getOutsideData.py
SBFRF 88bd6f8
Fix getWaveSpecModel to capture indexRef and compute ncfileindex
Copilot 4ed8da2
Fix inconsistent indexing in getWaveSpecModel - use ncfileindex for a…
Copilot c43e8e7
Merge branch 'fix/ncml-indexing-and-satellite-bounds' into copilot/su…
SBFRF 00629a0
Fix inconsistent indexing and remove unused variables
Copilot 1b981a9
Merge pull request #29 from SBFRF/copilot/sub-pr-27-again
SBFRF 462985e
Merge pull request #30 from SBFRF/copilot/sub-pr-27-another-one
SBFRF da12d44
Merge branch 'fix/ncml-indexing-and-satellite-bounds' into copilot/su…
SBFRF f49a642
Merge pull request #28 from SBFRF/copilot/sub-pr-27
SBFRF bae4feb
Merge pull request #31 from SBFRF/copilot/sub-pr-27-yet-again
SBFRF d3e1919
Merge pull request #27 from SBFRF/fix/ncml-indexing-and-satellite-bounds
SBFRF 18eaa58
Update murgtools/getdata/getDataFRF.py
SBFRF 373568b
Initial plan
Copilot 8eb4765
Update tests/test_getDataFRF.py
SBFRF fa19570
Update murgtools/getdata/getDataFRF.py
SBFRF 7055046
Update tests/test_getDataFRF.py
SBFRF 0168ee7
Update murgtools/getdata/getDataFRF.py
SBFRF 03363ff
Update tests/test_getDataFRF.py
SBFRF d9bf24a
Add parameter validation to roundDatetimeToInterval function
Copilot 4195893
Initial plan
Copilot 3cceb43
Merge pull request #34 from SBFRF/copilot/sub-pr-26
SBFRF 531d0f4
Merge pull request #35 from SBFRF/copilot/sub-pr-26-again
SBFRF 5f30064
Update murgtools/getdata/getOutsideData.py
SBFRF File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,134 @@ | ||
| """Centralized configuration for murgtools library. | ||
|
|
||
| This module contains all hardcoded URLs, server addresses, and configuration | ||
| values used throughout the murgtools package. Centralizing these values makes | ||
| it easier to update endpoints and ensures consistency across modules. | ||
| """ | ||
|
|
||
| import socket | ||
|
|
||
| # ============================================================================= | ||
| # THREDDS Server URLs | ||
| # ============================================================================= | ||
|
|
||
| # FRF Local THREDDS (internal network only) | ||
| THREDDS_FRF_LOCAL = 'http://134.164.129.55:8080/thredds/dodsC/' | ||
| THREDDS_FRF_LOCAL_ALT = 'http://134.164.129.55/thredds/dodsC/' | ||
| THREDDS_FRF_LOCAL_FRF = 'http://134.164.129.55/thredds/dodsC/FRF/' | ||
|
|
||
| # CHL Public THREDDS (externally accessible) | ||
| THREDDS_CHL_PUBLIC = 'https://chldata.erdc.dren.mil/thredds/dodsC/' | ||
| THREDDS_CHL_ALT = 'https://chlthredds.erdc.dren.mil/thredds/dodsC/frf/' | ||
|
|
||
| # CMTB (Coastal Model Test Bed) THREDDS | ||
| THREDDS_TESTBED = 'http://134.164.129.62:8080/thredds/dodsC/CMTB' | ||
| THREDDS_CRUNCH = 'http://134.164.129.55:8080/thredds/dodsC/cmtb/' | ||
|
|
||
| # Survey data URLs | ||
| SURVEY_TRANSECTS_URL = 'http://134.164.129.55/thredds/dodsC/FRF/geomorphology/elevationTransects/survey/surveyTransects.ncml' | ||
| WAVE_8M_ARRAY_URL = 'http://134.164.129.55/thredds/dodsC/FRF/oceanography/waves/8m-array/2017/FRF-ocean_waves_8m-array_201707.nc' | ||
|
|
||
| # ============================================================================= | ||
| # Imagery URLs | ||
| # ============================================================================= | ||
|
|
||
| # Argus coastal imaging system | ||
| ARGUS_BASE_URL = 'https://coastalimaging.erdc.dren.mil/FrfTower/Processed/Orthophotos/cxgeo/' | ||
|
|
||
| # STAC (SpatioTemporal Asset Catalog) endpoints for satellite imagery | ||
| STAC_URLS = { | ||
| 'element84': 'https://earth-search.aws.element84.com/v1/search', | ||
| 'planetary-computer': 'https://planetarycomputer.microsoft.com/api/stac/v1/search' | ||
| } | ||
|
|
||
| # Planetary Computer signing endpoint | ||
| PLANETARY_COMPUTER_SIGN_URL = 'https://planetarycomputer.microsoft.com/api/sas/v1/sign' | ||
|
|
||
| # ============================================================================= | ||
| # External Data Sources | ||
| # ============================================================================= | ||
|
|
||
| # NCEP WaveWatch III forecast data | ||
| NCEP_DATA_URL = 'http://nomads.ncep.noaa.gov/pub/data/nccf/com/wave/prod/' | ||
|
|
||
| # ============================================================================= | ||
| # Network Configuration | ||
| # ============================================================================= | ||
|
|
||
| # IP prefixes that indicate FRF internal network | ||
| FRF_IP_PREFIXES = ('134.164', '10.0.0') | ||
|
|
||
| # Default timeout for network requests (seconds) | ||
| DEFAULT_TIMEOUT_SECONDS = 60 | ||
|
|
||
| # Maximum retry attempts for network operations | ||
| MAX_RETRY_ATTEMPTS = 3 | ||
|
|
||
| # ============================================================================= | ||
| # Time Configuration | ||
| # ============================================================================= | ||
|
|
||
| # Argus imagery is available at 30-minute intervals | ||
| ARGUS_IMAGE_INTERVAL_MINUTES = 30 | ||
|
|
||
| # Default time rounding (seconds) | ||
| DEFAULT_TIME_ROUND_SECONDS = 60 | ||
|
|
||
| # NetCDF time units standard | ||
| TIME_UNITS = 'seconds since 1970-01-01 00:00:00' | ||
|
|
||
| # ============================================================================= | ||
| # Argus Image Types | ||
| # ============================================================================= | ||
|
|
||
| ARGUS_IMAGE_TYPES = ('timex', 'var', 'snap', 'bright', 'dark') | ||
|
|
||
| # ============================================================================= | ||
| # Helper Functions | ||
| # ============================================================================= | ||
|
|
||
|
|
||
| def get_thredds_server(server=None, ip_address=None): | ||
| """Select appropriate THREDDS server based on network location. | ||
|
|
||
| Args: | ||
| server (str, optional): Force server selection. 'FRF' for local, | ||
| 'CHL' for public. If None, auto-detect based on IP. | ||
| ip_address (str, optional): IP address to check. If None, | ||
| uses current machine's IP. | ||
|
|
||
| Returns: | ||
| tuple: (server_url, server_prefix) where server_prefix is 'FRF' or 'frf' | ||
| for use in constructing data paths. | ||
| """ | ||
| if ip_address is None: | ||
| try: | ||
| ip_address = socket.gethostbyname(socket.gethostname()) | ||
| except socket.error: | ||
| ip_address = '' | ||
|
|
||
| is_frf_network = ip_address.startswith(FRF_IP_PREFIXES) | ||
|
|
||
| if server == 'FRF' or (server is None and is_frf_network): | ||
| return THREDDS_FRF_LOCAL, 'FRF' | ||
| else: | ||
| return THREDDS_CHL_PUBLIC, 'frf' | ||
|
|
||
|
|
||
| def is_frf_network(ip_address=None): | ||
| """Check if the current machine is on the FRF internal network. | ||
|
|
||
| Args: | ||
| ip_address (str, optional): IP address to check. If None, | ||
| uses current machine's IP. | ||
|
|
||
| Returns: | ||
| bool: True if on FRF network, False otherwise. | ||
| """ | ||
| if ip_address is None: | ||
| try: | ||
| ip_address = socket.gethostbyname(socket.gethostname()) | ||
| except socket.error: | ||
| return False | ||
|
|
||
| return ip_address.startswith(FRF_IP_PREFIXES) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,147 @@ | ||
| """Custom exceptions for murgtools library. | ||
|
|
||
| This module defines a hierarchy of exceptions for better error handling | ||
| and more descriptive error messages across the murgtools package. | ||
| """ | ||
|
|
||
|
|
||
| class MurgToolsError(Exception): | ||
| """Base exception for all murgtools errors. | ||
|
|
||
| All custom exceptions in murgtools inherit from this class, | ||
| allowing users to catch all murgtools-specific errors with a single | ||
| except clause if desired. | ||
| """ | ||
| pass | ||
|
|
||
|
|
||
| class DataNotFoundError(MurgToolsError): | ||
| """Raised when requested data is not available. | ||
|
|
||
| This exception is raised when a data query returns no results, | ||
| such as when requesting data outside the available time range | ||
| or from a non-existent gauge. | ||
|
|
||
| Attributes: | ||
| message (str): Explanation of why data was not found. | ||
| """ | ||
| pass | ||
|
|
||
|
|
||
| class InvalidGaugeError(MurgToolsError): | ||
| """Raised when an invalid gauge name or number is specified. | ||
|
|
||
| This exception provides helpful information about valid gauge options | ||
| when an invalid gauge identifier is provided. | ||
|
|
||
| Attributes: | ||
| gauge_name: The invalid gauge identifier that was provided. | ||
| valid_gauges: List of valid gauge options (if available). | ||
| message (str): Full error message. | ||
|
|
||
| Examples: | ||
| >>> raise InvalidGaugeError('invalid-gauge', valid_gauges=['waverider-26m', 'waverider-17m']) | ||
| InvalidGaugeError: Invalid gauge: 'invalid-gauge'. Valid options: ['waverider-26m', 'waverider-17m'] | ||
| """ | ||
|
|
||
| def __init__(self, gauge_name, valid_gauges=None, message=None): | ||
| """Initialize InvalidGaugeError. | ||
|
|
||
| Args: | ||
| gauge_name: The invalid gauge identifier. | ||
| valid_gauges: Optional list/tuple of valid gauge identifiers. | ||
| message: Optional custom message. If not provided, a default | ||
| message is generated from gauge_name and valid_gauges. | ||
| """ | ||
| self.gauge_name = gauge_name | ||
| self.valid_gauges = valid_gauges | ||
| if message is None: | ||
| message = f"Invalid gauge: '{gauge_name}'" | ||
| if valid_gauges: | ||
| message += f". Valid options: {valid_gauges}" | ||
| self.message = message | ||
| super().__init__(self.message) | ||
|
|
||
|
|
||
| class InvalidTimeRangeError(MurgToolsError): | ||
| """Raised when time range is invalid or contains no data. | ||
|
|
||
| This exception is raised when: | ||
| - Start time is after end time | ||
| - The time range contains no data | ||
| - The time range is outside available data bounds | ||
|
|
||
| Attributes: | ||
| start_time: The start of the requested time range. | ||
| end_time: The end of the requested time range. | ||
| message (str): Explanation of the time range error. | ||
| """ | ||
|
|
||
| def __init__(self, message, start_time=None, end_time=None): | ||
| """Initialize InvalidTimeRangeError. | ||
|
|
||
| Args: | ||
| message: Description of the time range error. | ||
| start_time: Optional start time for context. | ||
| end_time: Optional end time for context. | ||
| """ | ||
| self.start_time = start_time | ||
| self.end_time = end_time | ||
| self.message = message | ||
| super().__init__(self.message) | ||
|
|
||
|
|
||
| class NetworkError(MurgToolsError): | ||
| """Raised when network operations fail. | ||
|
|
||
| This exception wraps network-related errors such as connection failures, | ||
| timeouts, or server errors when accessing THREDDS or other data servers. | ||
|
|
||
| Attributes: | ||
| url (str): The URL that failed (if applicable). | ||
| original_error: The underlying exception that caused the failure. | ||
| message (str): Description of the network error. | ||
| """ | ||
|
|
||
| def __init__(self, message, url=None, original_error=None): | ||
| """Initialize NetworkError. | ||
|
|
||
| Args: | ||
| message: Description of the network error. | ||
| url: Optional URL that failed. | ||
| original_error: Optional underlying exception. | ||
| """ | ||
| self.url = url | ||
| self.original_error = original_error | ||
| self.message = message | ||
| super().__init__(self.message) | ||
|
|
||
|
|
||
| class InvalidParameterError(MurgToolsError): | ||
| """Raised when a function parameter is invalid. | ||
|
|
||
| This exception is raised when a function receives an argument | ||
| that doesn't meet its requirements (wrong type, out of range, | ||
| invalid format, etc.). | ||
|
|
||
| Attributes: | ||
| parameter_name (str): Name of the invalid parameter. | ||
| value: The invalid value that was provided. | ||
| expected: Description of what was expected. | ||
| message (str): Full error message. | ||
| """ | ||
|
|
||
| def __init__(self, message, parameter_name=None, value=None, expected=None): | ||
| """Initialize InvalidParameterError. | ||
|
|
||
| Args: | ||
| message: Description of the parameter error. | ||
| parameter_name: Optional name of the invalid parameter. | ||
| value: Optional invalid value that was provided. | ||
| expected: Optional description of expected value/type. | ||
| """ | ||
| self.parameter_name = parameter_name | ||
| self.value = value | ||
| self.expected = expected | ||
| self.message = message | ||
| super().__init__(self.message) |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.