[MVP-1] Geodesy Notebook 1 - #3
Open
eshntmshra wants to merge 6 commits into
Open
Conversation
eshntmshra
changed the title
Contributor
Author
|
Added a new notebook that uses the EarthScope SDK for data retrieval. This is an introductory notebook that gives a brief introduction of GNSS data and teaches the users how to access it. |
sarahw-earthscope
requested changes
Jul 24, 2026
| "id": "35320cb4-aed2-4420-8f9c-b3deb365e792", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "**Version:** 1.0 | **Last updated:** 2026-07-09 | **Author:** Eshanta Mishra" |
Contributor
There was a problem hiding this comment.
Add affiliation: EarthScope Consortium, maintainer: EarthScope OnRamp Team, maintainer contact: help@earthscope.org
| "id": "811fc68e-1e81-482c-82dd-5efa2a89d715", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Learning Objectives\n", |
Contributor
There was a problem hiding this comment.
Is this meant to repeat what is in the introduction/what you will accomplish section?
Contributor
Author
There was a problem hiding this comment.
My idea is that the introduction/what you will accomplish section is like a summary of the sections/areas that the notebook demonstrates. Learning objective is what the user is expected to learn after completing this notebook. I agree that there is overlap here. I'm happy to merge them if needed.
| "\n", | ||
| "**Raw Observations vs. derived products**\n", | ||
| "\n", | ||
| "The data we retrieve in this notebook are raw observations. The geodetic results you may ultimately want such a station's position, displacement over time, etc. are derived products. These derived products are computed by processing many observations together. " |
Contributor
There was a problem hiding this comment.
minor text edits:
- such as
- etc.,
| "id": "52c5e185-8dca-42ff-8a9d-89c23297a761", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Since you use your EarthScope account to log into GeoLab, your EarthScope credentials are always available inside it. So, the client authenticates automatically. This removes the step of logging in or passing tokens manually, as you would do in a non GeoLab Environment." |
Contributor
There was a problem hiding this comment.
@tkbravo - How much authentication content is remaining in Cloud Foundations?
@eshntmshra I think we whould we add a note here, "if you are running this notebook outside geolab, you will need to follow steps (link) to add your authentication here"
| "id": "421226b8-42bc-4d91-a517-469c29fa6e85", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "`AsyncEarthScopeClient` is the asynchronous client. Async lets the SDK process a large query into many concurrent sub-requests, making large quantity of data retrieval more efficient. The SDK also comes with a synchronous `EarthScopeClient` if you prefer." |
Contributor
There was a problem hiding this comment.
Add hyperlink to sdk documentation where this is discussed
Contributor
Author
There was a problem hiding this comment.
I did, but the sdk documentation discussion is not very informative.
| "\n", | ||
| "**Why it matters**: Some requests such as an entire network for a week will not fit in memory at once. Query plans also limit how many requests hit the API at the same time. Each group's sub-requests run in parallel and are collected into a single table you can process before moving on.\n", | ||
| "\n", | ||
| "The expected output is a summary for each group (row count, time span, and stations), rather than one huge dataframe." |
Contributor
There was a problem hiding this comment.
When I ran this, all that printed was "AsyncGnssObservationsQueryPlan(unplanned)" - is this expected?
Contributor
Author
There was a problem hiding this comment.
You should be getting: AsyncGnssObservationsQueryPlan(requests=40, groups=40)
I reran and got the correct output as above.
| "\n", | ||
| "\n", | ||
| "def summarize(table):\n", | ||
| " # Print a quick summary of one group's table.\n", |
Contributor
There was a problem hiding this comment.
perhaps add a commented out line or two of code that would calculate something more exciting, or call out exactly where someone would add their processing steps
| "id": "8e88b724-65ec-4ef7-9e19-fb6bbf8c895b", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "### A rough guide for choosing a strategy" |
| "\n", | ||
| "2. **Isolate one signal:** In Section 5, request a single `obs_code` (e.g. `\"5Q\"`) with `field=\"snr\"`. How much smaller is the result?\n", | ||
| "\n", | ||
| "3. **Save your results:** Write a fetched dataframe to the scratch directory (*Hint: Use `os.environ[\"SCRATCH_BUCKET\"]`*) as Parquet with `df.write_parquet(...)`, then read it back. Parquet preserves types and is far smaller than CSV." |
Contributor
There was a problem hiding this comment.
Please write an example for this in one of the earlier steps, and maybe have them repeat it for a different dataframe here as an exercise. Writing to scratch and understanding different storage locations is critical. I know it doesn't show up in the plan until MVP2, but it makes sense to introduce here.
Contributor
There was a problem hiding this comment.
Is this meant to be a duplicate file?
Contributor
Author
There was a problem hiding this comment.
This is the main file actually. The other file was a hidden checkpoint file that got pushed alongside this one. Since they are identical, I will make changes according to the comments on this file and delete the checkpoint file during next commit.
There was a problem hiding this comment.
Please go ahead and remove the checkpoint file so we are all reviewing the same file.
hamilton-earthscope
left a comment
hamilton-earthscope
left a comment
There was a problem hiding this comment.
Overall, looks good. Some nitpicking on wording.
| "\n", | ||
| "**What this notebook does:** It instantiates the EarthScope SDK and allows you to retrieve GNSS observations for your station of interest.\n", | ||
| "\n", | ||
| "**Why it is useful:** This notebook provides a hands-on on how to use the SDK to retrieve GNSS observations from the cloud and load it into a dataframe. It provides the users an insight into how cloud based workflows can be more efficient compared to traditional methods of downloading and accessing GNSS observations as RINEX files.\n", |
There was a problem hiding this comment.
The SDK does not require the user to be in the cloud. Only the part about saving to scratch is really Geolab-specific. But they could just as easily write parquet out to local disk if running outside Geolab.
Suggested change
| "**Why it is useful:** This notebook provides a hands-on on how to use the SDK to retrieve GNSS observations from the cloud and load it into a dataframe. It provides the users an insight into how cloud based workflows can be more efficient compared to traditional methods of downloading and accessing GNSS observations as RINEX files.\n", | |
| "**Why it is useful:** This notebook provides a hands-on on how to use the SDK to retrieve GNSS observations from the cloud and load it into a dataframe. It provides the users an insight into how modern workflows can be more efficient compared to traditional methods of downloading and accessing GNSS observations as RINEX files.\n", |
Comment on lines
+66
to
+71
| "## Learning Objectives\n", | ||
| "\n", | ||
| "By the end of this notebook, you will be able to:\n", | ||
| "\n", | ||
| "1. Instantiate EarthScope SDK client\n", | ||
| "2. Retrieve and filter GNSS data from EarthScope" |
There was a problem hiding this comment.
The feels like a repeat of the What you will accomplish: section above
| "id": "9e03f7ee-00b4-4501-bd77-551d49ca52a6", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "A GNSS (Global Navigation Satellite System) is a constellation of satellites that broadcast timing signals. GNSS includes constellations such as GPS (United States), GLONASS (Russia), BeiDou (China) etc. A ground station (receiver plus antenna) records the data transmitted by these satellites several times per minute.\n", |
There was a problem hiding this comment.
Suggested change
| "A GNSS (Global Navigation Satellite System) is a constellation of satellites that broadcast timing signals. GNSS includes constellations such as GPS (United States), GLONASS (Russia), BeiDou (China) etc. A ground station (receiver plus antenna) records the data transmitted by these satellites several times per minute.\n", | |
| "A Global Navigation Satellite System (GNSS) is a constellation of satellites that broadcast timing signals. GNSS includes constellations such as GPS (United States), GLONASS (Russia), BeiDou (China) etc. A ground station (receiver plus antenna) records the data transmitted by these satellites several times per minute.\n", |
| "id": "52c5e185-8dca-42ff-8a9d-89c23297a761", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Since you use your EarthScope account to log into GeoLab, your EarthScope credentials are always available inside it. So, the client authenticates automatically. This removes the step of logging in or passing tokens manually, as you would do in a non GeoLab Environment. However, if you are running this notebook outside geolab, you will need to authenticate manually using your EarthScope account credentials through the EarthScope CLI (Detailed instructions for doing this can be found [here](https://gitlab.com/earthscope/public/earthscope-cli))." |
There was a problem hiding this comment.
Suggested change
| "Since you use your EarthScope account to log into GeoLab, your EarthScope credentials are always available inside it. So, the client authenticates automatically. This removes the step of logging in or passing tokens manually, as you would do in a non GeoLab Environment. However, if you are running this notebook outside geolab, you will need to authenticate manually using your EarthScope account credentials through the EarthScope CLI (Detailed instructions for doing this can be found [here](https://gitlab.com/earthscope/public/earthscope-cli))." | |
| "Since you use your EarthScope account to log into GeoLab, your EarthScope credentials are already available inside it. The client finds your credentials automatically. This removes the step of logging in again using the CLI or passing tokens manually, as you would do in a non-GeoLab Environment. If you are running this notebook outside geolab, you will need to authenticate using the EarthScope CLI (detailed instructions for doing this can be found [here](https://gitlab.com/earthscope/public/earthscope-cli))." |
| "id": "421226b8-42bc-4d91-a517-469c29fa6e85", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "`AsyncEarthScopeClient` is the asynchronous client. [Async](https://docs.earthscope.org/sdk/usage#async-usage) lets the SDK process a large query into many concurrent sub-requests, making large quantity of data retrieval more efficient. The SDK also comes with a synchronous `EarthScopeClient` if you prefer." |
There was a problem hiding this comment.
Suggested change
| "`AsyncEarthScopeClient` is the asynchronous client. [Async](https://docs.earthscope.org/sdk/usage#async-usage) lets the SDK process a large query into many concurrent sub-requests, making large quantity of data retrieval more efficient. The SDK also comes with a synchronous `EarthScopeClient` if you prefer." | |
| "`AsyncEarthScopeClient` is the asynchronous client. [Async](https://docs.earthscope.org/sdk/usage#async-usage) lets the SDK process a large query into many concurrent sub-requests, making large quantity of data retrieval more efficient. The SDK also comes with a synchronous `EarthScopeClient` if you prefer. The methods used below are identical on the synchronous client, just remove the `async` / `await` keywords." |
Suggest also linking to some other resource (not one we maintain) explaining the difference between python sync/async programming.
| "| `obs_code` | str | Which signal was measured, e.g. `1C`, `2W`, `2L` (decoded below). |\n", | ||
| "| `range` | float | Pseudorange in meters. It is the apparent satellite to receiver distance. |\n", | ||
| "| `phase` | float | Carrier phase in cycles. It is a precise but ambiguous range measurement. |\n", | ||
| "| `snr` | float | Signal strength as carrier-to-noise density ($C/N_0$), roughly in dB-Hz. Higher is cleaner. |\n", |
There was a problem hiding this comment.
Suggested change
| "| `snr` | float | Signal strength as carrier-to-noise density ($C/N_0$), roughly in dB-Hz. Higher is cleaner. |\n", | |
| "| `snr` | float | Signal strength as carrier-to-noise density ($C/N_0$), in dB-Hz. Higher values correspond to stronger signal. |\n", |
| "id": "c0a4e066-7b6b-4161-9260-a4a825637f2a", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "When you are working with RINEX files, the files hand you everything, all fields, constellations, satellites etc. Most analyses only need a small slice of these data. Passing filters to `gnss_observations()` in the SDK pushes that selection to the server, so only the data that you asked for is transferred.\n", |
There was a problem hiding this comment.
Suggested change
| "When you are working with RINEX files, the files hand you everything, all fields, constellations, satellites etc. Most analyses only need a small slice of these data. Passing filters to `gnss_observations()` in the SDK pushes that selection to the server, so only the data that you asked for is transferred.\n", | |
| "When you are working with RINEX files, the files include everything: all fields, constellations, satellites etc. Many analyses only need a small slice of these data. Passing filters to `gnss_observations()` in the SDK pushes that selection to the server, so only the data that you asked for is transferred.\n", |
| "source": [ | ||
| "When you are working with RINEX files, the files hand you everything, all fields, constellations, satellites etc. Most analyses only need a small slice of these data. Passing filters to `gnss_observations()` in the SDK pushes that selection to the server, so only the data that you asked for is transferred.\n", | ||
| "\n", | ||
| "The payoff is immediate: requesting a single signal from a single satellite over two months returns in a fraction of the time and requires smaller memory size, where full RINEX for the same window would be orders of magnitude larger and mostly discarded. You also skip RINEX parsing entirely because the results arrive as an Arrow table ready to load straight into a dataframe, rather than a set of daily text files you have to decode first.\n", |
There was a problem hiding this comment.
Suggested change
| "The payoff is immediate: requesting a single signal from a single satellite over two months returns in a fraction of the time and requires smaller memory size, where full RINEX for the same window would be orders of magnitude larger and mostly discarded. You also skip RINEX parsing entirely because the results arrive as an Arrow table ready to load straight into a dataframe, rather than a set of daily text files you have to decode first.\n", | |
| "The payoff is immediate: requesting a single signal from a single satellite over two months returns in a fraction of the time and requires less memory, where full RINEX for the same window would be orders of magnitude larger and mostly discarded. You also skip RINEX parsing entirely because the results arrive as an Arrow table, rather than a set of daily text files you have to decode first.\n", |
| "| Location | Path | Persistence | Use it for |\n", | ||
| "|---|---|---|---|\n", | ||
| "| **Home** | `/home/jovyan/` | Private, persistent, 50 GB limit | Notebooks, scripts, and results you want to keep while working on your project |\n", | ||
| "| **Shared** | `/home/jovyan/shared/` | Read-only | Datasets and starter notebooks placed there by instructors. You can copy files out, but not modify them |\n", |
There was a problem hiding this comment.
Suggested change
| "| **Shared** | `/home/jovyan/shared/` | Read-only | Datasets and starter notebooks placed there by instructors. You can copy files out, but not modify them |\n", | |
| "| **Shared** | `/home/jovyan/shared/` | Read-only | Datasets and starter notebooks placed there by instructors. You can copy files out, but not modify the original |\n", |
| "\n", | ||
| "**Try these modifications:**\n", | ||
| "\n", | ||
| "1. **Change the station:** Set `STATION` in the Configuration section to a different 4-character station ID and re-run Section 3. Does the station report the same constellations?\n", |
There was a problem hiding this comment.
Suggested change
| "1. **Change the station:** Set `STATION` in the Configuration section to a different 4-character station ID and re-run Section 3. Does the station report the same constellations?\n", | |
| "1. **Change the station:** Set `STATION` in the Configuration section to a different 9-character station ID and re-run Section 3. Does the station report the same constellations?\n", |
eshntmshra
marked this pull request as ready for review
August 6, 2026 17:40
eshntmshra
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft PR for the first geodesy notebook.
Based on Alex's comments on slack, this might not be as useful or the recommended way of accessing EarthScope GNSS data. We can discard this notebook if not required.