From 23cd45280a473bf42d1eaf903a81a9d4ef32db50 Mon Sep 17 00:00:00 2001 From: leewujung Date: Sun, 5 Jun 2022 14:51:12 -0700 Subject: [PATCH 1/8] add proposed rerog of ad2cp data --- docs/source/ad2cp_reorg.md | 176 +++++++++++++++++++++++++++++++++++++ 1 file changed, 176 insertions(+) create mode 100644 docs/source/ad2cp_reorg.md diff --git a/docs/source/ad2cp_reorg.md b/docs/source/ad2cp_reorg.md new file mode 100644 index 000000000..335ea48f2 --- /dev/null +++ b/docs/source/ad2cp_reorg.md @@ -0,0 +1,176 @@ +## Questions/comments: +- why check the version again? + - https://github.com/OSOceanAcoustics/echopype/blob/16a574dda792a61f6f7ae0583b70b4b87d787f12/echopype/convert/parse_ad2cp.py#L589-L593 + - https://github.com/OSOceanAcoustics/echopype/blob/16a574dda792a61f6f7ae0583b70b4b87d787f12/echopype/convert/parse_ad2cp.py#L630-L634 +- Move `AHRS_COORDS` def to `parse_ad2cp`? + - https://github.com/OSOceanAcoustics/echopype/blob/16a574dda792a61f6f7ae0583b70b4b87d787f12/echopype/convert/set_groups_ad2cp.py#L10-L14 +- the `beam` coordinate is missing in `rawtest.090.00015.nc`? +- `echosounder_raw_beam`, `echosounder_raw_echogram`: not sure what these correspond to +- `status`, `status0`: need additional parsing + + + + + + +# The original `Beam` group + +## Which mode goes to which group +Proposal/question: +- use specific group name for different mode +- this means that then some groups may not exist all together, i.e., there won't always be `Beam_group1` in the converted data file +- this conflicts with what we have for EK80, where there will always be `Beam_group1` but the content changes depending on what are in the file (power/angle samples only, complex samples only, or both) +- if we go with this approach, we can use the following: + - `Beam_group1`: average + - `Beam_group2`: burst + - `Beam_group3`: echosounder + - `Beam_group4`: echosounder raw samples + - the first ping is the raw transmit signal + - the rest of pings are the receive raw echoes + +## Variables common to all modes +Coordinates: +- `ping_time`: use the original `ping_time_average`, `ping_time_burst` and `ping_time_echosounder` in each of their own `Beam_groupX` but with the same name `ping_time` +- `range_sample`: use the original `range_sample_average`, `range_sample_burst` and `range_sample_echosounder` in each of their own `Beam_groupX` but with the same name `range_sample` +- `beam`: + - the actual physical beam activated in the setting (1, 2, 3, 4, 5) + - parsed from the data variable `data_set_description` using `parse_ad2cp._postprocess_beams` for the following packets: `BURST_AVERAGE_VERSION3_DATA_RECORD_FORMAT`, `BURST_AVERAGE_VERSION2_DATA_RECORD_FORMAT`, `BOTTOM_TRACK_DATA_RECORD_FORMAT` + - this is missing in `rawtest.090.00008.nc`?? + +Data variables: +- `number_of_beams` +- `coordinate_system` +- `number_of_cells` +- `blanking` +- `cell_size` +- `velocity_range` +- `echosounder_frequency` +- `ambiguity_velocity` +- `data_set_description` +- `transmit_energy` +- `velocity_scaling` +- `velocity`: separate `velocity_burst` and `velocity_average` into different `Beam_groupX` and use the name `velocity` +- `amplitude`: separate `amplitude_burst`, `amplitude_average` and `amplitude_echosounder` into different `Beam_groupX` and use the name `amplitude` +- `correlation`: separate `correlation_burst`, `correlation_average` and `correlation_echosounder` into different `Beam_groupX` and use the name `correlation` + +Attributes: +- `pulse_compressed` + - this probably should be a variable? + - should go to `Beam_group3` that stores the echosounder mode data + - can have a max length of 3 since there can be 3 echograms in the echosounder mode (aligned with `beam`): is this supported currently? + + +## Move from `Beam` to `Platform` group +- `figure_of_merit`: for bottom tracking +- `altimeter_distance` +- `altimeter_quality` +- `altimeter_spare` +- `altimeter_raw_data_num_samples` +- `altimeter_raw_data_sample_distance` +- `altimeter_raw_data_samples` +- `ast_distance` +- `ast_quality` +- `ast_offset_100us` +- `ast_pressure` + + + + + + + +# `Vendor_specific` group +## Remove +- `pulse_compressed`: this already is/will be in `Beam_group3` (echosounder mode data) + +## Move from `Vendor_specific` to `Platform` group +- `ahrs_rotation_matrix_mij` +- `ahrs_quaternions_wxyz` +- `ahrs_gyro_xyz` +- `std_dev_pitch` +- `std_dev_roll` +- `std_dev_heading` +- `std_dev_pressure` +- `compass_sensor_valid` +- `tilt_sensor_valid` + +## Move from `Vendor_specific` to `Environment` group +Coordinates: +- `time1`: this will be the combined ping_time from all modes + +Data variables: +- `temperature_of_pressure_sensor`: rename to "temperature_pressure_sensor" +- `magnetometer_temperature`: rename to "temperature_magnetometer" +- `real_ping_time_clock_temperature`: rename to "temperature_real_ping_time_clock" +- `pressure_sensor_valid` +- `temperature_sensor_valid` + +## Move from `Vendor_specific` to each of the `Beam_groupX` groups +Coordinates: `ping_time` for that group (not the combined one) + +Data variables: +- `data_record_version` +- `error` +- `status`: need additional parsing +- `status0`: need additional parsing +- `power_level` +- `nominal_correlation` +- `percentage_good_data` +- `battery_voltage` +- `ensemble_counter` + +## Move from `Vendor_specific` to `Beam_group4` group +Coordinates: +- `ping_time` for that group (not the combined one) +- `sample`: rename to `range_sample`; this is sample number along range for raw echosounder data +- `sample_transmit`: rename to `transmit_sample` (to be consistent with what's used for Simrad RAW4) + +Data variables: +- `echosounder_raw_samples_i` +- `echosounder_raw_samples_q` +- `echosounder_raw_transmit_samples_i` +- `echosounder_raw_transmit_samples_q` +- `echosounder_raw_beam`: not sure what this corresponds to, but couldn't find a variable that correpsonds to `beam` which is the physical beam used in transmission +- `echosounder_raw_echogram`: not sure what this corresponds to + + + + + + + +# The original `Environment` group +Coordinates: +- `time1`: this will be the combined ping_time from all modes + +Data variables: +- `sound_speed_indicative` +- `temperature` +- `pressure` + + + + + + + +# The original `Platform` group +Coordinates +- `time1`: this will be the combined ping_time from all modes + +Data variables: +- `heading` +- `pitch` +- `roll` + +## Move from `Platform` group to `Vendor_specific` group +Coordinates: +- `time1`: this will be the combined ping_time from all modes +- `xyz`: coordinate for `magnetometer_raw` + +Data variable: +- `magnetometer_raw` + + + + From 59b1bc679c7e9b58ce812c394b6b48c9598dfb34 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Sun, 5 Jun 2022 22:01:13 +0000 Subject: [PATCH 2/8] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/ad2cp_reorg.md | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/docs/source/ad2cp_reorg.md b/docs/source/ad2cp_reorg.md index 335ea48f2..eedfafc10 100644 --- a/docs/source/ad2cp_reorg.md +++ b/docs/source/ad2cp_reorg.md @@ -16,7 +16,7 @@ # The original `Beam` group ## Which mode goes to which group -Proposal/question: +Proposal/question: - use specific group name for different mode - this means that then some groups may not exist all together, i.e., there won't always be `Beam_group1` in the converted data file - this conflicts with what we have for EK80, where there will always be `Beam_group1` but the content changes depending on what are in the file (power/angle samples only, complex samples only, or both) @@ -170,7 +170,3 @@ Coordinates: Data variable: - `magnetometer_raw` - - - - From 465dc4fc63ca0cbe71074899d0fb830abcd638d6 Mon Sep 17 00:00:00 2001 From: leewujung Date: Mon, 13 Jun 2022 11:37:11 -0700 Subject: [PATCH 3/8] refactor the reorg proposal based on discussion last Thur --- docs/source/ad2cp_reorg.md | 121 ++++++++++++++++++++++--------------- 1 file changed, 72 insertions(+), 49 deletions(-) diff --git a/docs/source/ad2cp_reorg.md b/docs/source/ad2cp_reorg.md index eedfafc10..c592cbf04 100644 --- a/docs/source/ad2cp_reorg.md +++ b/docs/source/ad2cp_reorg.md @@ -1,66 +1,92 @@ -## Questions/comments: +## Questions/comments for @imranmaj: - why check the version again? - https://github.com/OSOceanAcoustics/echopype/blob/16a574dda792a61f6f7ae0583b70b4b87d787f12/echopype/convert/parse_ad2cp.py#L589-L593 - https://github.com/OSOceanAcoustics/echopype/blob/16a574dda792a61f6f7ae0583b70b4b87d787f12/echopype/convert/parse_ad2cp.py#L630-L634 - Move `AHRS_COORDS` def to `parse_ad2cp`? - https://github.com/OSOceanAcoustics/echopype/blob/16a574dda792a61f6f7ae0583b70b4b87d787f12/echopype/convert/set_groups_ad2cp.py#L10-L14 - the `beam` coordinate is missing in `rawtest.090.00015.nc`? -- `echosounder_raw_beam`, `echosounder_raw_echogram`: not sure what these correspond to -- `status`, `status0`: need additional parsing +## Overarching action items: +- Double check all raw to actual units conversion +- For all variables, put + - `long_name`: Field in spec sheet + - `comment`: Description in spec sheet + - `units`: following spec sheet and the units convention in [CF convention](https://cfconventions.org/Data/cf-standard-names/current/build/cf-standard-name-table.html) + - for those that are already converted to the actual units, put "but already converted to actual units here." + - for example: for the field `Temperature`, put + ``` + { + "long_name": "Field "Temperature" in the data specification. " + "comment": "Reading from the temperature sensor. " + "Raw data given as 0.01 °C but already converted to actual units here." + "units": "degree_C" + } + ``` # The original `Beam` group ## Which mode goes to which group -Proposal/question: -- use specific group name for different mode -- this means that then some groups may not exist all together, i.e., there won't always be `Beam_group1` in the converted data file -- this conflicts with what we have for EK80, where there will always be `Beam_group1` but the content changes depending on what are in the file (power/angle samples only, complex samples only, or both) -- if we go with this approach, we can use the following: - - `Beam_group1`: average - - `Beam_group2`: burst - - `Beam_group3`: echosounder - - `Beam_group4`: echosounder raw samples - - the first ping is the raw transmit signal - - the rest of pings are the receive raw echoes +- following SONAR-netCDF4 and our implementation of EK80 groups, we will use the following sequence when determining where data from each mode goes in `Beam_groupX`: + 1. average + 2. burst + 3. echosounder + 4. echosounder raw samples +- examples: + - example 1: file with average and burst mode: + - `Beam_group1`: average + - `Beam_group2`: burst + - example 2: files with burst and echosounder mode: + - `Beam_group1`: burst + - `Beam_group2`: echosounder + - example 3: files with echosounder mode and raw echosounder data: + - `Beam_group1`: echosounder + - `Beam_group2`: echosounder raw samples +- for echosounder raw samples + - the first ping is the raw transmit signal + - the rest of pings are the receive raw echoes ## Variables common to all modes Coordinates: - `ping_time`: use the original `ping_time_average`, `ping_time_burst` and `ping_time_echosounder` in each of their own `Beam_groupX` but with the same name `ping_time` - `range_sample`: use the original `range_sample_average`, `range_sample_burst` and `range_sample_echosounder` in each of their own `Beam_groupX` but with the same name `range_sample` - `beam`: - - the actual physical beam activated in the setting (1, 2, 3, 4, 5) - - parsed from the data variable `data_set_description` using `parse_ad2cp._postprocess_beams` for the following packets: `BURST_AVERAGE_VERSION3_DATA_RECORD_FORMAT`, `BURST_AVERAGE_VERSION2_DATA_RECORD_FORMAT`, `BOTTOM_TRACK_DATA_RECORD_FORMAT` - - this is missing in `rawtest.090.00008.nc`?? + - the actual physical beam activated in the setting (1, 2, 3, 4, 5): @imranmaj please double check this + - parsed from the data variable `data_set_description` using `parse_ad2cp._postprocess_beams` for the following packets: + - `BURST_AVERAGE_VERSION3_DATA_RECORD_FORMAT` + - `BURST_AVERAGE_VERSION2_DATA_RECORD_FORMAT` + - `BOTTOM_TRACK_DATA_RECORD_FORMAT` Data variables: - `number_of_beams` - `coordinate_system` - `number_of_cells` - `blanking` -- `cell_size` +- `cell_size`: this is conceptually equivalent to `sample_interval` for the other echosounders, just that `sample_interval` is defined in time (second) and `cell_size` is defined in space (meter) - `velocity_range` - `echosounder_frequency` - `ambiguity_velocity` - `data_set_description` - `transmit_energy` - `velocity_scaling` -- `velocity`: separate `velocity_burst` and `velocity_average` into different `Beam_groupX` and use the name `velocity` -- `amplitude`: separate `amplitude_burst`, `amplitude_average` and `amplitude_echosounder` into different `Beam_groupX` and use the name `amplitude` -- `correlation`: separate `correlation_burst`, `correlation_average` and `correlation_echosounder` into different `Beam_groupX` and use the name `correlation` +- `velocity` + - separate `velocity_burst` and `velocity_average` into different `Beam_groupX` and use the name `velocity` +- `amplitude` + - separate `amplitude_burst`, `amplitude_average` and `amplitude_echosounder` into different `Beam_groupX` and use the name `amplitude` +- `correlation` + - separate `correlation_burst`, `correlation_average` and `correlation_echosounder` into different `Beam_groupX` and use the name `correlation` Attributes: - `pulse_compressed` - - this probably should be a variable? - - should go to `Beam_group3` that stores the echosounder mode data - - can have a max length of 3 since there can be 3 echograms in the echosounder mode (aligned with `beam`): is this supported currently? + - this should be a variable in the `Beam_groupX` that stores echosounder mode data + - can have a max length of 3 since there can be 3 echograms in the echosounder mode (aligned with `beam`) + - @imranmaj: now sure what the current status of this is? -## Move from `Beam` to `Platform` group +## Move from `Beam` to `Vendor_specific` group - `figure_of_merit`: for bottom tracking - `altimeter_distance` - `altimeter_quality` @@ -83,7 +109,10 @@ Attributes: ## Remove - `pulse_compressed`: this already is/will be in `Beam_group3` (echosounder mode data) -## Move from `Vendor_specific` to `Platform` group +## Keep in `Vendor_specific` group +- Questions: + - flag for whether these data will be collected? + - when collected, what their expected dimensions are? - `ahrs_rotation_matrix_mij` - `ahrs_quaternions_wxyz` - `ahrs_gyro_xyz` @@ -94,44 +123,37 @@ Attributes: - `compass_sensor_valid` - `tilt_sensor_valid` -## Move from `Vendor_specific` to `Environment` group -Coordinates: -- `time1`: this will be the combined ping_time from all modes - -Data variables: -- `temperature_of_pressure_sensor`: rename to "temperature_pressure_sensor" -- `magnetometer_temperature`: rename to "temperature_magnetometer" -- `real_ping_time_clock_temperature`: rename to "temperature_real_ping_time_clock" +### Data collected for every ping in all modes +- Coordinate `time1`: + - this will be the combined `ping_time` from all modes +- `temperature_of_pressure_sensor`: rename to `temperature_pressure_sensor` +- `magnetometer_temperature`: rename to `temperature_magnetometer` +- `real_ping_time_clock_temperature`: rename to `temperature_real_ping_time_clock` - `pressure_sensor_valid` - `temperature_sensor_valid` - -## Move from `Vendor_specific` to each of the `Beam_groupX` groups -Coordinates: `ping_time` for that group (not the combined one) - -Data variables: - `data_record_version` - `error` -- `status`: need additional parsing -- `status0`: need additional parsing +- `status`: need additional parsing @imranmaj +- `status0`: need additional parsing @imranmaj - `power_level` - `nominal_correlation` - `percentage_good_data` - `battery_voltage` - `ensemble_counter` -## Move from `Vendor_specific` to `Beam_group4` group +## Move from `Vendor_specific` to `Beam_groupX` group Coordinates: - `ping_time` for that group (not the combined one) - `sample`: rename to `range_sample`; this is sample number along range for raw echosounder data -- `sample_transmit`: rename to `transmit_sample` (to be consistent with what's used for Simrad RAW4) +- `sample_transmit`: rename to `transmit_sample` (following the proposed new variable in PR#714) Data variables: -- `echosounder_raw_samples_i` -- `echosounder_raw_samples_q` -- `echosounder_raw_transmit_samples_i` -- `echosounder_raw_transmit_samples_q` -- `echosounder_raw_beam`: not sure what this corresponds to, but couldn't find a variable that correpsonds to `beam` which is the physical beam used in transmission -- `echosounder_raw_echogram`: not sure what this corresponds to +- `echosounder_raw_samples_i`: rename to `backscatter_r` +- `echosounder_raw_samples_q`: rename to `backscatter_i` +- `echosounder_raw_transmit_samples_i`: rename to `transmit_pulse_r` (following the proposed new variable in PR#714) +- `echosounder_raw_transmit_samples_q`: rename to `transmit_pulse_i` (following the proposed new variable in PR#714) +- `echosounder_raw_beam`: not sure what this corresponds to, but couldn't find a variable that correpsonds to `beam` which is the physical beam used in transmission @imranmaj +- `echosounder_raw_echogram`: not sure what this corresponds to @imranmaj @@ -162,6 +184,7 @@ Data variables: - `heading` - `pitch` - `roll` +- `tilt`: is there such a field defined in the spec sheet? I didn't see any in the example files @imranmaj ## Move from `Platform` group to `Vendor_specific` group Coordinates: From 1a432c24243df4b59c861d81e39e874dc80c01fc Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 13 Jun 2022 18:40:36 +0000 Subject: [PATCH 4/8] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/ad2cp_reorg.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/source/ad2cp_reorg.md b/docs/source/ad2cp_reorg.md index c592cbf04..eb9af0886 100644 --- a/docs/source/ad2cp_reorg.md +++ b/docs/source/ad2cp_reorg.md @@ -15,7 +15,7 @@ - `comment`: Description in spec sheet - `units`: following spec sheet and the units convention in [CF convention](https://cfconventions.org/Data/cf-standard-names/current/build/cf-standard-name-table.html) - for those that are already converted to the actual units, put "but already converted to actual units here." - - for example: for the field `Temperature`, put + - for example: for the field `Temperature`, put ``` { "long_name": "Field "Temperature" in the data specification. " @@ -57,7 +57,7 @@ Coordinates: - the actual physical beam activated in the setting (1, 2, 3, 4, 5): @imranmaj please double check this - parsed from the data variable `data_set_description` using `parse_ad2cp._postprocess_beams` for the following packets: - `BURST_AVERAGE_VERSION3_DATA_RECORD_FORMAT` - - `BURST_AVERAGE_VERSION2_DATA_RECORD_FORMAT` + - `BURST_AVERAGE_VERSION2_DATA_RECORD_FORMAT` - `BOTTOM_TRACK_DATA_RECORD_FORMAT` Data variables: @@ -110,9 +110,9 @@ Attributes: - `pulse_compressed`: this already is/will be in `Beam_group3` (echosounder mode data) ## Keep in `Vendor_specific` group -- Questions: +- Questions: - flag for whether these data will be collected? - - when collected, what their expected dimensions are? + - when collected, what their expected dimensions are? - `ahrs_rotation_matrix_mij` - `ahrs_quaternions_wxyz` - `ahrs_gyro_xyz` @@ -124,7 +124,7 @@ Attributes: - `tilt_sensor_valid` ### Data collected for every ping in all modes -- Coordinate `time1`: +- Coordinate `time1`: - this will be the combined `ping_time` from all modes - `temperature_of_pressure_sensor`: rename to `temperature_pressure_sensor` - `magnetometer_temperature`: rename to `temperature_magnetometer` From 04a84938f4b17d218aff93ac7f7091299965cbcc Mon Sep 17 00:00:00 2001 From: leewujung Date: Mon, 13 Jun 2022 11:51:08 -0700 Subject: [PATCH 5/8] add description for each beam_groupX --- docs/source/ad2cp_reorg.md | 37 ++++++++++++++++++++++++++++++++++--- 1 file changed, 34 insertions(+), 3 deletions(-) diff --git a/docs/source/ad2cp_reorg.md b/docs/source/ad2cp_reorg.md index c592cbf04..0c802ab82 100644 --- a/docs/source/ad2cp_reorg.md +++ b/docs/source/ad2cp_reorg.md @@ -19,8 +19,11 @@ ``` { "long_name": "Field "Temperature" in the data specification. " - "comment": "Reading from the temperature sensor. " - "Raw data given as 0.01 °C but already converted to actual units here." + "comment": ( + "Reading from the temperature sensor. " + "Raw data given as 0.01 °C " + "but already converted to actual units here." + ) "units": "degree_C" } ``` @@ -31,10 +34,38 @@ ## Which mode goes to which group - following SONAR-netCDF4 and our implementation of EK80 groups, we will use the following sequence when determining where data from each mode goes in `Beam_groupX`: - 1. average + 1. average: + ``` + "descr": ( + "contains echo intensity, velocity and correlation data " + "as well as other configuration parameters from the Average mode." + ) + ``` 2. burst + ``` + "descr": ( + "contains echo intensity, velocity and correlation data " + "as well as other configuration parameters from the Burst mode." + ) + ``` 3. echosounder + ``` + "descr": ( + "contains backscatter echo intensity and other configuration parameters from the Echosounder mode. " + "Data can be pulse compressed or raw intensity." + ) + ``` 4. echosounder raw samples + ``` + "descr": ( + "contains complex backscatter raw samples and other configuration parameters from the Echosounder mode, " + "including complex data from the transmit pulse." + ) + ``` + - we need a trickier implementation than what's used for EK80, since for EK80 the max number group is 2, and depending on if complex or power data exist, where they show up may be different: + - if complex data exists, it is always stored in `Beam_group1` + - if ONLY power data exists, it is stored in `Beam_group1` + - if BOTH complex and power data exist, complex data in `Beam_group1` and power data in `Beam_group2` - examples: - example 1: file with average and burst mode: - `Beam_group1`: average From b1b0ef85b94eba39a42773094f94ddce0f0189b5 Mon Sep 17 00:00:00 2001 From: leewujung Date: Mon, 13 Jun 2022 11:56:41 -0700 Subject: [PATCH 6/8] tidy up a few things --- docs/source/ad2cp_reorg.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/docs/source/ad2cp_reorg.md b/docs/source/ad2cp_reorg.md index 28a0ac0ca..77e25954f 100644 --- a/docs/source/ad2cp_reorg.md +++ b/docs/source/ad2cp_reorg.md @@ -10,6 +10,7 @@ ## Overarching action items: - Double check all raw to actual units conversion +- Move the sequence of variable handling to following the sequence of the data specification, for easier reference - For all variables, put - `long_name`: Field in spec sheet - `comment`: Description in spec sheet @@ -82,8 +83,10 @@ ## Variables common to all modes Coordinates: -- `ping_time`: use the original `ping_time_average`, `ping_time_burst` and `ping_time_echosounder` in each of their own `Beam_groupX` but with the same name `ping_time` -- `range_sample`: use the original `range_sample_average`, `range_sample_burst` and `range_sample_echosounder` in each of their own `Beam_groupX` but with the same name `range_sample` +- `ping_time` + - use the original `ping_time_average`, `ping_time_burst` and `ping_time_echosounder` in each of their own `Beam_groupX` but with the same name `ping_time` +- `range_sample` + - use the original `range_sample_average`, `range_sample_burst` and `range_sample_echosounder` in each of their own `Beam_groupX` but with the same name `range_sample` - `beam`: - the actual physical beam activated in the setting (1, 2, 3, 4, 5): @imranmaj please double check this - parsed from the data variable `data_set_description` using `parse_ad2cp._postprocess_beams` for the following packets: @@ -96,9 +99,11 @@ Data variables: - `coordinate_system` - `number_of_cells` - `blanking` -- `cell_size`: this is conceptually equivalent to `sample_interval` for the other echosounders, just that `sample_interval` is defined in time (second) and `cell_size` is defined in space (meter) +- `cell_size` + - this is conceptually equivalent to `sample_interval` for the other echosounders, just that `sample_interval` is defined in time (second) and `cell_size` is defined in space (meter) - `velocity_range` - `echosounder_frequency` + - the parsed values seem wrong: right now it shows as either 0 (these are probably for pings from other modes and not echosounder -- please verify) or 10000, but should be 1000000 (1 MHz) @imranmaj - `ambiguity_velocity` - `data_set_description` - `transmit_energy` @@ -106,7 +111,7 @@ Data variables: - `velocity` - separate `velocity_burst` and `velocity_average` into different `Beam_groupX` and use the name `velocity` - `amplitude` - - separate `amplitude_burst`, `amplitude_average` and `amplitude_echosounder` into different `Beam_groupX` and use the name `amplitude` + - separate `amplitude_burst`, `amplitude_average` and `amplitude_echosounder` into different `Beam_groupX` and use the name `backscatter_r` (since this "amplitude" is equivalent to what we have from EK60, AZFP and EK80 power data) - `correlation` - separate `correlation_burst`, `correlation_average` and `correlation_echosounder` into different `Beam_groupX` and use the name `correlation` From 44d639c4cfb59290f5ae658c998297dd06b251d2 Mon Sep 17 00:00:00 2001 From: imranmaj <49664304+imranmaj@users.noreply.github.com> Date: Mon, 13 Jun 2022 16:03:46 -0700 Subject: [PATCH 7/8] Add results from 6/13 discussion --- docs/source/ad2cp_reorg.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/source/ad2cp_reorg.md b/docs/source/ad2cp_reorg.md index 77e25954f..3939f0f05 100644 --- a/docs/source/ad2cp_reorg.md +++ b/docs/source/ad2cp_reorg.md @@ -2,8 +2,12 @@ - why check the version again? - https://github.com/OSOceanAcoustics/echopype/blob/16a574dda792a61f6f7ae0583b70b4b87d787f12/echopype/convert/parse_ad2cp.py#L589-L593 - https://github.com/OSOceanAcoustics/echopype/blob/16a574dda792a61f6f7ae0583b70b4b87d787f12/echopype/convert/parse_ad2cp.py#L630-L634 + - Safety check to make sure the version is correct + - Maybe there are multiple versions in 1 file? + - Default is to interpret as version 3, maybe the version is 2? - Move `AHRS_COORDS` def to `parse_ad2cp`? - https://github.com/OSOceanAcoustics/echopype/blob/16a574dda792a61f6f7ae0583b70b4b87d787f12/echopype/convert/set_groups_ad2cp.py#L10-L14 + - Chose to keep coordinate value logic in set_groups - the `beam` coordinate is missing in `rawtest.090.00015.nc`? From fc0ab2bbaffd48fc813c2444cf69ed10c35a8548 Mon Sep 17 00:00:00 2001 From: imranmaj <49664304+imranmaj@users.noreply.github.com> Date: Mon, 13 Jun 2022 16:14:10 -0700 Subject: [PATCH 8/8] Add results of beam discussion on 6/13 --- docs/source/ad2cp_reorg.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/source/ad2cp_reorg.md b/docs/source/ad2cp_reorg.md index 3939f0f05..78daeb6f3 100644 --- a/docs/source/ad2cp_reorg.md +++ b/docs/source/ad2cp_reorg.md @@ -9,6 +9,8 @@ - https://github.com/OSOceanAcoustics/echopype/blob/16a574dda792a61f6f7ae0583b70b4b87d787f12/echopype/convert/set_groups_ad2cp.py#L10-L14 - Chose to keep coordinate value logic in set_groups - the `beam` coordinate is missing in `rawtest.090.00015.nc`? + - Beam is currently not included because no variables in the beam group are indexed by beam + - It should be included anyway