BigEarthNet EDA

[taeil]

dataset format
GeoTiff
TFRecord.

The project seems to be using both formats.
This is a little confusing so started doing some discussion

For example, BigEarthNet dataset is available via tensorflow dataset catalog while CA dataset needs to be downloaded from Google Earth Engine support exporting as TFRecord or GeoTiff format.

• Overall tranining set description is here.

[taeil]

To download BigEarthNet from Tensorflow dataset as TFRecord format, use this notebook https://github.com/Berkeley-Data/irrigation_detection/blob/taeil/notebooks/0.2-tg-bigearthnet-eda.ipynb

1. bigearthnet-s2 v1.0: it can take a few hours
   curl http://bigearth.net/downloads/BigEarthNet-S2-v1.0.tar.gz -o data/raw/BigEarthNet-v1.0
2. bigearthnet-model
   1. download bigearthnet-model and unzip to /data/raw/bigearthnet-models
      curl "https://gitlab.tubit.tu-berlin.de/rsim/BigEarthNet-S2_43-classes_models/repository/master/archive.zip" -o "bigearthnet-models.zip"

[taeil]

it turns out it takes forever, I have been waiting for a few hours and progress shows 0... not sure if this is progress bar issue or it is simply big data. we need to convert to python script so that losing notebook connection does not interrupt the downloading task. If anyone is planning to download, please open the notebook and download the python.

[taeil]

downloading GeoTiff images takes around 2.5 hours on AWS US region.

[taeil]

tar -xf data/raw/BigEarthNet-v1.0.tar.gz data/raw

[taeil]

Eurosat https://github.com/phelber/eurosat

[taeil]

Need TFRecord format dataset.

[taeil]

From @oropezaev

[taeil]

We have ~13571 irrigation labels (Based on split csv files). This is 2.29% of the total BigEarthnet of 590326 images.
while BE paper say there is 13589.

[taeil]

Google Earth SAR (Synthetic Aperture Radar)