Skip to content

Update snapshot links #117

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
walnut-the-cat merged 6 commits into from
Feb 5, 2025
Merged

Update snapshot links #117

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
c7c4e3f
Update run-rpc-node-with-nearup.md
walnut-the-cat Feb 4, 2025
9b6cb7e
Update run-rpc-node-with-nearup.md
walnut-the-cat Feb 4, 2025
c179d17
Update run-rpc-node-without-nearup.md
walnut-the-cat Feb 4, 2025
5677573
Update run-rpc-node-without-nearup.md
walnut-the-cat Feb 4, 2025
bfda35e
Delete docs/src/redirects/external-sites.json
walnut-the-cat Feb 4, 2025
32d1896
Update snapshots.md
walnut-the-cat Feb 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 2 additions & 72 deletions docs/intro/snapshots.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,80 +6,10 @@ sidebar_position: 3
description: Node Data Snapshots
---


### !!!Note!!!
As the NEAR ecosystem continues to decentralize, Pagoda has cease operations and decentralize its functions into NEAR ecosystem teams and committees.
FastNEAR will be the sole provider of snapshots downloads starting Jan 1, 2025. Please visit https://docs.fastnear.com/docs/snapshots for more details.

## Overview

Before you start running a node, you must first sync with the network. This means your node needs to download all the headers and blocks that other nodes in the network already have. You can speed up this process by downloading the latest data snapshots from a public cloudfront endpoint.
Copy link
Contributor

@wacban wacban Feb 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realize this is a doc about snapshots - but it may be worth mentioning Epoch Sync as an alternative.

Copy link
Contributor Author

@walnut-the-cat walnut-the-cat Feb 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will take that as the next action item


Here are the available snapshots directories based on node type and network. Please note that the RPC snapshots are updated every 12 hours. Split-Storage Archival snapshots are updated every 72 hours due to the size of cold storage data.


| Node Type and Network| CloudFront Path |
| -------------------- | ---------------------------------------------------------------------------------------|
| RPC testnet | `https://dcf58hz8pnro2.cloudfront.net/backups/testnet/rpc/latest` |
| RPC mainnet | `https://dcf58hz8pnro2.cloudfront.net/backups/mainnet/rpc/latest` |
| Archival testnet | `https://dcf58hz8pnro2.cloudfront.net/backups/testnet/archive/latest_split_storage` |
| Archival mainnet | `https://dcf58hz8pnro2.cloudfront.net/backups/mainnet/archive/latest_split_storage` |

----

## Sample instruction

Prerequisite:

Recommended download client [`rclone`](https://rclone.org).
This tool is present in many Linux distributions. There is also a version for Windows.
And its main merit is multithread.
You can [read about it on](https://rclone.org)
** rclone version needs to be v1.66.0 or higher

First, install rclone:
```
$ sudo -v ; curl https://rclone.org/install.sh | sudo bash
```
Next, prepare config, so you don't need to specify all the parameters interactively:
```
mkdir -p ~/.config/rclone
touch ~/.config/rclone/rclone.conf
```

, and paste exactly the following config into `rclone.conf`:
```
[near_cf]
type = s3
provider = AWS
download_url = https://dcf58hz8pnro2.cloudfront.net/
acl = public-read
server_side_encryption = AES256
region = ca-central-1

```
Commands to run for RPC|Validator Data Download:
```
chain="mainnet" # or "testnet"
kind="rpc"
rclone copy --no-check-certificate near_cf://near-protocol-public/backups/${chain:?}/${kind:?}/latest ./
latest=$(cat latest)
rclone copy --no-check-certificate --progress --transfers=6 --checkers=6 \
near_cf://near-protocol-public/backups/${chain:?}/${kind:?}/${latest:?} ~/.near/data
```

Commands to run for Archival Split Storage Data Download:
```
chain="mainnet" # or "testnet"
kind="archive"
rclone copy --no-check-certificate near_cf://near-protocol-public/backups/${chain:?}/${kind:?}/latest_split_storage ./
latest=$(cat latest_split_storage)
rclone copy --no-check-certificate --progress --transfers=6 --checkers=6 \
near_cf://near-protocol-public/backups/${chain:?}/${kind:?}/${latest:?} ~/.near/data
```


The latest daily snapshots are made available to the public by FastNear, and can be used to set up a validator node or RPC.

>Got a question?
<a href="https://stackoverflow.com/questions/tagged/nearprotocol">
<h8>Ask it on StackOverflow!</h8></a>
For detailed instructions, please refer to [HERE](https://docs.fastnear.com/docs/snapshots).
49 changes: 3 additions & 46 deletions docs/rpc/run-rpc-node-with-nearup.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,50 +77,7 @@ Then run:
$ nearup stop
```

Retrieve a copy of the latest RPC snapshot from S3 using rclone:
Prerequisite:
### Retrieve a copy of the latest RPC snapshot:

Recommended download client [`rclone`](https://rclone.org).
This tool is present in many Linux distributions. There is also a version for Windows.
And its main merit is multithread.
You can [read about it on](https://rclone.org)
** rclone version needs to be v1.66.0 or higher

First, install rclone:
```
$ sudo -v ; curl https://rclone.org/install.sh | sudo bash
```
Next, prepare config, so you don't need to specify all the parameters interactively:
```
mkdir -p ~/.config/rclone
touch ~/.config/rclone/rclone.conf
```

, and paste exactly the following config into `rclone.conf`:
```
[near_cf]
type = s3
provider = AWS
download_url = https://dcf58hz8pnro2.cloudfront.net/
acl = public-read
server_side_encryption = AES256
region = ca-central-1

```

```bash
$ rclone copy --no-check-certificate near_cf://near-protocol-public/backups/testnet/rpc/latest ./
$ LATEST=$(cat latest)
$ rclone copy --no-check-certificate --progress --transfers=6 --checkers=6 \
near_cf://near-protocol-public/backups/testnet/rpc/${latest:?} ~/.near/data
```


Finally, run the following command and the node should start syncing headers:
```bash
$ nearup run testnet
```

>Got a question?
<a href="https://stackoverflow.com/questions/tagged/nearprotocol">
<h8>Ask it on StackOverflow!</h8></a>
The latest daily snapshots are made available to the public by FastNear, and can be used to set up a validator node or RPC.
For detailed instructions, please refer to [HERE](https://docs.fastnear.com/docs/snapshots).
78 changes: 5 additions & 73 deletions docs/rpc/run-rpc-node-without-nearup.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,42 +117,8 @@ This command will create the required directory structure and will generate `con

The node is ready to be started. However, you must first sync up with the network. This means your node needs to download all the headers and blocks that other nodes in the network already have.

Prerequisite:

Recommended download client [`rclone`](https://rclone.org).
This tool is present in many Linux distributions. There is also a version for Windows.
And its main merit is multithread.
You can [read about it on](https://rclone.org)
** rclone version needs to be v1.66.0 or higher

First, install rclone:
```
$ sudo -v ; curl https://rclone.org/install.sh | sudo bash
```
Next, prepare config, so you don't need to specify all the parameters interactively:
```
mkdir -p ~/.config/rclone
touch ~/.config/rclone/rclone.conf
```

, and paste exactly the following config into `rclone.conf`:
```
[near_cf]
type = s3
provider = AWS
download_url = https://dcf58hz8pnro2.cloudfront.net/
acl = public-read
server_side_encryption = AES256
region = ca-central-1

```

```bash
$ rclone copy --no-check-certificate near_cf://near-protocol-public/backups/testnet/rpc/latest ./
$ LATEST=$(cat latest)
$ rclone copy --no-check-certificate --progress --transfers=6 --checkers=6 \
near_cf://near-protocol-public/backups/testnet/rpc/${latest:?} ~/.near/data
```
The latest daily snapshots are made available to the public by FastNear, and can be used to set up a validator node or RPC.
For detailed instructions, please refer to [HERE](https://docs.fastnear.com/docs/snapshots).

### 5. Run the node {#5-run-the-node}
To start your node simply run the following command:
Expand Down Expand Up @@ -229,44 +195,10 @@ This command will create the required directory structure by generating a `confi

The node is ready to be started. However, you must first sync up with the network. This means your node needs to download all the headers and blocks that other nodes in the network already have.

Prerequisite:

Recommended download client [`rclone`](https://rclone.org).
This tool is present in many Linux distributions. There is also a version for Windows.
And its main merit is multithread.
You can [read about it on](https://rclone.org)
** rclone version needs to be v1.66.0 or higher
The latest daily snapshots are made available to the public by FastNear, and can be used to set up a validator node or RPC.
For detailed instructions, please refer to [HERE](https://docs.fastnear.com/docs/snapshots).

First, install rclone:
```
$ sudo -v ; curl https://rclone.org/install.sh | sudo bash
```
Next, prepare config, so you don't need to specify all the parameters interactively:
```
mkdir -p ~/.config/rclone
touch ~/.config/rclone/rclone.conf
```

, and paste exactly the following config into `rclone.conf`:
```
[near_cf]
type = s3
provider = AWS
download_url = https://dcf58hz8pnro2.cloudfront.net/
acl = public-read
server_side_encryption = AES256
region = ca-central-1

```

```bash
$ rclone copy --no-check-certificate near_cf://near-protocol-public/backups/mainnet/rpc/latest ./
$ LATEST=$(cat latest)
$ rclone copy --no-check-certificate --progress --transfers=6 --checkers=6 \
near_cf://near-protocol-public/backups/mainnet/rpc/${latest:?} ~/.near/data
```

### 6. Run the node {#5-run-the-node}
### 5. Run the node {#5-run-the-node}
To start your node simply run the following command:

```bash
Expand Down
3 changes: 0 additions & 3 deletions docs/src/redirects/external-sites.json
View file
Open in desktop

This file was deleted.