Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions vignettes/CIDA_BIOS_Cluster.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ Because HPC clusters are intended to serve a group of people (i.e. A biostatisti
When a user is ready to run something (an analysis, processing pipeline, etc) on the cluster, they will submit a new *job* to the cluster. The cluster will then schedule and run the job as soon as compute resources are available.

```{r, echo=FALSE, out.width="80%", fig.align="center"}
knitr::include_graphics("cluster_basic_diagram.png")
knitr::include_graphics("figures/CIDA_BIOS_Cluster/cluster_basic_diagram.png")
```

The example figure above shows an example cluster with three users. Each user connects to the Head Node to submit their jobs.
Expand All @@ -50,7 +50,7 @@ HPC clusters are useful for:
The CSPH Biostats cluster consists of four nodes. The `csphbiostats.ucdenver.pvt` node serves as both a head node and one of the compute nodes (i.e. Submitted jobs may also run on this node), and the `cidalappc[1-3].ucdenver.pvt` nodes serve as compute nodes.

```{r, echo=FALSE, out.width="100%", fig.align="center"}
knitr::include_graphics("Biostats_HPC_diagram.png")
knitr::include_graphics("figures/CIDA_BIOS_Cluster/Biostats_HPC_diagram.png")
```

The CIDA/Biostats server uses the [SLURM](https://slurm.schedmd.com) system to manage job scheduling and resource management on the cluster.
Expand All @@ -67,7 +67,7 @@ Once approved, an account will be created for you on the server.
To log in to the CSPH Biostats Cluster, you can use SSH from the command line or an SSH client of your choice.

```{r, echo=FALSE, out.width="80%", fig.align="center"}
knitr::include_graphics("biostats_hpc_ssh.png")
knitr::include_graphics("figures/CIDA_BIOS_Cluster/biostats_hpc_ssh.png")
```

If you are connecting from the command line (like the above example), run:
Expand Down Expand Up @@ -135,7 +135,7 @@ sbatch my_batch.sh
```

```{r, echo=FALSE, out.width="80%", fig.align="center"}
knitr::include_graphics("cluster_sbatch.png")
knitr::include_graphics("figures/CIDA_BIOS_Cluster/cluster_sbatch.png")
```

When we execute the `sbatch` command, SLURM will assign the job an ID and schedule the job to execute. In this case, our job is assigned ID `6179` (first arrow in the figure).
Expand Down Expand Up @@ -177,7 +177,7 @@ will schedule a new job to run the script `./my_script.sh`.
The `--exclude` flag tells SLURM to schedule the job on any node except the node(s) listed. In this case, we exclude the `csphbiostats.ucdenver.pvt` node to ensure our job runs on one of the `cidalappc[1-3].ucdenver.pvt` nodes (and the first line of the script output shows our job ran on `cidalappc01`).

```{r, echo=FALSE, out.width="80%", fig.align="center"}
knitr::include_graphics("cluster_srun.png")
knitr::include_graphics("figures/CIDA_BIOS_Cluster/cluster_srun.png")
```

#### Interactive Jobs
Expand Down Expand Up @@ -213,7 +213,7 @@ At this point, we can execute any commands or scripts normally.
To end the interactive job, type `exit`.

```{r, echo=FALSE, out.width="80%", fig.align="center"}
knitr::include_graphics("interactive_job.png")
knitr::include_graphics("figures/CIDA_BIOS_Cluster/interactive_job.png")
```

#### When to use `srun`
Expand All @@ -233,21 +233,21 @@ The below screenshots show an example of connecting to `csphbiostats.ucdenver.pv
First, open Cyberduck and click the 'Open Connection' button on the top bar.

```{r, echo=FALSE, out.width="80%", fig.align="center"}
knitr::include_graphics("cyberduck_1.png")
knitr::include_graphics("figures/CIDA_BIOS_Cluster/cyberduck_1.png")
```

Next, ensure that the 'SFTP' option is selected in the dropdown, then input your SSH credentials.

```{r, echo=FALSE, out.width="80%", fig.align="center"}
knitr::include_graphics("cyberduck_2.png")
knitr::include_graphics("figures/CIDA_BIOS_Cluster/cyberduck_2.png")
```

If successful, you should see a file browser interface showing your home directory on `csphbiostats.ucdenver.pvt`.

You can use the interface to navigate and download and existing files. You can also drag-and-drop files from your local machine to upload them to the cluster.

```{r, echo=FALSE, out.width="80%", fig.align="center"}
knitr::include_graphics("cyberduck_3.png")
knitr::include_graphics("figures/CIDA_BIOS_Cluster/cyberduck_3.png")
```

### Networked Storage
Expand Down
4 changes: 2 additions & 2 deletions vignettes/coding_guidelines.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ Quote from R for Data Science:
Initially, you may consider your analysis to live in your R environment (e.g. the objects listed in the environment pane). However, it’s much easier to recreate this environment from an R script than it is to recreate an R script from the environment! Your analysis therefore lives in your code. So, if you haven’t already, you should instruct RStudio to never preserve your workspace between sessions to foster this attitude and to make your life easier in the long-term.

```{r, echo=FALSE, out.width="75%", out.height="75%", fig.align='center'}
knitr::include_graphics("coding_1.png")
knitr::include_graphics("figures/coding_guidelines/coding_1.png")
```

After changing this, you will notice when you restart RStudio that it will not remember the results of the code you ran last time, because remember – your analysis lives in your code.
Expand Down Expand Up @@ -146,7 +146,7 @@ If you are working within an Rstudio Project, you will not usually have to worry
Sample `.gitignore` file, stored at the project-level directory (the top level of your git repository):

```{r, echo=FALSE, out.width="50%", out.height="50%", fig.align='center'}
knitr::include_graphics("coding_2.png")
knitr::include_graphics("figures/coding_guidelines/coding_2.png")
```


Expand Down
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
40 changes: 18 additions & 22 deletions vignettes/git.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -17,17 +17,13 @@ vignette: >
---

```{r, echo=FALSE, out.width='70%', fig.show='hold'}
knitr::include_graphics("sph_cida_wm_blk.png")
knitr::include_graphics("figures/git/sph_cida_wm_blk.png")
```

---

```{r setup, include=FALSE}
library(CIDAtools)
find_png <- function(x) {
files <- list.files()
files[grepl(x, files)]
}
```

# Introduction
Expand Down Expand Up @@ -459,8 +455,8 @@ GitHub account associated with the CIDA GitHub. This will take you to the SSO
associated with the CIDA GitHub, which you will then authorize and sign onto.

```{r, echo=FALSE, out.width="100%", out.height="100%"}
knitr::include_graphics(find_png("figure0A"))
knitr::include_graphics(find_png("figure0C"))
knitr::include_graphics("figures/git/figure0A.png")
knitr::include_graphics("figures/git/figure0C.png")
```

Once you have logged into GitKraken with you GitHub account, you will need to
Expand All @@ -474,12 +470,12 @@ generate a token for 90 days, making sure to select "repo" (along with other
scopes as needed).

```{r, echo=FALSE, out.width="100%", out.height="100%"}
knitr::include_graphics(find_png("figure1A"))
knitr::include_graphics(find_png("figure1B"))
knitr::include_graphics(find_png("figure1C"))
knitr::include_graphics(find_png("figure2A"))
knitr::include_graphics(find_png("figure2B"))
knitr::include_graphics(find_png("figure2C"))
knitr::include_graphics("figures/git/figure1A.png")
knitr::include_graphics("figures/git/figure1B.png")
knitr::include_graphics("figures/git/figure1C.png")
knitr::include_graphics("figures/git/figure2A.png")
knitr::include_graphics("figures/git/figure2B.png")
knitr::include_graphics("figures/git/figure2C.png")
```

Ideally, you should be able to automatically connect to GitHub without
Expand All @@ -493,9 +489,9 @@ GitKraken on the lower left, and your screen will now display the SSH key
within GitKraken and under the Settings -> SSH and GPG keys tab on your GitHub:

```{r, echo=FALSE, out.width="100%", out.height="100%"}
knitr::include_graphics(find_png("figure3A"))
knitr::include_graphics(find_png("figure3B"))
knitr::include_graphics(find_png("figure3C"))
knitr::include_graphics("figures/git/figure3A.png")
knitr::include_graphics("figures/git/figure3B.png")
knitr::include_graphics("figures/git/figure3C.png")
```

Congratulations, you should be successfully linked to the CIDA GitHub!
Expand All @@ -505,15 +501,15 @@ using the, you guessed it, Open a repo, Clone a repo, and Start a local repo
options on the home screen:

```{r, echo=FALSE, out.width="100%", out.height="100%"}
knitr::include_graphics(find_png("figure4A"))
knitr::include_graphics("figures/git/figure4A.png")
```

If you are going to clone a repo, just make sure you are on the GitHub.com
section, and that you select the CIDA repository to clone, as well as where you
would like to clone it to (H: or P: drives, or elsewhere).

```{r, echo=FALSE, out.width="100%", out.height="100%"}
knitr::include_graphics(find_png("figure4B"))
knitr::include_graphics("figures/git/figure4B.png")
```

First, however, you will need to generate another SSH key on GitKraken to add
Expand All @@ -522,7 +518,7 @@ Preferences -> SSH and then generate a new Private/Public Key, or copy the SSH
Public Key if you already have one:

```{r, echo=FALSE, out.width="100%", out.height="100%"}
knitr::include_graphics(find_png("figure4C"))
knitr::include_graphics("figures/git/figure4C.png")
```

Then return to your GitHub account, and click on SSH and GPG keys under the
Expand All @@ -531,15 +527,15 @@ will then need to Authorize it with the SSO. After that, you should be able to
clone, open, and initialize your repos!

```{r, echo=FALSE, out.width="100%", out.height="100%"}
knitr::include_graphics(find_png("figure4D"))
knitr::include_graphics("figures/git/figure4D.png")
```

Once you have cloned a repo, you can open it to push/pull any changes. This is
done pretty simply via the push or pull commands on the top after opening the
repo.

```{r, echo=FALSE, out.width="100%", out.height="100%"}
knitr::include_graphics(find_png("figure5A"))
knitr::include_graphics("figures/git/figure5A.png")
```

For more details on how to manage pull, push, branching, and other features,
Expand All @@ -548,5 +544,5 @@ visit https://help.gitkraken.com/gitkraken-client/github-gitkraken-client/
---

```{r, echo=FALSE, out.width='70%', fig.show='hold'}
knitr::include_graphics(find_png("sph_cida_wm_blk"))
knitr::include_graphics("figures/git/sph_cida_wm_blk.png")
```