Skip to content

install

Jump to bottom
jmarkow edited this page Aug 20, 2012 · 10 revisions

The ephys package is comprised of two components: (1) a series of scripts for interactive data analysis to be used in MATLAB and (2) a pipeline comprised of bash scripts and compiled MATLAB scripts that enables automated, high-throughput analysis of data recorded using the Intan Technologies demo software.

Follow these steps to get started with the MATLAB component of the ephys package:

  1. Unpack the compressed archive of this repository (master branch) anywhere on your local hard disk
  2. Add the uncompressed directory (including subdirectories) to your MATLAB path
  3. Celebrate!

Overview

To install the pipeline portion of the package you need a few things: (1) a PC for data acquisition using the Intan Technologies, headstage, eval board and demo software (link), (2) a Mac or Linux machine for post-processing. The basic workflow is as follows:

Demo software-->pipeline front end-->post-processing machine-->pipeline back end

In words, the pipeline front end takes Intan files, detects vocalizations, and converts the Intan files to MATLAB-readable binaries for use with the backend. The files are synced over the network to a post-processing machine (preferably a Mac, Linux machine should work and PCs may be supported in the future). On the post-processing computer the pipeline back end carries out automated spike clustering and generates figures suitable for further manual analysis.

System requirements

Front end

  1. PC running Windows XP SP3 or later
  2. Intan Technologies demo software
  3. Inta Technologies head stage retrofitted with microphone and evaluation board.
  4. At least 3 GB of RAM to read the Intan files
  5. It is recommended to use a hard drive with at least 300 GB of space, though smaller hard drives can be used...painfully

Back end

  1. Mac OS X 10.6+
  2. 8 GB+ RAM
  3. Mac Pro 2010 or later recommended
  4. MATLAB 2011b+
  5. Matlab 2011b compiler runtime (standard with MATLAB 2011b)
  6. MATLAB Toolboxes: Bioinformatics (SVM), Statistics (GMM), Wavelet (Spike sorting)

User end

I highly recommend familiarizing yourself with the command line before trying to use the pipeline. While the instructions are meant to be (somewhat) foolproof, blind copy-and-paste jobs can lead to unintended consequences. First, a gentle introduction to command line stuff on Mac OS X can be found here. Next, I highly recommend going through the entirety of this classic guide to shell scripting.

Step-by-step instructions (Mac OS X)

Back end

  1. In the bash terminal navigate to the uncompressed archive directory and issue the following command

    sudo ./install.sh

    This will create symlinks for all binaries and bash scripts that comprise the pipeline in /usr/local/bin

  2. The install script will prompt you to enter a few directories:

    1. The network directory, which is the full path on your computer to the network share with the Intan files processed with the acquisition portion of the pipeline (intan_intmic_daemon.m)
    2. The local directory where you will store the Intan files and process them, this must be a full path to a directory that already exists

  3. Next, you will need to coerce cron into syncing your network directory with your local directory, to do so issue the following command in Terminal.app

    crontab -e

    Either you have a somewhat full crontab (like me) or an empty one, in which case I suggest you read a brief overview of cron, perhaps here. Anyway, you need to add the following line to your crontab

    */15 * * * * /usr/local/bin/data_sync.sh

    This will sync the files in your network directory to your local directory every 15 minutes.

  4. Next, you will need to add /usr/local/bin to your PATH environment variable, in Mac OS X the simplest way to do this is to create a .profile file in your user directory that looks like this

    export PATH=/usr/local/bin:$PATH

  5. Now, install the Matlab Compiler Runtime (2011b), and add the relevant paths to .profile so that it looks like this

    export PATH=/usr/local/bin:$PATH
export MCR=/Applications/MATLAB/MATLAB_Compiler_Runtime
export DYLD_LIBRARY_PATH=\
/Applications/MATLAB/MATLAB_Compiler_Runtime/v716/runtime/maci64:\
/Applications/MATLAB/MATLAB_Compiler_Runtime/v716/sys/os/maci64:\
/Applications/MATLAB/MATLAB_Compiler_Runtime/v716/bin/maci64:\
/System/Library/Frameworks/JavaVM.framework/JavaVM:\
/System/Library/Frameworks/JavaVM.framework/Libraries
export XAPPLRESDIR=/Applications/MATLAB/MATLAB_Compiler_Runtime/v716/X11/app-defaults

  6. Try running ephys_pipeline_wrapper.sh to fire up the pipeline, you should see something like the following output

    Starting smscore daemon:  ephys_pipeline_smscore_daemon.sh
Log:   /Users/jmarkow/.ephys_pipeline_smscore.log
Starting ephys daemons:   ephys_pipeline_aggregate_daemon.sh
Starting ephys daemons:   ephys_pipeline_sua_daemon.sh
Starting ephys daemons:   ephys_pipeline_mua_daemon.sh
Starting ephys daemons:   ephys_pipeline_lfp_daemon.sh
Log:   /Users/jmarkow/.ephys_pipeline_aggregate.log
Log:   /Users/jmarkow/.ephys_pipeline_sua.log
Log:   /Users/jmarkow/.ephys_pipeline_mua.log
Log:   /Users/jmarkow/.ephys_pipeline_lfp.log
Starting cluster daemon:   ephys_pipeline_soundcluster_daemon.sh
Log:   /Users/jmarkow/.ephys_pipeline_cluster.log
Press any key to kill Intan daemons and exit...

Front end

  1. On your data acquisition PC, place the uncompressed archive of this repository somewhere in your MATLAB path.

  2. Find a directory where you would like to store your data and run the MATLAB script ephys_pipeline_mkdirs.m, this will create the following directory structure

    pwd-->data-->intan_data
   \->staging-->processed
             \->unprocessed

    The raw Intan files will go in pwd/staging/unprocessed, which are then transferred to pwd/staging/processed. Sorted and analyzed data will do in pwd/data/intan_data. You must be sure to clean out pwd/staging/processed periodically, the pipeline will not automatically delete old files, this is a feature not a bug (for now)!

  3. Start the Intan demo software on your PC, and record to files with the following naming convention,

    [SUBJECTID]_[RECORDINGID]_mic[MICCHANNEL]

    If, for instance you are recording from subject LB97 in HVC with microphone channel 12, the filename should read

    lb97_hvc_mic12

    The filenames are automatically timestamped by the demo software.

Step-by-step instructions (Linux)

  1. I have not tested the back end on the Linux machine, but the back end should work as advertised though you will need to recompile the binaries (instruction to be added soon!).

Step-by-step instructions (PC)

  1. Since a lot happens under the hood using Bash PCs will not be supported for the back end.

Bonus material

There is some bonus material that includes a simple GNU Screen configuration to monitor the log files for the pipeline and a script that displays the number of song extractions and clustered units for a given experiment. For the GNU Screen configuration:

  1. If you are using a stock configuration of the pipeline, simply run the following command to fire of GNU Screen with the custom pipeline configuration file

    screen -c /usr/local/bin/screenrc

And for the pipeline configuration script:

  1. Try running pipeline_list.sh, if you see formatted output then the script works. To display these results on your desktop you can run pipeline_list.sh in cron, with a configuration like the following

    */5 * * * * pipeline_list.sh > ~/ephys_pipeline_list.txt

    This will run the script every five minutes and store the results in ~/ephys_pipeline_list.txt, to display the file on your desktop, use either GeekTool or Nerdtool.

Clone this wiki locally