User Tools

Site Tools


biac:cluster

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
biac:cluster [2012/06/15 14:54]
mm265 [Tutorials]
biac:cluster [2023/04/06 18:31] (current)
cmp12
Line 3: Line 3:
 ===== User guides ===== ===== User guides =====
  
-  * [[biac:cluster:account|Requesting a cluster account]]   +  * [[biac:cluster:architecture System Architecture ]] 
-  * [[biac:cluster:access|Accessing the cluster]]   +  * [[biac:cluster:account Requesting cluster access]]   
-  * [[biac:cluster:passwords|Cluster password]]  +  * [[biac:cluster:access Accessing the cluster]]   
-  * [[biac:cluster:submit|Job submission]] +  * [[biac:cluster:submit |Job submission]] 
-  * [[biac:cluster:packagesList of packages installed]]  +  * [[biac:cluster:modules Software]] 
-  * [[biac:cluster:examples| BIAC Sample Scripts]] +  * [[biac:cluster:examples | BIAC Sample Scripts]] 
-  * [[biac:cluster:interactive| Interactive jobs]]+  * [[biac:cluster:interactive | Interactive jobs]]
   * [[huettel:using_the_biac_cluster | Advanced Bash/Python Submission Scripts]] - Provided by Dr. Scott Huettel's lab, the scripts included here will check FSL parameters to make sure jobs do not get "stuck" running on the cluster, and submit multiple jobs in a smart way so individual users do not tie up the cluster at any one time.      * [[huettel:using_the_biac_cluster | Advanced Bash/Python Submission Scripts]] - Provided by Dr. Scott Huettel's lab, the scripts included here will check FSL parameters to make sure jobs do not get "stuck" running on the cluster, and submit multiple jobs in a smart way so individual users do not tie up the cluster at any one time.   
  
  
-===== Tutorials =====+=== Basic Primer ===
  
-A [[http://wiki.biac.duke.edu/suppl/intro.htm|"Getting Started"]] tutorial on setting up access to the cluster and basic job related tasks ([[biac:cluster:submit#job submission|submission]], [[biac:cluster:submit#job status|status checking]] and [[biac:cluster:submit#job delete|deletion]] +The BIAC cluster is a Linux-based processor put in place to analyze any type of data.  You will work on the cluster though a terminal window by writing commands one line at a time In order to set up access to the cluster on your computer, you need to be behind the DHE firewall.  Being at BIAC or on a DHE wireless network puts you within the medical center firewall.  If you are not within the med center you can request access to the DHE VPN via biac-help@duke.edu. 
-Or see: {{:biac:infrequent_setup.doc|}}+                
 +To connect to the cluster, you will need XWin/Putty from windows, or Terminal with Xquartz from Mac 
 +  * [[biac:cluster:access Accessing the cluster]]  
  
-=== Hugin for dummies === 
  
-                   The BIAC cluster, Hugin, is a Linux base processor put in place to analyze any type of data.  You will work on Hugin though a terminal window by writing commands one +== The Interactive Node ==
-                   line at a time.  In order to set up access to Hugin on your computer, you need to be behind the firewall.  Being at BIAC or on "Diamonds" or "Clubs" puts you +
-                   behind this firewall. +
-                              +
-                             First, you will need X-Win.  On BIAC computers, X-Win comes standard but it can also be downloaded through OIT’s website  +
-                             here http://oit.duke.edu/comp-print/software/index.php (select “Browse and order software”). +
-                              +
-                             You will also need a cluster account. Please send an email to help@biac.duke.edu requesting a cluster account. +
-                             Include your BIAC Windows username in the email (see http://wiki.biac.duke.edu/biac:accounts). +
-                    +
-                    +
- +
-== the Interactive Node ==+
  
 qinteract is also known as the ‘interactive node,’ because it allows you to interact directly with your data on Munin (or whatever server).  It also contains the “packages” (FSL, CMTK, FreeSurfer) that BIAC has installed.  You can access the GUI’s for these packages by typing in their appropriate commands once in qinteract. qinteract is also known as the ‘interactive node,’ because it allows you to interact directly with your data on Munin (or whatever server).  It also contains the “packages” (FSL, CMTK, FreeSurfer) that BIAC has installed.  You can access the GUI’s for these packages by typing in their appropriate commands once in qinteract.
  
   * List of packages: http://wiki.biac.duke.edu/biac:cluster:packages   * List of packages: http://wiki.biac.duke.edu/biac:cluster:packages
-  * Getting on: log onto Hugin, in the command line (username@hugin: ___) type “qinteract” if prompted, type yes, and then enter your cluster password.+  * Getting on: log onto the cluster, in the command line (username@cluster: ___) type “qinteract” if prompted, type yes, and then enter your cluster password.
   * Congratulations!  You are now on the “interactive node.”  Access your study data by typing “cd `findexp Study.01`”   * Congratulations!  You are now on the “interactive node.”  Access your study data by typing “cd `findexp Study.01`”
  
 == Useful Commands == == Useful Commands ==
- +  * ls - "list scripts" displays the contents of the directory 
-Command lines +  ls -ltr - ls w/ time and date 
-ls +  cd - "change directory," to be followed by the name of the directory you want to go to 
-ls -ltr +  * mkdir - "make directory," to be followed by the name of the directory you want to make 
-mkdir +  * gedit - will open a text editor, to be followed by the name of the text file you'd like to edit 
-cd +  rm - "remove" followed by what you'd like to remove - be careful! 
-.. +  cp - copy 
-Tab to complete +  qdel - delete job 
-Gedit . nedit +  * qstatall - display active jobs
- Commenting out +
- saving +
-+
-bash +
-rm +
-Up arrow +
-cp +
-qdel +
-qstatall +
-Blue “directories” are equivalent to folders, “cd”ing is like double clicking +
-Getting files from munin onto hugin, and mounting experiments?  Findexp +
-Print from terminal window +
  
  
 +== EXTRA TIPS ==
 +  * If you command line is gone in the cluster, as in you don’t have the prompt to type something, make sure nothing is open in gedit (for example batchMakeFSL_Data is open using gedit – click out), click ctrl+c
 +  * In order to avoid this, follow commands with an &
 +  * Hit the up arrow in order to go to the previous written command
 +  * Hit tab in order to fill in a file, script, or directory name
 +  * If there are multiple files with that name it’ll fill in up to where they differ
 +  * If you’d like to work the command on all files with that particular prefix, immediately follow the prefix with * (for example “gedit batch*” would open all scripts that start with batch)
 +  * To cd to previous folder type cd ..
 +  * Scripts with tildes (~) are there as a backup service, ignore, don’t use, can be deleted if you make sure the proper script is current (example batch_QA versus batch_QA~)
 +  * By typing qstatall (no folloing arguments necessary), you can look at the active jobs on the cluster
 +  * By typing ls –ltr you’ll have a list of files with a timestamp/ more details
 +  * In order to paste from windows into linux (example gedit or the FSL GUI), copy using right click and paste using the center scroll button
 ===== Administration guides ===== ===== Administration guides =====
  
-  * [[biac:cluster:admin:architecture|System architecture]] +  * <del>[[biac:cluster:admin:installation|Installation]]</del> 
-  * [[biac:cluster:admin:installation|Installation]]+
   * [[biac:cluster:admin:manage|Management]]   * [[biac:cluster:admin:manage|Management]]
   * [[biac:cluster:admin:syslog|System Log]]   * [[biac:cluster:admin:syslog|System Log]]
biac/cluster.1339772077.txt.gz · Last modified: 2014/08/04 16:03 (external edit)