The Psychophysics toolbox (abbreviated as “Psychtoolbox”) is a set of functions, all located in various subfolders of the \\Gall\Programs\User_Scripts\huettel\PsychToolbox\ directory, which allow you to display many types of stimuli on a computer screen from a MATLAB script. The advantage of this is that the display can be interactive, changing based on subject responses, instead of an invariant movie. You can find the Psychtoolbox web page at http://psychtoolbox.org/.

Copy the scripts ‘addpsychtoolbox.m’ and ‘rmpsychtoolbox.m’ from \\Gall\Programs\User_Scripts\huettel to your user directory (\\Gall\Users\[username]). Your user directory is always in your path, so you should be able to simply type addpsychtoolbox or rmpsychtoolbox at the command line to add or remove the correct directories to your path.

There is an issue with using the function ‘copyfile’ while Psychtoolbox is added to your path. There is both a MATLAB version as well as a Psychtoolbox version and they’re somewhat incompatible. The problem will crop up whenever you use readmr (which calls the MATLAB copyfile) with Psychtoolbox added. Just remove psychtoolbox (using rmpsychtoolbox) before calling readmr.

This is probably the most important function in Psychtoolbox. It lets you display your stimuli on the screen, draw objects, etc. There are a whole lot of options and subfunctions in it; I will try to go through the commonly used ones. Note that none of these commands will work if you haven’t run addpsychtoolbox.

At the command line, type screen and hit enter. This will list a lot of options and usage information. For help with subfunctions, type screen [subfuncname]? (Yes, include the question mark). This format will also work: screen(‘[subfuncname]?’)

An on-screen window is visible on the monitor. An off-screen window exists only in memory and is usually copied to the on-screen window for display (see info about this in the ‘CopyWindow’ section below).

You only need to open one of these once when you start your script. You will copy all of your stimuli and such to this window. Open it using the following command: [window,screenRect]=SCREEN(0,’OpenWindow’,[],[],32) The zero tells it to open on screen # zero, which is the monitor you’re currently using. The first set of brackets indicates color, and leaving it empty like this defaults to white. The second set of brackets indicates the size of the window(its “rect”), and leaving it empty defaults to the current screen’s size. 32 is the pixel size. So, this command opens a white on screen window (the whole size of the monitor) and creates two important variables. The variable ‘window’ is what you have now named your on-screen window and is where you will copy other screens and such. The variable ‘screenRect’ is a 1×4 matrix that indicates the pixel locations of the upper-left and lower-right corners of ‘window’. So, if you’re at 1024×768 resolution, screenRect = [0 0 1024 768]. For more information on rects, see below.

Psychtoolbox is able to copy from off- to on-screen windows very quickly. This means that you can preload a pretty large number of windows off-screen, then just quickly copy them to the on-screen window. Preloading like this will save time, especially if you’re doing things like putting jpegs on the screen.
To open an off-screen window, use something similar to the following example:
blank_screen = SCREEN(window,’OpenOffscreenWindow’,[0 0 0],screenRect);
This line will make an off-screen window called blank_screen. This screen is black, as indicated by the RGB color code [0 0 0] (see section on this below), and fills the entire size of the screen, as indicated by the use of screenRect.

To close only one window type
SCREEN([windowname],’Close’)

And to close all windows type:
SCREEN(‘CloseAll’)

• Psychtoolbox uses “rects” (short for rectangles) to determine where to place things on your screens. They are always 1×4 matrices in the following format: [upperleft_x upperleft_y lowerright_x lowerright_y]. The values correspond to screen pixels. So, for example, a rect defined as [0 0 400 600] would cover the left half of a 800×600 window.
• Clarification: X coordinates increase as you go from Left to Right on your monitor. Y coordinates increase as you go from Top to Bottom on your screen.
• There are a number of helper functions for manipulating rects, although we seldom use them. An example of this is the CenterRect function, which is used to center one rect inside another. This could be useful for placing something small in the center of the screen, for example. To find out more about these type help psychrects.

Psychtoolbox uses standard RGB (red, green, blue) codes to indicate on-screen colors. What you’re doing is indicating the individual amounts of red, green, and blue you want to display. Values can range from 0 to 255, so [0 0 0] is black and [255 255 255] is white. Blue would then be [0 0 255], etc. Here's a random website I found that lists a ton of different colors and their RGB codes

There are a ton of these, but I’ll try to get through the common ones. Note that most of them have very similar syntax, so if you understand one, you’ll understand nearly all of them.

As mentioned previously, you’ll use this a lot. If you’re copying full-screen sized windows to the on-screen window it’s really easy to use. If you’re copying smaller rects to the on-screen window it can get tricky, so I’ll try to explain.

Say, for example, that you’ve created a full size off-screen window with a circle in the center that represents your stimulus. This window will be called circle_screen , and to display it to the subject, you would use copywindow as such:
SCREEN(‘CopyWindow’,circle_screen,window,screenRect,screenRect)
This tells Psychtoolbox that your source screen is the off-screen ‘circle_screen’ and that you want to copy it to the on-screen ‘window’ (the destination). The two screenRect arguments indicate that you’re copying the information located in screenRect in your source to screenRect in your destination.

Ok, this is kinda confusing. I’ll go through an example to illustrate. The difficulty arises when you define an off-screen window that is smaller than screenRect.

1. Say you want to display different colored squares in the center of your screen because each color means something different to the subject. However, you have some instructions elsewhere on the screen that need to stay put while the color of the square changes. You could create a ton of different full-sized off-screen windows for each combination of instructions and square color, and then just copy the whole window. But maybe you have 20 different types of instructions and 5 colors…..that’s a lot of windows. So, it would be easier if you could just create 5 different off screen windows for the 5 different-colored squares and worry about the instructions separately. Got all that? Good.
2. So, first you’d create the 5 off-screen windows. Say you define the blue square window as such (we’ll say the squares are 100×100 pixels in size):
   blue_square = SCREEN(window,’OpenOffscreenwindow’,[0 0 0],[350 250 450 350]);
   SCREEN(blue_square,’FillRect’,[0 0 255], [350 250 450 350]);
   This puts a blue square in a rectangle defined as [350 250 450 350], which would be 100 pixels on a side in the center of an 800×600 screen.
3. You would think then, that when you go to copy this to the on-screen window, that you would define the source rect and destination rect as the same [350 250 450 350]. Unfortunately, you would be wrong. If you define an off-screen window as something smaller than screenRect, no matter where you define it, psychtoolbox will always act like you placed it in the upper-left hand corner at 0, 0. So, the correct way to copy this to the center of your on-screen window would be:
   SCREEN(‘CopyWindow’,blue_square,window,[0 0 100 100],[350 250 450 350]);
   
4. So, to eliminate (I hope) the confusion about this sort of thing, here’s the easiest way to go about this. No matter where you want your object to eventually end up on the display, ALWAYS define smaller-than-screenRect-sized off-screen windows in the upper left-hand corner. So, the two lines of code from above would change to this:
   blue_square = SCREEN(window,’OpenOffscreenwindow’,[0 0 0],[0 0 100 100]);
   SCREEN(blue_square,’FillRect’,[0 0 255], [0 0 100 100]);
   

This way there is consistency in all of your locations. Copying it over to the on-screen window is still the same as above, except now it makes conceptual sense…..you defined something in the rect [0 0 100 100], then you copy from that same rect to a different one of the same size ([350 250 450 350]).
SCREEN(‘CopyWindow’,blue_square,window,[0 0 100 100],[350 250 450 350]);
Voila!

This function draws text on whatever window you define. It’s pretty quick, so if you need to write stuff on the screen in real time it’s not a big deal (i.e. you don’t need to use CopyWindow). Call it like this: SCREEN(window,’DrawText’,’This is the string you want to display’,xstart,ystart,[R G B]) The xstart and ystart variables are the starting coordinates for the “pen”. It is the lower left hand corner of the text you are writing.

Changes the size of the text written by DrawText. For example, change to 40pt font like this:
SCREEN(window,’TextSize’,40)

Changes the font. Example, changing to Arial.
SCREEN(window,’TextFont’,’Arial’)

1. Many times you’ll want to center your text in the middle of the screen. This is most easily done by taking advantage of the TextWidth function, which calculates the length of a given string. Example:
   widthofstring = SCREEN(window,’TextWidth’,’How long is this?’)
   The variable widthofstring will now contain the length of the string “How long is this?”
2. Draw it in the middle of the screen. You can determine the center of the width of your screen using your screenRect like this:
   centerWidth = (screenRect(3)-screenRect(1))/2;
   This is useful for creating a script that is interchangeable between resolutions. To draw it:
   SCREEN(window,’DrawText’,’How long is this?’,centerWidth-widthofstring/2,ystart,[R G B])

• This function draws solid rectangles (that is, they are “filled-in” with the color you specify. See the blue square example above in the CopyWindow section.
• Note that FrameRect operates almost identically and draws rectangles that are NOT filled-in. FrameRect includes an extra argument or two to define “pen-width”….how wide the border of your rectangle will be.

Similar to FillRect but draws ovals/circles. The shape (ovular or circular) is defined by the shape of the rectangle you specify. If the rectangle is a square, you get a circle. If your rectangle is actually rectangular you get an oval. Example of a red circle with radius 50 in the upper left corner:
SCREEN(window,’FillOval’,[255 0 0],[0 0 100 100]);
FrameOval operates in the same fashion as FrameRect.

Kind of like FillOval in that it draws ovals and circles, but you have to specify the angles through which to draw. That is, it’s easy to draw portions of circles and ovals. Example of the top half of a circle in the upper left-hand corner of the screen:
SCREEN(window,’FillArc’,[255 0 0],[0 0 100 100],-90,180)
The first angle argument (-90) specifies the starting angle, with 0 being vertical. So, -90 is horizontal to the left. The second angle argument (180) specifies the angle sweep to draw. Thus, this will draw 180 degrees (half) of a circle.
There is also a FrameArc function that is similar.

Often times you’ll want to display .jpg images on the screen. Note that jpg’s work the best in our experience. Other formats like .bmp probably work ok, but GIF’s are definitely screwy. Avoid them or convert them to jpg’s (I’ll explain how to do that below).

• This function loads your graphic into a MATLAB matrix. Use it like this:
  [img] = imread(‘filename.jpg’,’jpg’);
  You now have a variable called img that contains the information.
• If you must deal with GIF’s, load them as such:
  [img,map] = imread(‘filename.gif’,’gif’);
  

1. The most common reason to save images is if you need screenshots of your experiment for a poster or something (use in conjunction with GetImage; see below). Example:
   imwrite(img,’screenshot1.jpg’,’jpg’);
   This will write the image data in the variable img to a jpg file called screenshot1.
2. To convert a GIF into a jpg:
   imwrite(img,map,’newjpgimage.jpg’,’jpg’);
   

This is another SCREEN subfunction. Pretty easy to use:
SCREEN(window,’PutImage’,img,[0 0 200 200])
This would put the image data from img into the indicated rect, [0 0 200 200]. If the image is a different size than the specified rect it will be scaled to fit.

Couple this with the imwrite function above to grab and save screenshots. Example:
img = SCREEN(window,’GetImage’,screenRect);
This would grab the whole screen and save the image data in the variable img. You can also easily grab smaller rects if you wish.

Psychtoolbox has a few built-in functions for keeping track of experiment timing.

Gets the time in seconds since the computer started, with 4-decimal place precision. Thus, you will generally use it by first creating some sort of start time variable and using it as a reference. For example, you might want to record subject response time, so you could create a variable startsecs = getsecs right before they are prompted for a button press. Then, when the subject actually presses the button, you could record a new variable, responsetime = getsecs-startsecs.

Waits for the number of seconds specified. This is good for when you need some sort of delay between two different aspects of your display. For example, you might show a box indicating the selection a subject made, and then two seconds later indicate that they won or lost money. You could use the command waitsecs(2); to wait for those two seconds.

This is just a snippet of code that will hold the program in a loop until the start of the next available TR. Assume in this example that the variable startsecs references the time at very beginning of the experiment, and that a variable TR has been defined as well (in seconds). Specifically this will wait until the getsecs clock is within .001 seconds of the next TR onset.

while mod((getsecs-startsecs),TR) > .001
end

For your experiment you will have a set run time (set time that the scanner will be collecting images) based upon the number of trials and such that you wish to accomplish per run. You will need to synch your script with this timing. This means it won’t be sufficient to just set up a for loop for the number of trials and let the thing go. Instead you will need a while loop that incorporates scan run time.
This while loop set up will be in the following general form:

currenttrialnum = 0;
while (getsecs-startsecs)<totalruntime & currenttrialnum < maxtrialnum
     ALL THE TASK STUFF GOES HERE
     currenttrialnum = currenttrialnum+1;
end
while getsecs-startsecs<totalruntime
end
SCREEN(‘Closeall’)

This ensures two things. 1) If the subject goes quickly and finishes all of the trials before time runs out, the second while loop will wait until the scanner stops before closing all of the psychtoolbox windows. 2) If the subject is slow and doesn’t get through all of the trials in the allotted time, the experiment will still stop at the end of the scan time (or fairly soon after).

In practice and testing scripts you will generally be using the keyboard to record subjects’ responses. The commonly used code for this is below. Using the KbCheck function, it waits for a subject to press one of the specified buttons. The buttons are specified according to their ASCII code, which you can determine using KbName.

Simply call this at the command line with the key you want to know the ASCII code for. Examples KbName(‘left’) will return 37 for the left arrow. KbName(‘z’) returns 90 for the Z key.
Some commonly used ASCII codes:

• left = 37
• right = 39
• esc = 27
• space = 32

press = 0;
while press==0
  [z,b,c] = KbCheck;
  if (find(c)==37)  %they chose left
	  press = 1;
  elseif (find(c)==39 %they chose right
	  press = 2;
  elseif (find(c)==27 %they chose esc to bail out
	  press = 3;
	  Screen(‘Closeall’)
	  return
  end
end

Notice that the variable ‘press’ is used to bail out of the while loop as well as to keep track of which button was pressed. Also notice the inclusion of the escape sequence. It will close all open Psychtoolbox screens so that you get your MATLAB command line back. You can obviously modify this code to keep track of other items as well. You can also add a timeout feature by adding the following: make a variable ref_time = getsecs right before the while loop, then change the while loop expression so that it reads while press==0 & getsecs-ref_time<timeout_time

In the actual scanner, the subjects will not be able to use a keyboard, but will instead use a buttonbox. Thus, you need to make a separate experiment script that checks the buttonbox for responses instead of the keyboard. The buttonbox can be checked for responses if an activeX control object referencing the buttonbox is created in MATLAB.

The Mabry software that is used to get responses from the buttonboxes is installed on the scanner computer. This software is currently installed on all scanner control computers, but it would need to be reinstalled if we got new computers, or if the computers were formatted. This is also the reason why this code will not work on the lab computers. If for some reason it needs to be reinstalled, you can find it in \User_Scripts\huettel\Mabry_Software.

The button box has four buttons and a joystick on it. The Mabry Joystick software that is used to record presses regards the joystick-top button as button #1, and the four actual buttons as buttons #2-5 going from left to right. Because the activeX control software only allows for the use of 4 buttons, you cannot access the fourth button, so it actually only goes to button#4.

1. Inserting the following code at the top of your script will create the needed ActiveX control object:
   h = actxcontrol('Mabry.JoyStkCtrl');
   
2. This code will open a blank figure. Do not close that figure! The activeX control is embedded within that figure. You can close the figure during breaks between runs if you want, but it is not necessary.

You’ll use essentially the same code as above for recording keyboard input with some slight modifications.

press = 0;
while press==0
  [z,b,d]=KbCheck;
  c = [h.button2 h.button3]  %we’re recording the first two buttons
  c = sum(find(c));
  if c==1  %user pressed first button
	  press = 1;
  elseif c==2  %user pressed second button
	  press = 2;
  elseif find(d)==27  %still include escape sequence
	  press = 3;
	  SCREEN(‘closeall’)
  end
end

The gamepad should be located in the mock scanner room. It’s got a USB cable on it, so all you need to do is plug into the USB port on the front of the computer (under the little silver panel that says Dell on it). The gamepad uses the same activeX control as the buttonbox. The code for setting up the activeX control is identical. Button numbers are slightly different on the gamepad. There are four buttons on the right side of the controller. Starting at the leftmost button and going clockwise, the buttons are numbered 1 through 4. So for a decision between something on the ‘left’ and something on the ‘right’ you’d probably access h.button1 and h.button3.

Generally we’ll be recording output in text files. You’ll have to decide, for the particular experiment, what outputs you should be recording, so I won’t go into that here. However, this should get you started.

In almost all cases you will place output into the Data\Behavioral\ directory for your subject. Also, it’s customary to have one output file per run, named something like ‘scanner_output_[subject#]_[run#].txt’.

This function opens a filestream that you can write to. Open one like this:

fid = fopen(‘outputfilename.txt’,’w’);

Fid stands for “file input destination.” Change the filename to whatever you want, and make sure to add the ‘w’ argument which opens the file with write access.

This function prints information to a fid. You have to tell it exactly what to print and where. The general format is as such:

fprintf(fid,‘[format rules]’,arg1,arg2,arg3…..)

I’ll discuss format rules below. The args are all the variables you want to print to the file.

You need to tell fprintf what type of variable you are printing (double, string, etc.) and how to display it. Each variable type has a single letter designation that must always be preceded by a %. The common ones are below. If you need more, Google something like “fprintf formatting rules.”

1. %s – string
2. %d or %i – double
3. %f – floating point decimal
4. %c – single character

In the above formatting rules for numbers you can specify things how many decimal places you want to print to. An example of the formatting for a floating point decimal printed to 4 places would be %.4f It is recommended that you use fwrite, as the syntax is simpler. To find out more typle help fwrite into the matlab command window

1. Just placing a space between two data type signifiers will actually put in a space. For example ‘%s %d’ would print a string followed by a space followed by a number.
2. \t – inserts a tab
   • You’ll generally want to insert tabs between columns of your output file (i.e. between your args), because tab-separated files are more convenient for programs like Excel or Matlab to read.
3. \r – carriage return
4. \n – new line
   • Because of the legacy of typewriters, where both a carriage return and a new line was needed to begin a new line, to actually get to a new line in MATLAB you need to use both \r and \n in conjunction. So a typical formatting string would end with ‘\r\n’. An example: Say I wanted to print three variables separated by tabs for each trial in an experiment: resp_time, resp_acc, and money_won. Resp_time will be a floating point that should carry out to 4 decimal places. Resp_acc will be a string that reads “correct” or “incorrect”. Money_won will be a dollar value to two decimal places. Do it like this:
     fprintf(fid,’%.4f\t%s%.2f\r\n’,resp_time,resp_acc,money_won);
     

Closes the file (actually the fid) you’re currently writing to. Called as such:

fclose(fid);

It is important to close every fid that you open or else you’ll run into odd problems when you try to open a new one.

• In most cases we won’t want to make things in our experiment entirely random. That is, say you had four different types of trials in your experiment and you’d like to have them appear in equal numbers but in a random ordering. Furthermore, assume that these trial types are designated as type 1, type 2, etc. In your script you’d most likely create a vector of 1’s, 2’s, 3’s, and 4’s so that on each trial, the script knows what type of trial to display. Here’s how to make that vector quickly and easily:
• Say you want to do 10 of each trial type. Create a vector called trial_types as such:
  trial_types = [ones(1,10) 2*ones(1,10) 3*ones(1,10) 4*ones(1,10)];
  The function ‘ones’ creates a matrix of 1’s that is the size you specify. In our case each one of these is 1×10. By multiplying each of these vectors by a number (2,3,or 4) we change all the values in the vector. Then we concatenated them together. So, right now we have a vector with 10 1’s followed by 10 2’s etc.
• Randomize it. Do this by using randperm. Randperm will create a randomized vector of numbers in the range you tell it. So randperm(8) would create a randomized vector of the numbers 1 through 8. To randomize your trial_types vector:
  trial_types = trial_types(randperm(length(trial_types)));
  
• There are other cases in which you’d like to pseudo-randomize things like this. This is just one useful instance.

In the mock scanner, stimuli are presented to the subjects using mirrored glasses and a flat-screen monitor located above their head. Due to this location, all stimuli will be appear to be left-right flipped. If your stimuli are completely symmetrical about a vertical axis, this isn’t a big deal. However, if you have things that need to be in a single orientation to make sense (the most obvious example being letters or words), you’ll need to manually flip them.

The basic strategy here is to utilize the GetImage and PutImage functions discussed earlier to grab a screenshot of an off-screen window, left-right flip the resulting matrix, then copy that flipped matrix to the on-screen window.

Make an off-screen window called something like window2:

window2 = SCREEN(window,‘OpenOffscreenWindow’,[],screenRect);

Place all of your stimuli here instead of the on-screen window. When you’ve placed everything in window2, grab a screenshot of it. Use GetImage to do this:

img_array = SCREEN(window2,’GetImage’);

Calling it like that will ensure you’re grabbing the full screen.

Fliplr is a built in MATLAB function that will flip a matrix from left to right.

img_array = fliplr(img_array);

This is now just like placing an image (like a .jpg or something) on the screen. Use PutImage as described before:

SCREEN(window,’PutImage’,img_array);

GetImage is not the fastest function, especially when you’re grabbing an image of the entire screen. As a result, mirroring stimuli like this WILL have an effect on your experiment timing. This really shouldn’t be a huge issue since you’ll only need to flip things for use in the mock scanner, where timing isn’t super important. Regardless, to keep an experiment running smoothly (and close to the timing you want), try to cut down on the number of times you call GetImage. Place everything into an off-screen window first, then call GetImage, flip, and display.

If your subjects are going to be making decisions using the gamepad, make sure that you switch the button designations left-to-right. If you don’t, they’ll press the left button, meaning to select whatever it is they see on the left, except they’ll have chosen what is actually on the right side of the screen. (Confused yet?).

It appears that remote desktop limits the “pixelsize” to 16, so in order for screen to function through remote desktop, you need to set pixelsize to 16, rather than 32. A simple workaround for this so that you don't have to go and change all of you code is to use the line:

   eval('[monitor, screenRect] = Screen(0,''OpenWindow'',[],[],32)',...
  '[monitor, screenRect] = Screen(0,''OpenWindow'',[],[],16)') ;

Which will attempt to use 32, and if that fails will use 16.

If you use remote desktop, it will alter the settings on the computer you are logging onto, potentially creating keyboard input problems. If you restart the computer, these should be fixed. (This is a very tentative problem and solution, but it worked once).