======================================= Stimulus Syncing ======================================= When recording a scan there are 3 computers involved: * The `stimulus` computer has psychtoolbox and takes care of showing the images in the monitor. A stimulus is made of many trials and each trial lasts for one or more 'flips' (see Glossary below). This computer records the time of each flip in each trial in stimulus.Trial in `stimulus clock` time (i.e, time measured in this computer, in seconds). * The `scanimage` computer which takes care of scanning and saving the scan. During scanning this sends a `frameSync` signal (see Glossary) which gets routed to the behavior computer (this is a cable so there's no latency). * The `behavior` computer is the master controller. This runs 2pMaster and takes care of starting the stimulus, starting the cameras (pupil, posture), start recording treadmill, audio and temperature, and sending an start signal to the scanimage computer. This computer takes care of saving the videos, inserting scan information in experiment.Scan and saving the `behavior file`. The behavior is an h5 file that records the scanimage's frameSync signal, the photodiode signal (see Glossary), some behavioral measures (wheel position, audio, temperature); all of them timestamped in `behavior clock` time (or `master clock` time). Behavior clock time is an integer counter that increases every 0.1 usecs and wraps around at 2 ** 32 - 1. This time is VERY exact. Glossary =========================================================== frameSync signal: The scanning laser turns on at the start of each frame and turns off at the end (while the objective is moved to the start of the next frame); this signal is recorded by scanimage as an analog signal (~0V for laser off, ~5V for laser on) and sent directly to the behavior computer. .. image:: images/frameSync.png photodiode signal: During stimulus presentation, a little square in the upper left corner of the screen flashes going from white-to-gray, gray-to-white, black-to-gray and gray-to-black; we have a photodiode sensor recording those intensity sensors taped to that corner of the screen. The photodiode signal is the signal recorded from this photodiode. This signal encodes flips. .. image:: images/photodiode_signal.png flip: A flip is a change of the photodiode from white-to-gray, gray-to-white, black-to-gray or gray-to-black (for instance that photodiode signal above has around 19). A flip time is the time at the flip. These are generated by psychtoolbox and are used to make sure stimulus was correctly presented (they have to follow a specified order) and to synchronize what was shown on the screen to all other behavioral measures (because we have times for this in the `stimulus clock` recorded in stimulus.Trial and in the `behavior clock` recorded in the behavior file). *Goal of syncing:* Because the stimulus flip times are recorded in `stimulus clock` time while everything else is recorded in the `behavior clock` time, we have to synchronize both so we can know what stimulus was happening during x recording frame or x behavior measure. How syncing works: =============================================================== stimulus.BehaviorSync ---------------------------------------------------------------- For each recorded frame in the scan, find its time in `behavior clock`. This requires reading the frameSync signal, find all the rising edges (when the scan of that frame start) and record all their timestamps in `behavior clock` (this timestamps are already in the behavior file so it's a matter of finding the indices of each rising edge). stimulus.Sync ---------------------------------------------------------------- Find frame times in `behavior clock` (same way as BehaviorSync), call it `frame_times`; find flips in photodiode signal and their respective time in `behavior clock` (again finding the indices of the flips and indexing the timestamp vector), call it `photodiode_flip_times`; get all flip times from stimulus.Trial in `stimulus clock` time, call it `trial_flip_times`; fit a line between `photodiode_flip_times` and `trial_flip_times` to find the `stimulus clock` time equivalent to the `behavior clock time` of zero; and add this to `frame_times` to have the frame times in `stimulus clock`. Some other details that are taken care in code ------------------------------------------------------------------------------------------------------------------------ * Time for the analog signals in the behavior file (frameSync, photodiode, wheel) is recorded in packets (i.e., each 1K samples are given the same timepoint corresponding to the exact time the last of the 2000 samples was recorded). We interpolate to get the right time for all samples before any syncing (see pipeline.utils.h5.ts2sec). * We transform the `behavior clock` time into seconds (from counter times). So behavior file has time in counters (0-2**32) but all saved stuff is in seconds (0-infinity). (See pipeline.utils.h5.ts2sec) * I haven't explained the way flips are encoded or decoded (see pipeline.utils.h5.find_flips) * All of the code dealing with behavior files is able to deal with lost packets or NaNs in the behavor file by interpolating with NaNs.