Last week was successful in including all the necessary steps of performing Principal Component Analysis. The first issue was that the covariance matrix needed to be scaled to account for the SNR boosting step in which 3 epochs were averaged into a single epoch, therefore the covariance matrix needed to be divided by 3. Then I had to ensure that the average epochs were being processed in the same manner that my original epochs were processed. Next, a projection operator had to be applied to the forward matrix according to the epochs.info structure. The API is working successfully and re-producing the ground truth as produced from my command line model, within a small level of tolerance. However, the data scaling function that I had gotten to work earlier in the summer, when I was not using a ground truth to check my work, is not working now. I recall being able to reproduce the scaling function using MNE functions, so this week will focus on replacing that function with MNE functions. Then I will work to publish this script to MNE-Python's example data scripts for use of the functional connectivity model.

Although I only completed 1 of 2 major contributions that were intended for this summer, I learned a ton in regards to the MNE-Python functions, collaborations using Github, building objects and classes, and most importantly to use a ground truth when converting code into a new environment to ensure that each step is being transformed correctly and the frequent use of sanity checks. 

The first step of the API, simply loading the epoch data from a folder, was successful. I was also successful in implementing the SNR boost step which works to average a small number of epochs in order to increase the SNR per epoch input in to the model. I confirmed that the bootstrapped epoch output from the in-progress API was equivalent to the bootstrapped epoch output from the command line model. Then I got stuck on the PCA step. I first discovered that the epochs were being trimmed before the bootstrap step in my API, and after bootstrapping in the command line model. I made it so these were the same but the PCA step is producing a different number of principal components in the API than in command line. The remainder of the inputs seem to be the same but I will investigate deeper this week. After confirming the PCA output is the same, I will fit the model. 

Last week was successful in altering the pre-processing steps so that the model would process data in regards to the 4 labels included with the sample dataset instead of the 12 labels I had used for a previous dataset in past usage of the model. The model also ran successfully in the conda environment used by MNE-Python which includes several dependencies specific to MNE-Python. I had to add a single dependency used for calculating derivatives and performing back propagation. Moving forward, I will begin to reconstruct the API I had working in week 8 one step at a time and ensuring that I can compute the same output after each processing stop. The first step is an SNR boosting step in which a small number of epochs (per experimental condition) are averaged together in order de-noise the data. Once I check that the averaged epochs are the same from my API output compared to my original model output (computed strictly using command line), I can move forward toward computing PCA on the data. I will ensure that the same number of principal components are computed from my API and command line, and again, ensure from this step that I can produce the same output as produced from my model in command line.

Last week I was able to re-structure the sample dataset so that it would work with my original connectivity model. The model currently outputs connectivity measures for the 12 regions of interest used in my PhD connectivity project. This week I will work to get this model to work in the conda environment that MNE-Python uses to develop their tools. The following step will be to get the model to work with the 4 regions of interest used with the sample dataset from MNE. Then I can re-create the API I constructed over the last several weeks while ensuring that my output is as expected with each iterative step.

Last week was challenging. I was able to successfully change the scaling data function so that it incorporated the MNE functions to accomplish the same goal. The challenge arose when processing the 16 subjects' data that was meant to act as a ground truth to compare the model's output to an expectation. The output is currently pretty far from the expectation. I, more or less, need to start from the beginning and check step-by-step that each part of the model is producing something that I would expect. The biggest challenge being that it's so far in the summer and I am concerned I won't be able to get to the second part of my project. My plan for this week is to run the sample dataset, frequently used on the MNE-Python website to be used in examples, on my already existing pipeline which uses primarily command line and configuration files to process data. Once that is working I can do a step by step comparison of each step's output between the command line pipeline and the API I have been working on for this project.