https://blogs.python-gsoc.orgUpdates on different articles published on Adam2392's BlogenFri, 20 Aug 2021 00:43:20 +0000https://blogs.python-gsoc.org/en/adam2392s-blog/final-check-in-2/This final week, I've managed to finish the deprecation PR, merge the VAR model implementation and fix some bugs to allow for backwards compatibility. I also discussed future potential improvements that could be made in the package. I've thus been adding additional examples to fully document the capabilities that were added to the package during GSoC. Towards the end of the week, I will make a v0.2 release on pypi.adam2392@gmail.com (Adam2392)Fri, 20 Aug 2021 00:43:20 +0000https://blogs.python-gsoc.org/en/adam2392s-blog/final-check-in-2/https://blogs.python-gsoc.org/en/adam2392s-blog/weekly-check-in-6-23/This week, I've still been now finalized the deprecation PR, but waiting on mentor(s) and other developers to chime in regarding a very weird numpydoc error that comes up in our sphinx build. This to mine and my mentor's eyes doesn't look like it is caused by my PR changes. I have also added comparisons to statsmodels VAR model implementation with order-2 and order-1, thus validating that our implementation is just as good for VAR models.adam2392@gmail.com (Adam2392)Fri, 13 Aug 2021 02:04:52 +0000https://blogs.python-gsoc.org/en/adam2392s-blog/weekly-check-in-6-23/https://blogs.python-gsoc.org/en/adam2392s-blog/weekly-blog-post-5-2/What Have I Been Working On: This week, I’ve finished the deprecation of the connectivity sub-module in mne-python. It was actually more complicated then I originally thought and each time there was additional CI failures due to the interactions in various tutorials to the connectivity module. I have also now been working on comparing the vector auto regression to the implementation on statsmodels in both accuracy and also runtime performance. What Have I Been Struggling With: Mostly I have been struggling with CI failures that are difficult to debug due to my inexperience. There are various pytest and sphinx complexities that I was not used to, but it's been quite informative learning about various ways to customize pytest and sphinx in order to perform more complex unit testing and documentation building.adam2392@gmail.com (Adam2392)Fri, 06 Aug 2021 16:29:41 +0000https://blogs.python-gsoc.org/en/adam2392s-blog/weekly-blog-post-5-2/https://blogs.python-gsoc.org/en/adam2392s-blog/weekly-check-in-5-23/This week, I've been wrapping up examples showing the VAR model and incorporating mentor comments. I've also finalized the deprecation of mne.connectivity within the mne-python module.adam2392@gmail.com (Adam2392)Thu, 05 Aug 2021 01:01:32 +0000https://blogs.python-gsoc.org/en/adam2392s-blog/weekly-check-in-5-23/https://blogs.python-gsoc.org/en/adam2392s-blog/weekly-blog-post-4-4/What Have I Been Working On: This week, I’ve been working on finishing the vector autoregressive model addition, connectivity examples showing off the VAR model and adding a "connectivity for classification" example. This involves working a lot with scikit-learn's API to assist transformation of functions into scikit-learn compatible pipelines. The other examples require me to learn how to best simplify explanations for new readers. My plan for the upcoming week is to finish these functional improvements, taking in feedback from my mentors. I intend on then adding additional tests, and code that deals with bivariate statistical tests, or graph statistical tests. What Have I Been Struggling With: Writing good examples that are useful for package users is difficult, so there's a lot of iteration that I have been working on with my mentors.adam2392@gmail.com (Adam2392)Mon, 26 Jul 2021 16:25:28 +0000https://blogs.python-gsoc.org/en/adam2392s-blog/weekly-blog-post-4-4/https://blogs.python-gsoc.org/en/adam2392s-blog/weekly-blog-post-5-1/TBD.adam2392@gmail.com (Adam2392)Mon, 26 Jul 2021 06:45:57 +0000https://blogs.python-gsoc.org/en/adam2392s-blog/weekly-blog-post-5-1/https://blogs.python-gsoc.org/en/adam2392s-blog/weekly-checkin-4/This week, I've been implementing a vector auto-regressive (VAR) model with unit tests, and examples. Some portions of the code were ported from ``scot``, which had a BSD license. Next up, I am planning on adding bivariate/network based statistics, such as Granger causality, or independence testing between two time series from ``statsmodels`` and ``hyppo`` packages. In addition, I plan on adding posthoc graph statistics using ``graspologic``.adam2392@gmail.com (Adam2392)Fri, 16 Jul 2021 02:05:26 +0000https://blogs.python-gsoc.org/en/adam2392s-blog/weekly-checkin-4/https://blogs.python-gsoc.org/en/adam2392s-blog/weekly-blog-post-3-1/What Have I Been Working On: This week, I’ve been working on finishing the "Connectivity" data structure class container. I've successfully used "xarray" as the underlying data structure and implemented improvements in the API to simplify user experience when dealing with different connectivity structures. I've added additional unit tests to bring up the coverage from 79 to 82%. Now I am working on adding additional functionality to the saving/reading of the connectivity data structures as "netCDF" files. In addition, I'm working on improving the API for "envelope correlation" function. Finally, I've implemented storage optimizations for RAM for "symmetric" connectivity arrays. My plan for the upcoming week is to finish these functional improvements and then add in other connectivity functions that were previously implemented in the open-source package "scot". What Have I Been Struggling With: Getting the functional improvements working was difficult, especially with respect to reading/writing files. This is due to some minor bugs with respect to data types. What Solutions Have I Found: I'm trying to convert most of the data types stored within xarray to basic data types. I've also been backtracking my steps to see which changes caused this error.adam2392@gmail.com (Adam2392)Sun, 11 Jul 2021 17:24:29 +0000https://blogs.python-gsoc.org/en/adam2392s-blog/weekly-blog-post-3-1/https://blogs.python-gsoc.org/en/adam2392s-blog/weekly-check-in-3-25/This week, I've been implementing and making the API very nice and robust for a general Connectivity data structure that is also memory efficient. It's been very exciting because now we're packaging up all these connectivity functions to return a singular multi-array data structure with semantics for each of its dimensions. This makes use of "xarray", which has been very fun and interesting to learn about. Next up, I will be updating new examples to show usage of this Connectivity data structure and also porting over connectivity functions from the open-source package, "scot".adam2392@gmail.com (Adam2392)Fri, 02 Jul 2021 04:05:49 +0000https://blogs.python-gsoc.org/en/adam2392s-blog/weekly-check-in-3-25/https://blogs.python-gsoc.org/en/adam2392s-blog/week-3-blog-post-1/What Have I Been Working On: This week, I’ve been working on developing a "Connectivity" data structure class container for time-varying connectivity, frequency-varying connectivity, or spectrotemporal connectivity, and their counterparts over "Epochs". I have decided to use "xarray" as a base class for storing the data, since it provides naming and structure for multi-dimensional arrays. This has involved a bit of reading through documentation of "xarray" and discussing with mne developers. In addition, I've made an official v0.1 pypi release, so that I could deprecate the MNE-Python.connectivity submodule. So far, I have successfully updated the unit tests and documentation to work with the new connectivity containers. What Have I Been Struggling With: By switching the codebase to return this new connectivity container class, I may have messed up some memory optimizations that occur in some of the docs building. What Solutions Have I Found: This problem was isolated to be caused by a "sparse" array being represented as a dense array, so now I have to determine the best way to construct a sparse array and pass it as an underlying data representation to the connectivity data structure.adam2392@gmail.com (Adam2392)Mon, 28 Jun 2021 14:09:25 +0000https://blogs.python-gsoc.org/en/adam2392s-blog/week-3-blog-post-1/https://blogs.python-gsoc.org/en/adam2392s-blog/week-2-check-in-7/I've successfully coordinate with one of my MNE mentors in getting the CI setup correctly, such that docs are being deployed, unit tests are occurring and build tests are occurring during every commit to a new PR. The next step is figuring out how to make a release properly by having the CI build the docs and deploy them for a release when wanted. In terms of coding, I have been discussing and perusing the existing mne.connectivity code base to understand what sort of "Connectivity" data structure containers I want to implement to provide to the user. We have decided on a few and I will be implementing their functionality and tests this week!adam2392@gmail.com (Adam2392)Mon, 21 Jun 2021 03:50:23 +0000https://blogs.python-gsoc.org/en/adam2392s-blog/week-2-check-in-7/https://blogs.python-gsoc.org/en/adam2392s-blog/week-2-blog-post-2/What Have I Been Working On: This week, I’ve been primarily working on refactoring existing MNE-Python code in their connectivity submodule into the new “mne-connectivity” repository. In order to simplify the code changes, I have kept everything in terms of the API and examples the same name without changing any real code functionality. Instead, this week is dedicated to setting up the repository successfully. Thus, I have been working on setting up the documentation via sphinx, and setting up the continuous integration (CI) and unit tests. What Have I Been Struggling With: For the CI, I have ported over the code from an existing MNE repository, “mne-realtime”. This has multiple CI pipelines, such as Azure, Github Actions and CircleCI. However, getting the CI to properly work is a bit tedious since each pipeline run requires a Github push. Thus, a major time investment is on perusing over CI code line-by-line to either i) get rid of any unnecessary procedures that only mne-realtime needed, or ii) adjust the pipeline to be mne-connectivity specific. What Solutions Have I Found: I have also been looking at other MNE repositories, such as “mne-bids”, which has a very simple CI and documentation setup. Thus, to directly port the CI and documentation code into “mne-connectivity” is a bit easier. I have been using bits and pieces of code from “mne-bids” in order to i) get the unit tests working locally, ii) simplify the documentation setup via sphinx and then iii) simplifying the CI workflows.adam2392@gmail.com (Adam2392)Mon, 14 Jun 2021 01:48:07 +0000https://blogs.python-gsoc.org/en/adam2392s-blog/week-2-blog-post-2/https://blogs.python-gsoc.org/en/adam2392s-blog/initial-week-getting-started/I've been talking with my MNE mentors and reviewing some of the existing connectivity code for neural data. I have begun refactoring existing code from the "mne.connectivity" submodule into a new repository "mne-connectivity" housed under the mne-tools organization. Next week, I plan on finalizing this push with a working CI pipeline that confirms equivalent functionality to the original mne-python code. Things are progressing as expected so far!adam2392@gmail.com (Adam2392)Thu, 10 Jun 2021 07:15:03 +0000https://blogs.python-gsoc.org/en/adam2392s-blog/initial-week-getting-started/