End of my GSoC journey - 31/08/2020
My GSoC'20 journey end today. This summer was a truly an amazing journey and surely an unforgettable one!
I would like to thank my mentors John Andersen for guiding me and being a very patient & supportive mentor, Yash Lamba for being supportive & understanding and Sudharsana for being helpful throughout my journey. Their guidance and support is the reason why this project was successful. I have learnt a ton of important things from them.
I'd also like to thank my fellow GSoC students Himanshu Tripathi and Aghin Shah Alin for helping me during the summer.
Thank you to Google and Python Software Foundation for providing this opportunity!
What did you do this week?
I finalized the Custom Neural Networks PR, so to summarize what this PR adds are the following stuff:
1. Create custom neural networks using config files for performing image processing tasks.
2. Replace last layer of torchvision pre-trained models to perform classification using powerful architectures.
3. Entrypoint style loading of Loss functions to modify their parameters.
4. Accommodate image processing tasks other than image classification.
5. Tests and example tutorial for creating networks using DFFML to classify Rock Paper Scissors hand poses.
Final Report and Future Work
Add ways of training and testing machine learning models in DFFML on image datasets and perform image processing and computer vision tasks via DFFML.
The project is divided into 2 parts:
- Wrapping the image processing libraries namely OpenCV and Scikit-Image.
- High Level Operation Workflow, i.e., Custom Operations which will act as high level operations implementing a predefined flow of OpenCV and Scikit-Image functions.
Modifications to the proposal and scope of the project
During the summer, the proposed work was modified to achieve better results. The finalized work done in the project:
- Wrapping important image pre-processing functions as DFFML operations.
- PyTorch based pre-trained Convolutional Networks for image classification.
- Custom Neural Networks for image pre-processing and classification tasks via DFFML.
Project Tasks Completed
Sources for reading and pre-processing of image datasets
To enable working with image datasets and pre-process these datasets, the Directory source and DataFlow pre-processing source were added. The Directory source is used to read images stored in a directory format and DataFlow pre-processing source is used to modify the data using DFFML operations and creating a flow of operations to run the data through. The edit command uses DataFlow pre-processing Source to modify records using operations and provides an option to overwrite modified records on old records.
- DataFlow pre-processing Source
- Doctestable Example for DataFlow pre-processing Source
- Edit command to edit records present in a source
- Directory Source
Image Processing Operations
Added DFFML operations that wrap functions from OpenCV for pre-processing images after reading the image dataset from the source provided. The operations are put in a flow through which the image data runs by and is pre-processed before they are feeded to a machine learning model in DFFML for performing various tasks such as image classification!
- Added OpenCV functions as DFFML operations
- PyPi: https://pypi.org/project/dffml-operations-image/
- Added more pre-processing functions and support for default values in operations
Convolutional Neural Networks
Visual data being very complex and containing high dimensions of features of same type in different orientation can be very hard to classify using classification models such as RandomForestClassifier, KNearestNeighbours, etc. This is where powerful deep learning models come into the picture. Deep learning networks like Convolutional Neural Networks offer great accuracy and versatility for tasks such as image classification, object detections, etc.
By adding Convolutional Neural Networks to DFFML, it becomes very easy to perform image processing techniques and get great results by using Transfer Learning or creating custom networks.
- Transfer learning PyTorch based models with dynamic loading for image classification & add documentations and tests for the same
- PyPi: https://pypi.org/project/dffml-model-pytorch/
- Added example usages for classifying flower species using OpenCV and Transfer Learning approach and added tests for the examples
- Custom Neural Networks, custom layer addition support, loss entrypoint classes along with their example usage and tests
Many other important features that aided the addition and proper functioning of image processing tasks in DFFML are listed below:
- Improve String Representation for better viewing of image records
- Added support for loading images from fle formats namely JPEG, PNG and TIFF
- PyPi: https://pypi.org/project/dffml-config-image/
- Add ability to add operations while creating a dataflow with different names so as to use same operations for different tasks in a single dataflow
- Ability to load YAML/JSON file formats as dictionaries via DFFML CLI
DFFML is a machine learning based project which aligns with my interest in the field, so I will be more than happy to stay a part of the community to keep contributing and learning!
Goals for future contributions:
- Adding more image processing techniques.
- Adding image processing examples via DFFML's PyTorch model plugin such as image colorization.
- Connecting Transfer Learning models with custom Neural Networks to use powerful architectures for different Computer Vision tasks.
- Contribute to DFFML's Web UI.
Link to read the complete final report: GSoC2020FinalReport.md
Thank you for reading!