What did you do this week?

This week I had to travel back to my college since the summer vacations have ended. I had a coding session with Serge this week, we modified the exporter function and looked at some bugs I faced.

We managed to use the io.load_io function. There was a strange issue with loading png files. I had to convert the PIL Image to RGB format, and it fixed the issue, turns out png images are stored as P (pallet) mode.

I also added the glb format support to the importer. While loading a .glb model, I noticed that the image data is also stored in the buffer and there can be a bufferview index to get the buffer.

During this time, I also tested the glTF loader with all models in the KhronoosGroup/glTF-samples repository. Here's the table of models working well: see file.

What is coming up next week?

• Adding tests and merging export function PR.
• Start working on Simple Animations.

Did you get stuck anywhere?

• To create a texture, we needed the RGB values, However .png images were returning a 2D array when read using PIL. It is fixed by

   

• pygltflib's load method doesnot handle glb files very well. It does not contain the buffer uri. I used glb2gltf method as of now.

What did you do this week?

• The first task for this week was to fix the glTF fetcher. We noticed that while running tests it reaches the API limit. So we decided to use a JSON file that contains the download URLs for all models.
• Created a function to generate JSON files with download URLs for all models, which can be found here.
• Modified the tests and fetcher.py to download using the JSON URLs and merged the PR #616.
• Added docstring for all functions in #600. Wrote tests for transformation functions.
• Multiple actor support in glTF export. Added tutorial and created a new branch for the same. Here's an example: (The glTF model below is created using FURY and rendered back using the glTF class)

What is coming up next week?

I will be doing the following:

• We still need to figure out how to get indices from vtkSource actors
• I'll be finishing the exporting functions and adding tests for the same and I should be able to create a mergeable PR for the exporting function.

Other tasks will be decided after the meeting.

Did you get stuck anywhere?

No

What did you do this week?

This week I worked primarily on the glTF fetcher and exporting scenes to a glTF file. I added tests and docstrings for all functions. I modified the fetch_gltf function to return the downloaded files. As we planned, the PR for the glTF fetcher was merged this week.

I fixed the color issue of the glTF exporter by manually appending the color data from the actor to the polydata. But there's another issue raised while using actors from vtkSource. The utils.get_polydata_triangles(polydata) method only works with primitives and it raises an error when used with vtkSource actors.

Textures and Cameras can now be added to the glTF file. However, it supports baseTexture only. I'll be working on materials support later on.

What is coming up next week?

• Saving all models download link (from the Khronos glTF-Samples-Models repository) to a JSON file, create a separate branch and add the download script.
• Add tests and docstring for PR #600.
• Create a PR for glTF exporter.

Did you get stuck anywhere?

• I managed to fix colors in polydata by adding them manually, but it raised another issue with indices (triangles) of the actor weren't of the correct shape. We decided to take a look at it later.
• Due to Github's API limit, it raised an error due to limits exceeding. We decided to save a JSON file with all model names and their download links. Then use that to download the model.

What did you do during the Community Bonding Period?

In the community bonding period I met with my mentor Serge Koudoro, along with other mentors in FURY and gsoc contributors. We discussed about my project and set up project timeline on github projects section. As my project (glTF Integration) and Keyframe animations were related so we discussed on that too.

I read documentation of various glTF loading libraries (pygltflib, panda3D-gltf and pyrender) in python and tried to understand how do they work. I started creating a basic glTF loader, it was using dataclasses to store json data. I created polydata for each primitive in a mesh, each polydata will contain the vertices, triangles, normals. To apply textures properly, I had to apply texture coordinates to polydata and flip the texture along X axis by 90 degrees.

What did you do this week?

After discussing on pros and cons of various glTF libraries we decided to use pygltflib to handle json to python dataclass conversion. This week I reshaped PR #600 to use pygltflib. I also modified the code to handle multiple base textures. While experimenting with textures, I accidentally applied normals to the polydata and discovered that it makes the surface look much smoother, however in few models it results in a darker model which reflects almost no light. So I made it optional for now to apply normals in a model.

I also created a basic fetcher (PR #602) to get glTF models from Khronos group's glTF sample repository. I also made this function asynchronous, as Serge suggested me to use asyncio and aiohttp to make the API callbacks asynchronous and it decreased the downloading time of multiple models.

I am also working on exporting scene to a glTF file. Using pygltflib I am converting the python dataclass to a json like structure.

What is coming up next week?

Create a PR for the fetcher function, add tests, fix bugs and merge it by the end of the week. Fix the colors issue in the glTF exporter. Add texture and camera in glTF exporter and create a PR for it.

Did you get stuck anywhere?

• var: list[int] was causing the error TypeError: 'type' object is not subscriptable since the ability to use the [] operator on types like list was added in 3.9+. I had to modify the dataclasses and use Typing.List instead.
• Texture in actors weren't applied correctly. I tried flipping the texture image by 90 degrees using gimp and the texture worked nicely. The reason for this issue was the coordinate system of glTF texture format which has origin (0, 0) at top-left and (1, 1) in the bottom-right corner, Where as in vtkTexture we want coordinate (0, 0) and (1, 1) at bottom-left and top-right corner respectively. Here's an example of the issue:

• Praneeth told me that some models with multiple nodes and multiple meshes weren't loading correctly. The reason for it was the way SCALAR data was extracted from the binary blob, I had to change some variables in get_accessor_data method and everything started to work just fine.

About Me

Hi! I'm Shivam, currently pursuing my bachelor's (expected 2024) in Production and Industrial engineering from the Indian Institute of Technology (IIT) Roorkee.

I was introduced to the C programming language through the Introduction to programming course in my first year of college. I always liked computers, and I was overwhelmed by printing "hello world" in the terminal. The course continued to teach us data structures and algorithms. It was a lot of fun competing with friends over a programming question. I learned python on my own and did a lot of projects. After learning basic programming, I participated in many hackathons and won some, but lost a lot. Coding was enjoyable, and I knew this was what I wanted to do for the rest of my life. In my second semester, I learned about open-source, git, and GitHub. I came across some computer graphics enthusiasts in my college; they told me that they created a 2D game engine on their own using OpenGL. This gave me enough motivation to learn OpenGL and basic computer graphics.

Intro to Open-Source and GSoC

In October 2021, I participated in Hactoberfest and completed it successfully. I learned a lot about Free and Open Source Software during this time. I heard about GSoC around this time from one of my seniors and asked him about the program.

I went through the previous year's accepted organizations list and shortlisted 2-3 organizations. I started contributing to a few projects based on android and Kotlin, but I lost interest after some time. It was about December, and I hadn't chosen any organization yet.

I heard about FURY from one of my seniors. I started looking at its docs. Picked up some books from the library to brush up on my concepts of computer graphics. The documentation of FURY is remarkable. I created my first pull request by improving one of the tutorials, and It got merged in a few days.

I started looking at the source code of FURY and tried to understand it again. I read the API reference part of FURY docs this time along with the documentation of VTK. I also tried to solve some of the open issues. I created a few pull requests for the same. My first contribution was in the fury primitive class, I created a sphere primitive (inspired by the sphere in Blender).

I started looking at bugs and improvements that can be made to the source code and created PRs for the same. During this time I was also learning how computer graphics work from the UCSC lectures & OpenGL by Victor Gordon.

After the accepted organizations were announced, I was so happy that FURY got selected this year for GSoC and went straight to the Ideas list. The first project idea was glTF Integration. I heard about the glTF file format before but didn't know how it works. So I went straight through the reference materials provided on the wiki. I read a lot of documentation by the Khronos group. I also tried to implement a basic glTF loader during this time using VTK's built-in glTFReader.

I also liked the Improve UI drawing project idea (I had a basic understanding of line drawing algorithms and rasterizations at that time) and thought I'll make a proposal for that too. After completing my proposal for glTF integration I started my research on this one too.

I started writing the proposal early so that I could get my proposal reviewed at least once with the project mentors. I almost forgot the fact that I had my end-term examinations at hand (Somehow I managed to pass all of my courses), I kept contributing to the project till the end of April 2022.

Code contributions:

1. [https://github.com/fury-gl/fury/pull/520]
2. [https://github.com/fury-gl/fury/pull/525]
3. [https://github.com/fury-gl/fury/pull/533]
4. [https://github.com/fury-gl/fury/pull/547]
5. [https://github.com/fury-gl/fury/pull/556]
6. [https://github.com/fury-gl/fury/pull/559]

The Day

May 18: I was a bit anxious since on May 18th the GSoC website was showing my proposal for glTF integration had been rejected. (p.s. maybe it was a bug of some sort that they fixed later on).

May 20: I woke up very early in the morning and started checking the contributor's profile. I kept checking for new emails but didn't receive any emails that day. "The result will be announced at 11:30 PM, isn't it? " My dad said, It was 8:00 AM IST. I called my friends in the night to join me on discord, I was talking to them and refreshing the GSoC site at the same time. One of my friends shouted that he got selected in NumFocus. I refreshed the page again, my proposal for glTF Integration was accepted. I can't express what I felt at that moment. I told my friends and my parents, they were happy for me and I got a lot of blessings :). I received an official email the next day.