Articles on m-agour's Blog

Week 14: Introducing standalone Animations

mo.aggour@gmail.com (m-agour) — Mon, 26 Sep 2022 19:28:46 +0000

What did you do this week?

PLACEHOLDER

What is coming up next week?

PLACEHOLDER

Did you get stuck anywhere?

PLACEHOLDER

Week 13: Keyframes animation is now a bit easier in FURY

mo.aggour@gmail.com (m-agour) — Tue, 20 Sep 2022 05:50:08 +0000

What did you do this week?

Added the ability to have the ShowManager stored inside the Timeline. That way the user does not have to update and render the animations because it will be done internally.
Added a record method to the Timeline that records the animation and saves it as either GIF or MP4 (requires OpenCV). This record functionality has the option to show/hide the PlaybackPanel which makes it better than recording the animation using a third-party software.
Fixed some issues that Serge mentioned while reviewing PR #665

What is coming up next week?

Instead of adding the ShowManager to the Timeline, doing it the other way around is a better choice and makes the code more readable.
Add tests for the Timeline's record method.
Add tests for the billboard actor to test consistency among different approaches..

Did you get stuck anywhere?

I didn't get stuck this week.

Week 12: Adding new tutorials

mo.aggour@gmail.com (m-agour) — Tue, 13 Sep 2022 05:52:06 +0000

What did you do this week?

Restructured tutorials to be more readable and more focused on a specific topic.
Replaced the old animation introductory tutorial with a lot simpler one. And added tutorial to explain keyframes and interpolators.
Simplified setting lighting uniforms for the geometry-based-billboard actor by getting the Scene from the actor using actor.GetConsumer(scene_idx).

What is coming up next week?

Allow the Timeline to take the ShowManager as an argument to reduce the amount of code user has to write every time using FURY animations.'
Fix some typos in the tutorials and write some info about Slerp.
Find a way to fix the shader-callback problem of not being executed when the actor is out of the camera's frustum.

Did you get stuck anywhere?

I didn't get stuck this week.

Week 11: Improving tutorials a little

mo.aggour@gmail.com (m-agour) — Mon, 05 Sep 2022 09:25:43 +0000

What did you do this week?

Fixed some issues in the hierarchical order animation support PR that we discussed during last week’s meeting (mostly naming issues).
Explained the introductory tutorial a little. But it is not good suitable beginners. So, I will spend time improving tutorials this week.
Added extrusion to vector_text to allow the z-scaling to be functional.
Fixed lightColor0 being hard-set to (1, 1, 1). Instead, it’s now using the Scene to set the lighting uniforms.

What is coming up next week?

Improve tutorials.
Find out how to get the Scene from the actor instead of manually assigning it.
If I have time, I will try implement recording animation as GIF or as a video.

Did you get stuck anywhere?

I didn’t get stuck this week.

Week 10: Supporting hierarchical animations

mo.aggour@gmail.com (m-agour) — Mon, 29 Aug 2022 13:59:29 +0000

What did you do this week?

Implemented hierarchical order support for animations using matrices in this PR.
Improved the API of a PartialActor by adding some methods to control the scales, positions and colors.
Added a new example of using the new hierarchical feature to animate an arm robot.
Improved vector_text a little by adding options to control its direction.

What is coming up next week?

Finish the hierarchical order animation support PR.
Explain tutorials in more detail. See this issue.
Fix issues discovered by Serge in this review.
Fix lightColor0 being hard-set to (1, 1, 1). Instead, find a way to get this property the same way other polygon-based actor gets it.
Try to get PRs and issues mentioned above merged, closed or ready for a final review.

Did you get stuck anywhere?

I didn't get stuck this week.

Week 9: Animating primitives of the same actor

mo.aggour@gmail.com (m-agour) — Thu, 25 Aug 2022 05:58:03 +0000

What did you do this week?

Made two tutorials that show two approaches on how to animate primitives of the same FURY actor using the Timeline.
Tried sending all keyframes at once as uniforms, but I faced a performance issue doing this.
Set uniforms that are not being sent by VTK for the billboard actor.

What is coming up next week?

Alter the Timeline to use matrices instead of setting values directly to allow hierarchical transformation.
Improve the API of the PartialActor to act almost like a normal actor.

Did you get stuck anywhere?

I had some issues but didn’t get stuck.

Week 8: Back to the shader-based version of the Timeline

mo.aggour@gmail.com (m-agour) — Mon, 15 Aug 2022 10:30:35 +0000

What did you do this week?

First, I made some modifications to the main animation PR. Then as Filipi requested, I integrated the vertex shader That I was implementing a few weeks ago to work with the new improved version of the Timeline.
As for how keyframes are sent to the GPU, the method being used is to send the needed keyframes for each draw. This is heavy because we only roll out the interpolation part, which with linear or step interpolation won’t make any difference! Instead, I tested hard-setting all the keyframes as a constant variable by manipulating the shader program’s string. This also was slow to initialize, and the shader was getting bigger and slower to bind and unbind. to solve this problem, I made a uniform that holds all the keyframes of the animation and sent data as vectors with was faster than string manipulation, also it was faster to render since data are now stored directly in memory and the shader program was a lot more compact. But this method had an issue; uniforms do not keep data stored as expected! If two or more actors have the same uniform name in their shader program. And only one of them was set, the other actor will get this value as well. A way around this is to change the names of the uniforms so that they maintain their data.
Tested animating 200K billboard spheres animation but this time using translation as well. And they performed very well.

What is coming up next week?

Fix issues I encountered this week working with GLSL shades.
Implement slerp in GLSL as well as figure out a way to optimize sending keyframes to the shader program.
Figure a way to animate primitives of the same actor by different timelines.

Did you get stuck anywhere?

I had two issues, one mentioned above which was uniforms not being abler to hold data. The second one is that VTK does not send specular-related uniforms and lightColor0 to the dot primitive, which are needed for light calculations.

Week 7: Billboard spheres and implementing interpolators using closures

mo.aggour@gmail.com (m-agour) — Thu, 04 Aug 2022 19:04:35 +0000

What did you do this week?

Restructured the keyframe animation interpolators using closures to be functions instead of classes #647. Now it is easier to implement new interpolators with the help of the functions existing in fury/animation/helpers.py. Also, now unit tests can be added for the latter functions.
Added two examples explaining how to implement a custom interpolator that can be used by the Timeline, one using classes and the other using closures.
Fixed rotation issue that Shivam discovered while using the Timeline to animate glTF models. So, rotation in VTK is done by rotating first around Z-axis, then X-axis, then finally around Y-axis, which was not the order I was using to convert from quaternions to Euler degrees.
Made changes requested by Javier and Filibi on the billboards using geometry shader PR, and made an example of how to use this billboard to show an impostor sphere which looks almost like a real high poly sphere. Also benchmarked using this version of billboard vs using a quad-based billboard, and how they both affect the FPS of the animation.

What is coming up next week?

Document the new closure-based interpolators. And make some slight renaming changes that we discussed in today’s meeting.
Add tests to the functions inside fury/animation/helpers.py.
Make changes to the geometry-shader-based billboard to make it more like the quad-based billboard actor while maintaining the new features.
Integrate the already-implemented shader functionality to the new Timeline in a separate draft or PR.

Did you get stuck anywhere?

I got stuck trying to get and modify the vertices (centers) of the billboard actor.

Week 6: Fixing the Timeline issues and equipping it with more features

mo.aggour@gmail.com (m-agour) — Sat, 30 Jul 2022 18:19:06 +0000

What did you do this week?

Improved the PlaybackPanel by adding speed control and the ability to loop the animation. Also, fixed the lagging issue of play and pause buttons and composed them into a single play/pause button.
Updated the old tutorials’ syntax to match the other tutorials and added a new tutorial on position animation using spline interpolation. Added unit tests for the PlaybackPanel and the newly added color converters in colormap.py.
Added more hooks to the 2D sliders to cover two more states:
1. on_value_changed, which gets called whenever the value of the slider is changed without interacting with the slider.
2. on_moving_slider, which gets called when the position of the slider is changed by user interaction. #634.
- The reason for adding these two hooks is that there was only the on_change hook, which always gets called when the value of the slider is changed without considering how the value is changed, hence, the functionality of the slider was limited.
Provided the ability to add static actors to the Timeline, which might be needed in the animation part of Shivam’s glTF project #643.
- If an actor is added to the Timeline as a static actor, it won’t be animated by the Timeline, but it will get added to the scene along with the Timeline.
Implemented a custom evaluator for the Timeline’s properties.
- A custom evaluator uses a user-provided function that takes time as input and evaluates the property at that time. This feature is yet to be discussed more in today’s meeting.
Fixed camera rotation and the view-up issue when interacting with the scene.

What is coming up next week?

Make a tutorial on how to implement a new custom Interpolator to work with the Timeline.
Add motion path visualization feature for both position and color properties.
Add custom evaluation functions directly, without using the CustomInterpolator.
Implement an already existing complex interpolator using closures instead of classes.

Did you get stuck anywhere?

I didn’t get stuck this week.

Week 5: Slerp implementation, documenting the Timeline, and adding unit tests

mo.aggour@gmail.com (m-agour) — Wed, 20 Jul 2022 09:38:16 +0000

What did you do this week?

Implemented Slerp (spherical linear interpolation) for rotation keyframes.
Controlling the speed of the animation is now an option.
Added the tests and documented the Timeline.
Used the geometry shader to generate billboard actors using minimal set of calculations #631.

What is coming up next week?

This week I will do the following:

Focus on finalizing the PR #626 to be merged.
Make some upgrades to the playback panel to make it more functional and responsive.

Did you get stuck anywhere?

I didn’t get stuck this week.

Week 4: Camera animation, interpolation in GLSL, and a single Timeline!

mo.aggour@gmail.com (m-agour) — Sat, 16 Jul 2022 08:35:59 +0000

What did you do this week?

Managed to implement a single Timeline using the Container class. So, instead of having two classes: Timeline and CompositeTimeline, now the Timeline can have multiple Timeline objects and controls them as in the code below
```
main_timeline = Timeline()
sub_timeline = Timeline(actors_list)
main_timeline.add_timeline(sub_timeline)
```
Implemented the HSV, Lab, and XYZ color interpolators in GLSL.
Added support for animating two camera properties: position and focal position, which can be interpolated using any general Interpolator.
Allowed all actors inside all timelines to be added to the Scene automatically when the main (parent) Timeline is added to the Scene.
Fixed the PlaybackPanel, added a counter to display the animation time as in the video above, and added an option to attach a PlaybackPanel to the Timeline.
```
main_timeline = Timeline(playback_panel=True)
```

What is coming up next week?

This week I will do the following:

Start Implementing slerp interpolation using quaternions in both Python and GLSL. And use slerp to apply camera rotation.
Add tests for the previous work.
Make a PR to merge the non-shader-based version of the Timeline.
Test how shader attributes can be used to hold keyframes for each vertex and benchmark it.
Study ‘colormaps’ and implement some ‘colormaps’ in GLSL.

Did you get stuck anywhere?

Uniforms don’t maintain their data after shaders are unbounded and another uniform with the same name in a different shader is set.

Week 3: Redesigning the API, Implementing cubic Bezier Interpolator, and making progress on the GPU side!

mo.aggour@gmail.com (m-agour) — Thu, 14 Jul 2022 16:05:24 +0000

What did you do this week?

Redesigned the keyframe animations API to optimize having a lot of timelines by composing them into a parent Timeline called CompositeTimeline while maintaining playing individual timelines separately.
Implemented the cubic Bezier Interpolator using two control points for each keyframe. Also made a tutorial for it below:
Also Implemented linear and cubic Bezier in GLSL interpolations to be computed by the GPU by sending two keyframes as uniforms and the current animation time.
Composed the playback panel as a single component PlaybackPanel (slider does not function yet).
As we tried to come up with a way to do keyframe animations on a partial subset of the actor’s primitives not all of them, we found that there is no currently implemented way to get the primitives count of a single actor. So I made this PR #617 so that the primitives’ count is saved inside the polydata of the actor as suggested by Filipi and Javier so that the primitives’ count can be used to distinguish the vertices of different primitives.

What is coming up next week?

This week I will do the following

Finish up the PlaybackPanel.
Implement all other interpolators in GLSL (color interpolators included).
Start Implementing slerp interpolation using quaternions in both Python and GLSL.
Add tests for the previous work.
Now that I made some progress with the keyframe animation API, I should be able to make a mergeable PR!

Did you get stuck anywhere?

Couldn’t get the LineSlider2D to work inside my new PlaybackPanel.

Week 2: Implementing non-linear and color interpolators

mo.aggour@gmail.com (m-agour) — Mon, 04 Jul 2022 20:17:09 +0000

What did you do this week?

Implemented some other general interpolators such as n-degree spline and cubic spline interpolators. Also implemented HSV and LAB color interpolators.

PRs #612 and #613
Added animation slider to seek a particular time and visualize the timeline in real-time.
Managed to do the transformation on the GPU side using GLSL using matrices. And did some digging about how and when we interpolate the camera and also how to do this on the GPU side.

What is coming up next week?

This week I will do the following

Allow FURY actors to maintain the number of primitives as an object property so that it can be used to manipulate only a subset of primitives in a single actor.
Change the structure of the Animation API to a newer one designed by Filipi to solve performance issues when creating a large number of timelines.
Implement the Bézier curve interpolation.

Did you get stuck anywhere?

I got stuck trying to fix the clipping plans not being appropriately set. Which caused actors to disappear at some point.

Week 1: Implementing a basic Keyframe animation API

mo.aggour@gmail.com (m-agour) — Wed, 29 Jun 2022 16:24:09 +0000

What did you do during the Community Bonding Period?

During the community bonding period, we had weekly meetings in which I got to know the mentors better and bonded with Shivam and Praneeth, my fellow contributors. We talked about the importance of open-source contributions and discussed scientific visualization, how important it is, and how it differs from game engines despite having some similarities. We also discussed how important the code review is and how to do a proper code review.

What did you do this week?

I continued implementing the Timeline class. Added some functionalities according to what we discussed in the weekly meeting, such as handling Fury Actors, adding, removing, translating, and scaling them according to the keyframes. Then made two tutorials for it with simple UI to control the playback of the animation.

Reviewed Bruno’s PR #424 as Filip advised to help me figure out how to change uniforms’ values during runtime. I found it straightforward. I also found a similar method already used in some tutorials and experiments to set the time uniform, which is based on the same idea. They both are using shaders callback functions. Going through the VTk’s documentation, I found a more efficient way to do this. It’s newly implemented in VTK 9, does not require callbacks, and is easy to use on individual actors. I had to verify first that Fury does not support an older version of VTK and it does not. I tested it and ran into some issues As Filip instructed I don’t waste a lot of time with uniforms so I postponed it for the next week.

Also, when I was attending Shivam’s meeting, he showed us a smooth glTF model which differed from the last time I saw it. When I asked him he said he applied the normal. Him saying that reminded me of how we can use normals to smooth shade shapes. I applied this trick to the sphere actor in this PR #604 and worked as expected.

What is coming up next week?

If the implemented timeline is good to go, I will write the associated tests, finish up any missing methods, and document it properly. I will probably continue implementing the other interpolation methods as well.

Did you get stuck anywhere?

I got stuck changing the color of some actors.

My journey till getting accepted into GSoC22

mo.aggour@gmail.com (m-agour) — Tue, 28 Jun 2022 07:12:11 +0000

A Little About Myself

My name is Mohamed and I’m from Egypt. I am pursuing a Bachelor of Engineering in Computer Engineering and Automatic Control (expected: 2023), Tanta University, Egypt.

I’ve been around computers since 2008 when I had my first PC with 128MB RAM and ran on Windows XP (that’s almost all I could remember about it).

Around 2013, I had some questions about how games are made and how to make them myself! There was no one to answer these questions for me.

My english wasn’t any good and the game development arabic community was very hard to find on the internet. But eventually, I came across a forum with some people speaking about some stuff such as game engines, 3D models, textures, animations, and a lot of fun stuff. That made sense back then and answered some of my questions. Then It was time to get involved with this amazing community. I was lucky enough to start with 3ds Max 2009 Edition, with that little view cube begging me to touch it. It was just love at first sight.

I learned enough to model a basic house with no interior or a low poly human face.

I was very curious about these game engines. Back then, the top three game engines available for the public were CryEngine 3, UDK, and Unity.

I was advised to use CryEngine 3 since it required no coding experience to use it. I started designing and texturing terrains and adding 3D models (I remember adding a pyramid and had a hard time scaling it down).

My first coding experience

It started with C programming language, which I came across while taking Harvard’s CS50. I then used it in competitive programming as my primary language.

When I heard about OpenGL API for the first time from a senior and that it is being used to develop game engines, I started learning it along with GLSL.

And it finally hit me: It is all math!

I then started practicing pipelines, lighting models such as Blinn and Blinn-Phong models, cameras, texturing, fog, skybox, shadows, etc…

Developing a 3D game

In the Graphics course, OpenGL was being taught to us using python and there was a course project!

I started preparing my notes and the shaders I wrote during that summer, only for the Professor to restrict us to use OpenGL v1.0 and only use PyOpenGL and Pygame.

So, no Numpy, no PyGLM, fixed pipelines, no custom shaders, and each vertex had to be sent individually to the GPU.

At first, I got disappointed, but while developing the game, I had a lot of optimization challenges that made me have fun figuring out how to still make the 3D game I’ve always wanted to make!

I searched on GitHub for similar projects to the one I’m trying to make and did not find any 3D FPS game using this version of OpenGL (at least not listed publicly).

I ended up implementing almost everything from scratch.

Real-world physics inspired the physics of the game. This means that I have collected data about walking speed, running speed, initial jumping velocity, and gravity. Even the sound Sound intensity is inversely proportional to the square of the distance between the source and the player.

I used Interpolation (bilinear interpolation) to interpolate the terrain’s height (y) at any given (x-z) coordinates so that entities can move on 64*64 pixels height map based terrain smoothly to reduce the number of triangles. I also implemented for this game an animation system that used obj sequence animations, then optimized it to load faster, and consume 95% less disk space compared to the non-optimized form.

Source code of the game: MummyIsland

My journey to GSoC22

I first knew about GSoC from a friend last year. I then went through the organization’s list. There were a lot of familiar names that I recognized such as Anki, Godot, Blender.

Then I glanced at the Python software foundation. I had to go in and scroll through the sub-organizations only to come across FURY. The luminous horse got my attention. I felt the connection right away.

I went through the docs of both FURY and VTK.

I tried to contribute back then by adding the option to use the FXAA anti-aliasing algorithm for the window (I didn’t notice it has already been implemented as a method for the window).

So instead, I implemented the first mentioned shader in the GSoC 21’s project idea “Add new shader effects” (the fire shader) in GLSL using Gaussian noise.

The next step was to implement the shader to make it work with FURY actors but then I knew I’m having a mandatory training required by the University last summer. I was disappointed, and I decided to postpone until next year’s GSoC.

This year when the accepted organizations were announced, I didn’t bother to go through the list since I knew exactly which organization I’m going to apply for. So I just went directly to read the Ideas list for FURY.

The keyframe animation system was the perfect match! I have been around keyframes for half of my life to even guess the required features before I read them! I then started to contribute and one contribution led to the other one.

Code contributions:

The day I got accepted

I made it my mission for the day to keep watching the email for any changes. When It got 8:00 pm in the Cairo timezone, I didn’t see any changes. So I started searching for any news whatsoever and I couldn’t find any. The GSoC dashboard doesn’t say anything (story of my life. I didn’t even get any promotions or spam mails that day). That’s when I gave up. But, something doesn’t feel right! Right? I went again to check the dashboard to find that my proposal got accepted. I couldn’t even believe it. I would better not jump to conclusions and wait for the official email before celebrating. Shortly after, I received the official email. It was a whole new feeling I had never experienced ever before! I reached a whole new level of happiness.