Geometry Shaders – An Introduction

It is well accepted that Shaders have played a very important role in fast and effective visualization effects. After Vertex Shaders and Fragment Shaders, we have a very important Shader – Geometry Shader.

Fragment Shaders and Vertex Shaders are the building blocks of any rendering scene. But it would be very cumbersome and difficult to construct a complicated shape using ONLY vertex shaders and fragment shaders. This is because it is very difficult to code details of each and every vertex. Hence, we have with us the Geometry Shader. A Geometry Shader takes the whole primitive as input and produces a geometry which can then be used as a block rather than handling each and every vertex separately.

A geometry shader can be thought of as vertex amplifier, that is, it helps in rendering more vertices than what it actually would take to do it using Vertex and Fragment Shaders.  Thus as given in the example here, only a single instance of Points would render different shapes. This is helpful in complicated rendering scenes, like rain, fireworks etc.

But all of this comes with some limitations. A geometry shader is slow compared to normal rendering and this is not that big a surprise. Also, as described here,  there is a limit to the number of primitives I can use per input to Geometry Shader. The limit given here is 1024. This may vary over systems but the message is that we can not use it for too big data.

Is this useful for us?

dipy requires rendering complicated data with a large number of vertices. I have not fully explored the possibilities in Geometry Shader but I am sure this would be very helpful to us. GS would make it easy to code the vertices of the data. This does compromise the speed but that must be marginal. More would be known after working with actual datasets.

My approach now

Currently, I am trying to replicate the example given here using python-vtk. This is a little bit tricky as I have never read examples of GS in python-vtk. I guess this would be completed after having a discussion with mentors and reading relevant documentation.

Shaders – Resources and Examples

Shaders are probably the best tool for start-of-art visualizations. Having shaders in our dipy would be a cool addition.  Apart from the implementation, I also spend time getting to know the shaders better. Fragment shaders and Vertex Shaders are not very trivial and it takes a bit of time to understand and digest completely. Below are some points about Shaders and some easy examples of how they work.

Vertex Shaders

As the name suggests, Vertex Shaders contains attributes about a particular vertex in a shape. All the information about a vertex is contained in it including the color and texture. As mentioned here, there is always a 1:1 correspondence between the input vertex and the output vertex. Just the shader does a particular transform on the attributes.

Fragment Shaders

Fragment Shaders gives the color and the depth value of an input pixel. The fragment shader is something which is independent of the movement of the object. It is very important to note that each Fragment Shader has its own “window space” which acts as the output of the stream.

It requires examples to properly understand the difference between the two and how both of them when combined, can render interesting shapes.

Now, look at the sample code given below:

#ifdef GL_ES

precision medium float;

#endif

void main() {

fl_FragColor = vec4(1.0, 0.0, 1.0, 1.0);

}

Clearly, this deals with the fragment shader. Why so? It is because of the fact that we just want to see the color of the output and no vertex is involved here. Hence, Fragment Shaders makes sense here. The color of the screen is set to Red and Blue, which yields Pink.

There are a lot of interesting examples which uses the combination of two to get eye-catching examples and visualizations.

Inbuilt Variables in Shaders

It is very very important to know what all predefined variables comes along with shader so that you don’t have to keep looking for where it was first used. So, here is a resource which gives major focus on the predefined variables and what it does. It also has important data types required for visualization tasks.

*NOTE* This blog will be updated regularly as I read more about Shaders and try them out.

More into my experiments on vtk

Hey!!
It’s time for another update of the work.  After having spent a lot of time exploring shaders, I came up with beautiful illustrations in my last blog. This time around, I will discuss the brief extension of the work done and will also talk about my recent Pull Request being merged.

vtk Ellipsoid with adjustable semi-major axis length

So, last week I introduced a sphere which can change colors based on the color input by the user. The next step is to move this further and make a sphere which can dynamically adjust the parameters while rendering continuously. The following example uses a slider to set the value of the semi-major axis. In our case, we are only changing the x coordinate to adjust the ellipsoid.

Initially, I had issues with dynamically rendering the screen. On discussion with Ranveer, we had in our mind two possible solutions:-

  • Destroy the renderer and render afresh every time. It is clear that this is not an elegant solution to the problem.
  • Update the renderer every time. This was an acceptable solution and quite logical.

This was discussed with Elef during our periodic conversation and he helped me sort out the problem of assigning the updated value back to the vtk ellipsoid.

The code is available here.

My Pull Request to Add Sphere object to the sphere class is merged.

My work on adding Sphere objects as a parameter to the sphere function got merged. The pull request can be seen here. I added faces and vertices parameter to the sphere. Now instead of providing the function every time with centers and radii, we can simply provide the faces and vertices which we get from the Sphere function of dipy.

Also, I am working on resolving my doubts with the function MapDataArrayToVertexAttribute. After today’s meet, it is clear and I will soon come up with the corrected version of my code.

The has been a good journey so far. I hope to say this till the end of my summers. Shaders are quite interesting and exciting. Looking forward to more fun.

thechargedneutron

Interesting world of Shaders and vtk

Hey folks,
The last two weeks were full of fun. Sitting near my screen exploring various shaders was fun. With the help of Elef and Ranveer, I could do a lot of different examples and all this helped me build some really cool visualization examples (At least I was amazed 😛 ).
So let me walk you through few things I achieved over the past two weeks.

  • Displaying Frame-Rate of vtk Renderer
    The example renders ten small unit radii spheres and displays the number of times the screen is rendered per second.  It is calculated by finding the inverse of the last rendering time (in seconds of course). And we obtain last rendering time using a callback function GetLastRenderTimeInSeconds. As the name suggests, this function returns the time delay between last rendering time and the present rendering time.
    A small demo of the work is available here:

    The documented code is available here.

  • Taking user inputs to Shaders using SetUniformf
    This task was aimed at understanding how to use user input to set shader properties. I came up with a code which takes as an input our choice for RGB values and returns a sphere of that color.
    SetUniformf is a very important tool which magically acts as an interface between the OpenGL world of shaders and our Python world. As we move on to more complicated tasks, this function will be increasingly useful for sure.
    A small demo of the code is available here:

    The documented code can be found here.

  • Constructing Sphere using fundamental blocks – Vertices and Faces
    It was very interesting to know how any 3D shape is rendered using triangles as the fundamental block. I tried to visualize the same in a more general way. The rendered shape can be changed to ellipsoid very easily. I am currently working on including a slider which changes the dimension of the sphere as per the slider input. The sphere looks deformed (intentionally) to emphasize the use of triangles to form the sphere. I have used as less as 24 triangles to get a sphere.
    The sphere looks like this:

    The documented code can be found here.

  • Extending the to-be-added sphere function
    Elef is working on a Pull Request to add sphere function. The sphere would help to produce 3D Glyphs. The purpose of my work is to allow for additional parameters -vertices and faces, which would act as an alternative to the traditional center and radii thing. The Pull Request is under review and can be seen here.

It is really fun working with my mentors and the awesome community at dipy. I strive to learn new things and will try more such interesting stuff as we proceed. I’m loving GSoC 🙂 . Looking forward to more fun this summer.

thechargedneutron

End of Community Bonding Period

Hey folks!!

We have come to the end of the first stage of GSoC project, the Community Bonding Period. Official coding begins NOW!!

I spent my Community Bonding Period very well. I got a very good exposure to the project and this helped me a lot in understanding the work and starting off correctly. I got introduced to vtk, GLSL language and obviously the vtk library of Python and how to communicate between them.

I had a Video Chat with Elef, Ranveer, David, and Gabriel on 8th May. The talk was very smooth and I got myself familiar with their expectations of this project. They are very much interested in improving the visualization module of dipy which I also think is the need of the hour. We discussed the importance of Shaders and its relevance in this field of study.

As a part of the Community Bonding Period, I worked on the following tasks to gain more insight into the project.

1. Experimenting with the vtk-shaders code. This was the initial code developed and maintained by David. I went ahead and made several modifications to the given code to obtain interesting shapes. I understood the structure of the code and different aspects of using a Python wrapper around python vtk.

2. Understood the basic functioning of Streamlines. The strategies used to perform various operations on the graph input.

3. Spherical Harmonics are used for the visualization of spheres. I tried out experimenting with that and now I would be spending some more time trying to figure out the Spherical Harmonics and the use it has to produce visualizations of the datasets.

With an important part of the project ending soon, I am very much excited to get into the Coding Period of the program and look forward to having great fun ahead. In the next few days, I’ll be again having a Video chat with Elef and my mentors regarding future works that need to be done.

Looking forward to more exciting times.

thechargedneutron

 

Random Git tricks

Here are few times when I looked up to help on the internet for help regarding git.

  • `git pull upstream master` caused many merge conflicts which I did not wanted to handle at that time. To undo the effect use `git reset –merge`

More things will be added as I proceed.

thechargedneutron

Selected for GSoC with dipy, Python Software Foundation

Phew!!! Selected for GSoC-2018 🙂 .

This was definitely not the time I wanted the result to come. After having an exam till 5 and another due tomorrow afternoon, this result business kept me out of focus. Sitting with my headphones on, refreshing PSF GSoC page, and drinking water regularly, this is what I was at from 9.00pm IST. Now, it’s all sorted, the stage is set, the time has started to execute what has been thoroughly planned for this summers.

With my End Semesters examination still on, I am briefly getting familiar with the work. I have started using playing around with shaders. As we enter the Community Bonding Period, I have the following goals in coming days

  1. To start writing blog, which I guess is on track after having a good start like this 😉
  2. Participating more on the Gitter Channels.
  3. Continue with the small contributions that I have been doing.
  4. Look for more visualization related issues (#1492 and #1355).
  5. Participate in Brainhack. Although I cannot physically attend the event, but I’ll be keeping track of the talks, to get introduced this field.

While there are a few issues I am working on and hope to complete that soon, I would join the discussions on issues related to visualization soon.

It’s a good start to this interesting endeavour and would love to keep the spirits high. <3

Will be back with more interesting stuff.

thechargedneutron

(NOTE: This post was written on 23rd April 2018 and published after I got rights of this blog.)

Checking my brand NEW Blog

Hey there!!

I got this cool blog from PSF. This is highly customizable and I would like to spend more time playing around with this cool layout.

Stay tuned to hear more from me!!!!

thechargedneutron