demvessias's Blog

Weekly Check-In #7

Published: 07/19/2021

Hi everyone! My name is Bruno Messias currently I'm a Ph.D student at USP/Brazil. In this summer I'll develop new tools and features for FURY-GL Specifically, I'll focus into developing a system for collaborative visualization of large network layouts using FURY and VTK.

What did I do this week?

  • PR fury-gl/helios#16 (merged): Helios IPC network layout support for MacOs
  • PR fury-gl/helios#17 (merged): Smooth animations for IPC network layout algorithms

    Before this commit was not possible to record the positions to have a smooth animations with IPCLayout approach. See the animation bellow

    After this PR now it's possible to tell Helios to store the evolution of the network positions using the record_positions parameter. This parameter should be passed on the start method. Notice in the image bellow how this gives to us a better visualization

  • PR fury-gl/helios#13 (merged) Merged the forceatlas2 cugraph layout algorithm

Did I get stuck anywhere?

I did not get stuck this week.

What is coming up next?

Probably, I'll work more on Helios. Specifically I want to improve the memory management system. It seems that some shared memory resources are not been released when using the IPCLayout approach.
View Blog Post

Post #3: Network layout algorithms using IPC

Published: 07/12/2021

Hi all. In the past weeks, I’ve been focusing on developing Helios; the network visualization library for FURY. I improved the visual aspects of the network rendering as well as implemented the most relevant network layout methods.

In this post I will discuss the most challenging task that I faced to implement those new network layout methods and how I solved it.

The problem: network layout algorithm implementations with a blocking behavior

Case 1: Suppose that you need to monitor a hashtag and build a social graph. You want to interact with the graph and at the same time get insights about the structure of the user interactions. To get those insights you can perform a node embedding using any kind of network layout algorithm, such as force-directed or minimum distortion embeddings.

Case 2: Suppose that you are modelling a network dynamic such as an epidemic spreading or a Kuramoto model. In some of those network dynamics a node can change the state and the edges related to the node must be deleted. For example, in an epidemic model a node can represent a person who died due to a disease. Consequently, the layout of the network must be recomputed to give better insights.

In described cases if we want a better (UX) and at the same time a more practical and insightful application of Helios layouts algorithms shouldn’t block any kind of computation in the main thread.

In Helios we already have a lib written in C (with a python wrapper) which performs the force-directed layout algorithm using separated threads avoiding the GIL problem and consequently avoiding the blocking. But and the other open-source network layout libs available on the internet? Unfortunately, most of those libs have not been implemented like Helios force-directed methods and consequently, if we want to update the network layout the python interpreter will block the computation and user interaction in your network visualization. How to solve this problem?

Why is using the python threading is not a good solution?

One solution to remove the blocking behavior of the network layout libs like PyMDE is to use the threading module from python. However, remember the GIL problem: only one thread can execute python code at once. Therefore, this solution will be unfeasible for networks with more than some hundreds of nodes or even less! Ok, then how to solve it well?

IPC using python

As I said in my previous posts I’ve created a streaming system for data visualization for FURY using webrtc. The streaming system is already working and an important piece in this system was implemented using the python SharedMemory from multiprocessing. We can get the same ideas from the streaming system to remove the blocking behavior of the network layout libs.

My solution to have PyMDE and CuGraph-ForceAtlas without blocking was to break the network layout method into two different types of processes: A and B. The list below describes the most important behaviors and responsibilities for each process

Process A:

  • Where the visualization (NetworkDraw) will happen
  • Create the shared memory resources: edges, weights, positions, info..
  • Check if the process B has updated the shared memory resource which stores the positions using the timestamp stored in the info_buffer
  • Update the positions inside of NetworkDraw instance

Process B:

  • Read the network information stored in the shared memory resources: edges , weights, positions
  • Execute the network layout algorithm
  • Update the positions values inside of the shared memory resource
  • Update the timestamp inside of the shared memory resource

I used the timestamp information to avoid unnecessary updates in the FURY/VTK window instance, which can consume a lot of computational resources.

How have I implemented the code for A and B?

Because we need to deal with a lot of different data and share them between different processes I’ve created a set of tools to deal with that, take a look for example in the ShmManagerMultiArrays Object , which makes the memory management less painful.

I'm breaking the layout method into two different processes. Thus I’ve created two abstract objects to deal with any kind of network layout algorithm which must be performed using inter-process-communication (IPC). Those objects are: NetworkLayoutIPCServerCalc ; used by processes of type B and NetworkLayoutIPCRender ; which should be used by processes of type A.

I’ll not bore you with the details of the implementation. But let’s take a look into some important points. As I’ve said saving the timestamp after each step of the network layout algorithm. Take a look into the method _check_and_sync from NetworkLayoutIPCRender here. Notice that the update happens only if the stored timestamp has been changed. Also, look at this line helios/layouts/, the IPC-PyMDE implementation This line writes a value 1 into the second element of the info_buffer. This value is used to inform the process A that everything worked well. I used that info for example in the tests for the network layout method, see the link helios/tests/


Until now Helios has three network layout methods implemented: Force Directed , Minimum Distortion Embeddings and Force Atlas 2. Here docs/examples/viz_helios_mde.ipynb you can get a jupyter notebook that I’ve a created showing how to use MDE with IPC in Helios.

In the animation below we can see the result of the Helios-MDE application into a network with a set of anchored nodes.

Next steps

I’ll probably focus on the Helios network visualization system. Improving the documentation and testing the ForceAtlas2 in a computer with cuda installed. See the list of opened issues

Summary of most important pull-requests:

View Blog Post

Weekly Check-In #5

Published: 07/05/2021

What did you do this week?

fury-gl/fury PR#437: WebRTC streaming system for FURY

  • Before the 8c670c2 commit, for some versions of MacOs the streaming system was falling in a silent bug. I’ve spent a lot of time researching to find a cause for this. Fortunately, I could found the cause and the solution. This troublesome MacOs was falling in a silent bug because the SharedMemory Object was creating a memory resource with at least 4086 bytes indepedent if I've requested less than that. If we look into the MultiDimensionalBuffer Object (stream/ before the 8c670c2 commit we can see that Object has max_size parameter which needs to be updated if the SharedMemory was created with a "wrong" size.

fury-gl/helios PR 1: Network Layout and SuperActors

In the past week I've made a lot of improvements in this PR, from performance improvements to visual effects. Bellow are the list of the tasks related with this PR:

  • - Code refactoring.
  • - Visual improvements: Using the UniformTools from my pull request #424 now is possible to control all the visual characteristics at runtime.
  • - 2D Layout: Meanwhile 3d network representations are very usefully for exploring a dataset is hard to convice a group of network scientists to use a visualization system which dosen't allow 2d representations. Because of that I started to coding the 2d behavior in the network visualization system.
  • - Minimum Distortion Embeddings examples: I've created some examples which shows how integrate pymde (Python Minimum Distortion Embeddings) with fury/helios. The image bellow shows the result of this integration: a "perfect" graph embedding

What is coming up next week?

I'll probably focus on the heliosPR#1. Specifically, writing tests and improving the minimum distortion embedding layout.

Did you get stuck anywhere?

I did not get stuck this week.
View Blog Post

Post #2: SOLID, monkey patching a python issue and network layouts through WebRTC

Published: 06/28/2021

Hi everyone! My name is Bruno Messias and I'm a PhD student working with graphs and networks. This summer I'll develop new tools and features for FURY-GL Specifically, I'll focus on developing a system for collaborative visualization of large network layouts using FURY and VTK.

These past two weeks I’ve spent most of my time in the Streaming System PR and the Network Layout PR In this post I’ll focus on the most relevant things I’ve made for those PRs

Streaming System

Pull request fury-gl/fury/pull/437

Code Refactoring

Abstract class and SOLID

The past weeks I've spent some time refactoring the code to see what I’ve done let’ s take a look into this fury/blob/b1e985.../fury/stream/, the FuryStreamClient Object before the refactoring.

The code is a mess. To see why this code is not good according to SOLID principles let’s just list all the responsibilities of FuryStreamClient

  • Creates a RawArray or SharedMemory to store the n-buffers
  • Creates a RawArray or SharedMemory to store the information about each buffer
  • Cleanup the shared memory resources if the SharedMemory was used
  • Write the vtk buffer into the shared memory resource
  • Creates the vtk callbacks to update the vtk-buffer

That’s a lot and those responsibilities are not even related to each other. How can we be more SOLID[1]? An obvious solution is to create a specific object to deal with the shared memory resources. But it's not good enough because we still have a poor generalization since this new object still needs to deal with different memory management systems: rawarray or shared memory (maybe sockets in the future). Fortunately, we can use the python Abstract Classes[2] to organize the code.

To use the ABC from python I first listed all the behaviors that should be mandatory in the new abstract class. If we are using SharedMemory or RawArrays we need first to create the memory resource in a proper way. Therefore, the GenericImageBufferManager must have a abstract method create_mem_resource. Now take a look into the ImageBufferManager inside of stream/server/, sometimes it is necessary to load the memory resource in a proper way. Because of that, the GenericImageBufferManager needs to have a load_mem_resource abstract method. Finally, each type of ImageBufferManager should have a different cleanup method. The code below presents the sketch of the abstract class

from abc import ABC, abstractmethod

    def __init__(
            self, max_window_size=None, num_buffers=2, use_shared_mem=False):
    def load_mem_resource(self):
    def create_mem_resource(self):
    def cleanup(self):

Now we can look for those behaviors inside of and that does not depend if we are using the SharedMemory or RawArrays. These behaviors should be methods inside of the new GenericImageBufferManager.

# code at:

class GenericImageBufferManager(ABC):
    def __init__(
            self, max_window_size=None, num_buffers=2, use_shared_mem=False)
        self.max_window_size = max_window_size
        self.num_buffers = num_buffers
        self.info_buffer_size = num_buffers*2 + 2
        self._use_shared_mem = use_shared_mem
         # omitted code
    def next_buffer_index(self):
        index = int((self.info_buffer_repr[1]+1) % self.num_buffers)
        return index
    def buffer_index(self):
        index = int(self.info_buffer_repr[1])
        return index
    def write_into(self, w, h, np_arr):
        buffer_size = buffer_size = int(h*w)
        next_buffer_index = self.next_buffer_index
         # omitted code

    def get_current_frame(self):
        if not self._use_shared_mem:
        # omitted code
        return self.width, self.height, self.image_buffer_repr

    def get_jpeg(self):
        width, height, image = self.get_current_frame()
        if self._use_shared_mem:
        # omitted code
        return image_encoded.tobytes()

    async def async_get_jpeg(self, ms=33):
       # omitted code
    def load_mem_resource(self):

    def create_mem_resource(self):

    def cleanup(self):

With the GenericImageBufferManager the RawArrayImageBufferManager and SharedMemImageBufferManager is now implemented with less duplication of code (DRY principle). This makes the code more readable and easier to find bugs. In addition, later we can implement other memory management systems in the streaming system without modifying the behavior of FuryStreamClient or the code inside of

I’ve also applied the same SOLID principles to improve the CircularQueue object. Although the CircularQueue and FuryStreamInteraction was not violating the S from SOLID the head-tail buffer from the CircularQueue must have a way to lock the write/read if the memory resource is busy. Meanwhile the multiprocessing.Arrays already has a context which allows lock (.get_lock()) SharedMemory dosen’t[2]. The use of abstract class allowed me to deal with those peculiarities. commit 358402e

Using namedtuples to grant immutability and to avoid silent bugs

The circular queue and the user interaction are implemented in the streaming system using numbers to identify the type of event (mouse click, mouse weel, ...) and where to store the specific values associated with the event , for example if the ctrl key is pressed or not. Therefore, those numbers appear in different files and locations: tests/, stream/, steam/server/ This can be problematic because a typo can create a silent bug. One possibility to mitigate this is to use a python dictionary to store the constant values, for example

	“ mouse_move” : 2, “mouse_weel”: 1, ….
But this solution has another issue, anywhere in the code we can change the values of EVENT_IDS and this will produce a new silent bug. To avoid this I chose to use namedtuples to create an immutable object which holds all the constant values associated with the user interactions. stream/

The namedtuple has several advantages when compared to dictionaries for this specific situation. In addition, it has a better performance. A good tutorial about namedtuples it’s available here


My mentors asked me to write tests for this PR. Therefore, this past week I’ve implemented the most important tests for the streaming system: /fury/tests/

Most relevant bugs

As I discussed in my third week check-in there is an open issue related to SharedMemory in python. This"bug" happens in the streaming system through the following scenario

1-Process A creates a shared memory X
2-Process A creates a subprocess B using popen (shell=False)
3-Process B reads X
4-Process B closes X
5-Process A kills B
4-Process A closes  X
5-Process A unlink() the shared memory resource 
In python, this scenario translates to

from multiprocessing import shared_memory as sh
import time
import subprocess
import sys

shm_a = sh.SharedMemory(create=True, size=10000)
command_string = f"from multiprocessing import shared_memory as sh;import time;shm_b = sh.SharedMemory('{}');shm_b.close();"
p = subprocess.Popen(
    [sys.executable, '-c', command_string],
    stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False)

Fortunately, I could use a monkey-patching[3] solution to fix that; meanwhile we're waiting for the python-core team to fix the resource_tracker (38119) issue [4].

Network Layout (Helios-FURY)

Pull request fury-gl/helios/pull/1

Finally, the first version of FURY network layout is working as can you see in the video below

In addition, this already can be used with the streaming system allowing user interactions across the internet with WebRTC protocol.

One of the issues that I had to solve to achieve the result presented in the video above was to find a way to update the positions of the vtk objects without blocking the main thread and at the same time allowing the vtk events calls. My solution was to define an interval timer using the python threading module: /fury/stream/, /fury/stream/ /fury/stream/


  • [1] A. Souly,"5 Principles to write SOLID Code (examples in Python)," Medium, Apr. 26, 2021. (accessed Jun. 28, 2021).
  • [2]"[Python-ideas] Re: How to prevent shared memory from being corrupted ?" (accessed Jun. 28, 2021).
  • [3]“Message 388287 - Python tracker." (accessed Jun. 28, 2021).
  • [4]“bpo-38119: Fix shmem resource tracking by vinay0410 · Pull Request #21516 · python/cpython," GitHub. (accessed Jun. 28, 2021).
View Blog Post

Weekly Check-In #3

Published: 06/21/2021

Hi everyone! My name is Bruno Messias. In this summer I'll develop new tools and features for FURY-GL Specifically, I'll focus into developing a system for collaborative visualization of large network layouts using FURY and VTK.

What did you do this week?

  • PR fury-gl/fury#422 (merged): Integrated the 3d impostor spheres with the marker actor
  • PR fury-gl/fury#422 (merged): Fixed some issues with my maker PR which now it's merged on fury
  • PR fury-gl/fury#432 I've made some improvements in my PR which can be used to fine tuning the opengl state on VTK
  • PR fury-gl/fury#437 I've made several improvements in my streamer proposal for FURY. most of those improvements it's related with memory management. Using the SharedMemory from python 3.8 now it's possible to use the streamer direct on a jupyter without blocking
  • PR fury-gl/helios#1 First version of async network layout using force-directed.

Did I get stuck anywhere?

A python-core issue

I've spent some hours trying to discover this issue. But now it's solved through the commit devmessias/fury/commit/071dab85

The SharedMemory from python>=3.8 offers new a way to share memory resources between unrelated process. One of the advantages of using the SharedMemory instead of the RawArray from multiprocessing it’s that the SharedMemory allows to share memory blocks without those processes be related with a fork or spawm method. The SharedMemory behavior allowed to achieve our jupyter integration and simplifies the use of the streaming system. However, I saw a issue in the shared memory implementation.

Let’s see the following scenario:

1-Process A creates a shared memory X
2-Process A creates a subprocess B using popen (shell=False)
3-Process B reads X
4-Process B closes X
5-Process A kills B
4-Process A closes  X
5-Process A unlink() the shared memory resource X
This scenario should work well. unlink() X in it's the right way as discussed in the python official documentation. However, there is a open issue which a think it's related with the above scenario. Fortunately, I could use a monkey-patching solution to fix that meanwhile we wait to the python-core team to fix the resource_tracker (38119) issue.

What is coming up next?

I'm planning to work in the fury-gl/fury#432 and fury-gl/helios#1.
View Blog Post