Week (3) - What we've made so far?

Hi everyone,
Today I will share with you some in-depth details about what we've done in the last couple of weeks.

The road to the new version

We've migrated and improved all the old APIs, previously the application was using a singly entry point, where all server configurations, routes and controllers were in the same file, since we're going to add a lot of new features. We've decided that we need to organize it a little bit, decided that we will use a folder per component structure, where all the files related to one component (say, Users) will be located in the same folder, the controller, model, interfaces, router, and tests, This would help us in easily locating any file if any changes needed to be made, and also would help us in scaling if we wanted to move any component to elsewhere.

Then, we've worked on making the improvements, many of them were focusing on making the application behaves faster in a scale, and this was done by converting the synchronous calls that wouldn't make difference if it called asynchronously, and also using async and await for some calls instead of sync, You might say that this doesn't change anything, in-fact, calling functions asynchronously and then awaiting on them will help in not blocking the entire event loop while these functions are working, but instead it will only block the context at which this function is called in, this would lead to a huge difference that can be noticed with concurrent requests specially because we have many I/O calls.

What about Continuous integration

This was one of the most important things I've learned in the previous weeks, it was my first time trying to implement a small continuous integration process. We want to apply all the tests on each new PR or a commit to to our main branch, this was very needed and important to be made, to make sure that with each pull request, All the APIs are working as expected, the process might seems so easy to go! but it wasn't, I've faced some struggles to make it done and that's because we're using postman tests for Version 1 APIs, postman tests can be ran using newman command line, the problem was that we need to start the server, run the tests using newman and then, terminating the server. Github actions need the previous step to be terminated or finished before starting a new step or command, and we needed to start the server (step 1), keep the server running and then test the APIs (step 2) without terminating the step 1!, how would we even start the step 2 and step 1 is not terminated yet! Github will stay waiting for step 1 forever. That's being said I needed to figure another way, I've created a script to run the testing script from the server itself, this way we will only make the step 1 from Github, and then making the server dealing with newman command lines, and then terminating the server after the execution of the tests, with an appropriate status code to let the Github knows if the tests are passed or not. Worked like a charm!!

What I will be working on in the next 10 days?

The next 10 days, including this weekend and the next week, I will be working on establishing the process of updating the database icons, we've decided that we will use a MongoDB as a source of truth, that's being said, we need to handle the case if any icons are added to the main repository on Gitlab, This couldn't be done without using Gitlab actions, kinda interesting? we my discover it later in the future blog posts :'p Also, by end of this 10 days, All the new APIs will be created, an improved APIs that will do the same functionality but in a better manner, no new featured will be added, but probably we will decrease the number of APIs, and have a unified responses. Anyway, new features are in the plan, stay tuned!


That's for now!
Thank you for reading.