Summer Rewind

Let’s rewind to the beginning of this year. We had started working on this application way before GSoC had even started. The goal was to have a working application which PSF will be able to use for this year’s GSoC for the management of their students. In this way, we will be able to make sure that students actually use it and we get a clear idea if the application is serving its purpose. I’m glad the plan worked out, because tons of bugs were reported and we could fix them. We also received valuable feedback from all users.

Schedulers and Builders

Allow me to introduce you to some of the most important modules of our system. Without any doubt the first on the list is our Scheduler, which can perform particular tasks from sending an email to archiving webpages. Well, the most powerful feature of this module is that it can perform those tasks at any particular date and time. Need to remind students that they have not written a blog on time? Not a big deal, Scheduler can do that for you. Now think of this, many students who are not like me publish their blogs on time, so we don’t really need to spam them with emails. Thus we built the module Builder which in turn builds Scheduler on different conditions.

Blogging Platform

We didn’t have to create a blogging platform as we integrated aldryn-newsblog but we had to tweak it a lot to fit into our system. Something that we had to work on was setting up custom permissions for each user so that they only have access to their blogs. We achieved this with the help of django admin which allows us to set add, view, change permissions based on querysets! Sanitizing the artilce contents was another challenge that we faced because aldryn-newsblog uses an editor which injects HTML so that users can customize their blog posts. Our system currently allows only particular tags like

,

, etc. Other tags are sanitized conditionally, like for iframes we only render iframes for YouTube videos so that users can add YouTube videos to their articles.

We also tweaked the article list templates to include our own reddit-styled comment system which makes use of recursion on django templates to display different threads. For more information on how to achieve this, check out the article Creating Reddit Styled Comment System with Django .

Tweaking Django

We used django-forms wherever we could, but there were cases where we needed to customize the forms to an extent which wasn’t supported out of the box. We have a form which lets suborg admins and admins add selected students to the system. Generally, this includes adding a lot of students (~50). Typing out the emails one by one is still okay, but selecting the GSoC year or the Suborg one by one for each student? Ask my mentor, and he will let you know what a pain it is. So, we tweaked the django form to add buttons which would let the admin select a particular Suborg, year for all the mentioned users.

We have RSS feeds for each blog separately (all the articles published by a student) and also for all the articles published on this platform. Django has Syndication Feed Framework which allows customizable RSS feeds, but the all articles feed was too long and took seconds to load. We needed to paginate the feed, and we were out of luck as django didn’t support this out of the box. This was a challenge, as we had to take in the request object and parse the url to get the page number and render the blogs accordingly. We also added the year argument which takes in the year and displays the blogs of that particular GSoC. The current feed url looks something like this https://blogs.python-gsoc.org/en/feed/?y=2019&p=2.

For most of the other admin features, we heavily relied upon django admin. The admin portal lets admins

• Add new Schedulers and Builders
• Check blog post histories of different articles to track changes
• Add the current GSoC timeline which gets pushed to the schedule page in the github site repository
• Send custom emails to users as admins
• Review comments and delete them if necessary
• Disable a user profile if the student fails mid-way

Integration with Github

We annotated some of the manual work that an admin has to put in to maintain the static site on Github. Our system creates pull requests adding new Suborgs in the Ideas page whenever a new Suborg Application is added, it also archives current pages when the GSoC program ends. These pulls can then be reviewed and merged to master by the admin.

Fixing Bugs

This was really a major part of the whole summer and it went hand in hand with the whole developmene process. There were bugs that were found by the users, and the others figured out by the mentors and me. There are a ton of “Bugfix” PRs which were basically bug fixes.

There was a time when we pushed some changes and it made the system send emails to all the users regardless of whether they have blogged or not. Yes, basically we spammed a lot of users unintentionally. This was another challenge that we had faced and overcame eventually by making a flag which would disable all notifications to any user. We also followed a strict push cycle to avoid any disturbance to students blogging at the end of the week.

Wrapping Up

We ran accessibility tests on our websites and fixed issues which decrease the accessibility of the website, like fixing contrast ratios of texts and background, adding alts to images, etc. We also worked on boosting the loading speed of out website on mobiles. In built tools provided by Chrome and Firefox gave us a list of issues after analyzing the website which we could work on.

We also ended up using a cache server to cache the data to fasten up the whole loading process. We also needed to manually override caching in some pages like the comments page, which would not show the new comment as the old one was cached. This is the issue which describes more about this bug and how we solved it.

Future Plans

Currently, the platform provides most of the functionality required for a smooth GSoC run at PSF, but there are features that would make it even smoother for the admins and make their lives a bit less painful. One of them is adding the mentors to the GSoC site automatically from the system’s database. This can be another nasty manual work (typing in the emails, names, etc. one by one) and needs to be done automatically. For more details, check this issue out.

We also need to write unit and integration tests for features that are not provided by django or any third-party packages.

I would love to work on these in future even when this GSoC ends, fix more bugs as and when they come up and be a part of this great community!

Credits

First of all, any of this would not have been possible without my mentors and other members of the PSF community. So a huge shout-out to them for helping me whenever I needed and for guiding me when I was clueless on how to proceed. While I was busy coding, my mentor would look for potential bugs in the system and point them out to me. This really kept me busy throughout the summer as I always had bugs to fix, and helped me make the system more stable.

Next, I would thank Google for organising such an amazing program for students who are passionate about coding and giving them an opportunity to gain some hands-on experience.

Last but not the least, I would thank my fellow applicants who also worked on building the application with me to bring it to a stage where it could be used in this year’s GSoC run.

Apologies

There are a lot of mistakes that I made and learnt from them. In the beginning I was not testing stuff through before making a PR, sometimes trying to do things faster and the other times just being lazy. It only made me spend more time on a particular feature as there were things that would not work.

Another thing that I should apologize for is being very irregular about posting blogs. This shouldn’t have come from me, as I was the student working on the very blogging platform itself.

What did I do in these two weeks?

There were a lot of bugs posted, kind of like the last set of major fixes that we need to make.

1. be able to add gsoc_year
2. add username to password reset email
3. prepopulate have you partisipated before
4. 'Settings' object has no attribute 'ADMIN_EMAIL' on suborg form since we removed that key
5. github username for org admins/mentors
6. add additional org admins as part of suborg form
7. cut out top part of suborg form
8. make a pr on suborg submit, dont auto commit
9. cookie notice
10. add mentors/suborg admins button
11. add psf privacy policy
12. checkbox on signing up for PSF terms and another for "opt-in to receive emails"
13. nuke all student users unless they click a link in an email, ie remove email name etc then readd with get request if they click at end of gsoc (gdpr)

In this week we mostly worked on all of these. Terri posted some issues with the comment notification emails that we had totally overlooked. I fixed those too. We also worked on fixing the existing feed and adding blog wise feeds for every blog.

What's next?

Publishing the final report and making some final changes to improve the accessibility of the site, like fixing the contrast ratios, etc.

Django has a great framework for generating feeds, but sadly it doesn't support pagination out of the box. Currently we have tons of blogs and the feeds page was loading especially slow with all the pages in it. I was hoping django will have some class variable to enable pagination but I was out of luck. But it had a method which took in the request, so I knew I had to parse the page number from there.

from django.contrib.syndication.views import Feed

class BlogsFeed(Feed):
    ...
    def get_objects(self, request):
        ...
        page = request.GET.get("p", 1)
        ...
        return queryset_from_page

So in the get_objects method we need to get the page number from the GET args and then return the queryset according to the page number.

That seems to be enought right? Even I thought so. But the RSS standards say that we also need to add entries like url for last page number, first page number and the current page number. Well, we can even do that. The class BlogsFeed takes in a class variable called feed_type and we need to set it to the feed type class and django provides a class for that too called DefaultFeed. Well we will inherit this DefaultFeed and make out own type of Feed which will include the page numbers too.

from django.utils.feedgenerator import DefaultFeed

class PaginateFeed(DefaultFeed):
    content_type = "application/xml; charset=utf-8"

    def add_root_elements(self, handler):
        super(CorrectMimeTypeFeed, self).add_root_elements(handler)
        if self.feed["page"] is not None:
            if not self.feed["show_all_articles"]:
                if (
                    self.feed["page"] >= 1
                    and self.feed["page"] <= self.feed["last_page"]
                ):
                    handler.addQuickElement(
                        "link",
                        "",
                        {
                            "rel": "first",
                            "href": f"{self.feed['feed_url']}?y={self.feed['year']}&p=1",
                        },
                    )
                    handler.addQuickElement(
                        "link",
                        "",
                        {
                            "rel": "last",
                            "href": (
                                f"{self.feed['feed_url']}?y={self.feed['year']}"
                                f"&p={self.feed['last_page']}"
                            ),
                        },
                    )
                    if self.feed["page"] > 1:
                        handler.addQuickElement(
                            "link",
                            "",
                            {
                                "rel": "previous",
                                "href": (
                                    f"{self.feed['feed_url']}?y={self.feed['year']}"
                                    f"&p={self.feed['page'] - 1}"
                                ),
                            },
                        )
                    if self.feed["page"] < self.feed["last_page"]:
                        handler.addQuickElement(
                            "link",
                            "",
                            {
                                "rel": "next",
                                "href": (
                                    f"{self.feed['feed_url']}?y={self.feed['year']}"
                                    f"&p={self.feed['page'] + 1}"
                                ),
                            },
                        )

This code pretty much explains itself. But there is one catch here too self.feed dict does not have a 'page' or a 'year' key. We need to pass that from our BlogsFeed class. Let's see how.

class BlogsFeed(Feed):
    ...
    feed_type = CorrectMimeTypeFeed
    ...
    def feed_extra_kwargs(self, obj):
        return {
            "page": self.page,
            "last_page": self.last_page,
            "show_all_articles": self.show_all_articles,
            "year": self.year,
        }

That's it guys. Now you have your own paginated feed as per the RSS standards.

What did I do this week?

We have come to the last phase of the summer. The application is almost ready but we needed to fix more bugs this time to fine tune the application. Some of those which we worked on this week were as follows.

• Fixed the sitemap to include each blog post and also different pages for the paginated blog
• Handling exceptions on sitemap
• Use django for sending mails to admins when any exception is raised on the main server
• Fixed a very very old issue by enhancing the user add feature. Now the admin can use single dropdowns to select the default values for multiple fields.

What will I do next week?

Will move towards finishing up whatever is left and work on more issues as and when they come up!

That will be all folks!

Have you ever tried to embed a Google Calendar in your site only to find it's not responsive for mobiles and you need to scroll left and right? Well, I'm gonna share a simple solution which will work.

There's an agenda mode in google calendar which seems to fit properly in a mobile screen, so how about changing it to the agenda mode when the screen size is smaller? Simple HTML and CSS will be enough for this.

<style type="text/css">@media (max-width: 550px) {
     .big-container {
         display: none;
     }
 }
 @media (min-width: 550px) {
     .small-container {
         display: none;
     }
 }
 /* Responsive iFrame */
 .responsive-iframe-container {
     position: relative;
     padding-bottom: 56.25%;
     padding-top: 30px;
     height: 0;
     overflow: hidden;
 }
 .responsive-iframe-container iframe,   
 .vresponsive-iframe-container object,  
 .vresponsive-iframe-container embed {
     position: absolute;
     top: 0;
     left: 0;
     width: 100%;
     height: 100%;
 }</style>

<iframe height="600" src="null" style="border-width: 0;" width="800"></iframe>
<iframe height="600" src="null" style="border-width: 0;" width="800"></iframe>

This is be good enough for a responsive calendar. Want another pro tip? How about changing timezones of the calendar according to the client timezone?

Well we will need a bit of js for getting the client's timezone.

var offset = new Date().getTimezoneOffset();

This line of code gets the offset integer in minutes, so -330 for +05:30 timezone and so on. Then we somehow need to convert this to the region name as google calendar link takes an argument named ctz for that. Include the moment.min.js and the moment-timezone-with-data-10-year-range.min.js. These libraries will help us get the region name.

String.prototype.replaceAll = function(search, replacement) {
    var target = this;
    return target.split(search).join(replacement);
};
var timezone = moment.tz.guess(offset).replaceAll('/', '%2F')

Now you just need to add a new argument to the url.

newUrl = `${oldUrl}?ctz=${timezone}`
document.getElementById('calendar').src = newUrl;

That's it guys and you will be good to go.

Reference: https://answers.squarespace.com/questions/54774/how-to-embed-a-google-calendar-in-a-responsive-way.html