PyCon India 2016

PyCon India 2016

Day 0

“This is awesome!”, this was my first reaction when I boarded my first flight to Delhi. I was having trouble in finding a proper accommodation Kushal, Sayan and Chandan helped me a lot in that part, I finally got honour of  bunking with Sayan , Subho and Rtnpro which I will never forget. So, I landed and directly went to JNU convention center. I met the whole  Red Hat intern gang . It was fun to meet them all. I had proposed Pagure for Dev Sprint and I pulled in Vivek to do the same.

The dev sprint started and there was no sign of Vivek or Saptak, Saptak is FOSSASIA contributor and Vivek  contributes to Pagure with me. Finally it was my turn to talk about Pagure on stage , it was beautiful  the experience and the energy.  We got a lot of young and new contributors and we tried to guide them and make them send at least one PR.  One of them was lucky enough to actually make a PR and it got readily merged.

I met a lot of other contributors and other mentors and each and every project was simply amazing. I wish I could help all of them some day. We also met Paul, who writes code for PyCharm, we had a nice discussion over Vim v/s PyCharm.

Finally the day ended with us Vivek, Sayan , Subho  , Saptak and me going out to grab some dinner. I bunked with Sayan and Subho and we hacked all night. I was configuring my Weechat and was trying all the plugins available and trust me there are a lot of them.

Day 1

I was a session chair in one of the lecture room and it was a crazy experience from learning to write a firmware for a drone, using generators to write multi-threaded program and also to work with salt stack. The food was really good but the line for food was equally “pythonic” as the code should be.

There were a lot of stalls put up and I went to all of them and had a chat with them. My favorite one was PyCharm because Paul promised me to teach me some neat tricks to use PyCharm.

The Redhat and Pyladies booth were also there which also were very informative and they were responsible making people aware about certain social issues and getting women in tech.

We had two keynotes on this day one by BG and the other by VanL and trust me both of the keynotes were so amazing the they make you look technology from a different view point altogether.

One of the amazing part of such conferences are Open Space and Lightning talks. There are few open spaces which I attended and I found them really enthralling. I was waiting for the famous Stair case meeting of Dgplug.  We met Kushal’s mentor, Sartaj and he gave a deep insight in what and why we should contribute to open source. He basically told us that even if one’s code is not used by anyone he will still be writing code for the love of doing it.

After this we went for Dgplug/Volunteers  dinner at BBQ nation, it was an eventful evening😉 to be modest.

Day 2 

The last day of conference I remember myself wondering how a programming language translates into philosophy and how that philosophy unites a diverse nation like India. The feeling was amazing but I could sense the sadness. The sadness of parting from friends who meet once in an year. I could actually now relate all IRC nicks with their faces. It just brings a lot more on the table.

At last we all went to the humdrum of our normal life with the promise to meet again. But I still wonder how a technology bring comradeship between people from all nook and corners of life. How it relates from a school teacher to a product engineer . T his makes  me feel that this is more than just a programming language , this is that unique medium that unites people and give them power to make things right.

With this thought fhackdroid signs out!

Happy Hacking!



Recently I have been to pycon-india (will blog about that too!) there Sayan and Vivek introduced me to weechat which is a terminal based IRC client, from the time I saw Sayan’s weechat configuration I was hooked to it.

The same night I started configuring my weechat , it’s such a beautiful IRC client I was regretting why did I not use it before. It just transforms your terminal into IRC window.

For fedora you need to do:

sudo dnf install weechat

Some of the configuration and plugins you need are :

  1. buffer
  2. notify-send

That’s pretty much it but that doesn’t stop there you can make that client little more aesthetic.  You can set weechat by using their documentation.

The clean design kind of makes you feel happy, plus adding plugin is not at all a pain. In the weechat window you just say /script install and it just installs it in no time.  There are various external plugin in case you want to use them and writing plugin is actually fun , I have not tried that yet.


I also use to use bigger font but now I find this size more soothing to eyes. It is because of weechat I got to know or explore about this beautiful tool called tmux ,  because on normal terminal screen weechat lags , what I mean by lag is the keystroke somehow reach after like 5-6 seconds which makes the user experience go bad.  I pinged people on IRC in #weechat channel with the query the community is amazing they helped me to set it up and use it efficiently , they only told me to use tmux or screen . With tmux my session are persistent and without any lag.

To install tmux on fedora:

sudo install tmux

tmux is a terminal multiplexer which means it can extend one terminal screen into many screen . I got to learn a lot of concepts in tmux like session, pane and windows. Once you know these things in tmux its really a funride. Some of the blogs I went through for configuring and using tmux the best I found was hamvoke , the whole series is pretty amazing . So basically my workflow goes for every project I am working on I have a tmux session named after it, which is done by the command:

tmux new-session -s <name_session>

Switching between two session can be done by attach and detach. And I have one constant session running of weechat. I thought I have explored every thing in tmux but that can’t be it , I came to know that there is a powerline for tmux too. That makes it way more amazing so this is how a typical tmux session with powerline looks like.


I am kind of loving the new setup and enjoying it. I am also constantly using tmux cheatsheet :P because it’s good to look up what else you can do and also I saw various screencast on youtube where  tmux+vim makes things amazing.

Do let me know how you like my setup or how you use it .

Till then, Happy Hacking!🙂


GSoC: Final Submission

GSoC: Final Submission

This summer has been really amazing, I learnt a lot and worked crazy hours it has been a crazy yet amazing ride. I am not going to stop working on open source projects and with Pagure it is something really close to my heart.

There are a few things left but I can conclude that I am able to achieve what I wanted to at the beginning of this program , but there is never a feeling of satisfaction it is just like you want to achieve the best possible and most beautiful solution.

Pagure has CI integration which was one of my major goals to achieve and with the coming release it will be out and will be usable to people. This gives me immense pleasure to say that the foundation of CI was laid by me although Pingou kind of wrote a lot after that but that helped me to learn the depth of thinking one needs to have when you are working on a feature like this. Selection_027

I also worked on Private Repo feature which took more time than expected and it was pretty challenging to achieve , this feature is in  feature branch and it may get merged after it is checked in the staging first. Selection_028

It was so challenging that I got stuck on a data fetching problem from the database , we use Sqlalchemy as ORM in Pagure. I went through a lot of ups and downs at times I was about to give up but then I get some small part of it and Pingou has been so amazing mentor he never spoon fed me instead he asked the right question the moment he ask something the idea bulb use to glow.

I still remember struggling with  Xapian and Whoosh. This was again a very big task and still is , it requires a lot of time to optimize it to a level where it doesn’t slow the site. I gave a lot of time on it but since I few other goals and various issue to solve so I eventually moved on to those just to come back.

Pagure pages is one of the last goals that I worked on recently and there are discussion pending over it.

At a glance I was able to achieve a lot of the big goals on my proposal and still work has to be done, and I will continue to work on achieving various other goals. Few links that I want to share :

Commits made to the master branch

Commits on private-repo branch on pagure 

Pull-request for static page hosting

This kinds of makes me feel happy that I have around 102 commits on the master branch now and I believing I will be working a lot more on Pagure to bring a lot of cool and useful feature to it. In case you have any suggestion feel free to file issues on Pagure.

To be really frank I am not at all sad that GSoC is getting because I have received so much love and inspiration from Fedora Community that contributing to projects has actually become my daily routine the day I don’t commit code, review patches or comment on issues I start feeling something is missing .

And some of my fellow GSoCers said That’s all folks!  ;)

Happy Hacking!


Docs in Pagure

Docs in Pagure

I took this week to hack on this feature called Docs which gives you the ability to host documentation of your project in Pagure. I have never explored this feature before so I started to hack on it .

This feature is pretty straight forward to use. Once you have your project up and running you can go to Setting of the project and under  Project Option  click on Activate Documentation this will actually activate a Doc tab in the main project. This can be used to host your docs specifically now this is a little tricky because you need to clone and push to a different URL, the docs are maintain in a separate location this is due to security concerns. When you activate the Project option you are provided with a Doc specific URL, you need to push your document or static pages to that URL and automatically any page named as index will be taken as the first page.


You have to click on the more button beside GIT URLs to get your Docs URL and then you are good to go to host your static page.

For people who want to hack on Docs in Pagure you need to pull a few tricks to do that.

First and foremost is you need to get the code from and then after setting up Pagure for development, you need to run two servers :

  1. Pagure server
  2. Doc server

The script corresponding to them are  runserver and rundocserver.

So if you have ever hacked on Pagure then you will know that you have to log in make repo  and follow the same steps mentioned above to see the Doc tab.

Under pagure/ a new conf key has to be added which is

DOC_APP_URL = 'https://localhost:5001'

This tells Pagure that this instance supports Doc.

Now comes the tricky part, if you need to see Doc there should be a <project_name>.git created in the docs repo which is not there you just need to copy the file from repo directory to docs. Once this is done you need to clone the project repo from docs delete all the files there put the files you want in the static page , we support a lot of formats like md, rst etc. Add, commit and push and voila you will see them in your local instance.

I am actually working on issue 469 in which Ryan has suggested to make docs more specific to static page  hosting with the architecture that docs is based on this is actually a straight forward task but a really beautiful one which need a bit of deliberation on things we want to achieve.  Hope it gave you insight in what I am trying to to do.

More documentation on this can be found in the usage section of Pagure Docs.

Happy Hacking!🙂

Search for Code in Pagure

Search for Code in Pagure

I was trying to get into code search in Pagure, thing that I land up on got really interesting and amazing.  If you want to have a code searching mechanism in your website you need to look into something called Indexing.

The way search happens in some E-commerce sites like Amazon or be it the search happening on Google, with Google its web scrapping and then indexing on the results. The point being the response time , while you are searching for something you get results in few microseconds.

Now imagine going through such a huge database and going through them in few micro second how much ever power you have but what you need is a clever way to manage it. I was looking at a CS50 video in which Mark Zuckerberg was telling about how he managed his DB, the first architectural design he took was have different MySql instance for different school so that they reduce time taken to search and form relation.

That was a really clever move.

While I was searching for ways to have code search feature on Pagure, I landed up on a pyhton based library called Whoosh. It blew me off with the way it was doing its searches and maintaining the database. I actually looked for a lot for tutorials on how one can understand indexing.

I landed up on Building Search Engines using Python and the way he explained things like N-grams , edge N-grams and how different files store different index words with the frequency and path to documents. I am yet to analyze git grep v/s whoosh.

While I was going through whoosh I saw that it has performance issues and then I started contemplating on the fact that if search is not fast enough then there is no point in having it. I actually looked into HyperKitty I figured out they were using Whoosh before and I assumed even they suffered form performance issues or may be because Django introduce Haystack . As the name suggest you can also use this to find the needle in haystack.

Yeah! you are right, I started looking for Haystack in Flask and I found Flask-whoosh. Again the draw back I had was it use to search through databases and not files, where as my application was to search through files on the system

There came the xapian there are a lot of core concepts involved while using or writing utilities in xapian. I went through the documentation for Xapian. They have covered a lot of concepts and have given examples of it, the bottleneck still persist when it comes to file searching and performance. I found a nice application Building Document Search which might give me some hope but still a lot of work is required there.

The whole concept being you need to do two things on a really high level:

  1. Indexing
  2. Search


Indexing is required to go through the each file or record and build something called Index which has the search words filtering  stop words and the new database is build having the frequency and location of the word , this is the most time consuming process.


This comprises of forming a query and searching through the formulated database and return the document in which word or phrase is found.

If you need to see a demo.

Till then Happy Coding an Bingo!

Setting Postgres For Pagure

Setting Postgres For Pagure

I normally use Sqlite for development because of the ease you get to see your file , browse through it and edit it. Having said that sqlite is good for development and not for production one of the foremost reason being it doesn’t support multi-thread querying.

The other disadvantage was sqlite doesn’t give a damn if you have dangling Foreign Key references, I land up on this problem recently. The way we categorize fork project in Pagure is on the basis of parent_id so if a project has parent_id its a fork and if it doesn’t then its not a fork.

This works out quite well unless recently we figured out a flaw , what if the main project is deleted, the expected behavior is the fork should be accessible but because of the parent_id  dependency the fork was getting inaccessible this was because as you delete the main repo , the FK references with the fork gets modified and becomes Null.

This creates anomaly because now the project is no more a fork , its a main repo and its treated like it which leads to a lot of repo path chaos. The relation of Postgres came here because I was able to have a dangling FK reference here in sqlite but when I try to achieve the same thing in Postgres it throws an integrity error.

Pagure uses Sqlalchemy as the ORM so I just need to set up postgres on my system and provide the URL in pagure/  and ORM magic makes all the queries just work.

Setting up Postgres is really easy because of the amazing documentation provided in fedora-wiki . The only thing you need to care or a little tricky about is you need to be a superuser  before you change to user postgres .  So first sudo su and then su - postgres. Then the follow the steps in the wiki and create a user and create a database name pagure.

Private Repo on Pagure

Private Repo on Pagure

One of my proposal for Pagure was to have private repositories. Private repositories are basically repositories which are visible to you and people you give permission to.

To be honest , I thought it would require a few tweaks and I will be good to go, but that wasn’t the case and the insights I got working on this feature was amazing. I fiddle with this project on primarily  three stages. Each stage was a challenge in its own.

The three stages were:

  1. UI
  2.  Database Query
  3. Tests


The UI  was suppose to have a checkbox saying “Private”  and when a user ticks it the existing project becomes private or the new project is private from the time it is conceived.

Achieving this was a joy ride, with flask I just need to make changes in the form and setting page UI and Voilla!

I introduced a column Private in the project table and that was pretty much it. Nice and beautiful.


This was the most challenging part for me , since I have not worked with databases, and this was out of my comfort zone, I actually went back to my database basics to see if I am doing things right.

We in Pagure use Sqlalchemy as the ORM layer, ORM stands for object relation mapper. It basically use to map databases to object-class model of representing data. Sqlalchemy is a really powerful tool.

While figuring out ways to get all admins who can view private projects , I struggled a lot since I was working with a function which forms the core of Pagure so if things go wrong with this function the whole Project will take a hit.

So the challenge was to make minimum changes which are independent so that it doesn’t compromise the existing functionality and yet able to introduce a new one. I struggle to achieve it I failed a lot of time , was working hard to get it working , constantly moving to the board to figure out a solution on paper. Then switching back to my screen to code it out.

I was so desperate to get this working that I even pinged Armin on IRC to ask my doubt about flask and Sqlalchemy.  All this while the best support I got was from my mentor Pingou.

Finally after struggling a lot I got a very beautiful solution and done !

Just when I thought I am done , there comes a question of writing tests. Since I have altered a very major functionality that means I need to test every aspect of it.



Testing was a herculean task since I have not done a lot of testing, I actually got a lot to learn for starting the DB used for testing is a in-memory DB and not the one used by the app.

The session maintained has to be replicated in a way to use them in the test and how to use pygit to actually initialize a repo with git init and use it.

Towards the end of this PR my development evolve from writing code and testing it , to write the test and then introduce code or write code that pass the test. It has been really amazing working on this feature and hope it will be integrated soon.

I think may be a little more work is required on this feature maybe. It feels really amazing to do this work.

The link to the branch on Pagure.

The link to the current Pull-Request.

Happy Hacking!