Investigating Python

Investigating Python

I have been trying to implement private projects on Pagure, while doing that I was struggling with certain design of a function and while doing that I constantly have to switch between shell, editor and at times browser.

I am not saying it is a bad thing or  a good thing but this lead me to find a debugger I thought that might ease my task of finding what is going wrong and where and it actually helped.

I used a python debugger called pudb. This looks like Turbo C but its a lot more useful. This can be used in one of the two ways:

  1. You can directly debug a script  pudb <your_script_name>.py
  2.  You may need to call certain function when you are working with big projects in that case you can just put import pudb; pu.db

 

The most beautiful thing is, it just pops an IDE out of no where, it gives a deep insight about what code i s doing, how it is doing , what stack it is maintaining and what are the various values of variables.

You can always set breakpoints so that you can investigate about the code,  you actually play a detective. This is actually an important  point in developing in an opensource project since there are a lot of functions doing a lot of Hocus Pocus.

This is one of those tools that might even help you to understand the code base better , it really helped me to design the code better.

This is the script that I am trying to debug, the screen looks like this and ‘n’ can let you to go to the next line which can be investigated using the stacks shown.

Selection_017

These are the few windows that you can navigate to and see what is going on.

Selection_018

Not only this but you can jump between different modules and you can set breakpoints.

Selection_019

 

This might help you get some cool insight about the project.

Happy Hacking !

Pagure CI

Pagure CI

As my GSoC project one of the first goals is getting CI to Pagure. In my previous post I have been blogging about getting Fedmsg to work, Configuring Jenkins and my favorite Poor Man’s CI. Well, Poor Man’s CI evolved into Slightly Richer Man’s CI and now  Pagure CI.

This is how it started I was trying to test how Poor Man’s CI works and how can I actually integrate it, while doing that I hit a deadend when Poor Man’s CI would not respond to local Pagure instance because there is no GIT server running. So one cannot clone git repo with a normal URL, I landed up on a really amazing utility called Git Daemon, it gave me some hope but it didn’t last long, that also has disadvantage that the URL should be a git:// URL.

I somehow thought I needed to set my Pagure as an application and then I can make common clone commands. I tried using that and I am still in the process of doing it , I am still setting gitolite on that front.

While doing that and because of my lazy nature I found a neat hack and it was , my Poor Man’s CI was running on localhost and so was my Jenkins but the I made a repo on pagure.io.  The only thing to understand here is the Fedmsg should listen to all the msgs and Fedmsg plugin should be enabled in the repository on Pagure.

So, recap you have :

1. fedmsg-relay running and listening to all the production messages.

2.  fedmsg-tail  –really-pretty running to check the message is being read.

3. fedmsg-hub running in Poor man’s CI which is consuming the message and configure        to consume new pull request.

Voilla! it worked and there you go I got a build there and the flag is attached to it.

Selection_016

 

After talking to Pierre we arrived a on conclusion that we need a way to either merge the two application or make Pagure talk to Poor Man’s CI. I had a lot of doubts to achieve it, later Pierre gave me “Food for thought ”  which was

“If we are able to gather all information from Pagure and Poor Man’s CI could get that database access would it not do the deal?”

The way PMCI handles it is the consumer written for fedmsg-hubs parses the message received and according to the message received it triggers Jenkins build and does all the work and it just queries a single table.

Pierre wanted no individual UI for PMCI and so we made it a hook , now a person can enable the hook and add the useful information required. I brought the whole consumer as it is from PCMI. Consumers need to get registered in setup.py and dependencies needs to be build up using python setup.py develop which does a lot of magic behind the scenes.

One of the dependencies is python-jenkins library, which provides various functionality. The whole PMCI is reduced to three files in Pagure. The most important being consumer.py , I gathered a lot of functionalities into a lib/pagure_ci.py and made a model in hooks/jenkins_hook.py.  Since I got lot of functionalities in place the only thing then I struggled with was because of my poor understanding of Sqlalchemy. I read up a lot and was able to debug the code and make it work.

 

The most peculiar thing I came across was even the server needs to be run in the virtualenv, since Jenkins library is installed in that. Once that is done its like a train of dominoes , one thing will lead to another . I am trying it to show you diagrammatically.

hook