Google Summer of Code 2018 Final Report – Minkush Jain
Author: Minkush Jain <minkushjain@gmail.com>
Wiki: https://wiki.debian.org/MinkushJain
Website/Blog: http://minkush.me/
Project: Wizard/GUI helping new interns/students get started
Project Wiki: https://wiki.debian.org/NewContributorWizard
GSoC Work
Repository: https://salsa.debian.org/minkush-guest/gsoc-2018-experiments
GSoC 2018 Final Report Blog: http://minkush.me/gsoc-2018-final-report/
This blog post contains a detailed description of all tasks with their video demonstration of the features.
?CardBook Package (in Debian directory): https://salsa.debian.org/minkush-guest/CardBook/tree/debian/sid
Commits:
https://salsa.debian.org/minkush-guest/gsoc-2018-experiments/commits/master
https://salsa.debian.org/minkush-guest/new-contributor-wizard/commits/master
https://salsa.debian.org/minkush-guest/CardBook/commits/debian/sid
Mentors: Daniel Pocock
Co-mentors: Urvika Gola, Jaminy Prabharan, Sanyam Khurana
Introduction
The project aims at developing programs and tools that would simplify the process for new applicants in the open source community get the required setup. It would consist of Wizard with scripts to setup various communication and development tools like PGP key, subscribe to mailing lists, source code scanner, XMPP, get started with IRC etc.
I created several scripts for the project to automate tasks for a user so that the user can get started with open source development right away.
Tasks Assigned
I was initially assigned the following modules:
Communication module:
- email list recommendations, joining
- mail filter setup/labels assistant
contact management and ?CardBook package
- IRC setup
- XMPP setup
- enable top-posting configuration and other settings in Thunderbird
'Start Here' Module:
- Salsa account registration
- New Github account creation
For final evaluation (after 2nd evaluation), the following tasks were assigned to me:
?CardBook Debian Package review, aim for final upload on mentor
- add profile adjustment feature in Thunderbird script
- extend IRC script for channel selection and joining them
- Adding test scripts, check dependencies
- extracting data and telephone numbers from email
- Filter user’s emails in mailbox in folders
- create Salsa script using HTTP Post
- Source code scanner: scanning user home directory and finding preferable language
Progress
Mailing List subscription
I created Python script to automate subscription to important Debian mailing lists for a user. The script also automates its confirmation process by replying user-unique code to the list mail.
Thunderbird Setup
Created a script to help a user get started with Thunderbird’s setup. The program edits Thunderbird configuration file inside the user’s system after killing its running process. Extended script to add the feature for Profile selection for modifying settings.
IRC Setup
The aim for this task was to help a newcomer get started with Internet Relay Chat and its setup.
I created a script for registering a new IRC nickname on Freenode server and checking for already registered nicknames.
Also added feature to display the list of available Debian IRC channels and joining channels.
CardBook Debian Package
The primary task was to simplify contact management for a user in Thunderbird.
I created a Debian Package for ?CardBook, a Thunderbird Add-on, for its easy installation in Debian OS.
Package repo: https://salsa.debian.org/minkush-guest/CardBook/tree/debian/sid
Commits: https://salsa.debian.org/minkush-guest/CardBook/commits/debian/sid
It closes Request for Package bug RFP #881379
Bug Report: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=881379
I have signed the package and uploaded it on mentors.debian.net
xul-ext-cardbook package: https://mentors.debian.net/package/cardbook
I spent significant time learning the packaging process, using packaging libraries and scripts, creating versions, removing Lintian errors and uploading it.
Salsa Account Registration
Selenium
Made a program to automate new account registration on Salsa using Selenium Web Driver automation.
HTTP Post
Created Salsa account registration script using HTTP post method (with Requests and Beautiful Soup library)
Github Account Registration
Created a script to automate the process to create a new Github account. Wrote tests scripts for Salsa and Github programs using Pytest.
Source Code Scanner
I created a script to explore user’s home or project directory and analyse it for various file extensions and scripts. The program would find a user’s preferable programming language and number of lines for each language.
Extract Phone numbers and data from Emails
Wrote a program to iterate over all emails of a user and extract telephone numbers from them. All phone numbers would be saved in an external text file with their Sender, Location, Time zone, Carrier details.
Filter Emails
I created a script which analyses and filters a user's emails into various folders in Gmail.
Research
XMPP
I researched on XMPP API like sleekxmpp and slixmpp in Python. Tested a script to create XMPP bot using sleexmpp to log in, send and receive messages. Wrote a script using pywebview to display HTML website inside GUI.
Mail Filter Setup
Researched on Procmail and Seive for customising mail filters.
Weekly Reports – Mailing list and Blogs
Community Bonding Period Blog: http://minkush.me/gsoc-community-bonding/
Week 1 & 2 Progress
Weekly report: https://lists.debian.org/debian-outreach/2018/05/msg00046.html
Blog: http://minkush.me/gsoc-week-1-and-2/
Week 3 Progress
Weekly report: https://lists.debian.org/debian-outreach/2018/06/msg00013.html
Blog: http://minkush.me/gsoc-week-3/
Week 4 Progress
Weekly Report: https://lists.debian.org/debian-outreach/2018/06/msg00051.html
Blog: http://minkush.me/gsoc-week-4/
Week 5 Progress
Weekly Report: https://lists.debian.org/debian-outreach/2018/06/msg00083.html
Week 6 Progress
Weekly Report: https://lists.debian.org/debian-outreach/2018/06/msg00119.html
Blog for week 5 and 6: http://minkush.me/cardbook-debian-package/
Week 7 and 8: https://lists.debian.org/debian-outreach/2018/07/msg00035.html
Week 10 & 11: https://lists.debian.org/debian-outreach/2018/08/msg00000.html
Week 12: https://lists.debian.org/debian-outreach/2018/08/msg00035.html
Summary of Weekly Progress (till 2nd evaluation)
Week |
Goal |
Activity |
Code |
0 |
Community Bonding |
Actively participated in Debian community & project's discussions through mails. |
|
0 |
Community Bonding |
Attended Kubecon + ?CloudNativeCon, Copenhagen |
|
0 |
|||
1 |
Thunderbird config |
Research configuration API and extensions |
|
1 |
Mail filters |
Researched Sieve and Maildrop |
|
1 |
|||
2 |
XMPP, JID |
Learned about Jabber and XMPP chat. Researched sleekxmpp documentation, JID registration and finding ways to setup for user through script |
|
2 |
|||
3 |
XMPP |
Searched ways to register Jabber account on webpages. Tested a script to log in, send messages to Jabber Users |
|
3 |
Mail filters |
Researched on Procmail and its syntax, and ways to filter emails on Thunderbird. Studied Mutt and Fetchmail servers. |
|
3 |
Open webpages |
Wrote scripts for opening webpage inside GUI for Jabber registration, Thunderbird & Pidgin Download |
|
3 |
Mailing List Subscription |
Wrote scripts to automate mailing list subscription & its email reply for ten Debian lists. Created a Pull request for the code. |
|
3 |
Start_Here Module |
Tested Salsa, Github automation python scripts using 'requests'. |
|
3 |
Salsa Registration |
Wrote a script using Selenium to automate new account registration on Salsa. |
|
3 |
|||
4 |
Server Program |
Learned socket programming in Python through server-client model. Created a twitter bot to test data transfer on servers. |
|
4 |
IRC Nickname |
Created a script to automate new IRC nickname registration on Freenode server. |
|
4 |
Github account |
Wrote a script using Selenium to automate new account registration on Github. |
|
4 |
Merge Request |
Worked on PR to improve scripts, all discussions resolved |
|
4 |
Contact Management |
Searched on Lightning, ?SoGo, ?CardBook in Thunderbird for managing contacts and calender. |
|
4 |
CardDAV/CalDAV |
Learned CardDAV, CalDAV Protocols and DAVdroid app |
|
4 |
|||
5 |
Packaging and license |
Did extensive research on Debian Packaging, studied DFSG license for package. |
|
5 |
?CardBook Debian Package |
Created Debian Package for ?CardBook (Thunderbird add-on). |
|
5 |
Policy errors |
Used Lintian to check and fix Debian Policy and Packaging errors. Studied DPM, DMG guides for it. |
refer blog |
5 |
Local Debian Repo |
Created a Local Debian Repository for testing ?CardBook Package. Changed apt-config for testing installation process |
refer blog |
5 |
Project Environment |
Setup Project's environment in system with dependencies, linter |
|
5 |
|||
6 |
Thunderbird process |
Tested some scripts to kill Thunderbird process in the system. Made one using 'psutil' |
|
6 |
Thunderbird Config |
Learned all mail, news settings and 'prefs.js' file in Mozilla Thunderbird. Created a script to customise Thunderbird settings for a user like top-posting config in mail |
|
6 |
Update Package |
Created new Cardbook version, uploaded latest 2 versions on a branch of cloned ?CardBook repository |
|
6 |
IRC Nickname |
Refactored IRC nickname script to make it work as a stand-alone script. |
|
6 |
|||
7 |
Profiles in Mozilla |
Researched on 'profiles.ini' in Mozilla and how new profiles are coded |
|
7 |
Salsa suggestions |
Tested new account script and mailed Maintainers for suggestions |
|
7 |
GUI |
Studied basics of GUI creations with Kivy, and how to add it in python scripts |
|
7 |
|||
8 |
Thunderbird config |
Updated script to add feature for finding file location is users's system dynamically |
|
8 |
Virtual env, Pipenv |
Learned about Virtual environments and Pipenv in python. Added the dependencies required with my scripts in Pipefile and Pipefile.lock |
|
8 |
Testing scripts |
Learned unittest, pytest. Working on adding pytest testing to scipts. |
|
8 |
Summary of Work (for Final evaluation)
Task |
Activity |
Code |
Mozilla devscripts |
Changed the package architecture to xul-ext with devscripts |
Remove Lintian errors, Sign it |
Fixed all errors in cardbook v1.3.0 and signed the package with keys |
RFP #881379 |
Changed RFP bug report to ITP and solved it |
Thunderbird script |
Added feature to extend scripts for profile selection |
Extend IRC script |
Added feature for channels display and joining them |
Source code scanner |
Wrote script for analysing system and finding preferable language |
Extract phone numbers |
Wrote script to fetch all mails and extract phone numbers with their details |
Filter mailbox |
Script to filter and move emails in new label in Gmail |
Test scripts |
Added test scripts using Pytest |
HTTP Post Salsa |
Made script to create new Salsa account using HTTP Post |
What I learned?
- Python APIs, libraries
- Debian Packaging
- Email inbox scripting with Python
- HTTP client scripting in Python
- IRC/socket programming
- Writing tests
- Automation scripts
?PyQt GUI
- Data scraping
- Git, Shell scripting
Summary
All the tasks assigned to me have been completed, but there is always a scope left to add additional features to the already made scripts. I plan to continue contributing even after completion of GSoC.
Acknowledgement
I would like to thank Debian and Google for giving me this opportunity to work on this project.
I am grateful to my mentors Daniel Pocock, Urvika Gola, Jaminy Prabharan and Sanyam Khurana for their constant help throughout GSoC.
Special Mention
I would like to thank Carsten Schönert and Andrey Rahmatullin for their help with Debian packaging.