|Grimoire Lab-GitHub - Stats On Steroids|
|Written by Nikos Vaggalis|
|Tuesday, 06 June 2017|
Grimoire Lab is an open source toolkit built on Python, Elasticsearch and Kibana. It taps into GitHub's raw data through Perceval, a module designed for retrieving data from repositories related to software development.
Perceval forwards the data to another tool for filtering, the so called Sorting Hat, responsible for managing and merging identities that correspond to the same real person/commiter and potentially come from different sources, before finally rendering the data manageable and accessible through rich UI Kibiter dashboards. Kibiter, a fork of Kibana, is what enables the user to create and edit visualizations as well as perform queries facilitated by the underlying Elasticsearch REST APIs.
Note the small-print; Perceval is "specialized in retrieving data from repositories related to software development". This means it is not restricted just to GitHub but can attach to a variety of disparate data sources able to host either source code, or code reviews, mailing lists and forums, meetups, tickets, issues, chats, you name it. Concrete examples are Bugzilla, JIRA, Gerrit, mbox, pipermail, StackExchange and Discourse. In fact makers Bitergia, offer enterprise packages for businesses which want to hook up their datasources to extract information.
When Perceval is applied to GitHub you end up with a powerful Open Source Community Analytics platform which can answer a multitude of inquiries revolving around an open source community; "How big? How many?", "Where from? Organizations? Gender?", "How old? New one? Inactive ones?", "How fast?". In other words, through metrics like these and many others, Grimoire can provide indications and measurements on a project's success.
As proof of concept, in Grimoire's repository we find a good use case on how to use Perceval to retrieve data from a GitHub project (git repository and associated GitHub issues), upload it to Elasticsearch, and produce a Kibana-based dashboard with it. You can find detailed instructions on how to set it up here, but suffice to say that the procedures are mostly Python related :
You first create and activate virtualenv for Python:$ virtualenv -p /usr/bin/python3 gitlab
$ source gitlab/bin/activate
Then install the necessary dependencies:
and so on.
To make it even easier, Bitergia has setup Cauldron exposing Grimoire as a service so that you can monitor your GitHub repo without having to set up anything! Each GitHub user may ask for up to five dashboards, each corresponding to a GitHub owner (user or organization).
For example let's have a look at the Eclipse Foundation projects' dashboard, which monitors the foundation's mailing lists as well as its Git, Gerrit and Bugzilla repositories.
Best of all and as already said, Grimoire is open source to the core and itself available on GitHub. As it isn't a single product but an ecosystem comprising of distinct components, each one comes under its own license. Perceval and Sorting Hat are under the GNU General Public License (GPL), version 3, while Kibiter is under Elasticsearch's Apache License, Version 2.0. so that you can use them in your own setups and applications.
or email your comment to: firstname.lastname@example.org
|Last Updated ( Tuesday, 06 June 2017 )|