Attracting and retaining Debian contributors
Many projects struggle with attracting and retaining contributors; Debian is no different in that regard. At DebConf24, Carlos Henrique Lima Melara and Lucas Kanashiro gave a presentation about efforts that the Brazilian Debian community has made to increase participation. Their ideas and the lessons learned can be applied more widely, both for other Debian communities and for other projects.
Kanashiro introduced himself as a software engineer, Debian developer since 2016, and an Ubuntu core developer working on Ubuntu Server at Canonical. Melara said that "everybody calls me Charles", and that is his name within Debian; he is a computer engineer interested in security and operating systems, working on embedded systems at Toradex. He has recently become a Debian developer.
History
Eight or nine years ago, a local Debian community was formed in Brasília, which is the capital of Brazil, "right in the middle of the country", Kanashiro said. He was working on Debian without any local colleagues at the time and wanted to try to get others involved. So he and his former professor began holding events and installfests, which worked pretty well, "but in the end we were not able to engage people" and retain them in a local community.
Later, he was able to work with some university students who were interested and they formed a group that met monthly, sometimes in conjunction with a local Linux users group. His goal, though, was not to be a users group; instead he wanted to have a local community of Debian contributors. But, then, along came the pandemic, and everything stopped.
A few months after the start of the pandemic, the group started to have online meetings. That is "when things started to get some traction", largely because people from other cities were able to join in. The group also started to be able to contribute to Debian; it now has four Debian developers, five Debian maintainers, and those numbers increase every semester.
Brazil now has an engaged and active community of Debian contributors. He showed a picture of attendees at the MiniDebConf Belo Horizonte that was held in April; there were around 225 participants, Kanashiro said. He also showed a picture of all of the Brazilian attendees at last year's DebConf in Kochi, India; there were 14 people in the picture. The Brazilian Debian community likes to promote what it is doing, by showing up and participating in events like those, but it is also interested in improvement and in helping other communities to improve as well.
To that end, Melara asked the audience how many were Debian developers (lots) and maintainers (two); then he asked how many "have a thriving local community where you are living?" There were a few hands raised, which is "nice", he said, but is indicative of the problem that they have been trying to solve, both in the city of Brasília and in the whole country of Brazil.
There are two things that they think are important to overcome in order to create and build a new local community. The first is to lower the barriers for newcomers to contribute. But once you have attracted a newcomer and gotten them to contribute, how can they be retained so that they become long-term contributors or even become Debian maintainers and developers? The main goal of the talk, he said, is to relate what the Brazilian community has done "to make those two things a reality".
Barriers and problems
Kanashiro described some of the barriers for newcomers that the Brazilian community has encountered. Documentation for newcomers is often missing, or is spread around in various places if it exists, so it is hard to find. Another problem is that there are too many different tools and workflows for Debian packaging, which is confusing to newcomers. Searching for packaging tools, or talking to different Debian developers about it, will lead to too many options. Communicating via mailing lists and IRC is the norm in Debian, but "that does not work well" for the "new generation"; if you want to bring those folks into the project, there needs to be some kind of workaround. "Of course, we will not change everything, because the community works like that, but, for sure, we need to find a way to lower this barrier."
English is not an easy language for many potential newcomers in Brazil, but most of the documentation that is available is written in English, so that needs to be addressed. People, especially new participants, are shy about asking for help; they do not know who the right people to ask about how to contribute are and worry that, even if they did, those people are so busy that a newcomer should not bother them. The final barrier that he relayed is the technical jargon, terms, and acronyms that pervade Debian and its packaging; newcomers find it hard to understand what people are even saying when they are talking about those topics.
Melara said that the list of barriers is long; he had some more to add to the pile. Newcomers obviously already know about Git, rebasing, packaging tools, using the command line, and more, right? "Yeah, no, that's not true." Most of the new people are coming from Windows, so they are not familiar with the command line and only have a general sense of how computers and operating systems work. As noted earlier, the diversity of tools and workflows is intimidating as well.
Once you get someone interested, engaged, and perhaps even doing some packaging work or fixing a bug, he asked, how do you encourage them to continue to contribute in the long term? "It's difficult to make this transition from a first-time contributor to a long-term one."
Once a contributor gets to the point where they know how to use the tools, fix bugs, and do packaging tasks, local community members help them to become Debian maintainers. But, there is a similar problem when that happens, he said; they get upload permissions for the packages they maintain, but there is no clear path forward. The goal is for the Debian-maintainer role not to get stale.
The last problem is with visibility for local groups. For those doing packaging, there are various statistics that can show the work that is going on, which can help increase the visibility of the group. But there are other tasks that are not really measured that way, including putting together events like the MiniDebConf. Those things are important for the community but are not as visible.
So, with those barriers and problems in mind, Melara said, they wanted to talk about the solutions that the local group has come up with. These things have been worked out over the last four or five years; they would be presenting what they are doing now, which "seems to work for us".
Solutions
Kanashiro said that the documentation problems are being addressed with a wiki for Debian Brasil. They had encountered a number of problems with using the official Debian wiki due to IP-address blocks, which restricted access and required pinging people to get addresses unblocked. Beyond that, the Debian wiki is not particularly modern, while the Debian Brasil wiki, which is based on Wiki.js, has Markdown-editing capability and allows editing pages side-by-side with their rendering, both of which make it easier to use.
All of the wiki content is in Portuguese, since it is targeting Brazilians. There are some basic tutorials on setting up an environment to do packaging work, the structure of a Debian package, using packaging tools, and so on. There are also videos from previous events on topics of interest. "Everything is there, everything is in Portuguese". Since it is a wiki, it can be edited, of course; if someone has a problem when they follow the steps on the wiki, they are asked to edit the item to improve the document for the next person.
This helps address two of the barriers: lack of documentation for newcomers and the prevalence of documentation in English. They could have translated the existing documentation, he said, but what they have is easier and works better for the Brazilian community, at least for now. There has been some interest in updating the Debian wiki to something modern, which might make it easier to merge the two at some point.
The tool-proliferation problem has been fixed by the simple expedient of choosing "a very opinionated set of tools" for newcomers to use for packaging, Melara said. These are the same tools that he, Kanashiro, and others use, so they can easily review the work that is being done and "make sure that everything is OK for uploading", Melara said. The process uses sbuild, git-buildpackage (or gbp), DEP-14 Git layouts, and the Salsa GitLab instance for Debian.
All of the review comments are picked up by a bot to send to the Matrix channel for the group. That allows others to see what changes people are asking for in review and to learn from that. There is a "very, very detailed workflow", which is well-documented and that only requires learning two tools and Git; that is the starting point for newcomers, though they can use other tools and workflows once they get their feet under them. This helps address the problems with too many tools and workflows; "it's pretty easy for newcomers".
The Brazilian community is using modern tools for communication, Kanashiro said. It is difficult to make big changes in the overall Debian community, but their local community can make its own way. IRC is the default chat mechanism for Debian, but younger people do not use IRC; in Brazil, they use Telegram instead. It is not practical to expect them to leave Telegram and move to IRC, so Debian Brasil has created a bridge that connects IRC, Matrix, and Telegram so that people can choose the one that works best for them "and everyone will be there discussing things".
One of the big complaints about IRC is that messages get lost when people are not logged in, so Debian Brasil provides a Convos service for its users. Convos is an web client and server-side "bouncer" for IRC, which handles message persistence so that users can always get the messages sent. These tools have been helping with the communication barrier that was identified, but also helps with long-term commitment to the community since people can track and review all of the different comments being made on package reviews, merge requests, and other activities.
Helping people to transition from proprietary tools to free ones ("as in free speech, not free beer") is another thing that the group does, Melara said. There is an ongoing effort to convince people to move from Telegram to Matrix or IRC. It takes a long time, he said, but he thinks that progress is being made there.
Another way to help keep people involved is to keep in touch frequently. Debian Brasil has weekly online meetings using Jitsi. As might be guessed, those calls are in Portuguese. Normally the first hour and a half or so are spent on problems that people are having; after that, it becomes a "happy hour" where people talk about other things. It is a bonding experience that helps make newcomers more comfortable by seeing the other members in a social setting.
It is important to be consistent and not skip any weeks, because if people try to attend and the meeting is not held, that may make them less likely to come the next time. So Melara tries to be there every week; his slide had a picture from DebCamp the previous week of a few Brazilians attending the Jitsi meeting from Korea at 7am Friday, which was the usual 7pm Thursday meeting time in Brazil.
Debian Brasil has a well-defined process for packaging contributions, Kanashiro said; it is based around using issues and merge requests on Salsa. Newcomers often know about using GitHub, so it is easy to get them using similar features on Salsa. Instead of going to a Debian mailing list to ask for help in English, they can get help in Portuguese via chat. There is a real effort to provide feedback quickly, he said, rather than having them wait weeks or months, which can happen in other parts of the Debian project. They have been successful in getting new people to contribute this way, he said.
The process can be easily tracked because tags can be assigned in GitLab, Melara said. That allows the progress to be tracked by email or via the web client; reviews can be made and checked in email and the comments get relayed to the Matrix channel so that people can keep up with what others are doing. "Yeah, it's pretty nice."
Reviews are handled by any available Debian developer or maintainer, but they have also developed a mentorship program to try to help new Debian maintainers to keep contributing. The mentor is a Debian developer who gets assigned to the maintainer to help them, on a one-to-one basis, explore different possibilities of things to work on. This program is new to Debian Brasil; it is still being worked on, including in a discussion in an informal birds-of-a-feather (BoF) session earlier in DebConf, Melara said.
But the plan is for it to be more than just working with the mentor, since they may be unavailable at times. So there will also be group discussions to share knowledge and look for tasks to work on beyond just the handful of packages that a maintainer is responsible for. That way, there will be faster responses with input from more people, which is especially helpful for harder questions and problems.
Application
Other local communities can use some of what they have learned, Kanashiro said. He suggested that Debian could create "virtual environments for local groups" that come with "a bunch of different tools that helped us a lot". Debian Brasil has people available to set up and run all of those tools, but other local communities may not.
A local group could ask to have this environment set up for its members. They would get a wiki area for the group with some amount of storage reserved for them. The IRC bouncer would be configured and started. There would be a way to generate statistics for the activities of the group. Providing "an easy way to do all of this for any local community" would be "great to have" for Debian.
The younger generation ("Gen Z") is not interested in lengthy videos, Melara said, so they have been working on three-to-four-minute videos explaining basic packaging concepts, answering FAQs, and so on. The localization team created videos a ways back, which were well-received, so they are trying to do something similar for packaging.
Eventually, they would like to collect them all into a video course on packaging, Kanashiro said. It would present their opinionated view on packaging tools and processes; much of it is already done and is being used. The goal is to start with everything in Portuguese, naturally, but eventually to also have them in English. Getting the video content onto the Debian PeerTube is in the works as well.
There are some open questions, Melara said. He wondered if the contribution and mentorship processes would make sense elsewhere; can those be applied or adapted for other local communities? Are the problems that have been identified mostly Brazilian in nature or are they more widespread? There was not a lot of time left in the session at that point, so those questions mostly went unanswered.
A packaging-team member from the audience said that regular calls were useful, but that it is hard to do when people are distributed across the globe. For local groups, that have participants in similar time zones, it is much easier. Mentoring people using GitLab issues "is a really good idea", he said, which makes it easier to see "what's going on and what's not". Kanashiro suggested that recording the calls and making them available can help those who cannot attend them live.
Samuel Henrique, who is part of the Debian Brasil group, pointed out that there are participants in the call from elsewhere, including him from Ireland; they have managed to find a time that works, though "it's not perfect" since he ends up staying up until 3am most Fridays. Melara said that some people come early, some stay late, and some do both.
There is a question of how to promote the work that local groups do, Kanashiro said. One way is to do more frequent blog posts and make them available on planet.debian.org so that others can see what the group has been doing. They have also been talking with the people working on contributors.debian.org about adding data sources for things like event organization, meeting attendance, and various group activities. Currently, those things are not tracked.
The final item in their presentation was about new Debian maintainers getting more exposure to the whole project and, in particular, to people outside of the Brazilian community. The mentors are often encouraging these people to broaden their horizons and to get in touch with the project-wide Debian mentors organization. The intent is to get them working on more complex projects once they come up to speed on packaging. But the local mentoring is a new program for Debian Brasil, so they are still working it all out and are interested in what others are thinking.
With that, time expired on the session, which provided some ideas that may well be applicable to other Debian local groups—and beyond. It is interesting to note that some of the choices that are being made at the local level (e.g. opinionated packaging, moving away from IRC-only) are going to be difficult to push at the distribution-project level, at least for some time to come. The relative success of local organizations at attracting and retaining new people could—slowly—start pushing Debian as a whole toward more modern approaches. Time will tell.
A WebM video of the talk is available for those who are interested.
[I would like to thank the Linux Foundation, LWN's travel sponsor, for its assistance in visiting Busan for DebConf24.]
Index entries for this article | |
---|---|
Conference | DebConf/2024 |
Posted Sep 9, 2024 22:03 UTC (Mon)
by serzan (subscriber, #8155)
[Link] (2 responses)
a quick search suggests that gitlab only supports vanilla git tags and CI tags (to select which runner runs for any given commit).
presumably this was referring to project labels (https://docs.gitlab.com/ee/user/project/labels.html), which support email notifications? still unclear to me though whether MR emails are just "you have a MR to review" vs actually contain the patch, and in the latter case whether one can do the review exclusively via email
https://docs.gitlab.com/ee/user/profile/notifications.htm...
Posted Sep 10, 2024 9:31 UTC (Tue)
by taladar (subscriber, #68407)
[Link] (1 responses)
Posted Sep 10, 2024 14:35 UTC (Tue)
by serzan (subscriber, #8155)
[Link]
right. but if gitlab does support seamless web/email integration, then both can be happy, and newcomers can benefit from everyone's feedback (including from old timers')
the discussion of dep-18 (https://lwn.net/Articles/986480/) is to a large extent bogged down on the assumption that such an integration does not exist, but the speaker's remark seems to contradict that assumption
Posted Sep 14, 2024 9:01 UTC (Sat)
by azumanga (subscriber, #90158)
[Link] (5 responses)
About 10 years ago, and then again about 2 years ago, I tried to look into packaging. The problem is the instructions are a mess, and like this article says, there seems to be 20 different ways to do things.
Also, if you don't use autoconf (and I don't want to touch autoconf ever again if I can help it), the instructions are even less useful.
I see many people maintain a long list of packages -- I'm sure once you are 'in the zone' it's easy to keep a bunch more packages, but it's annoyingly hard to do one or two -- particularly getting things in the right directory.
Lots of programs nowadays are just a single executable, linked to some pre-installed libraries -- but it seems hard to say "build like this, then there is the executable, put it where you like", as far as I could find at least.
Posted Sep 14, 2024 9:12 UTC (Sat)
by bluca (subscriber, #118303)
[Link]
For what it's worth it, if you use Meson things pretty much just work (TM) thanks to the work of Niels and others
Posted Sep 16, 2024 10:24 UTC (Mon)
by highvoltage (subscriber, #57465)
[Link]
Posted Sep 19, 2024 23:25 UTC (Thu)
by technomancy (guest, #170546)
[Link] (2 responses)
My package is very simple; 5000 lines of architecture-independent code with only one extraordinarily stable dependency, and it still took over a month of work to get the package to the point where it could successfully build without warnings. But even that on its own doesn't get you anything; you still have to find a sponsor for your package; I've been waiting on that for 8 months and would not be at all surprised if it takes 18 more.
I was struck by the vast difference between the experience of Debian as a user (flawless anarchist utopia, every action you take succeeds with effortless grace, boundless opportunities beckon at a single command) and Debian as a packager (trudge thru rancid error messages, documentation assumes you are already an expert packager who wants to package a program that uses gnu autotools, every tool you encounter is a rewrite of a rewrite of a rewrite, jargon includes many words deployed as the opposite of their normal meaning)
Posted Sep 25, 2024 1:23 UTC (Wed)
by voltagex (subscriber, #86296)
[Link] (1 responses)
The Lintian URLs were broken at some point due to infrastructure moves and it was never clear what warnings I needed to worry about vs what were noise.
Posted Oct 2, 2024 9:30 UTC (Wed)
by taladar (subscriber, #68407)
[Link]
Posted Sep 21, 2024 6:49 UTC (Sat)
by MantraRay (guest, #173582)
[Link]
Really now, attracting and retaining Debian contributors? I'm switching to another OS as soon as I get some other projects out of the way.
tag-based email notifications?
tag-based email notifications?
tag-based email notifications?
Debian seems to want you to be 'all in' on contributing
Debian seems to want you to be 'all in' on contributing
Debian seems to want you to be 'all in' on contributing
Debian seems to want you to be 'all in' on contributing
Debian seems to want you to be 'all in' on contributing
I've been trying on and off to write a metapackage that configures new Debian VMs so that I can get up and running sooner, and I thought I'd try to make them Lintian clean.
Debian seems to want you to be 'all in' on contributing
Really?