CodeRefinery - Celebrating eight years

Continuing eight years of collaboratively teaching good enough research software engineering practices

• CodeRefinery started 8 years ago
• What is CodeRefinery?
• CodeRefinery workshops
• Bring your own classroom
• Sharing experiences and support for doing your own thing
• More than just workshops
• Reaching out
• The future

CodeRefinery started 8 years ago

In October 2024 the CodeRefinery project was celebrating the conclusion of its 8th year of existence. During that time about 9 online and 28 in-person workshops were held by ~30 instructors, organizers and facilitators teaching good enough research software engineering practices to ~3000 participants from ~20 countries, of many scientific fields, career stages and preferred programming languages. We would like to celebrate this achievement with you, by looking back at the history and achievements unlocked over the years and also ask you to share your best memories. Spoiler: Even though this is a "looking back" post, CodeRefinery will not cease to exist anytime soon if we can help it. We'd just like to celebrate the project and highlight the contributions of key members.

The project grew out of an initially local course (given at KTH Stockholm in 2014 and 2015) and in 2016 became a Nordic project. Rossen Apostolov (KTH Stockholm) submitted a proposal to the Nordic e-Infrastructure Collaboration (NeIC) in 2015 and NeIC quickly recognized it as a potentially impactful project worth co-funding. The project was started in 2016, under the leadership of Radovan Bast, project manager of CodeRefinery for the whole duration of the project.

Thor Wikfeldt remembers: "[...] I have had the privilege of working with Radovan Bast since 2016 when the CodeRefinery project was launched, and I joined under his leadership. Among all the highly competent individuals I have encountered in my career, Radovan stands out for his relentless dedication and selfless commitment to making a meaningful impact on the world. He has inspired me to not only become a better teacher and developer of training materials but also to be more productive, collaborative and generous in my professional life. Radovan took CodeRefinery from a mere concept to a pioneering educational project that is now well- known not only in the Nordic region (where it started) but globally. [...] "

What is CodeRefinery?

CodeRefinery acts as a hub for FAIR (Findable, Accessible, Interoperable, and Reusable) software practices. It currently focuses on the Nordic/Baltic countries, but aims to expand beyond this region. CodeRefinery aims to operate as a community project with support from academic organizations. The project started in 2016 and has developed a broad curriculum of openly maintained and reviewed lessons, has taught hundreds of participants across all academic disciplines, and has managed to build a community of instructors, learners, team leads (who help learners during exercises), expert helpers (who support team leads), local organizers and partner organizations.

The project idea/directive grew out of two courses given at PDC/KTH in 2014 and 2015, which focused on research software engineering tools and techniques. The courses were popular and it was clear that the demand is not limited to the Stockholm region and we approached NeIC to bring this project to a Nordic level, both to have more impact, but also to connect instructors across Nordic borders. The first CodeRefinery workshop was given late 2016 and since then the lesson material has evolved a lot and we have delivered many more workshops, both in-person and online.

CodeRefinery has established itself as a highly successful initiative that improves coding skills at an intermediate level, bridging the gap between Software Carpentry for beginners, and the more advanced/bespoke training offered by other universities and HPC/computational research initiatives.

The objectives of the CodeRefinery project are:

• Organize and deliver workshops and events
• Develop and maintain a lesson portfolio
• Build a community and network of instructors and volunteer helpers
• Operate a Nordic GitLab service
• Support the community of Nordic research software engineers

CodeRefinery workshops

The CodeRefinery project provides open, reusable and self-learning ready lesson materials developed by experts from different countries, organizations and scientific backgrounds. CodeRefinery focuses on maintaining collaboration in lesson development, teaching and workshop organization. Workshops with multiple roles especially highlight the value of collaborative efforts.

The workshops are focused around exercises and discussions and participants are encouraged to form teams for these sessions. The learning outcomes for each lesson are defined and shared in the beginning of each lesson.

We kindly request feedback from participants after each workshop day. Feedback is gathered using known tools with no separation between workshop and feedback. If necessary and possible, given feedback is already implemented for the next workshop day.

The CodeRefinery project maintains manuals with a collection of work processes and ideas (https://coderefinery.github.io/manuals/). It summarizes how meetings, workshops and other topics work and serves as basis for e.g. the helper onboarding for the workshop.

Bring your own classroom

When switching from in-person to online workshops the CodeRefinery team put a lot of effort in embracing the online workshop format. A lot of thought has been put into our online hand-on, demo and screen sharing setups in order to provide the best possible learning experience to participants.

Since 2020 we had multiple local classrooms join our workshop and have adapted the format to accommodate these special circumstances.

Paula Martinez Lavanchy shares: "[...][W]e have been ‘bringing our own class’ to the CodeRefinery workshops by joining the streaming of the lessons from the classroom with our participants and helpers. TU Delft researchers provided very positive feedback about this initiative reflected in comments such as:

“It's great and extremely useful. If it was it for me I would make it mandatory knowledge. It's really important that TU Delft continues promoting these workshops.”

“Excellent workshop: the graduate school would be so much better with more of these practical & technical workshops”

The CodeRefinery initiative has helped us and benefit TU Delft researchers in several ways:

• The possibility of joining the workshops allowed us to advance with the implementation of our Vision for Research Data & Software management training and the implementation of TU Delft Research Software Policy by providing high quality and well-received training on FAIR software practices.•
• The involvement of our data stewards, software engineers and trainers as helpers in the CodeRefinery workshops have also provided them with a great opportunity to continuously improve their skills and learn from this great community.
• The CodeRefinery learning materials are openly available and of excellent quality. We often refer our researchers to use them as consultation materials on our websites and/or guides. [...]"

Sharing experiences and support for doing your own thing

But the CodeRefinery project does not only focus on own workshops, it also wants to make it easier for others to provide clean and functional lesson materials with all the features needed for computational topics by providing a public lesson template (https://github.com/coderefinery/sphinx-lesson). In addition, the ways that teaching has worked well for CodeRefinery are shared through train the trainer workshops, which have been presented in different forums and to various groups (https://coderefinery.github.io/train-the-trainer/).

The lesson development process always involves multiple experts. All discussions and reviews are public and can be found on our GitHub pages (https://github.com/coderefinery).

Since it is free and open to reuse, there is no full overview about who has reused CodeRefinery materials. But when talking to people at conferences and other events we often get to hear that teachers are happily reusing the CodeRefinery materials for their lectures.

Two larger programs that have been built on top of CodeRefinery materials are the Netherlands eScience Center workshop on "Good practices in research software development" (https://www.esciencecenter.nl/event/good-practices-in-research-software-development-5/) and the "EuroCC best practices in HPC training" program lead by ENCCS Sweden (https://enccs.github.io/instructor-training/).

Mateusz Kuzak and his team from the Netherlands eScience center explain:

"[...] At the Center, we have been successfully using the training materials developed by the CodeRefinery project since 2020. At that time, CodeRefinery filled the gap in the intermediate research software skills for researchers. We already delivered training based on the Carpentries and were looking for more advanced content for researchers already doing some programming. What I appreciate about the project is that Radovan and others didn’t just reinvent everything. They build the Trainer the Trainer programme on top of excellent Carpentries Instructor Training. They also realised that the pedagogical methodology used by the Carpentries, heavily dependent on live coding, would not work that well for intermediate audiences. They developed a curriculum rich in independent work on complex exercises. At the eScience Center, we found that approach more effective for less novice learners. CodeRefinery was also very innovative, introducing a distributed online approach with many helpers supporting locally or in online breakout rooms. I believe that helps with scaling the course and reaching a new audience that otherwise wouldn't be able to access this training. We at the eScience Center will continue reusing and contributing to the CodeRefinery project. [...]"

More than just workshops

The main CodeRefinery workshop is organized twice a year and it is free and open for everyone. Everyone is encouraged to ask their questions and discuss the topics that interest them in a collaborative document. Instructors have a variety of different scientific and cultural backgrounds and are in different stages in their career.
After each workshop participants are encouraged to join the community which mainly lives in the CodeRefinery Zulip chat that to date is a home to 446 people with about 10% being really active. The chat is also home to the Nordic-RSE and Nordic-HPC communities which are tightly knit with CodeRefinery. The chat serves as a space for planning, support and discussions around different topics. Participants of the CodeRefinery workshops are encouraged to use the chat also beyond the workshop to ask their questions around workshop topics and beyond. Some participants even have found their way into the project this way.

While the workshops are the main event for CodeRefinery, it is also a community with an open heart for supporting research computing and providing courses on a researchers level. Research Software Hour was born from the community, and has brought topics of Research Software Engineering that you cannot teach in a class to the research community (https://researchsoftwarehour.github.io/).

A Zenodo community is available to collect all CodeRefinery and CodeRefinery related outputs: https://zenodo.org/communities/coderefinery/records?q=&l=list&p=1&s=10&sort=newest

CodeRefinery is also active on social media: LinkedIn (347 followers), X (844 followers) and Mastodon (313 followers).

Reaching out

The CodeRefinery project has been mentioned alongside other successful programs such as the Carpentries in Research Software Engineering related publications:

• I. A. Cosden, K. McHenry and D. S. Katz, "Research Software Engineers: Career Entry Points and Training Gaps," in Computing in Science & Engineering, vol. 24, no. 6, pp. 14-21, Nov.-Dec. 2022, doi: 10.1109/MCSE.2023.3258630 or on arxiv; page 7

• US Research Software Engineer Association, & IEEE Computer Society. (2023). Research Software Engineers: Creating a Career Path—and a Career. Zenodo. https://doi.org/10.5281/zenodo.10073233 ; page 19

• Barker, M., Breitmoser, E., Broadbent, P., Chue Hong, N., Hettrick, S., Lampaki, I., Quinn, A., & Taylor, R. (2024). Software and skills for research computing in the UK. Zenodo. https://doi.org/10.5281/zenodo.10473186 ; page 15

• Software Sustainability Institute Blog post on Free resources for technical staff: Career development and research software training by A. Nenadic and S. Aragon: https://www.software.ac.uk/blog/free-resources-technical-staff-career-development-and-research-software-training

• Goth F, Alves R, Braun M et al. Foundational Competencies and Responsibilities of a Research Software Engineer [version 1; peer review: awaiting peer review]. F1000Research 2024, 13:1429 https://doi.org/10.12688/f1000research.157778.1

A collection of reports (https://coderefinery.org/about/reports/) and presentations (https://coderefinery.org/about/presentations/) about the project are collected on our website.

Project members have been actively seeking opportunities to spread the word and share the experiences from running the CodeRefinery workshops at conferences in the research computing world. Among others the project has been presented at Supercomputing (SC) conference in the US, International SuperComputing (ISC) in Germany, SIAM, RSECon and CarpentryCon in recent years.

The future

We are currently in the third round of funding by NeiC (one person to coordinate the efforts, other partners support in-kind) and considering our next steps.

One thing is clear: CodeRefinery will not end or cease to exist.

We are in contact with funders and past and potential future organizations to make these efforts go on and likely we will continue the funded coordination + in-kind partners model. For other structures we may collaborate with other organizations and projects.

If this project and its mission sounds like something you would like to join or support, we have multiple ways to get involved from spreading the word as an ambassador, supporting material and workshop format development, becoming an instructor and teaching with us to organizing workshops with us or on your own. Please contact support@coderefinery.org or join our chat (https://coderefinery.zulipchat.com/) and we can discuss in more detail.