Collaborations Workshop 2018 (CW18) Report – Culture Change, Productivity and Sustainability

The Collaborations Workshop 2018 (CW18) took place at The School of Mathematics, Cardiﬀ University from the 26th to 28th March 2018. 90 people attended the event to discuss the themes of the workshop – culture change, productivity and sustainability. With a mix of lightning talks, keynotes, Q&A panels, discussions with speed blogging, collaborative ideas sessions, mini workshops & demos, a social programme and a Hack Day in the agenda, it was a feature-packed and immersive event.


Culture Change
Kirstie Whitaker is a well-known open science advocate who works as a Research Fellow at the Alan Turing Institute in London.Her inspirational talk (Whitaker 2018) touched on some of her personal experiences when trying to convince researchers to do things in the open or change her colleagues' minds during the University and College Union (UCU) 2018 industrial action.
Kirstie mentioned three important activities to promote culture change: shape the agenda, mobilise citizens and invest in leaders.All three are important when doing the hard task of "turning around an 800 year-old ship" and you need to engage everyone onboard of the ship.
An important warning included in the talk was that "the idea of letting perfect stand in the way of progress or stand in the way of good is very very very damaging to the open community".People seeking perfection are important for culture change because they are the ones pushing the ship forward, but everyone must remember to be kind, constructive and respectful of the effort other people are also making towards the change.

Productivity
John Hammersley is the CEO and co-founder of Overleaf, an online collaborative writing and reviewing tool for LaTeX documents that helps two million users worldwide to be more productive, and one of the sponsors of Collaborations Workshop.John explained that sharing LaTeX source files by email is not the most efficient way to collaborate due to technical barriers, e.g.users having different versions of LaTeX packages and synchronisation constraints -one author needs to wait for the other to send the document back before working on it again.Having an online central place to collaborate on LaTeX documents makes users more productive.
Researchers' productivity is not only measured by the number of articles they write.The number of times one article is cited also plays an important role in the current system of measuring productivity in research.John presented the audience with a chart (Hammersley 2018), see Fig. 1, from The Royal Society that shows the direct correlation between the number of citations and the number of different countries the co-authors are based in.Given that more international co-authors increase the likelihood of citations, we should invest in lowering the barrier to international collaboration.This is not only limited to access to online platforms but also funding to support, build and maintain international collaborators.

Talks on Sustainability Perspectives
There were three short talks covering different facets related to sustainable software in the context of culture change and productivity.We had Daniel S. Katz (National Center for Supercomputing Applications (NCSA), University of Illinois Urbana-Champaign), Naomi Penfold (eLife) and Matthew Upson (former UK Government Digital Service) speaking and representing the views of researchers, publishers and citizens, respectively.

Researchers' Perspective
Daniel provided updates on the Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE) series and the US Research Software Sustainability Institute (URSSI).WSSSPE is an international community-driven organisation that promotes sustainable research software by addressing challenges related to the full lifecycle of research software through shared learning and community action.Daniel presented a diagram (Katz 2018), see Fig. 2, showing stakeholders of research software and their relationship and how the diagram ended up being covered by the WSSSPE4 working groups and WSSSPE5.1 speed blogs.WSSSPE6.1 will be a one-day workshop during the IEEE eScience 2018 conference, on 29th October 2018 in Amsterdam, and everyone is welcome to attend the event.

Publishers' Perspective
Naomi's talk (Penfold 2018) focussed on the way research is shared, the barriers that need to be removed, and how sharing contributes to sustainability.She highlighted that "shared data is often not machine-readable" because authors formatted it for human reading, see Fig. 3.
Naomi also talked about literate programming -the inclusion of code and data within the article -as one way to write sustainable research narratives and the challenge to longterm preservation of research code.

Citizens' Perspective
Matthew started his talk (Upson 2018) by stating that the UK Government is a great open source user and contributor but this is not uniform across all offices.When you promote open source and you have a lot of data, the natural next step would be to make your data openly available, something that the UK Government already does at https://www.gov.uk/government/statistics.Matthews showed in one of his slides, see Fig. 4, a simplified version of the pipeline to have the data published in gov.ukwhere all the manual and hardto-replicate steps are "hidden".
In 2016-2017, Matthew worked on his (crazy) "Reproducible Analytical Pipelines" idea (Upson 2017) that is now gaining traction in government and focusses on making the data published at gov.uk reproducible.Matthew's take-home message was that sustainability is a social problem, and, if you want to change the landscape, you need to talk a lot about your "crazy" sustainability ideas to people.The organisers expected the productivity debate to expand around practices and softwares that facilitate or deter personal and team productivity -e.g.continuous integration -, but the debate showed that, as Neil Chue Hong summarised, "productivity is defined by you as the person and is something personal to the way that you want to achieve something rather than being something defined as part of a much wider 'this is what the world needs to do'" [sic].The audience contributed with some interesting examples of different perspectives on productivity; for example, not worrying about food because the company where you work provides free catering and the fact that development tools only really work if they are an agreed choice amongst project team members.

Q&A on Sustainable Software Practice
Neil Chue Hong, Institute Director, chaired this Q&A panel consisting of Daniel S. Katz, Aly s Brett, Carina Haupt, Louise Brown and Joseph Parker.The panellists explored their personal experience leading Research Software Engineers teams or other groups/ organisations that provide training and guidance on sustainable software practice.Carina reminded the audience of Kirstie Whitaker's take-home message: every little helps because incremental progress is still progress.
The panellists mentioned the importance of highlighting success stories -e.g.The Carpentries long-term assessment results (Jordan et al. 2017) showed that people are doing things differently and being more productive.The importance of listening to your users was also highlighted by more than one panellist, focussing on the fact that listening could save many hours of software development.

Lightning talks
CW18's lightning talks offered a quick (read two minutes) presentation covering a topic from someone very closely involved with the work.Lightning talks are an excellent way to seed potential collaborators and introduce areas of interest to those attending.With 90 attending, it's not possible or desirable that everyone presents a lightning talk.However, we had 32 lightning talks in total.There were an array of talks from pet software projects, simple rules, life experience to future events -all very much in line with the workshop themes.A scan through the slides (Software Sustainability Institute 2018) is a great way to get up to speed with what was presented.

Discussion Sessions
These sessions give attendees a chance to discuss matters of interest within the scope of the workshop themes.The possible topics are collected during registration and during the workshop to allow both long-standing and nascent ideas to surface.If questions from previous CW's appear, we add them and the associated blog(s) to the CW FAQ page.Table 1 has the topics available for discussion at CW18.Groups of four to six people had 45 minutes to discuss their topics and then 45 minutes to write a blog post about it.This method of creating outputs from the discussion sessions is called speed blogging.

Topic Sub Topic
Culture Change What should a culture change case study include?
How do we influence more senior staff to promote culture change?Four speed blogs were published as outputs of the discussion sessions: "Sharing reproducible research -minimum requirements and desirable features", "Establishing RSE groups at different levels", "Five failed tests for scientific software" and "Archiving code and software shared with research: journal, author and re-user perspectives".

Collaborative Ideas
The collaborative ideas session at CW18 brought together researchers software engineers and people who work in similar domains.We created the groups, and each group had an hour to identify a problem they faced in their work or domains that could use better software and then outline a proposed solution, preferably something they could make progress on in the 24 hours of the Hack Day.There are strong networking and social aspects to the Collaborative Ideas sessions: they are much more about coming up with a Hack Day idea as getting to know and working with people from different institutions, focus and career stages.After the collaborative ideas session, the ideas are printed, and attendees get to vote on their preferred ideas.The winners were the "Providing evidence for software and data citations in published papers".Runner-ups were "Simplifying the use of JupyterLab as a teaching tool" (second place), "Automated Software-Assisted Provisioning" (third place).Each member of one of the winning teams was awarded an Amazon voucher.

Mini-workshops & Demos
Mini-workshops and demo sessions are 30-minute parallel sessions that allow more depth on a particular topic, tool, or technique.CW18 hosted mini-workshops and demos on Overleaf, citation file format, scientific code manifesto, (research) software engineering initiatives and testing.In addition to that and in the spirit of being an unconference, James Baker decided to chair an additional session to look at the newly released JupyterLab.
The full list of mini-workshops and demos is in Table 3.At the end of the Hackday, each team had five minutes to explain how they met the judging criteria and show the features and functionality of their new project.Was the entry novel?Was it potentially impactful?Were best practice in infrastructure used?Were the demo and presentation informative?Was their transparency in what had been done and what was the future potential?
It was a challenge for the judges as they were choosing among excellent entries, and all teams put in masterful efforts.The winners were "Productivity Approximator" -each team member won a Raspberry Pi 3 kit + Google AIY Voice Kit + sensor packs full size.Runnerups were "Keeping up with the Joneses: Providing evidence for software and data citations in published papers" (second place), "Code is Science Manifesto" (third place) and "Carpentry Translation Infrastructure (CTI)" (fourth place).

Social Programme
In addition to the workshop dinner at Jurys Inn Cardiff, the Institute also organised a few walks around the centre of Cardiff offering a change of scene for those attending and gave them an opportunity to explore local culture.The city centre tour before the workshop dinner was lead by Nikoleta E. Glynatsi, who also facilitated the visit to Bute Park on the morning of the second day, and to Alexandra Gardens on the afternoon of the 27th March.
The Collaborations Workshops social programme is beneficial for attendees, see Fig. 5, as it allows them to explore the city which they would otherwise miss while networking or working hard at the event.They get to know more about the place from our local contacts; thus getting a deeper insight into the places where CW takes place.

Conclusions and Take-Home Messages
Collaborations Workshop 2018 was a great success, described by attendees as "a great opportunity to learn about programming for research" and "the best ideas-sharing workshop around".At the end of three immersive days with many conversations about best practice and tooling, it was clear that there is now an increased focus on interoperability and human factors in software sustainability.Many discussions concluded with the idea that making tools work together would increase productivity, that culture change is inherently about people, and that if we get the human factors right (e.g.providing resources, training and support to allow people to advocate for better software practice), then the change around research software is more likely to be successful.
Many people came together to offer advice, resources and help running the workshop in addition to the Institute staff.Without them (and the rest of the CW delegates), the Collaborations Workshop would not be able to take place.
We extend our thanks to everyone who contributed to making CW18 a successful event: • Our sponsors: figshare and Overleaf

Figure 1 ."
Figure 1."Such collaboration helps the work spread" slide from John Hammerley's presentation during Collaborations Workshop 2018.

Figure 3 .
Figure 3. Data presentation in spreadsheets slide from Naomi Penfold's presentation during Collaborations Workshop 2018.

Figure 4 .
Figure 4. gov.uk report pipeline slide from Matthew Upson's presentation during Collaborations Workshop 2018.
Interactively explore the similarities and differences between how you can accomplish the same task in different languages Understand, translate, or even refactor Fortran code Simplifying the use of JupyterLab as a teaching tool Code review across projects for software sustainability Teaching programming to researchers across all research domains Education and career choices in the absence of relevant or up-to-date information Finding the right library to fit your project Time to download the entire dataset and pre-process it locally
Table 2 has the list of collaborative ideas that were submitted.
Can we work together and crowdsource to provide some standard responses and examples?Automated Software-Assisted Provisioning Documentation of research software, its use and connections to datasets and narrative research outputs

Table 3 .
Mini-workshops and Demos Sessions hosted at Collaborations Workshop 2018 short space of time often within a new environment and language.Having worked effectively with a new set of people often translates into future collaborations.