Lean software programming

The Lean software programming is a general method of software development that started in the early 2000s [1]. Primarily adapted from industrial process, it relies on several general principles:

1Eliminate waste
2Amplify learning
3Decide as late as possible
4Deliver as fast as possible
5Empower the team
6Build integrity in
7Optimize the whole

The lean philosophy grew and inspired the Agile software development practices. Agile development started with a Manifesto listing the values and principles to follow :

1 Individuals and interactions over processes and tools
2 Working software over comprehensive documentation
3 Customer collaboration over contract negotiation
4 Responding to change over following a plan

Additional principles were listed alongside, of which I’d like to highlight the followings (that apply best to scientific research work):

  • Projects are built around motivated individuals, who should be trusted.

  • Continuous attention to technical excellence and good design.

  • Simplicity is essential.

  • Regularly, the team reflects on how to become more effective, and adjusts accordingly.

The original goals of the Agile method is to give the “power” back to developers (people doing the work) and let them self-organize, the management being there to give general direction, and lift obstacles. I believe many physicists will find that this goal will resonate with their own aspirations.

Practical implementation

From the general principles of the Lean and Agile manifestos, methods and process have been derived and implemented in software development. Obviously, scientific research is not exactly like developing a software in a commercial context. However, many methods can be used and adapted in our research projects. The general idea is to avoid micromanaging the tasks (and the team members), and having supervisors enabling the work done by the team, which is generally what a Ph.D advisor should do with their student(s).

The first practical idea is to break down large tasks and goals into smaller items (just like a main quest is divided in intermediate goals), organize them by dependencies and priorities, and select the ones to work on first for the coming time period (see below Sprint). The individuals in the team then choose the tasks they will work on (and/or their shift schedule), and the team self-organizes to complete the list of tasks in the given time frame, after which the next set of goals will be determined.

The list of tasks to achieve is called backlog in the Agile methods. Important goals in the backlog could be related to milestones or deliverables in scientific projects. (The backlog is where the quests are listed.)

Note

During the writing of this manuscript, as well as the development of the related code, a backlog-like to-do list has been used.

The unit of time over which the work is done is call a sprint and may last from a few days to a few weeks. The set of goals to achieve during the sprint is fixed, and the team should focus only on these during that time. Hence, at the end of the sprint, a significant step forward in the project should have been reached.

During the sprint, the team organization revolves around the daily meeting [3] and [4] (sometime also called stand-up), which is intended to be limited in length (around 15 minutes). The daily should be an opportunity for all members to states what they did previously, what they plan to do next and what issue they face.

Important

The Sprint / daily organization is well adapted to experiments (limited time, a certain number of tasks, …) or mentoring a student (when specific goals can be set over a sprint time of a week or two, with daily meetings to assess the progress).

In Agile development, the productions of the team are called artifacts, which relates well to what the scientific projects call deliverable. Deliverables are usually less numerous than Agile artifacts, but internally, teams should hold themselves to produce more deliverables (in particular in the context of Open Science where more than just peer-reviewed articles should be published: data and code should also be counted as deliverables).

Footnotes