Welcome to the 69th edition of Git Rev News, a digest of all things Git. For our goals, the archives, the way we work, and how to contribute or to subscribe, see the About Git Rev News page on git.github.io.

This edition covers what happened during the month of October 2020.

• [PATCH] userdiff: support Bash

  Victor Engmark sent a patch to add support for Bash and POSIX shell to the userdiff mechanism. This mechanism is used by the diff code to make diffs more informative and better suited to the content, in this case Bash or POSIX shell programs.

  As explained in the documentation, diffs contain sections called hunks that look like:

  @@ -k,l +n,m @@ TEXT
  

  where k, l, n and m are numbers indicating the lines that are concerned, and TEXT, which is called the "hunk-header", is a part of a line of the file to further help identify the related context of the diff in the file.

  The best thing for programs is often to have the name of the enclosing function, or method, in the hunk-header. As detecting functions is programming language specific, it's the role of the userdiff mechanism to provide a regex (regular expression) that can be used to do that.

  Another role of the userdiff mechanism is to provide a regex to customize word diffs.

  Victor's patch then mainly consisted in adding regexes for Bash and POSIX shells to userdiff.c, along with some documentation and a lot of tests.

  Junio Hamano, the Git maintainer, replied to Victor, by commenting a bit on the tests and a lot on the complex regex to detect a function. He wondered if it correctly accepts white spaces where they are allowed, and suggested for more clarity to break it down like this:

  "^[ \t]*"                    /* (op) leading indent */
  "("                          /* here comes the whole thing */
  "(function[ \t]+)?"          /* (op) noiseword "function" */
  "[a-zA-Z_][a-zA-Z0-9_]*"     /* identifier - function name */
  "[ \t]*(\\([ \t]*\\))?"      /* (op) start of func () */
  "[ \t]*(\\{|\\(\\(?|\\[\\[)" /* various "opening" of body */
  ")",
  

  Victor then sent a version 2 of his patch implementing Junio's suggestions and answering his comments.

  Johannes Sixt then suggested a number of improvements especially about the tests, and about the regex to customize word diffs. Victor replied about the latter asking for pointers to look at, as it seemed that there were no tests for that.

  Victor nevertheless sent a version 3 implementing Johannes' suggestions, and Johannes indeed replied that he was happy with the result. Junio was also happy with the result after fixing a typo in the commit message.

  The patch was later merged to the next and then master branches, so Git should soon support shell scripts in a better way, while it itself has been developed for a long time using shell scripts.

• Who are you and what do you do?

  I'm a scientist at the Canadian Center for Meteorological and Environmental Prediction, in Montréal, Québec, Canada. I'm part of the development team for the numerical model we use to predict the movement, growth and melt of sea-ice, the frozen parts of the Arctic and Antarctic oceans. Like most computer models in the field of weather forecasting, this software is written in good old Fortran, and runs on our supercomputers. We use Git extensively to track changes across all layers of our technological stack, and I quickly developed an interest in Git's inner workings.

• What would you name your most important contribution to Git?

  Since I started contributing to the project a little more than one year ago, I've mostly been trying to fix the bugs I encounter in the course of my daily work. So I'm not sure which one of my topics has had the biggest impact on other users. However, I can say that the contribution I'm most proud of is fixing git checkout --recurse-submodules to correctly switch between branches when one branch has nested submodules and the other branch has no submodules at all. I learned enormously during the process of developing this fix, not only about how Git "unpack trees" to keep the index, working directory and HEAD consistent, but also how fork and exec calls work and especially how to debug such spawned processes using GDB and LLDB.

• What are you doing on the Git project these days, and why?

  Right now I'm working on a fix to prevent git checkout --recurse-submodules from losing uncommitted work in submodules. Although the documentation says this shouldn't happen, I've found a few cases where it does, and since it's never a nice experience to lose work, I'd like to fix that.

• If you could get a team of expert developers to work full time on something in Git for a full year, what would it be?

  Apart from rewriting the whole thing in C++, you mean? Jokes aside, I would like for more work to be put towards better submodule support. There was a colossal effort a couple of years ago to add --recurse-submodules flags to several Git commands, so that submodules worktrees stay up to date when switching between superproject commits. Unfortunately this effort has died off due to core contributors to the submodule code changing jobs, and some porcelain commands still lack that capability.

• If you could remove something from Git without worrying about backwards compatibility, what would it be?

  I think it's unfortunate that the dot-dot vs. dot-dot-dot syntax mean different things in git diff than in the rest of Git commands. It's another one of those tricky things that users have to remember. The fact that checkout and reset have so many different modes of operation also make them confusing for beginners. The introduction of git switch and git restore should help with that, though.

• What is your favorite Git-related tool/library, outside of Git itself?

  Since I've heard of it I've been using diff-so-fancy as a way to make Git diffs more easily readable. Apart from that I mostly stick to the Git command line. Recently I discovered git-crecord, an ncurses interface for, among other features, interactive line-by-line staging.

• Git 2.29.2
• Git for Windows 2.29.2(2), 2.29.2(1)
• GitHub Enterprise 2.22.4, 2.21.12, 2.20.21, 2.22.3, 2.21.11, 2.20.20, 2.19.26
• Bitbucket Server 7.8
• Gerrit Code Review 3.2.5.1, 3.2.5, 3.1.10, 3.0.15, 2.16.25, 2.15.21, 2.14.22
• GitLab 13.6.1, 13.6, 13.5.4, 13.5.3, 13.5.2
• GitKraken 7.4.0
• GitHub Desktop 2.6.0, 2.5.7

Various

• Announcing the Release of the Git Experience in Visual Studio; the new Git tooling is now the default source control experience in Visual Studio 2019, beginning with version 16.8.
• Kernel.org manager Konstantin Ryabitsev announced that signed git pushes are now supported by the kernel.org system. Among other things, push signatures can be preserved in a commit transparency log, which is also now provided by kernel.org.

Light reading

• Git before GitHub by Tarun Batra is a beginner-friendly article explaining how to submit patches using git's built-in tools without using a platform like GitHub.
• How to keep your Git history clean with interactive rebase by Tobias Günther on GitLab Blog. Interactive rebase is one of Git's most versatile tools. Here's how to use it to correct commit messages, fix mistakes, and more.
• How I remember everything I learn by Aurelio Ogliari; he uses his personal wiki, which is nothing more than a Git repo (on GitHub), divided in folders with a bunch of READMEs in each of them.
• GitHub Actions for semi-static web sites by Dave Cross on Dev.To explains how to keep a site up-to-date where there are only a few ways that it can change, like CPAN Dashboard. (GitHub Actions, then beta, were covered in Git Rev News #44).
• What adding CI to 30 projects revealed by Gabor Szabo on Dev.To. Among others it found developer forgetting to add file to Git repository, or pushing some changes, or that build script depends on unreleased code or local configuration, or that not all dependencies were declared.
• Nbdev: A literate programming environment that democratizes software engineering best practices by Hamel Husain on GitHub Blog. (See also nbdev's mention in the 'Git tools and sites' section below.) nbdev builds on top of Jupyter notebooks to encompass the entire software development life cycle: automated generation of docs, Continuous Integration (CI), writing tests directly in notebooks, etc. Many of those features are integrated with GitHub (GitHub Actions, GitHub Pages, GitHub Codespaces).
• Git is simply too hard: a fancified excerpt of Mislav Marohnić's unpopular opinion on Go Time #153 podcast. See also Gitless, the experimental reworking of Git, based on research on difficulties in Git, mentioned in Git Rev News #20.
• Gerrit Code Review fixes two important security vulnerabilities, Issue 13621 and Issue 13514. They have been found during the test migrations of the OpenDev Code Review service to the latest version v3.2. The issue was particularly sensitive for all those setups hosted in Europe, because of the potential exposure to other people's personal information, such as e-mail and external ids, and was impacting only those very old setups that had recently migrated to v2.14 or later.

Git tools and sites

• repositorch is a VCS repository analysis engine written in C#, which supports Git. It allows to get some basic and advanced statistics including LOCs, Burndown, Code ownership, Defect density and more.
• Githru is an interactive visual analytics system that enables developers to effectively understand the context of development history through the interactive exploration of Git metadata. It uses novel techniques (graph reconstruction, clustering, and Context-Preserving Squash Merge (CSM) methods) to abstract a large-scale Git commit graph.
• nbdev is a library that allows you to develop a Python library in Jupyter Notebooks, putting all your code, tests and documentation in one place. (See also nbdev's mention in the 'Light reading' section above.) Among other features it provides a robust, two-way sync between notebooks and source code, and tools for merge/conflict resolution with notebooks in a human readable format. It is set-up out of the box for integration with GitHub Pages and GitHub Actions. Compare with nbdime -- a tool for diffing and merging of Jupyter Notebooks, mentioned in Git Rev News #37, and jupytext -- a plugin for Jupyter that can save Jupyter notebooks as Markdown documents, or Julia, Python or R scripts, or Review Notebook GitHub App (free for Open Source and Education).
• Flagsmith (formerly Bullet Train) is an open source service to help release features with confidence by managing feature flags across web, mobile and server side applications. Hosted service is free for smaller projects.
• Tara.AI is a product development service that helps provide clarity around the release cycle by connecting Git to tasks and sprints (with GitHub integration). Base tier free forever plan has unlimited tasks, sprints and workspaces, with no user limits; paid for features are planned.
• Git Credential Manager Core (GCM Core) is a secure Git credential helper built on .NET Core that runs on Windows and macOS (Linux support is in an early preview). It replaces the .NET Framework-based Git Credential Manager for Windows (GCM), and the Java-based Git Credential Manager for Mac and Linux (Java GCM).
• Git-completion is a friendly fork of the official Git completion and prompt scripts for Zsh and Bash. It's a sister project of the Oh My Zsh gitfast plugin.
• git-crecord is a git subcommand (written in Python), which allows users to interactively select changes to commit or stage using a ncurses-based text user interface.
• ForgeFed - Federation Protocol for Forge Services. ForgeFed (formerly GitPub) is an upcoming decentralized federation protocol based on the W3C's ActivityPub, which extends ActivityStream 2.0. It provides a server to server API for pull request, forking and subscription of repositories provided by Git web services (services like GitLab, Gogs, Gitea or SourceHut).
• Visualizing Git Concepts with D3 is a website designed to help you understand some basic Git concepts visually, with sandboxes split by specific Git commands and a few specific real-world scenarios; adding/staging your files for commit is not covered.

This edition of Git Rev News was curated by Christian Couder <christian.couder@gmail.com>, Jakub Narębski <jnareb@gmail.com>, Markus Jansen <mja@jansen-preisler.de> and Kaartic Sivaraam <kaartic.sivaraam@gmail.com> with help from Philippe Blain, Semyon Kirnosenko, Tarun Batra, Philip Oakley and Luca Milanesio.