Git: submodules

You can use submodules for nesting other repositories within the folder structure of your own repository. The most common reason for wanting to do this is that your project has dependencies on other code bases (libraries for example), and you want to track those from their original sources rather than duplicating the files within your own repository. By default, submodules will add the subproject into a directory named the same as the repository.

If you run git status, you’ll notice a few things. First you should notice the new .gitmodules file. This is a configuration file that stores the mapping between the project’s URL and the local subdirectory you’ve pulled it into. If you have multiple submodules, you’ll have multiple entries in this file. It’s important to note that this file is version-controlled with your other files, like your .gitignore file. It’s pushed and pulled with the rest of your project. This is how other people who clone this project know where to get the submodule projects from. When you clone such a project, by default you get the directories that contain submodules, but none of the files within them yet.

The submodule directory is there, but empty. You must run two commands: git submodule init to initialize your local configuration file, and git submodule update to fetch all the data from that project and check out the appropriate commit listed in your superproject. There is another way to do this which is a little simpler, however. If you pass --recursive to the git clone command, it will automatically initialize and update each submodule in the repository.

Working on a Project with Submodules

It is possible to work on the code in the submodule at the same time as you’re working on the code in the main project (or across several submodules).

When you run the git submodule update command to fetch changes from the submodule repositories, Git would get the changes and update the files in the subdirectory but will leave the sub-repository in what’s called a ‘detached HEAD’ state. This means that there is no local working branch (like ‘master’, for example) tracking changes. So any changes you make aren’t being tracked well.

To set up your submodule, you need to go into each submodule and check out a branch to work on. Then you need to tell Git what to do if you have made changes and then git submodule update --remote pulls in new work from upstream. The options are that you can merge them into your local work, or you can try to rebase your local work on top of the new changes.

# If there was a change on the server for this submodule: it gets merged in
# If you forget the --merge, you're local changes will be lost (if another user changed the same committed file)
git submodule update --remote --merge

If you haven’t committed your changes in your submodule and you run a submodule update that would cause issues, Git will fetch the changes but not overwrite unsaved work in your submodule directory. If you made changes that conflict with something changed upstream, Git will let you know when you run the update. You can go into the submodule directory and fix the conflict just as you normally would.

So, if we commit in the main project and push it up without pushing the submodule changes up as well, other people who try to check out our changes are going to be in trouble since they will have no way to get the submodule changes that are depended on.

In order to make sure this doesn’t happen, you can ask Git to check that all your submodules have been pushed properly before pushing the main project. The git push command takes the --recurse-submodules argument which can be set to either ‘check’ or ‘on-demand’. The check option will make push simply fail if any of the committed submodule changes haven’t been pushed.

git push --recurse-submodules=check
git push --recurse-submodules=on-demand

Check will give us some helpful advice on what we might want to do next. The simple option is to go into each submodule and manually push to the remotes to make sure they’re externally available and then try this push again. The on-demand value, which will try to do this for you.

Leave a Reply