Git push with submodules: a how-to guide
This article is intended as a spiritual successor to Abhijit Menon-Sen’s excellent primer, Using Git to manage a website.
It is broken up into three parts; skip ahead to part 3 if you’re just looking for the solution!
- Using Git to manage a website
- Using Git to manage a website with submodules: the wrong way
- Using Git to manage a website with submodules: the right way
Using Git to manage a website
Like a lot of developers, my workflow consists of developing locally (MAMP PRO + Coda 2 for me), tracking changes with Git, and then pushing files to my web server with git push.
You’ll hear a lot of fear and loathing in the Git community about never pushing to a non-bare remote. So, originally, that’s how I had things set up.
First, on the remote server, I created a bare repository:
[aaron@aaronadams]$ cd ~/private/git/ [aaron@aaronadams]$ mkdir staging.git && cd staging.git [aaron@aaronadams]$ git init --bare Initialized empty Git repository in /var/www/vhosts/aaronadams.ca/private/git/staging.git/
Next, I set up a post-receive hook, to automatically check out the repository into my web server’s staging directory every time it receives a push:
[aaron@aaronadams]$ cat > hooks/post-receive #!/bin/sh # Read standard input or hook will fail while read oldrev newrev refname do : done # Set Git environment export GIT_WORK_TREE=/var/www/vhosts/aaronadams.ca/sites/staging.aaronadams.ca # Force checkout git checkout --force [aaron@aaronadams]$ chmod +x hooks/post-receive
Finally, in my local repository, I configured the remote, and did my first push:
Aaron$ git remote add staging aaron@aaronadams.ca:private/git/staging.git Aaron$ git push staging master
This was easy, and it worked well. It wasn’t perfect, as it required each repository to have a unique post-receive hook file; but that was easy enough to manage.
Eventually, I decided I wanted to use CodeIgniter as a submodule, to more easily keep it synchronized with the 3.0-dev branch on GitHub; and I wanted the remote server to automatically update the submodule with each git push.
And that’s where everything started to implode.
Using Git to manage a website with submodules: the wrong way
First, I tried appending git submodule update to hooks/post-receive:
[aaron@aaronadams]$ cat >> hooks/post-receive # Force update submodules git submodule update --init --recursive --force
But this caused an error:
Aaron$ git push staging remote: You need to run this command from the toplevel of the working tree.
Sigh. Really? I specified a working tree. Why do I need to be in it?
Fine, whatever. Let’s revise hooks/post-receive, adding a line that changes the current directory (and exits on failure):
[aaron@aaronadams]$ cat > hooks/post-receive #!/bin/sh # Read standard input or hook will fail while read oldrev newrev refname do : done # Set Git environment export GIT_WORK_TREE=/var/www/vhosts/aaronadams.ca/sites/staging.aaronadams.ca # Force checkout git checkout --force # Change directory; exit on failure cd $GIT_WORK_TREE || exit # Force update submodules git submodule update --init --recursive --force
Okay, let’s try that git push again.
Aaron$ git push staging remote: fatal: Not a git repository: '.'
Crap.
It turns out that, by default, when a hook runs, GIT_DIR=. – so as soon as we change the current directory, Git doesn’t know where the repository is anymore.
So let’s revise post-receive again, this time explicitly setting GIT_DIR to point to our repository:
[aaron@aaronadams]$ cat > hooks/post-receive #!/bin/sh # Read standard input or hook will fail while read oldrev newrev refname do : done # Set Git environment export GIT_DIR=$(pwd) export GIT_WORK_TREE=/var/www/vhosts/aaronadams.ca/sites/staging.aaronadams.ca # Force checkout git checkout --force # Change directory; exit on failure cd $GIT_WORK_TREE || exit # Force update submodules git submodule update --init --recursive --force
Let’s try again. What else could possibly go wrong?
Aaron$ git push staging remote: fatal: working tree '/var/www/vhosts/aaronadams.ca/sites/staging.aaronadams.ca' already exists. remote: Clone of 'git://github.com/EllisLab/CodeIgniter.git' into submodule path 'codeigniter' failed
It looks like setting GIT_WORK_TREE is messing up the submodule update.
So, let’s see if we can refactor post-receive, in order to work around the need to set GIT_WORK_TREE:
[aaron@aaronadams]$ cat > hooks/post-receive #!/bin/sh # Read standard input or hook will fail while read oldrev newrev refname do : done # Set Git environment export GIT_DIR=$(pwd) # Change directory; exit on failure cd /var/www/vhosts/aaronadams.ca/sites/staging.aaronadams.ca || exit
# Force checkout git checkout --force # Force update submodules git submodule update --init --recursive --force
One last git push. What fun new errors will we encounter this time?
Aaron$ git push staging remote: fatal: This operation must be run in a work tree remote: fatal: /usr/local/libexec/git-core/git-submodule cannot be used without a working tree.
Git doesn’t realize it’s in a work tree anymore, and everything fails.
Having now lost several hours to this problem, I’d officially lost my patience. I’m sure there was a way to get this configuration working, but I couldn’t help wondering: why not just come up with a simpler configuration?
Here is that simpler configuration.
Using Git to manage a website with submodules: the right way
First, let’s create a universal post-receive hook, one that I won’t need to change on a per-repository basis:
[aaron@aaronadams]$ cat > /usr/local/share/git-core/templates/hooks/post-receive.sample #!/bin/sh # # An example hook script to update the working tree, including its # submodules, after receiving a push. # # This hook requires core.worktree to be explicitly set, and # receive.denyCurrentBranch to be set to false. # # To enable this hook, rename this file to "post-receive". # Read standard input or hook will fail while read oldrev newrev refname do : done # Unset GIT_DIR or the universe will implode unset GIT_DIR # Change directory to the working tree; exit on failure cd `git config --get core.worktree` || exit # Force checkout git checkout --force # Force update submodules git submodule update --init --recursive --force [aaron@aaronadams]$ chmod +x /usr/local/share/git-core/templates/hooks/post-receive.sample
Now let’s go ahead and break all the rules.
We’re going to initialize a non-bare Git repository, right in our website directory; make sure it can receive from git push; explicitly set its working tree to its parent directory; and enable our hook we just created.
[aaron@aaronadams]$ cd /var/www/vhosts/aaronadams.ca/sites/staging.aaronadams.ca [aaron@aaronadams]$ git init && git config --bool receive.denyCurrentBranch false && git config --path core.worktree ../ && mv .git/hooks/post-receive.sample .git/hooks/post-receive Initialized empty Git repository in /var/www/vhosts/aaronadams.ca/sites/staging.aaronadams.ca/.git/
Finally, on our local machine, we’ll change our remote to reflect the location of our new repository, and push.
[aaron@aaronadams]$ git remote set-url staging aaron@aaronadams.ca:sites/staging.aaronadams.ca [aaron@aaronadams]$ git push staging master remote: Submodule 'codeigniter' (git://github.com/EllisLab/CodeIgniter.git) registered for path 'codeigniter' remote: Cloning into 'codeigniter'... remote: Submodule path 'codeigniter': checked out 'fd24adf31255822d6aa9a5d2dce9010ad2ee4cf0' To aaron@aaronadams.ca:sites/staging.aaronadams.ca * [new branch] master -> master
Holy crap, it worked!
Not only is this method compatible with submodules, it also requires just one command to set up a new remote repository (which, okay, consists of four commands). It also keeps the repository and the working tree in the same place; and with no absolute paths required in our configuration or hook files, it’s now completely portable as well.
Please let me know if you encounter any issues with this configuration, or if you’d like to recommend any improvements! And thank you to all the contributors to the litany of Stack Overflow articles that helped me figure all of this out.