aaronadams.ca

My little corner of the Web

6 notes

Git push with submodules: a how-to guide

This article is intended as a spiritual successor to Abhijit Menon-Sen’s excellent primer, Using Git to manage a website.

It is broken up into three parts; skip ahead to part 3 if you’re just looking for the solution!

  1. Using Git to manage a website
  2. Using Git to manage a website with submodules: the wrong way
  3. Using Git to manage a website with submodules: the right way

Using Git to manage a website

Like a lot of developers, my workflow consists of developing locally (MAMP PRO + Coda 2 for me), tracking changes with Git, and then pushing files to my web server with git push.

You’ll hear a lot of fear and loathing in the Git community about never pushing to a non-bare remote. So, originally, that’s how I had things set up.

First, on the remote server, I created a bare repository:

[aaron@aaronadams]$ cd ~/private/git/
[aaron@aaronadams]$ mkdir staging.git && cd staging.git
[aaron@aaronadams]$ git init --bare
Initialized empty Git repository in /var/www/vhosts/aaronadams.ca/private/git/staging.git/

Next, I set up a post-receive hook, to automatically check out the repository into my web server’s staging directory every time it receives a push:

[aaron@aaronadams]$ cat > hooks/post-receive
#!/bin/sh

# Read standard input or hook will fail
while read oldrev newrev refname
do
:
done

# Set Git environment
export GIT_WORK_TREE=/var/www/vhosts/aaronadams.ca/sites/staging.aaronadams.ca

# Force checkout
git checkout --force
[aaron@aaronadams]$ chmod +x hooks/post-receive

Finally, in my local repository, I configured the remote, and did my first push:

Aaron$ git remote add staging aaron@aaronadams.ca:private/git/staging.git
Aaron$ git push staging master

This was easy, and it worked well. It wasn’t perfect, as it required each repository to have a unique post-receive hook file; but that was easy enough to manage.

Eventually, I decided I wanted to use CodeIgniter as a submodule, to more easily keep it synchronized with the 3.0-dev branch on GitHub; and I wanted the remote server to automatically update the submodule with each git push.

And that’s where everything started to implode.

Using Git to manage a website with submodules: the wrong way

First, I tried appending git submodule update to hooks/post-receive:

[aaron@aaronadams]$ cat >> hooks/post-receive
# Force update submodules
git submodule update --init --recursive --force

But this caused an error:

Aaron$ git push staging
remote: You need to run this command from the toplevel of the working tree.

Sigh. Really? I specified a working tree. Why do I need to be in it?

Fine, whatever. Let’s revise hooks/post-receive, adding a line that changes the current directory (and exits on failure):

[aaron@aaronadams]$ cat > hooks/post-receive
#!/bin/sh

# Read standard input or hook will fail
while read oldrev newrev refname
do
:
done

# Set Git environment
export GIT_WORK_TREE=/var/www/vhosts/aaronadams.ca/sites/staging.aaronadams.ca

# Force checkout
git checkout --force

# Change directory; exit on failure
cd $GIT_WORK_TREE || exit

# Force update submodules
git submodule update --init --recursive --force

Okay, let’s try that git push again.

Aaron$ git push staging
remote: fatal: Not a git repository: '.'

Crap.

It turns out that, by default, when a hook runs, GIT_DIR=. – so as soon as we change the current directory, Git doesn’t know where the repository is anymore.

So let’s revise post-receive again, this time explicitly setting GIT_DIR to point to our repository:

[aaron@aaronadams]$ cat > hooks/post-receive
#!/bin/sh

# Read standard input or hook will fail
while read oldrev newrev refname
do
:
done

# Set Git environment
export GIT_DIR=$(pwd)
export GIT_WORK_TREE=/var/www/vhosts/aaronadams.ca/sites/staging.aaronadams.ca

# Force checkout
git checkout --force

# Change directory; exit on failure
cd $GIT_WORK_TREE || exit

# Force update submodules
git submodule update --init --recursive --force

Let’s try again. What else could possibly go wrong?

Aaron$ git push staging
remote: fatal: working tree '/var/www/vhosts/aaronadams.ca/sites/staging.aaronadams.ca' already exists.
remote: Clone of 'git://github.com/EllisLab/CodeIgniter.git' into submodule path 'codeigniter' failed

It looks like setting GIT_WORK_TREE is messing up the submodule update.

So, let’s see if we can refactor post-receive, in order to work around the need to set GIT_WORK_TREE:

[aaron@aaronadams]$ cat > hooks/post-receive
#!/bin/sh

# Read standard input or hook will fail
while read oldrev newrev refname
do
:
done

# Set Git environment
export GIT_DIR=$(pwd)

# Change directory; exit on failure
cd /var/www/vhosts/aaronadams.ca/sites/staging.aaronadams.ca || exit
# Force checkout git checkout --force # Force update submodules git submodule update --init --recursive --force

One last git push. What fun new errors will we encounter this time?

Aaron$ git push staging
remote: fatal: This operation must be run in a work tree
remote: fatal: /usr/local/libexec/git-core/git-submodule cannot be used without a working tree.

Git doesn’t realize it’s in a work tree anymore, and everything fails.

Having now lost several hours to this problem, I’d officially lost my patience. I’m sure there was a way to get this configuration working, but I couldn’t help wondering: why not just come up with a simpler configuration?

Here is that simpler configuration.

Using Git to manage a website with submodules: the right way

First, let’s create a universal post-receive hook, one that I won’t need to change on a per-repository basis:

[aaron@aaronadams]$ cat > /usr/local/share/git-core/templates/hooks/post-receive.sample
#!/bin/sh
#
# An example hook script to update the working tree, including its
# submodules, after receiving a push.
#
# This hook requires core.worktree to be explicitly set, and
# receive.denyCurrentBranch to be set to false.
#
# To enable this hook, rename this file to "post-receive".

# Read standard input or hook will fail
while read oldrev newrev refname
do
:
done

# Unset GIT_DIR or the universe will implode
unset GIT_DIR

# Change directory to the working tree; exit on failure
cd `git config --get core.worktree` || exit

# Force checkout
git checkout --force

# Force update submodules
git submodule update --init --recursive --force
[aaron@aaronadams]$ chmod +x /usr/local/share/git-core/templates/hooks/post-receive.sample

Now let’s go ahead and break all the rules.

We’re going to initialize a non-bare Git repository, right in our website directory; make sure it can receive from git push; explicitly set its working tree to its parent directory; and enable our hook we just created.

[aaron@aaronadams]$ cd /var/www/vhosts/aaronadams.ca/sites/staging.aaronadams.ca
[aaron@aaronadams]$ git init && git config --bool receive.denyCurrentBranch false && git config --path core.worktree ../ && mv .git/hooks/post-receive.sample .git/hooks/post-receive
Initialized empty Git repository in /var/www/vhosts/aaronadams.ca/sites/staging.aaronadams.ca/.git/

Finally, on our local machine, we’ll change our remote to reflect the location of our new repository, and push.

[aaron@aaronadams]$ git remote set-url staging aaron@aaronadams.ca:sites/staging.aaronadams.ca
[aaron@aaronadams]$ git push staging master
remote: Submodule 'codeigniter' (git://github.com/EllisLab/CodeIgniter.git) registered for path 'codeigniter'
remote: Cloning into 'codeigniter'...
remote: Submodule path 'codeigniter': checked out 'fd24adf31255822d6aa9a5d2dce9010ad2ee4cf0'
To aaron@aaronadams.ca:sites/staging.aaronadams.ca
 * [new branch]      master -> master

Holy crap, it worked!

Not only is this method compatible with submodules, it also requires just one command to set up a new remote repository (which, okay, consists of four commands). It also keeps the repository and the working tree in the same place; and with no absolute paths required in our configuration or hook files, it’s now completely portable as well.

Please let me know if you encounter any issues with this configuration, or if you’d like to recommend any improvements! And thank you to all the contributors to the litany of Stack Overflow articles that helped me figure all of this out.

Filed under nerd plesk git

  1. brendonwbrown reblogged this from aaronadams
  2. ronnorthrip reblogged this from aaronadams
  3. aaronadams posted this