Removing Large Files From You Git Repository

I’ve resisted moving my projects onto GitHub. When GitHub first opened it’s doors, it surprised me. Why would anyone build an UI on top of version control? It just seems like such a simple idea, which had already been done many times over. So what made GitHub different?

GitHub Logo

As it turns out, GitHub is different. They have a wonderful product.

I expected the switch to be uneventful, but things don’t always go as we expect. My previous git provider didn’t have filesize restrictions. During the push into GitHub, I received a warning at 50 megs. At 100 megs, it turned into a roadblock.

Luckily, GitHub has detailed instruction on how to remove the large files.

First, if it’s a pending check-in, you can simply remove the file from the cache.

git rm --cached giant_file
# Stage our giant file for removal, but leave it on disk

Commit the change.

git commit --amend -CHEAD
# Amend the previous commit with your change
# Simply making a new commit won't work, as you need
# to remove the file from the unpushed history as well

Push your changes to GitHub.

git push
# Push our rewritten, smaller commit

If it’s not in a pending check-in, but is a part of your repo’s history things get interesting. There is a utility, BFG Repo-Cleaner, that makes this process a breeze.

The command (from GitHub documentation).

bfg --strip-blobs-bigger-than 50M
# Git history will be cleaned - files in your latest commit will *not* be touched

The GitHub documentation must assume you have BFG install, because the command didn’t work for me.

I downloaded the jar file from and ran it. Don’t forget to be in the root of your git repository.

java -jar bfg.jar --strip-blobs-bigger-than 50M

Here is my output

Scanning packfile for large blobs: 35170
Scanning packfile for large blobs completed in 251 ms.
Found 10 blob ids for large blobs - biggest=125276291 smallest=53640151
Total size (unpacked)=958626718
Found 1691 objects to protect
Found 1 tag-pointing refs : refs/tags/v0.1
Found 7 commit-pointing refs : HEAD, refs/heads/dev, refs/heads/master, ...

Protected commits
-----------------

These are your protected commits, and so their contents will NOT be altered:

 * commit a99dbf81 (protected by 'HEAD')

Cleaning
--------

Found 1093 commits
Cleaning commits:   100% (1093/1093)
Cleaning commits completed in 8,427 ms.

Updating 6 Refs
---------------

Ref  Before After
------------------------------------------------
refs/heads/dev | 02eeab40 | 8ad272d3
refs/heads/master  | a99dbf81 | 8008478b
refs/heads/prod| 15f1558b | dc52efeb
refs/heads/qa  | 15f1558b | dc52efeb
refs/remotes/origin/master | 0c71d31f | d992278d
refs/tags/v0.1 | fc78e278 | ba078ff6

Updating references:100% (6/6)
...Ref update completed in 45 ms.

Commit Tree-Dirt History
------------------------

Earliest  Latest
|  |
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD

D = dirty commits (file tree fixed)
m = modified commits (commit message or parents changed)
. = clean commits (no changes to file tree)

Before After
-------------------------------------------
First modified commit | 71ab4035 | 5963444b
Last dirty commit | 48c18598 | d7000b5a

Deleted files
-------------

Filename  Git id
----------------------------------------------------------------
GroverFull-06292014.zip | 2ace978f (117.2 MB)
GroverFull-07142014.zip | 3fb67bc6 (117.8 MB)
GroverFull-07312014.zip | edc34fe0 (118.3 MB)
GroverFull-08012014.zip | cf8b9f19 (118.5 MB)
GroverFull-08292014.zip | a41ce08a (119.5 MB)
Grover_be2.mdb  | 129a7cc8 (61.4 MB)
HELOS.zip   | d730b329 (62.6 MB)
Unify Theme.zip | 5fca437c (53.2 MB), 728b06a4 (51.2 MB)
products-WB0412697.zip  | ebe5f6cf (94.6 MB)

In total, 3191 object ids were changed. Full details are logged here:

C:\Projects\grover.bfg-report\2015-07-21\13-31-45

BFG run is complete! When ready, run: git reflog expire --expire=now --all && gi
t gc --prune=now --aggressive

Has the BFG saved you time?  Support the BFG on BountySource:  https://j.mp/fund
-bfg