GIT “Broken Pipe” And “Out of Memory” Error When Pushing to Repository

Last week we spent some time migrating SVN to GIT in BitBucket, most of the processes was quite smooth, but we did encounter some errors while pushing some of our big repositories with lots of history into GIT. We got the following error:

error: RPC failed; result=55, HTTP code = 0
fatal: The remote end hung up unexpectedly

This is due to the fact that we are pushing too much data that exceeds git’s http post buffer which is defined in the doc as:

Maximum size in bytes of the buffer used by smart HTTP transports when POSTing data to the remote system. For requests larger than this buffer size, HTTP/1.1 and Transfer-Encoding: chunked is used to avoid creating a massive pack file locally. Default is 1 MiB, which is sufficient for most requests.

. The fix was quite straightforward, simply fire the following command on you shell:

git config http.postBuffer 524288000

This allows GIT to send up-to 500MB of data to the server, if the error still returns, try to increase this value until it can go through. Of course you need to be aware of how much memory your system has before it will crash.

Remember to reset the value back to its original value, or some value that is more common in everyday commits, according to the docs that I mentioned  above, it is 1MB by default. I forgot to reset the value after my initial imports to GIT and my subsequent commits resulted the following out of memory error:

git-remote-https(52011,0x7fff7011acc0) malloc: *** mmap(size=18446744071938875392) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
git-remote-https(52011,0x7fff7011acc0) malloc: *** mmap(size=18446744071938875392) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
fatal: Out of memory, malloc failed (tried to allocate 18446744071938872320 bytes)

even though I only push a small change to the GIT server, I am not exactly sure why, but reset the value should resolve the issue:

git config http.postBuffer 5242880

I use 5MB just in case.

Make Use of Subversion Changelist

During my development time, I constantly need to only commit partial of all my changes in the current project directory and leave the rest of files untouched in order to fix some problems. Every time I do a “svn diff” or “svn commit”, I will have to specify the list of files that have changes in order to view diff or commit them. Luckily subversion comes with a handy feature that helps you in this senario.

Subversion’s changelist support allows you to create changelists by applying labels to files you want to be associated with that changelist, remove those labels, and limit the scope of the files on which its subcommands operate to only those bearing a particular label. To use this feature, simply issue the following commands:

$ svn changelist minor-fixes file-1.php file-2.php
Path 'file-1.php' is now a member of changelist 'minor-fixes'.
Path 'file-2.php' is now a member of changelist 'minor-fixes'.

Then “svn status” will tell you the changes:

$ svn status

--- Changelist 'minor-fixes':
M      file-1.php
M      file-2.php
$

To see diff:

$ svn diff --changelist minor-fixes

And when you commit, can you apply the same options:

$ svn ci -m "Fix minor bug and leave the rest un-touched"  \
--changelist minor-fixes

Then you can continue working with the rest of the files in the current project.