Thursday, June 18, 2009

Using Github Through Draconian Proxies (Windows And Unix)

Here is a pretty standard scenario at most corporations:

- All access to the internet is restricted to a proxy
- The proxy only allows connections out on port 80 and 443
- CONNECT method is only enabled for 443
- Proxy Authentication is required (NTLM or Basic)

I like to use both Windows and Unix environments. On Unix tunneling to Github is a bit easier because lots of tools are included.

Unix

1. Download Git. At the time I was writing this I was using Ubuntu so I simply did apt-get install git-core

2. Download and install corkscrew (http://www.agroman.net/corkscrew/). This is a tool for tunneling SSH through HTTP proxies.

3. Edit or create the file ~/.ssh/config and put the following:

ProxyCommand /usr/bin/corkscrew proxy.example.com 443 %h %p ~/.ssh/myauth

Host github.com
User git
Port 22
Hostname github.com
IdentityFile "/media/truecrypt1/Keys/GitHubKey.private"
TCPKeepAlive yes
IdentitiesOnly yes

Host ssh.github.com
User git
Port 443
Hostname ssh.github.com
IdentityFile "/media/truecrypt1/Keys/GitHubKey.private"
TCPKeepAlive yes
IdentitiesOnly yes


  • The ProxyCommand is invoked when ssh needs to make a connection. We are telling ssh to use /usr/bin/corkscrew. This is a 3rd party program that sets up a socket through the HTTP proxy.
  • The program /usr/bin/corkscrew takes as its 5th argument a file containing credentials for your HTTP proxy. Not all proxies need authentication but if you do just put in the file a single line formatted username:password.
  • The Host github.com indicates to ssh that if we are connecting to github.com to use these specific settings. There is nothing special here except we specify the location of the private key that corresponds to the public key we had over in http://www.github.com/
  • Notice we have another entry titled "Host ssh.github.com" . This is to get around proxies that only allow the CONNECT command over 443 (the truly locked down ones). To get around this github setup a whole separate host that listens on port 443. We add both entries here since they are both valid.
4. If everything is setup correctly you should be able to run:
# ssh github.com

Hi tachang! You've successfully authenticated, but GitHub does not provide shell access.
Connection to github.com closed.

If this doesn't work you can run
# ssh ssh.github.com

And get the exact same thing. If the first command didn't work it means you are using a proxy that blocks CONNECT on port 22. Almost no proxies block CONNECT on port 443 because you need that for SSL.


We get a no shell access message from github because we are trying to obtain a shell and github has it disabled. However this indicates everything is working. This concludes the setup for Unix.

------------------------

Windows


1. Download msysgit http://code.google.com/p/msysgit/

Some settings:

- "Run Git from the Windows Command Prompt"
- "Use OpenSSH" (this one is very important)
- Pick your line endings

2. Download connect.c
http://bent.latency.net/bent/darcs/goto-san-connect-1.85/src/connect.html

This tool deserves its own post mostly because of its utter simplicity. It mirrors the open source tool corkscrew and is used for tunneling through proxies. Yes the tool's name is really called "connect.c".

For Window's users, a pre-compiled binary is available:
connect.exe
I put my connect.exe in C:\Windows\connect.exe

3. Decide whether you like to use the Windows cmd.exe to do stuff or the Cygwin style shell. Or both.

Cygwin Git Bash Shell
For the Cygwin style shell start up the Git icon and edit the file ~/.ssh/config
*Make sure the file has no extension.

Put the following in that file:

ProxyCommand /c/windows/connect.exe -H username@proxy.example.com:443 %h %p

Host github.com
User git
Port 22
Hostname github.com
IdentityFile "/c/Keys/GitHubKey.private"
TCPKeepAlive yes
IdentitiesOnly yes

Host ssh.github.com
User git
Port 443
Hostname ssh.github.com
IdentityFile "/c/Keys/GitHubKey.private"
TCPKeepAlive yes
IdentitiesOnly yes

  • Notice the slash style in order to access the file system.
  • The proxy username is specified as part of the proxy setting. The password for the proxy is prompted for. Read more about connect.c to figure out how to get rid of this prompt.
At this point, using the Git Bash shell should yield:

$ ssh github.com

Hi tachang! You've successfully authenticated, but GitHub does not provide shell access.
Connection to github.com closed


Windows cmd.exe shell

Suppose you don't like the Git Bash shell. You prefer the cmd.exe interpreter.

- Go to your config file at C:\Documents and Settings\\.ssh\config
- Make a copy of it or make a new one. I called mine config-windows

Put the following in the file:


ProxyCommand C:/Windows/connect.exe -H username@proxy.example.com:443 %h %p

Host github.com
User git
Port 22
Hostname github.com
IdentityFile "C:\Keys\GitHubKey.private"
TCPKeepAlive yes
IdentitiesOnly yes

Host ssh.github.com
User git
Port 443
Hostname ssh.github.com
IdentityFile "C:\Keys\GitHubKey.private"
TCPKeepAlive yes
IdentitiesOnly yes

  • Notice the mixture of slash styles. I find this rather odd but it is what works. We have a forward slash style for the ProxyCommand but for the IdentityFile a forward slash or backward slash both work.
Running the command (making sure we run Git\bin's ssh.exe and not some other one in the PATH):

C:\Program Files\Git\bin>ssh.exe -F "C:\Documents and Settings\\.ssh\config-windows" github.com

Hi tachang! You've successfully authenticated, but GitHub does not provide shell access.
Connection to github.com closed


General Git Cloning

- Make sure you are using the right Git URL:

Suppose your Public Clone URL is: git://github.com/tachang/EyeFiServer.git

You should use the following URL that utilizes the SSH transport:

git clone ssh://git@github.com:22/tachang/EyeFiServer.git
git clone ssh://git@ssh.github.com:443/tachang/EyeFiServer.git

38 comments:

JW said...

Does corkscrew support SOCKS etc like connect.c?

Jeff Tchang said...

@JW

Corkscrew doesn't but that isn't really its purpose. Once you have ssh+corkscrew you can use the -D switch in SSH to create a SOCKS server.

-D basically creates a SOCKS server on a port you specify. So if you wanted to make a SOCKS server on 1080 you do:

ssh -D 1080 host.example.com

Bruno said...

Nice guide.

For those who already have an http server listening on the 443 port, you can use OpenVPN's port-share feature. It seem it don't work on Windows.

--port-share host port
When run in TCP server mode, share the OpenVPN port with another application,
such as an HTTPS server. If OpenVPN senses a connection to its port which is
using a non-OpenVPN protocol, it will proxy the connection to the server at
host:port. Currently only designed to work with HTTP/HTTPS, though it would
be theoretically possible to extend to other protocols such as ssh.
Not implemented on Windows.

Justin said...

You can do this with PuTTY if you like too - http://blog.codeslower.com/2008/8/Using-PuTTY-and-SSL-to-securely-access-GitHub-repositories-via-SSH

Jeff Tchang said...

@Justin

I tried to do it the way it indicates in that link but could not get it to work.

Something with pageant kept screwing things up. Also does that work if you use normal cmd.exe?

Justin said...

@Jeff

I use it from cmd.exe all the time. One tricky bit is that you have to supply the name of a PuTTY *Profile* when specifying the remote server. Maybe that is it? Feel free to email me offline - jgbailey at codeslower.com.

Elia said...

Can somebody send me a copy of corkscrew because (obviously) my proxy/firewall cuts out that site entirely...

Thanks in advance :)

knicker said...

FYI for whomseever is looking for this

I cannot access gitorious.org/github.com repositories when using ssh on the mac leopard (10.5.7) with rsa keys. DSA keys work fine.

Shin Guey said...

ProxyCommand /c/windows/connect.exe -H username@proxy.example.com:443 %h %p

Is it compulsory to use 443 as the proxy port? If I have a http proxy which has port 911, should I put this as the proxy port?

Bruno said...

@Shin

You only need to use the port 443 if every others ports are not allowed at your working place or school.

443 is https so it's allowed most of the time.

Shin Guey said...

@Bruno,

I am getting confused. At my office we have to use the http proxy to browse website and the proxy is something like proxy02.png.com:911(which will set on each browser also)

So, for the ProxyCommand, it should use the http proxy proxy02.png.com:911 right? Because all external website need to go through this proxy include the https 443.

Shin Guey said...

Hi Jeff Tchang,

Really, really thanks for your guide. I am now able to use Github through the proxy on 443 port now.

*Your guide is clear but I skip all the unix setup guide and directly start from the windows setup. Those I miss some detail which been told during setting up the unix. Now, everything work fine and I'm happy :)

Frank said...

Has anyone gotten this to work with an authenticating proxy on windows? How would we specify a domain account for the username? I don't think it is:

ProxyCommand /c/windows/connect.exe -H domain\username@proxy.example.com:443 %h %p

Jeff Tchang said...

@Frank

I have in my ~/.ssh/config file:

ProxyCommand /usr/local/bin/corkscrew proxy.example.com 80 %h %p /home/jeff/.ssh/auth


So basically I am pointing to a file called auth for my authentication where I stored my password.

Inside auth I put a single line:

jeff:password

I guess you can try

domain\jeff:password or domain\\jeff:password if you need to escape the backslash.

Frank said...

@Jeff

Thanks for the response. Actually looking at it some more, connect.c supports basic but not digest authentication. It doesn't say anything about negotiate/NTLM authentication, which is what I need. But given it doesn't support digest, I can't imagine it supports NTLM. :( Thanks anyhow

Kshitij Kunte said...

I want to clone a public repository on sourceforge.net . How can I use a modification of this method for cloning a git repository on sourceforge .

Thanks .

Kshitij Kunte said...

I wish to clone a public repository on sourceforge.net . How can I modify the above method to work with sourceforge ?

Thanks .

dejan said...

Does anyone have a solution on how to clone a proxy using git:// over a proxy?
I am using an authenticated proxy, have setup env var HTTP_PROXY but it still fails.
Any comments would be appriciated.

امیرعلی said...

I can't download connect.c/exe because links on the site are broken, any idea?

James Carr said...

Yeah.... connect.c cannot be downloaded anymore. ARGH!

mike_price said...

Try this:
www.meadowy.org/~gotoh/ssh/connect.c

ProggerPete said...

I've tried following the instructions, but when I type ssh github.com or ssh ssh.github.com I get the message..

ssh: Could not resolve hostname ssh.github.com: hostname nor servname provided, or not known.

Any ideas?

Jeff Tchang said...

Hey pete,
Are you working on windows or linux? Double check your .ssh/config file and make sure you have the entries right. Lots of other people were able to get it so don't lose hope! Email me directly if you need more help.

ProggerPete said...

Hey Jeff, thanks for trying to help me via Email. Thought I'd share the solution I came up with, which was to run CNTLM as a proxy.

CNTLM knows how to talk to NTLM proxies and the standard proxy tools in msysgit know how to talk to CNTLM.

I didn't get non-ssh working, but I can now

git clone git@ssh.github.com:ProggerPete/proj.git

So I'm happy.

Jeff Tchang said...

Just a quick update that I found a copy of connect.exe (compiled for Windows) and put it on my Dropbox:

http://dl.dropbox.com/u/2177278/connect.exe

Testers Google said...

Hi Tchnag,
I am using windows 7.
in your example it is given as username@proxy.example.com:443 %h %p

what should i put in username?
also Git bash is not recognising proxy.example.com.
I am from testing background can you please help??

Testers Google said...

Hi Tchang,

In your example username@proxy.example.com:443 %h %p

what should be put in username also Git bash does not understand proxy.example.com

Testers Google said...

Hi Tchnag,
I am using windows 7.
in your example it is given as username@proxy.example.com:443 %h %p

what should i put in username?
also Git bash is not recognising proxy.example.com.
I am from testing background can you please help??

Jeff Tchang said...

You should connect with what you authenticate to the proxy with.

Proverbs said...

@Jeff,

Thanks for the hack, but my problem isn't actually a proxy but a firewall blocking the port. So, how do I edit the config file in this case, to get around the firewall.

thanks!

Jeff Tchang said...

Are you able to visit HTTPS sites? If you are then your firewall is not blocking port 443.

Proverbs said...

@ Jeff

Sorry, I meant a firewall blocking port 22 not port 443. In this case how will I manipulate the config file since there isn't any proxy server!

Jeff Tchang said...

You can just omit the ProxyCommand lines in step 3. Also remove the entry for the Host github.com. Just use the ssh.github.com.

Then try to do
ssh ssh.github.com and see if it works.


Host ssh.github.com
User git
Port 443
Hostname ssh.github.com
IdentityFile "/media/truecrypt1/Keys/GitHubKey.private"
TCPKeepAlive yes
IdentitiesOnly yes

Proverbs said...

Hello Jeff, it works fine now thanks. But I am having a slight problem with authentication. github seems not to be recognizing my passphrase and even when I left my passphrase as "empty" it still denied me access.
What to do?

Thanks for the help.

Jeff Tchang said...

Make sure that your SSH public key is registered on Github. Empty passphrases are only when you generate an ssh-key and don't want to type it in all the time. Double and triple check you've entered the SSH public key on github.

Proverbs said...

still the same problem, see the debug logs I got while running the ssh -vT command

debug1: Server accepts key: pkalg ssh-rsa blen 277
debug1: PEM_read_PrivateKey failed
debug1: read PEM private key done: type
Enter passphrase for key '/c/Users/Proverbs/.ssh/id_rsa.pub':
debug1: PEM_read_PrivateKey failed
debug1: read PEM private key done: type
Enter passphrase for key '/c/Users/Proverbs/.ssh/id_rsa.pub':
debug1: PEM_read_PrivateKey failed
debug1: read PEM private key done: type
Enter passphrase for key '/c/Users/Proverbs/.ssh/id_rsa.pub':

Jeff Tchang said...

Why are you reading the public key? You need to be reading the private key when you connect. E-mail me directly if you want to troubleshoot more.

Proverbs said...

Now working!!! Merci beaucoup!