IMAP and SMTP over HTTP Proxy

The solution that I’m using for allowing Thunderbird (and if you really want, Kmail) to connect to my employer IMAP and SMTP servers, is not straightforward but it simply works…

For this to work you really need an external server where you can connect through ssh. This ssh server must be able to contact and connect to the required mail server, namely accessing their IMAP and SMTP ports.

Right now, I use a Linux VPS, located somewhere in the world ( 🙂 ) that I’ve bought through the lowendbox.com site. Great price per year (around 2€ per month).

I run ssh on this server on a non standard port.

The trick is simple:

Just open up two terminal sessions, and if you have ssh through corkscrew tunnelling working (see my previous posts: https://primalcortex.wordpress.com/2014/02/19/ssh-over-http-proxy/ ), it’s simple as executing this:

On terminal 1 and for IMAP (secure):

ssh -L 1993:imap.server.com:993 -p 12345 mysshserver

where imap.server.com is the name or external IP of the IMAP server and 993 is the secure IMAP port. The 1993 is the port at the local address 127.0.0.1 that is listening to connections from thunderbird. The -p 12345 is the port that my remote ssh server is running on and listening on for connections, and of course, mysshserver is the dns or ip address for the ssh server.

On terminal 2 and for secure SMTP:

ssh -L 1465:smtp.server.com:465 -p 12345 mysshserver.

When this two connections are established, then the local machine ports 1993 and 1465 connect through ssh and corkscrew tunnelling to the mail server… and thunderbird can now work as it should.

Just use as IMAP server the localhost and port 1993, and as SMTP server the localhost and 1465 port.

Of course for thunderbird to work, first is needed to create the tunnels.

SSH over HTTP Proxy that uses NTLM Authentication

As can be read on my post https://primalcortex.wordpress.com/2014/02/19/ssh-over-http-proxy/ we can use SSH to connect to a remote client, even when there is between the client and the server a HTTP Proxy.

But some proxys,like Microsoft ISA or Forefront, can require authentication, but only using the NTLM protocol for authentication and nothing else.

In this case the solution is to use TWO proxys where one of them is running on your own machine, that provides and negotiates the NTLM authentication to ISA/Forefront, and allows Firefox, Chrome and corkscreew to connect the internet using those proxys.

So what you need?

1) Install the cntlm proxy on your machine: apt-get install ctnlm

2) Edit the ctnlm.conf config file to config it: the upstream proxy and credentials. This file is normally located in /etc.

3) For example add/edit the following lines:

Username  mydomainusername
Domain  MSDomainName
Password cleartextpasswordP
Proxy upstreamproxy:port
Listen cntlmproxylistenport

A “real example”:

Username PrimalCortex
Domain  ACME
Password itsasecret
Proxy  corp_proxy.acme.com:8080
Listen 3128

Now, the cntlm proxy can be started: as root start the proxy /etc/init.d/cntlm start

Now you can point your clients to the local address 127.0.0.1:3128  (the port defined in the Listen config property), and the proxy access is automatic with the NTLM authentication running in the background.

So now corkscrew can work through a proxy that requires NTLM authentication, just edit the SSH config file and change the proxy address to the localhost and cntlm port:

Host 1.2.3.4
  ProxyCommand corkscrew 127.0.0.1 3128 %h %p

and that’s it.

SSH over HTTP Proxy

Using SSH to connecting to an host when an HTTP Proxy is between the client and the host, can not be done directly without some configuration.

On Linux based machines the solution is to install and run corkscrew, a program that can tunnel the SSH protocol through an HTTP Proxy.

So how to do the configuration?

1) First install the corkscrew program with your package manager. On Ubuntu family: apt-get install corkscrew

2) Then you need to configure SSH to use corkscrew when connecting to the host that has a http proxy between.

3) Goto to your home directory and change to the hidden directoy .ssh within a command shell window.

4) Create or edit a file named config. The name is just config. No extensions.

5) Add the following lines to the config file

Host <IP_of _remote_host>  
 ProxyCommand corkscrew <IP_of_HTTP_Proxy> <HTTP_Proxy_Port> %h %p <auth_file>

Where the <IP_of_remote_host> is the public ip address of the host where you wish to connect.

The <IP_of_HTTP_Proxy> and <HTTP_Proxy_Port>  are the IP address and Port of you local http proxy server that you wish to go through.

And finally, if your proxy server requires authentication, by username and password, just give a complete path to a file where Proxy credentials are stored, for example /home/primalcortex/.corkscrew_auth

This file content must be something like:

username:password

For example a complete config file example:

Host 78.0.1.3
    ProxyCommand 192.168.1.1 8080 %h %p /home/primalcortex/.corkscrew-auth

and the .corkscrew-auth file:

myproxyuser:rtwertjwe4

6) Just connect now:

ssh myremoteuser@78.0.1.3

or when not using the default ssh port:

ssh -p 12345 myremoteuser@78.0.1.3

7) Done!

So why we need this?

Well, first is of course, to access a remote machine, but ssh can forward local ports to remote ports, and this is important because, with this feature we can use Thunderbird to directly connect to a remote server by using the standard IMAP and SMTP protocols through an HTTP proxy.

NSLU2: Using sftp-server with bash as shell

I’ve installed the bash shell on my Linksys NSLU2, and replaced the root’s shell with bash by editing the /etc/passwd file.

It worked fine, until I noticed that scp’ing to the NSLU2 didn’t work anymore… Reverting back to the sh shell, scp started to work again… The error was right after startup of scp it replied with “Connection closed”. No info on logs, what so ever.

I’m running SlugOS, and according to this thread: http://tech.groups.yahoo.com/group/nslu2-general/message/6537 it looked like a bash version bug.

Indeed I did have version 3.2-r2 installed… and so it would not work.

Because I have the optware repository enabled ( http://www.nslu2-linux.org/wiki/Optware/Slugosbe) I’ve checked that the bash version on this repository was newer. So:

– ipkg remove bash

– ipkg-opt install bash

– cp /opt/bin/bash /bin/bash

And while mantaining an open session, changed the shell to bash on one user and tried to logon. It worked.

And scp? It worked also, so with my finger’s crossed, I’ve changed again the root shell to bash, and logon on a new session. No problem. Indeed scp worked right away also.

So if any problems with scp failing, just upgrade your bash version.

Key words: sftp-server bash connection closed failed error 0

Kubuntu as an Xwindows Server

When on Windows, I normaly use the XLive CD to connect to my Unix machines (they can be Linux and Solaris) and access remotely the X applications with their output showing on my Windows PC.

Well I’ve moved my work computer from XP to Kubuntu 8.04, and since then I struggle to do the same as I did with the XLive CD.

Basically with the XLive CD this is how it works:

– On your windows PC after starting the XLive CD, the X server starts and bind to the TCP X port to hear incoming connections. These incoming connections come from the client I try to access.

– On the client side I just do and export=ip_my_windows_machine:0.0 and start the X application.

– That’s it.

On Kubuntu I’ve tried several configurations to allow the KDE display manager to listen to network connections. I’ve searched a lot, changed the xorg.conf file, the kdm.rc file, and yes it didn’t work….

Well the issue was that I needed a paradigm change in how to connect to my client machines. I took me a while to see that I do not need to fiddle with my KDE configuration…

So how you can you do it?

Simple:

– Connect to your client machine through ssh (You’re using ssh, righ? ) and edit the file sshd.conf or ssh_config (It depends on your version) located in /etc/ or one of it’s subdirectories like sshd.

– Edit the file and make sure that you have a line ForwardX11 yes

– Save and restart the ssh deamon.

Now on your workstation instead of connecting with just ssh user@IP use ssh -X user@IP. With this command all X connections will be forwarded automatically to your Kubuntu desktop, just like you did an export DISPLAY and so on.

X-Windows access on your Windows PC

Sometimes you need to run graphical Unix/Linux applications on your Windows client PC. Text access is easy, there are several ssh clients available, and windows has a telnet client. What about X-Windows?

The solution is XLiveCD: http://xlivecd.indiana.edu/

This is a CD that just makes your Windows PC into a X-Windows client (Correct terminology would be Server… 😉 ). With this you can just start a ssh session into your remote Unix/Linux box, and from there just run any graphical application that you need. The application will display along your other Windows applications.

By the way if you have Nero Image or something similar, you even don’t need to burn a CD…

Also as a bonus the CD has a complete CYGWin set of commands, and so you also get tail, grep, uniq…

Cool utilities

Secure shell (for non commercial uses): http://www.ssh.com/support/downloads/secureshellwks/non-commercial.html

I use it to connect to my NSLU2. By clicking an icon on the tool bar it opens up a sftp  session in a window much like Windows Explorer with drag and drop support for file transfer.

Wireshark (etherreal): http://www.wireshark.org/ -> Cool network tracer tool that is multi platform. Trouble guessing what is going on that SOAP request? This is the tool. Not for the fainted of heart…