Spring Netflix Fegin, an alternative to the classic RestTemplate

I’ve been using the Netflix OSS stack for a while now; and I have to say I’m really impressed. These guys know how to do OpenSource and they know how to do it well.

Create a client to consume an external resource via fiegn? Easy

@FeignClient(name = "TodoClient", url = "https://api.todo.com/v1", configuration = TodoClientConfig.class)
public interface TodoClient {

	@RequestMapping(method = RequestMethod.GET, value = "/external-pojo")
	public List<Todo> findAll();
}

@Configuration
public class TodoClientConfig {

	@Bean
    public Decoder decoder() {
        ObjectFactory<HttpMessageConverters>objectFactory = () -> new HttpMessageConverters(new MappingJackson2HttpMessageConverter());
        return new ResponseEntityDecoder(new SpringDecoder(objectFactory));
    }
}

Need to add some authentication tokens to the header? pfft piece of cake:

@Component
public class AuthRequestInterceptor implements RequestInterceptor {

	@Override
	public void apply(RequestTemplate template) {
		template.header("my-header", "header-token");
	}
}

Checkout the Github Repo and the Spring Cloud Netflix Project.

Upload to S3 using Python

Uploading files to S3

A quick and dirty solution to uploading non-hidden files in the current directory to an S3 bucket using Python, works well as a post commit hook in git 🙂

Note: You will need to set your AWS access keys as environment variables in addition to specifying the name of the S3 bucket you wish to upload to. This script works for small files only.

import os
import glob
import boto
import sys
import os
from boto.s3.key import Key

AWS_ACCESS_KEY_ID = os.environ['AWS_ACCESS_KEY_ID']
AWS_SECRET_ACCESS_KEY = os.environ["AWS_SECRET_ACCESS_KEY"]

bucket_name = '<<YOUR_BUCKET_NAME_HERE>>'
conn = boto.connect_s3(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
bucketobj = conn.get_bucket(bucket_name)
k = Key(bucketobj)

def main():
    files = listdir_nohidden('.')
    for f in files:
        upload(f[2:].strip(), f)

def listdir_nohidden(path):
    return glob.glob(os.path.join(path, '*'))

'''''''''''''''
ONLY SMALL FILES KB! MAX
'''''''''''''''
def upload(name, path):
    print "Uploading File: name = " + name + ", path = " + path
    k.key = name
    k.set_contents_from_filename(path, cb=percent_cb, num_cb=10)
    print "Done\n"

def percent_cb(complete, total):
    sys.stdout.write('.')
    sys.stdout.flush()

if __name__ == "__main__": main()

Poor Man’s VPN – SSH Port Forwarding to Access Remote Resources – Linux Guide

Overview

In this article I will demonstrate how you can setup your own poor man’s virtual private network to access remote resources that are protected behind a firewall. You could potentially use this to access your office computer or your home computer anywhere!

Note – If you are in an environment where you have no say in matters of firewall rules relating to remote access then you should consult the network administrator before  trying to implement what’s in this article, chances are they will think you are up to no good and may kill outbound connections coming from resources protected behind the firewall.

So what’s the problem?

Say you wanted access to a particular resource remotely, for example you are stuck abroad and you want to access your office’s web server running your latest and greatest web application, but you know access to that particular machine is restricted behind a firewall.

What we can do to overcome this is to create a secure straight through tunnel from a remote machine to the machine that is protected behind the firewall through an intermediary server, a Proxy Server.

Unless you can negotiate something with the network administrator, chances are you cannot bypass the firewall and as a result you will not be able to access that particular resource remotely, i.e. you will not be able to ping it from a remote location as it will be assigned a local address.

Using SSH Port Forwarding as a solution

SSH Port forwarding creates a secure connection between a local computer and a remote machine through which services can be relayed.

For this particular example we will use two types of port forwarding schemes:

1. Remote Port Forwarding: This forwarding scheme will be by the machine that sits behind the firewall.
2. Local Port Forwarding:  The forwarding scheme that will be used by a remote machine.

Remote Machine ---> Proxy Server <---| Machine Behind Firewall 
 
-->    Local Forwarding Scheme 
<--    Remote Port Forwarding
|      Firewall

From the tunnels that we have created we can create an additional tunnel that would allow us to tunnel through to the machine behind the firewall.

For more information on Port Forwarding, you can refer to these links:

1. Ubuntu Help
2. Wikipedia

What do I need, Show me an example!?

For example I have some files I would like to view on my office’s computer.

I can access those files remotely because I have enabled remote access over SSH on port 22. This is a trivial example but can be generalised to other services that run on other ports for example a web-server running on port 80.

The first thing I would need to do is have an intermediary server that would relay the SSH on port 22 using any arbitrary free port.

1 – Setup a Proxy Server running OpenSSH

You should have access to an intermediary machine that is connected to the internet, we will use this machine to establish tunnels on both ends, i.e. A connection from the machine that is behind the firewall and a connection from a remote machine.

You could use an EC2 instance on AWS for example, this is made easier as AWS generates SSH key pairs on instance creation, these keys can be used to authenticate to the EC2 instance. If you are not using EC2 you could generate your own  pairs  using ssh-keygen tool.

Follow these instructions to install and configure OpenSSH on Linux.

By default OpenSSH is not configured to listen interfaces connected to the internet and thus we must specify this in the /etc/ssh/sshd_config file which is the system-wide configuration file for OpenSSH, this allows us to set options that modify the operation of the daemon.

At the bottom of the file add the following configuration:

 GatewayPorts yes

You would need to restart the service in order for these config changes to take place:

/etc/init.d/ssh restart

2 – Establish a Remote Port Forwarding Scheme (Firewall Protected Machine)

Now that we have setup our proxy server, we will need to establish the tunnels. The first tunnel that we will establish is a tunnel that is going from the machine that is protected behind the firewall i.e. establish a connection from the office machine to the proxy server.

Proxy Server <---|-- Resource Behind Firewall

We will setup a Remote Port Forwarding scheme to achieve this. In order to enable a remote port forwarding you would need to run ssh with the -R flag as follows:

 ssh -f -R 2222:localhost:22 user@proxy-server -i "path/to/proxy-server/key" -N

In the above snippet we are remotely forwarding any traffic from remote port 2222 to local port 22 which is the default ssh port. That means any traffic coming out of port 2222 of the proxy server will be mapped to port 22 locally.

3 – Establish a Local Port Forwarding Scheme (Remote Location)

Next we will need to establish a Local port forwarding scheme. Local port forwarding will forward  traffic from a specific local port to another port on a remote server.

We will enable local port forwarding by using the -L flag as follows:

ssh -f -L 1234:localhost:2222 user@proxy-server -i "path/to/proxy-server/key" -N

In the above snippet we are locally forwarding any traffic from local port 1234 to port 2222 on the proxy server.

Remember in the previous step we mapped port 2222 on the proxy server to 22. That means any traffic coming out of port 1234 from the remote machine will transitively reach port 22 of our machine that is behind the firewall.

All that is left to do is tunnel through using the port 1234.

4 – Tunnel Through

Run ssh with -p flag, where port is 1234

ssh -p 1234 remote-user@localhost

At this stage you should be challenged for the password, once successfully authenticated, you should see the content of your remote machine locally.

How can I improve this, I want to make this more secure?

In this article I only covered the basics of port forwarding however, these basics can be used as a basis to create a secure VPN. Cloud compute services such as AWS provide a Identity and Access Management offering, alongside the IAM service one could code use the API to whitelist a set of IP addresses that can connect to proxy servers.