Installing Deflect

Operating System

  • Deflect tarball has been compiled with Debian 7.x amd64

    • Install Debian with ssh access
    • Upgrade the server. In most cases this can be done like:

    sudo -i apt-get update ; apt-get dist-upgrade ; reboot

Deflect Controller

Download the Deflect Tarball

wget http://users.deflect.ca/deflect-v3.0.tar.bz2
md5sum deflect-v3.0.tar.bz2
394af03511919e15aab40f2fa5a38165  deflect-v3.0.tar.bz2

Install Some Prerequisites

Note: Some Debian installs do not install a few things by default and you will get some errors without these. Please check.

apt-get install bzip2 rsync time bind9 nagios3 nagiosgrapher

Setup DNS System

FIXME: bind software setup link.

  • Add your controller and any edges you intend to install to your DNS name server.

Install Deflect

  • Change to install directory location and extract the file.

    cd /usr/local tar jxvf /path/to/deflect-v3.0.tar.bz2

  • Move /usr/local/deflect-v3.0 to /usr/local/deflect.

Note: If you have a old version of deflect installed here, this will need to be moved before moving this to deflect directory.

mv /usr/local/deflect-v3.0 /usr/local/deflect

Create SSL Keys

  • Create your private/public keys for the Edges

Example: First one is for root on the edges and second is for the trafficserver user.

ssh-keygen -b4096 -t rsa -f /usr/local/deflect/etc/edge_id

If you do not need a passphrase for the key ROOTPRIVKEY in etc/deflect.config, press Enter twice to create a key without pass phrase.

ssh-keygen -b4096 -t rsa -f /usr/local/deflect/etc/id_rsa

If you do not need a passphrase for the key TSPRIVKEY in etc/deflect.config, press Enter twice to create a key without passphrase.

Configure Deflect

This is an example etc/deflect.config file.

[default]
# Must use -A. This is a good idea. If you only want to setup one config and ALWAYS use default you can
# comment FORCE_ALT out or set to anything other than 1. Do not confuse yourself, only use in default.
FORCE_ALT=1
# This file is read in from top to bottom. Leave [default] at top so any variable over writting can happen
### local things
# Deflect root
DBASE=/usr/local/deflect
# Edge root for pushing files
EDGEBASE=${DBASE}/edges/default
# File list of ALL edges (live and non-live)
EDGES=${DBASE}/etc/edges/edges.default
# File list of ALL LIVE edges
EDGES_LIVE=${DBASE}/etc/edges/edges.default.live
# Private key for user root (used on edges)
ROOTPRIVKEY=${DBASE}/etc/edge_id
# Public key for user root (used on edges)
ROOTPUBKEY=${DBASE}/etc/edge_id.pub
# Private key for user of traffic server
TSPRIVKEY=${DBASE}/etc/id_rsa
# Public key for traffic server
TSPUBKEY=${DBASE}/etc/id_rsa.pub
# The program name that called this
MYNAME=`basename $0`
# Deploy log file
DPLOGFILE=/var/log/deploy.log
# Bind zone file template location
ZTEMPLATE_DIR=${DBASE}/etc/zones/default
# Current UNIX time. (Seconds elapsed since January 1, 1970 00:00:00 GMT) Used
# to plot serial numbers in bind zones
TIME_S="`date +%s`"
# Default proxied domain to test
DHOST=www.equalit.ie
# Bind (Main Zone Template File)
BIND_MZTFILE=${ZTEMPLATE_DIR}/deflect.ca.zone
# Bind live directory
BIND_LDIR=/var/named
# Bind (Main Zone File)
BIND_MZTFILE=${ZTEMPLATE_DIR}/deflect.ca.zone
# Bind live directory
BIND_LDIR=/var/named
# Bind (Main Zone File)
BIND_MZFILE=${BIND_LDIR}/deflect.ca.zone
#Bind SOA record
BIND_SOA="@    300    IN    SOA   dns0.easydns.com. zone.easydns.com.  $TIME_S   43200  10800  1209600  300"
#Bind NS definitions. If need more edit zonegen.sh and all more here
#BIND_NS_1="@               IN      NS      dns1.easydns.com."
#BIND_NS_2="@               IN      NS      dns2.easydns.net."
#BIND_NS_3="@               IN      NS      dns3.easydns.ca."
BIND_NS_1="@            IN      NS      adns1.easydns.com."
BIND_NS_2="@            IN      NS      adns2.easydns.com."
BIND_NS_3=";;;"
BIND_NS_4=";;;"
BIND_NS_5=";;;"
BIND_NS_6=";;;"
# Location of ATS remap.config file in controller push directory
ATS_RMAP=${EDGESBASE}/usr/local/trafficserver/config/remap.config
#Nagios Host Template File
NAG_HTEMPLATE=${DBASE}/etc/nagios/host.template.cfg
# Nagios Serviceext Template File
NAG_STEMPLATE=${DBASE}/etc/nagios/serviceext.template.cfg
# Nagios Host Group Template
NAG_HGTEMPLATE=${DBASE}/etc/nagios/hostgroups_template.cfg
# File to be read in for nagios monitoring
NAG_EDGES=${DBASE}/etc/hosts.monitor

# remote things
# ATS UID
TSUSER=trafserv
TSUID=99
# ATS GID
TSGROUP=trafserv
TSGID=99
# Location of ATS installed director on edge
TSPATH=/usr/local/trafficserver
# Our network domain of edges
DEFLECT_DOM=deflect.ca
# edge CNAME definition to include all our live edges in DNS
DEFLECT_EDGE=edge
# various prereqs for ATS
TSPREREQS="autoconf automake libtool g++ libssl-dev tcl-dev expat libexpat1-dev libpcre3-dev libcap-dev libhwloc-dev libncurses5-dev libcurl4-openssl-dev libre2-dev libzmq1 libconfig9 libconfig++9 python-swabber"
# various prereqs for nagios
NAGIOS="nagios-plugins nagios-nrpe-server libnagios-plugin-perl libreadonly-perl"
# various prereqs for fail2ban
FAIL2BAN="fail2ban"
# various prereqs for misc. Add anythis extra we want installed here
MISC="sudo rsync ntp iftop tcpdump node virt-what"
REPOSTRING="deb http://users.deflect.ca/repo/ wheezy main"
# Location on misc in push directory
MISC_PATHS="etc/cron.d etc/sudoers"
# Use ENCfs for the trafficserver directory?
ENCFS=0

[your_DNET]
# Location for your_DNET
EDGEBASE=${DBASE}/edges/your_DNET
EDGES=${DBASE}/etc/edges/edges.your_DNET
EDGES_LIVE=${DBASE}/etc/edges/edges.your_DNET.live
ATS_RMAP=${EDGESBASE}/usr/local/trafficserver/config/remap.config
ZTEMPLATE_DIR=${DBASE}/etc/zones/your_DNET
BIND_MZTFILE=${ZTEMPLATE_DIR}/your_DNET.deflect.ca.zone
BIND_MZFILE=${BIND_LDIR}/your_DNET.deflect.ca.zone
DEFLECT_DOM=your_DNET.deflect.ca
ENCFS=0

PATH for Scripts

  • Add PATH to scripts:

    echo “export PATH=$PATH:/usr/local/deflect/scripts” >> ~/.bashrc

Then source the profile so your change takes effect.

source /etc/profile

Sometime this does not work depending on you environment. If the deflect PATH does not get added try this.

source ~/.bashrc

DNET Push Directory

Deflect has the ability to have many DNETs (Deflect Networks) controlled by one controller. It is recommended that you download the template and save to deflect/edges/default and then copy that directory to your dnetname. So we want to start out configuring the basics here.

cd  /usr/local/deflect/edges/
wget http://users.deflect.ca/default_push-v3.0.tar.bz2
md5sum default_push-v3.0.tar.bz2
c6364b50e290aa3b5d746921d83291cb  default_push-v3.0.tar.bz2
tar jxvf default_push-v3.0.tar.bz2
mv default_push_v3 default
cd /usr/local/deflect/edges/default
  • Basic push configuration
    • edit a few files. This will be your default setup that you use for all DNET setups

edit the following files to work with your setup. What counts most is to read the comments and change items, the controller IP address, etc.

  • etc/iptables.deflect
  • etc/nagios/nrpe_local.cfg
  • usr/local/trafficserver/modules/banjax/banjax.conf
  • copy in right etc/localtime for the timezone you want to run in

Dig through all the files and get comfortable where everything is located. When you are happy with your basic setup copy to you live dnet so you can start to add the websites the this dnet will protect.

  • Copy the default push directory to your DNET name

    cd /usr/local/deflect/edges/ cp -a default your_DNET

Nagios Monitor

You will be able to access the nagios monitor web interface from the controller at http(s)://CONTROLLER_DOMAIN_OR_IP/nagios3. If you configured nagios with an administrative password, you should see an apache auth request when visiting the above location.

Configuring Edge Monitoring

In order to register your edge with nagios, add the edges to the file /usr/local/deflect/etc/hosts.monitor, in the format.

edge1.deflect.ca IP_ADDRESS
edge2.deflect.ca IP_ADDRESS

Where IP_ADDRESS is the actual ip address of the edge.

Then run:

nagios_conf.sh -A your_DNET

It is possible that the script does not fully configure nagios and you may need to modify some configurations by tracing down any errors when nagios attempts to start.

Add more documentation for edge monitoring configuration.

Deflect Edge

  • Log into your edge (If you have ssh access you can ssh into your edge for example) .

  • Install Debian Wheezy with ssh access

  • Upgrade the server. In most cases this can be done like:

    sudo -i apt-get update ; apt-get dist-upgrade ; reboot

Note: The deploy.sh script does this for us but have found it is safer to manually do it unless you know it will work with your VPS provider.

  • Go back to your controller.
  • Add your edge to /usr/local/deflect/etc/edges/edges.your_DNET file on controller. This is a list of all your edges controlled by your controller server. Later we will setup edgemanage to do this.

On controller:

cd /usr/local/deflect/etc/edges
bvi edges.your_DNET

add the full host name and save file

Example:

edge1.testnet.deflect.ca
edge2.testnet.deflect.ca
  • Deploy. Example:

    cd /usr/local/deflect/scripts/ deploy.sh -A your_DNET -b -v -H edge2

Notes: (This installs and sets up, approx time is 15 to 30 minutes)

Note: nagios installs samba and the install sometimes asks for a DOMAIN: Not important unless you are using samba. Adding DEFLECT will work

  • Add edge to live edges

    cd /usr/local/deflect/etc/edges bvi edges.your_DNET.live

add the full host name and save file like we did in the edges file.

Configuration

Note: all configuration is done on the controller and pushed to the edges

ATS remap.config

This file is the heart of the Apache Traffic Server configuration. It is used to map URLs to origin server. You can find ours here

cd /usr/local/deflect/edges/your_DNET/usr/local/trafficserver/conf
bvi remap.config

Add your map to this file, specifying the site that will be served, and where the content will come from.

Example:

regex_map       http://^(www\.)?equalit.ie$/        http://example.xxx.deflect.ca/
regex_map       https://^(www\.)?equalit.ie$/       https://example.xxx.deflect.ca/

ATS https (SSL/SNI) ssl_multicert.config

ssl_multicert.config file is the file to do changes to point to the key/cert files for out https with SNI enabled. We put our key/certs in the ATS config directory under a directory called ssl. The directory can be adjusted in records.conf.

Below is a example of ssl_multicert.config

# Equalit.ie
ssl_cert_name=equalitie.crt ssl_key_name=equalitie.key
# Deflect.ca
ssl_cert_name=deflectca.crt ssl_key_name=deflectca.key
# This is how we add the default in case the browser or OS is not able to use SNI or a https map was defined in remap.config but does not have a key/cert
dest_ip=* ssl_cert_name=deflectca.crt ssl_key_name=deflectca.key

Push Configuration

Here we use our deploy.sh script again located under /usr/local/deflect/scripts. The command should be in your PATH.

# view what your options are
deploy.sh -A your_DNET -h
# deploy our ATS change to ALL our edges.
deploy.sh -A your_DNET -p ats -H ALL

Further Security

  • Edge firewall (iptable rules)

    • This can be found in /usr/local/deflect/edges/your_DNET/etc/iptables.deflect

    cd /usr/local/deflect/edges/your_DNET/etc/ bvi iptables.deflect uncomment lines 38 - 42 edit safesrc= and at minimum add your controllers FQDN then comment out lines 45 and 46 deploy.sh -A your_DNET -p firewall -v -H ALL This will push the change to all edges

  • Fail2ban

    • This can be found /usr/local/deflect/edges/your_DNET/etc/fail2ban
    • Add your jail in jail.local
    • Add your filter in filter.d/jail_name.conf

    cd /usr/local/deflect/edges/your_DNET/etc/fail2ban bvi jail.local add your jail information. Examples are in this file. Save cd filter.d bvi jail_name.conf add and save deploy.sh -A your_DNET -p fail2ban -v -H ALL this will push to all edges it is good practice to reload the firewall any time you do changes to fail2ban deploy.sh -A your_DNET -p firewall -v -H ALL