16 KiB

Czysty Bezpośredni odnośnik Wina Historia

Per-platform Installation

All installations are currently being built from fresh sources for reasons of development and keeping up to date with core, evolving features. As such, installations make take extensive time to build, especially on slower machines.

Raspbian

sudo apt-get install -y git
git clone https://github.com/alecmuffett/eotk.git
cd eotk
./opt.d/build-raspbian-stretch.sh

Ubuntu 20.04LTS

Install a ubuntu-20.04.2-live-server-amd64.iso server instance; and then:

git clone https://github.com/alecmuffett/eotk.git
cd eotk
./opt.d/build-ubuntu-20.04.sh

Ubuntu 18.04LTS

Install a ubuntu-18.04.2-live-server-amd64.iso server instance; and then:

git clone https://github.com/alecmuffett/eotk.git
cd eotk
./opt.d/build-ubuntu-18.04.sh

CentOS 8.2.2004

Install a minimal server and then:

sudo yum -y install git
git clone https://github.com/alecmuffett/eotk.git
cd eotk
./opt.d/build-centos-8.2.2004.sh

macOS Mojave

Install Homebrew; and then:

git clone https://github.com/alecmuffett/eotk.git
cd eotk
./opt.d/build-macos-mojave.sh

FreeBSD 12.1

Install a base server and then:

pkg install git
git clone https://github.com/alecmuffett/eotk.git
cd eotk
./opt.d/build-freebsd-12.1.sh

Other Platform Requirements

EOTK requires recent tor and also recent openresty with the ngx_http_substitutions_filter_module NGINX module enabled. They simply need to be accessible in $PATH. The above "build" scripts simply supply that.

Demonstration And Testing

After installation, you can do:

./eotk config demo.d/wikipedia.tconf
- or: ./eotk config demo.d/wikipedia-v3.tconf
./eotk start wikipedia
./eotk maps -a # and connect to one of the onions you've created

Be aware that you will suffer from HTTPS certificate errors until you buy a HTTPS certificate.

Creating HTTPS Certificates for Testing & Development

When connecting to the resulting onions over HTTP/SSL, you will be using wildcard self-signed SSL certificates - you will encounter many "broken links" which are due to the SSL certificate not being valid.

This is expected and proper behaviour; there are currently three ways to address this.

Install `mkcert`

The best solution for development purposes is to install mkcert onto the machine which will be running EOTK and configure your own personal Certificate Authority for the certificates that you will need.

You can then add set ssl_mkcert 1 to configurations, and your mkcert root certificate will be used to sign the resulting onion certificates. You can install that certificate into your local copy of Tor Browser; of course it will not work for anyone else.

Visit `/hello-onion/` URLs

The old solution was/is much more manual: EOTK will use OpenSSL to create certificates, and then, for any onion - eg: www.a2s3c4d5e6f7g8h9.onion - EOTK provides a fixed url:

https://www.a2s3c4d5e6f7g8h9.onion/hello-onion/

...which (/hello-onion/) is internally served by the NGINX proxy and provides a stable, fixed URL for SSL certificate acceptance; inside TorBrowser another effective solution is to open all the broken links, images and resources "in a new Tab" and accept the certificate there.

In production, of course, one would expect to use an SSL EV certificate to provide identity and assurance to an onion site, rendering these issues moot.

Buy a SSL Certificate from HARICA or Digicert

See below.

Buying a HTTPS Certificate from a Certificate Authority

If you choose to buy an Onion HTTPS certificate from (e.g.) HARICA, what will happen, and what will you need to do?

You will need to create a CSR (Certificate Signing Request)

I chose to buy:

a Server Certificate
with Domain-Level (DV) Trust
with a reasonable duration
using the in-browser generated CSR
using the ECDSA algorithm at 256 bits
important: remember the password!
important: download the private key!

The HARICA website provides an in-browser method of generating a CSR, if you are not already familiar. Make sure to generate a good passphrase, and remember it, because you will need it soon. Also: make sure to download the privateKey.pem file that is offered, and keep it in a safe place.

If you manually create the CSR on the server, you'll use the 'onionaddress.key' file generated by openssl.

You will need to prove ownership of the site, to the CA

For example: HARICA will tell you that you need to post a secret key at a particular URL on your onion site; the message will read something like:

Place the file FILENAME to http://ONIONADDRESS.onion/.well-known/pki-validation/

...and they will offer you a file to download.

Download this file, and open it with a text editor; the content will be a long secret string, like THISISAREALLYLONGHEXADECIMALSECRET

Add a line to your EOTK configuration, substituting the values where necessary:

set ssl_proof_csv /.well-known/pki-validation/FILENAME,THISISAREALLYLONGHEXADECIMALSECRET

Then do something like:

eotk config projectname.conf && eotk nxreload projectname

...to install the URL handlers.

HARICA has a process of validation which involves generating an onion-csr. You won't need to add anything to your configuration.

Optional: what if you have multiple Onion addresses?

You can put multiple path,value strings into ssl_proof_csv, space-separated; use trailing backslashes to put entries onto separate lines:

set ssl_proof_csv \
    /.well-known/pki-validation/key1,value1 \
    /.well-known/pki-validation/key2,value2 \
    /.well-known/pki-validation/key3,value3

Optional: what if your multiple "proof" URLs all have the SAME pathname?

The ssl_proof_csv hack works if all the proof URLs are different; but if Digicert (or whomever) were to give you the same pathname (e.g. /.well-known/pki-validation/fileauth.txt) for all of the onions, what do you do?

Answer: you use "splicing". If you have onion addresses named xxxxxxxxxxxxxxxx and yyyyyyyyyyyyyyyy, then you can create files:

templates.d/nginx-site-xxxxxxxxxxxxxxxx.onion.conf
templates.d/nginx-site-yyyyyyyyyyyyyyyy.onion.conf

...and into each put something similar to the following incantation; customise as necessary:

    location = "/.well-known/pki-validation/fileauth.txt" {
      return 200 "RESPECTIVE-PROOF-STRING-GOES-HERE";
    }

...then when you next eotk config and eotk nxreload, that code should be spliced into the correct configuration for each onion.

You will need to install the certificates for your project

For each certificate, HARICA will offer you several files to download; download the "PEM Bundle" file and copy it to your EOTK server. Also: copy the privateKey.pem file (mentioned above) to the EOTK server.

Next, change Directory into ~/eotk/projects.d/PROJECTNAME.d/ssl.d; you should see your development certificates, which will look like:

$ ls
ONIONADDRESS.onion.cert
ONIONADDRESS.onion.pem

There are two steps to installation:

Step 1: copy the PEM Bundle file from HARICA, on top of ONIONADDRESS.onion.cert

Step 2: unlock and extract (or rename) the private key, by doing:

openssl ec -in privateKey.pem -out ONIONADDRESS.onion.pem

...and typing in the password that you chose during the CSR setup, earlier; if you chose to use RSA as the algorithm, you will need to use openssl rsa ... instead.

If you manually created the CSR, then rename the 'onionaddress.key' file the CSR generated to 'onionaddress.onion.pem'.

Then: change directory back to the EOTK directory, and do eotk nxreload projectname, and test it.

Configuring Start-On-Boot, And Logfile Compression

Once you have installed EOTK (below) and configured and tested it for your project, run:

./eotk make-scripts

This will create two files:

./eotk-init.sh - for installing on your system as a startup script
./eotk-housekeeping.sh - for cronjob log rotation and other cleanup work

Please read the individual files for installation instructions; local setup is intended to be pretty simple, let me know if anything is confusing.

Using EOTK

I Want To Create My Own Project!

Okay, there are two ways to create your own project:

Easy With Automatic Onions

Create a config file with a .tconf suffix - we'll pretend it's foo.tconf - and use this kind of syntax:

set project myproject
hardmap %NEW_V3_ONION% foo.com
hardmap %NEW_V3_ONION% foo.co.uk
hardmap %NEW_V3_ONION% foo.de

...and then run

eotk config foo.tconf

...which will create the onions for you and will populate a foo.conf for you, and it will then configure EOTK from that. You should probably delete foo.tconf afterwards, since forcibly reusing it would trash your existing onions.

Slightly Harder With Manually-Mined Onions

Do eotk genkey - it will print the name of the onion it generates
- Do this as many times as you wish/need.
- Alternately get a tool like scallion, shallot, or eschalot and use that to "mine" a desirable onion address.
  - https://github.com/katmagic/Shallot - in C, for CPUs
    - Seems okay on Linux, not sure about other platforms
  - https://github.com/lachesis/scallion - in C#, for CPUs & GPUs (GPU == very fast)
    - Advertised as working on Windows, Linux; works well on OSX under "Mono"
  - https://github.com/ReclaimYourPrivacy/eschalot - in C, for CPUs
    - Works well under Linux, and BSD systems. Uses multiple threads. Can use a wordlist to generate onion addresses
  - Be sure to store your mined private keys in secrets.d with a filename like a2s3c4d5e6f7g8h9.key where a2s3c4d5e6f7g8h9 is the corresponding onion address.
Create a config file with a .conf suffix - we'll pretend it's foo.conf - and use this kind of syntax, substituting a2s3c4d5e6f7g8h9 for the onion address that you generated.

set project myproject
hardmap secrets.d/a2s3c4d5e6f7g8h9.key foo.com

...and then (IMPORTANT) run:

eotk config foo.conf

...which will configure EOTK.

Then Start Your Project

Like this:

eotk start myproject

What If I Have Subdomains?

When you are setting up the mappings in a config file, you may have to accomodate "subdomains"; the general form of a internet hostname is like this:

hostname.domain.tld # like: www.facebook.com or www.gov.uk
- or: hostname.domain.sld.tld # like: www.amazon.co.uk
hostname.subdom.domain.tld # like: www.prod.facebook.com
hostname.subsubdom.subdom.domain.tld # cdn.lhr.eu.foo.net
hostname.subsubsubdom.subsubdom.subdom.domain.tld # ...

...and so on, where:

tld = top level domain
- sld = second level domain
domain = generally the name of the organisation you are interested in
subdomain = some kind of internal structure
hostname = actual computer, or equivalent

When you are setting up mappings, generally the rules are:

you will map one domain per onion
you will ignore all hostnames
you will append all possible subdomain stems

So if your browser tells you that you are fetching content from cdn7.dublin.ireland.europe.foo.co.jp, you should add a line like:

hardmap %NEW_V3_ONION% foo.co.jp europe ireland.europe dublin.ireland.europe

...and EOTK should do the rest. All this is necessary purely for correctness of the self-signed SSL-Certificates - which are going to be weird, anyway - and the rest of the HTML-rewriting code in EOTK will be blind to subdomains.

Subdomain Summary

Subdomains are supported like this, for dev as an example:

set project myproject
hardmap secrets.d/a2s3c4d5e6f7g8h9.key foo.com dev

...and if you have multiple subdomains:

hardmap secrets.d/a2s3c4d5e6f7g8h9.key foo.com dev blogs dev.blogs [...]

My Company Has A Lot Of Sites/Domains!

Example:

www.foo.com.au
www.syd.foo.com.au
www.per.foo.com.au,
www.cdn.foo.net
www.foo.aws.amazon.com
...

Put them all in the same project as separate mappings, remembering to avoid the actual "hostnames" as described above:

set project fooproj
hardmap %NEW_V3_ONION% foo.com.au syd per
hardmap %NEW_V3_ONION% foo.net cdn
hardmap %NEW_V3_ONION% foo.aws.amazon.com

Onion mapping/translations will be applied for all sites in the same project.

Troubleshooting

The logs for any given project will reside in projects.d/<PROJECTNAME>.d/logs.d/

If something is problematic, first try:

git pull and...
eotk config <filename>.conf again, and then...
eotk bounce -a

Lots Of Broken Images, Missing Images, Missing CSS

This is probably an SSL/HTTPS thing.

Because of the nature of SSL self-signed certificates, you have to manually accept the certificate of each and every site for which a certificate has been created. See the second of the YouTube videos for some mention of this.

In short: this is normal and expected behaviour. You can temporarily fix this by:

right-clicking on the image for Open In New Tab, and accepting the certificate
or using Inspect Element > Network to find broken resources, and doing the same
or - if you know the list of domains in advance - visiting the /hello-onion/ URL for each of them, in advance, to accept certificates.

If you get an official SSL certificate for your onion site then the problem will vanish. Until then, I am afraid that you will be stuck playing certificate "whack-a-mole".

NGINX: Bad Gateway

Generally this means that NGINX cannot connect to the remote website, which usually happens because:

the site name in the config file, is wrong
the nginx daemon tries to do a DNS resolution, which fails

Check the NGINX logfiles in the directory cited above, for confirmation.

If DNS resolution is failing, PROBABLY the cause is probably lack of access to Google DNS / 8.8.8.8; therefore in your config file you should add a line like this - to use localhost as an example:

set nginx_resolver 127.0.0.1

...and then do:

eotk stop -a
eotk config filename.conf
eotk start -a

If you need a local DNS resolver, I recommend dnsmasq.

I Can't Connect, It's Just Hanging!

If your onion project has just started, it can take up to a few minutes to connect for the first time; also sometimes TorBrowser caches stale descriptors for older onions. Try restarting TorBrowser (or use the New Identity menu item) and have a cup of tea. If it persists, check the logfiles.

OnionBalance Runs For A Few Days, And Then Just Stops Responding!

Is the clock/time of day correct on all your machines? Are you running NTP? We are not sure but having an incorrect clock may be a contributory factor to this issue.

OnionBalance And Load-Balancing

NEW FOR 2019: OnionBalance as-of June 2019 is an flaky piece of software which is hard to run on modern Linux because an stale python crypto library; more than 90% of Onion sites will not practically need it - or, not initially, anyway - so I am deprecating OnionBalance in EOTK until it is majorly overhauled and also supports v3 onion addressing.

So, I recommend people to avoid OnionBalance and use hardmap (local) for the moment, until Tor fix it. Consider OB if and only if your system is under sustained high bandwidth and strongly demonstrates extended choking on throughput.

Video Demonstrations

These videos are instructive, but dated.

Commands have changed - but not very much - but please check the documentation rather than trust the videos; consider the videos to be advisory rather than correct.

16 KiB Czysty Bezpośredni odnośnik Wina Historia