https://commons.wikimedia.org/wiki/File:Onion_garden.jpg
Onion Garden via Wikimedia Madmad1234, CC BY-SA 4.0

Mirroring a site on a .onion address

I've made a .onion Tor hidden service mirror of this here blog. Using The Tor Browser, you can now find it at ygwdnjw2gt6sabcrhjqmpgsn4hhddomhipdpxbx6bru2zr2damxevqyd.onion. This post is just a run through of the route I took to do it and some notes on the steps someone could take to do likewise, and a few pain points on the way. Lastly, I've chucked in some final points on what the world needs if Hidden Service platforms are to become more common.

The what and the way

If the following is old hat, then skip this section and just scroll down to The Goods that follow. I still assume a lot of people aren't familiar with The Tor Network.

Sometimes you don't want your internet service provider (ISP) to know what you're up to on the web. This could be just a personal choice, it could be due to local laws, the internet router could be suspect if you're on a shared or unfamiliar network, or maybe you're in a country where looking up some topics are taboo if not just illegal. You may have heard of the Tor Browser, which is a web browser aimed at delivering users with strong privacy and anonymity and evade censorship. It bounces traffic through encrypted relays and delivers the site back through them as well. That's the pithy short version of what's going on. Find out more about it at the Tor Project website

Tor itself is a network. People can also make websites inside the Tor network, or create an alternate address in it, letting people find their site on a ".onion" address inside Tor instead of exiting out the other side. This helps to maintain the end-to-end encryption between you and the website you're visiting. It also can add another layer of anonymity for visitors.

You've likely heard of the 'Dark Web' horror stories of hit men for hire and drug markets, but also sites like the BBC offer .onion addresses to help users protect themselves while visiting potentially forbidden content according to whatever filtering may be happening locally. To visit a .onion address, you need the Tor Browser, but with it you can circumvent censorship and monitoring by visiting it at  bbcnewsv2vjtpsuy.onion.  I won't list all the examples here, but if you want to see more examples, check them out elsewhere. There are loads.

The internet is becoming a more surveilled experience from the moment anyone gets on it. the onramps, the services and search engines and more and more of the tools we use to communicate are finding that the key way of staying in business is to categorise us and find ways to better flog things we can buy, or sell on that data to others who may have some use for it. Few software makers are traveling in the opposite direction. The Tor Project is one of these that seek to make an open source protocol out of your right to privacy. I'm interested in ways publishing platforms can bypass censorship, but also respect their visitor's rights/needs to privacy or more anonymity. Potentially, I want to use this, or a flavour of it, in a project for a large number of sites.

There are a few other services I'm interested in as well. the i2P network runs another censorship circumvention service where content or service creators can run what they call eepsites.  I'm interested in how to mirror on these as well.  In the true spirit of mad scientists, I experiment on myself.

The Goods

Now on to the technical bits. This isn't meant to be the all-singing-all-dancing tutorial on mirroring everything on Tor, but more of a "here's what I did" run-through. I may have done some things wrong. There may be better ways to do it. But if it helps you out and increases people using Tor Hidden Services to help users, then that's fabulous.

So, the public site I mirrored is the site you're looking at right now. It's a simple, flat-file blog hosted on Github Pages. I don't know how well some custom functionality or any especially whiz-bang things you may have on your website will work out-of-the-box when subjected to the Tor Browser or this method of mirroring a site as a hidden service. The good news, is you'll get to find out really soon.

The ingredients

Firstly, we're going to need a website to mirror. Note, this is only for mirroring a website that's already available on the open, public web, not hosting something inside a .onion address exclusively. We aren't trying to hide the website or much anything about it. It's about making it accessible  from inside the Tor network.

For this I used a really nice tool by Alec Muffett, called EOTK, to do the .onion mirroring. To host the mirror I got a cheap VPS (there are suitable candidates for this on Linode, Digital Ocean, Vultr, etc.) For the SSL certificate I used harica.gr, which is one of only two certificate providers that cover hidden services (I have strong opinions on this, and so more editorialising when we get to that section below).

Like any good cooking show, I've got the finished soufflé here. We need the Tor Browser to look at that, but you're going to need that anyway to see how our dish is shaping up at different stages along the way. Also, the preparation is all taking place in the terminal. Everything here assumes you're able to use that as well as do the set-up of a server and user with its own ssh key.

I'm going to fast-forward a bit.  Just know I made a number of errors along the way and what follows represents just the correct turns, and not the bad ones or dead ends. I highly recommend reading through the bulk of the EOTK repo and all doc files, including but not limited to the how-to-install page. The Changelog is also really useful and can save a lot of time by reading before starting. There is also a lot of good information to be found in   This post isn't meant to be a replacement for any of that, but just a summary of my efforts.

The server

Our .onion mirrored website needs a server. I fired up a Ubuntu 20.04 VPS. This is a standard option you can find on most any VPS provider. EOTK has instructions for various flavours of server but the steps that follow here are assuming that one.

Creating a .onion mirror of our website

SSH into this new server. It's good to do an apt-get update and apt-get upgrade to start things off with the latest versions of things.

Now let's get the EOTK package from Alec's repo:
git clone https://github.com/alecmuffett/eotk.git

Next head to the EOTK directory which is where most everything is going to happen.
cd eotk

Here, were' going to build the environment based on the use of our Ubuntu 20.04 setup:
./opt.d/build-ubuntu-20.04.sh

Now we're going to generate the scripts (really do look at before moving on):
./eotk make-scripts

Now it's time to give EOTK the instructions it needs to mirror our website. I use nano to edit things. Use vim or whatever you want, but nano is easy:
nano your_project.tconf

In this new empty file, customise and add the following:

set project your_project
hardmap %NEW_V3_ONION% your_website.com

(Note where it specifies 'V3'. We need a v3 onion address as these are what the SSL certificates will require. Earlier versions are shorter, possibly easier, but won't work with SSL)

Save and exit that file.

Run the following:
./eotk config your_project.tconf

This is going to generate the .onion address. Copy that and save it in a txt file somewhere for pasting into the Tor Browser, and for the SSL stage later.

We no longer need the tconf file, so remove that. Notice there's now a .conf file with the same name. The rest of the effort will go in there:
rm your_project.tconf

Now let's start the new .onion website:
./eotk start your_project

Go to the Tor browser and visit this brand new .onion address. Here's what will see if everything happened right:

This is fine. The Tor Browser, built on Mozilla's Firefox, is checking for Valid SSL certificates for the site to make a secure https connection. EOTK's nginx configuration defaults to https, and generates some default certificates that obviously won't match our site's brand new address. Since we know we made this website just now, we'll click the advanced button to "accept the risk and continue" and check out whether the website mirror is otherwise behaving normally. Remember to remove the exception from the browser when we're finished.

Now let's make https work

The tasks above worked for me quite fast (about 10 minutes). Sorting out SSL on .onions presented a slight learning curve, some Googling, and a few questions back and forth with Alec in the EOTK repo issue queue, and the Harica helpdesk, both of which were patient and helpful. It's worth pointing out that in that time, Alec has written up a blog post on this very topic, read that for the official version, and for how to better use the EOTK conf file, which I can already see lends itself to my more ultimate goal on how to wrap everything into a scriptable and quickly repeatable process.


Read: Make your cleartext website #UNBLOCKABLE by adding a @torproject Onion address, using #EOTK and a HTTPS certificate from #HARICA


Some SSL ranting

I'm now several years into being used to using Let's Encrypt SSL for sites and haven't needed to go buy a certificate and deal with that whole manual validation process for a long time, and am sort of antagonistic about the whole thing. There are use cases for extended domain validation, getting that extra-verified green bar padlock or external validation "circle-of-trust" in the browser navigation bar, etc. I'm happy my bank has it. Facebook, Google, Amazon users should expect it. But I don't think it's necessary for the vast majority of the web. It's like buying air. Aside from making SSL free, to me the true gift of Let's Encrypt is the simplicity and the automation. But it doesn't run on Tor hidden service sites (yet).

Up until pretty recently, it seems the only SSL option for a .onion was Digicert, with a price tag of $344 (not including tax) for a year of https. Possibly not what most people have budgeted for their blog, or underfunded news site or small, scrappy nonprofit organisation. Enter Harica.gr, which drastically reduced the cost of entry to € 5.58. Harica is also funded by a nonprofit itself, so with that small cost I feel like I'm actually paying for a service and funding someone to answer my potentially tedious questions.

Okay, back to it

In the EOTK directly, let's take some time to review the different commands (if not already done).
./eotk -help

It's likely this is something useful to do at the start, but at this stage, we need to add some more things to our .conf file, restart EOTK once or twice and restart nginx at least once.

In Harica.gr, let's request a new certificate. Follow the steps there. We will need to validate the website. For this, I chose to generate my own csr instead of have Harica do it, and it's worth remembering this for later when we need to work on our .key file.  Do the following to make a .csr file:
openssl req -newkey rsa:2048 -keyout your_site.key -out your_site.csr

IMPORTANT(!): Save the password in your local txt file, we'll need it.

Submit the csr. Now Harica wants us to host a file on the site to complete the varification. Download that so we can copy the name and the content. Copy its name and content into the your_site.conf file.
nano your_site.conf

Paste in something that will look like this. I'm doing this for just one website, so this is pretty small. There are great examples of how to mirror loads of .onions at once in the EOTK docs, so for those, this entry would be much longer.

set ssl_proof_csv \
    /test,hello-world \
    /www/.well_known/pki-validation/FILENAME,CONTENT 

I did reach out for help at this stage as various attempts weren't working out. You can see that here. This other issue on overhauling the SSL proof mechanism is also worth tracking, as I am wanting to make a lot of .onion sites at some point. The/test,hello-world \ argument is simply for us to test that our conf file is making urls right.

Save that .conf file and then run these:
./eotk config your_site.conf
./eotk restart your_site

Go back to the Harica dashboard to see to the next part of the validation. 

Get and install the certificates

If the .onion site's validation checked out, then huzzah! We can now access the new files in the Harica dashboard. The certificates there will come in different formats. There's pem, der, PkCS#7 and a Pem Bundle. Download the Pem Bundle. We don't need the rest.

For the next bit, we need to make our encrypted key into a private key. Go back to where we'd made your .csr. and .key file. Do the following:
openssl pkcs8 -in your_site.key -out your_site-plain.key

Now, we're off to the SSL directory to add our files. cd eotk/projects.d/your_site.d/ssl.d/and add the your_site-plain.key and the .pem bundle you downloaded from Harica. Remove the default keys that were there, we don't need them, now.

Next, we're going to edit our nginx file to reflect the new files there. cd ../to the your_site.d directory and open the nginx.conf file. In the # SSL config section, update the names of the pem and key files to your new ones, and be careful to keep the path the same. Save and close that.

Back in our EOTK directory, restart nginx:
./eotk nxreload treacherous

Now go back to the .onion site in the Tor Browser and it should be working splendid.

Final notes

So, I'm pretty sure I took the long way around and there are a lot of good shortcuts, and demonstratively better uses of the .conf file to combine steps. I forgot to include a .www in this certificate. The latest changes in EOTK this week also include a lot of good SSL advice.

Laundry list:

  • I forgot to add a www version of the .onion address in the Harica SSl cert. You can make as many addresses as you want in the Harica dashboard fro your certificates. Could be useful for wildcards or running long lists of hidden service mirrors.
  • I understand (and agree with) the argument for driving people toward https on Tor hidden service sites, but think I'd it should be easier not to do this as an option until there's a Let's Encrypt option or something like it. I need to play more with EOTK to figure this out. 
  • I think this entire process is scriptable, possibly in python. I'd like to run a cli that made this even more automated, and possibly removed some of the human gaffes from the process. Again, it would require a Let's Encrypt SSL solution to really work in one go.
  • Cloudflare could wrap this entire service into a back-end and render it as a switch in the user dashboard. This would then allow it to more easily white-list Tor traffic instead of treating it as suspect.

For my actual project, I'd like to see how I can batch as much of this into a single script as possible as I'd potentially like to create .onions for 100+ mirrors in a manageable way with some other requirements not worth going into here. While I recommend and appreciate Harica, for my project to scale sustainably, I'd need Let's Encrypt or some kind of variant of it to support .onions.

Anyway, hope it helped. Many thanks to Harica's helpdesk person and Alec Muffett for their kind assists. Anything in this post that may be incorrect or problematic is mine.

Happy onions.

This article was updated on 26 June 2021