https://blog.manifold.co/do-you-know-where-your-apps-secrets-are-e3e472bd9edb


Development today relies on multiple teams, services, and environments all working in unison, and they’re difficult to orchestrate.

Mismanagement of secrets can land you on the front page of popular tech publications as part of an embarrassing data breach that results in loss of trust and future business. It happens to even the biggest of companies, for example: LinkedIn, Dropbox, Yahoo.

There is an ongoing tug of war between productivity and security which typically tips in favour of productivity. We take shortcuts and make mistakes resulting in compromised security posture.

You shouldn’t have to choose, so we’ve created Torus.

Allow me to demonstrate how easy it is to take a project (in this case, a readily available open source one) and configure it with Torus:

Choose your project

I need an application to start with, something that requires secrets.

I’ve chosen SlackIn, which is a utility that provides self-service sign-up for Slack teams. First we clone the repository:

$ git clone git@github.com:rauchg/slackin.git
$ cd slackin

Install the required dependencies:

$ npm install

And if you haven’t already, install Torus and signup:

$ npm install -g torus-cli
$ torus signup

Link it up

We want to link our Torus account to the project using torus link
This allows the codebase to be physically linked to the Torus project.

$ torus link
✔ Select organization: skywalker
✔ Create a new project: slackin
Project slackin created.
This directory and its subdirectories have been linked to:
Org: skywalker
Project: slackin
Use ‘torus status’ to view your full working context.

I now have a .torus.json file in the project directory. I’ll commit that to git:

$ git add .torus.json && git commit -m “Add .torus.json”

By committing the .torus.json file, people who collaborate with me on this project will automatically be linked to the skywalker org and the slackinproject.

Secrets and where to store them

SlackIn accepts four different environment variables as config:

  • SLACK_SUBDOMAIN
  • SLACK_API_TOKEN
  • SLACK_CHANNELS (optional)
  • SLACK_COC (optional)
  • PORT

We need to generate the two required SLACK items before going any further. We’re still in development, so let’s register a Slack team specifically for test purposes, and once we’re logged in as admin we’ll generate a new API token.

For demonstration purposes my development team name and token will be:

SLACK_SUBDOMAIN=skywalker-ranch-dev
SLACK_API_TOKEN=xoxp-8675309–9035768–1042534-r2d2

Now that I have these values in plain text (taken from the Slack website output), I need to store them for use in my application.

Inside every org each member is given their own development environment. This allows me to experiment, and develop without impacting secrets shared across multiple teams or services.

First I set the port that I want the process to listen on, in development:

$ torus set port 1337

I set the subdomain and token value using torus set:

$ torus set slack_subdomain skywalker-ranch-dev

Credential slack_subdomain has been set at /skywalker/slackin/dev-jeff/default/*/*/slack_subdomain
$ torus set slack_api_token xoxp-8675309–9035768–1042534-r2d2

Credential slack_subdomain has been set at /skywalker/slackin/dev-jeff/default/*/*/slack_api_token

I’ll throw away the plaintext values, because I’ve now encrypted them inside Torus.

If I need to view the plaintext values, I can retrieve and decrypt the values using torus view.

$ torus view
SLACK_SUBDOMAIN=skywalker-ranch-dev
SLACK_API_TOKEN=xoxp-8675309–9035768–1042534-r2d2
PORT=1337

Start your processes

I mentioned earlier that SlackIn requires two environment variables in order to run, if we try running the process without those, it fails to run:

$ ./bin/slackin
Usage: slackin [options] <team-id> <api-token>

Because I’ve laid out what my secrets look like in Torus I can simply call torus run, specifying the necessary environment (defaulting to my personal dev environment when not provided).

This will inject the configured values into the application.

$ torus run ./bin/slackin
fetching
listening on 0.0.0.0:1337
ready

Here I can test to make sure it does everything I need. I can make changes to the project as I see fit without disrupting production.

Sharing secrets

Torus prides itself on being a tool that can be used in development and all the way to production.

I have a development application and now I want to deploy it to production. The application will need all of the same configuration, but in some cases we want environment-specific values.

First I create a production environment for my Torus org:

$ torus envs create production
✔ Org name: skywalker
✔ Project name: slackin
✔ Environment name: production
Environment production created.

Next I need to set up my production Slack team just like I did earlier for development. My example team name and token for production will be:

SLACK_SUBDOMAIN=skywalker-ranch
SLACK_API_TOKEN=xoxp-7779311–9035768–1042534-bb8

Let’s take another look at what secrets I have set.
This time I’ll use torus ls, which allows me to explore the secrets across my environments by their path:

$ torus ls
Listing secrets within path: /skywalker/slackin/*/*/*/*
/skywalker/slackin/dev-jeff/default/*/*/port
/skywalker/slackin/dev-jeff/default/*/*/slack_api_token
/skywalker/slackin/dev-jeff/default/*/*/slack_subdomain

Two of my secrets are environment specific. I don’t want to mix and match the slack values between environments, but PORT on the other hand can be the same across all environments.

Port is currently set at the following path:

/skywalker/slackin/dev-jeff/default/*/*/port

Notice the third segment (dev-jeff) — this is the environment value. I stored the port in my personal development environment, which makes it inaccessible to other environments.

To rectify this I can set a new port, 5000, across all environments using:

$ torus set port 5000 --environment “*”
Credential port has been set at /skywalker/slackin/*/default/*/*/port

When I view the secret values again I see that port value is still 1337, why?

$ torus view
PORT=1337
SLACK_API_TOKEN=xoxp-8675309–9035768–1042534-r2d2
SLACK_SUBDOMAIN=skywalker-ranch-dev

Torus cascades the values depending on their specificity.

Originally I set a port in my personal development environment dev-jeff. This will override the globally specified portof 5000 with my personal value of 1337 because it is more specific.

I can switch to using the global value by unsetting the PORT in my development environment:

$ torus unset port
You are about to unset “/skywalker/slackin/dev-jeff/default/*/*/port”.
This cannot be undone.
✔ Do you wish to continue? [Y/n] Y
Credential port has been unset at /skywalker/slackin/dev-jeff/default/*/*/port.

Now we see 5000 as the PORT value:

$ torus view
PORT=5000
SLACK_API_TOKEN=xoxp-8675309–9035768–1042534-r2d2
SLACK_SUBDOMAIN=skywalker-ranch-dev

I can now finish setting my production specific slack secrets:

$ torus set slack_api_token xoxp-7779311–9035768–1042534-bb8 -e production
Credential port has been set at /skywalker/slackin/production/default/*/*/slack_api_token
$ torus set slack_subdomkain skywalker-ranch -e production
Credential port has been set at /skywalker/slackin/production/default/*/*/slack_subdomain

Access control

Thus far I have been using my personal Torus account to bootstrap the secrets. When we run this project in production I won’t want to use my personal credentials.

This is where Machines come in handy. We’re able to create an instance that acts like a user and interacts with configuration it is given access to.

We create a machine, placing it on a team for slackin instances. Torus suggests a name for this specific machine instance, which we will use.

$ torus machines create
✔ Org name: skywalker
✔ Create a new team: slackin
✔ Enter machine name: slackin-bb8r2d2
Machine team slackin created for org skywalker.
Creating machine “slackin-bb8r2d2”

Machine created.
You will only be shown the secret once, please keep it safe.
Machine ID: 04bvea[redacted]y91v8
Machine Token ID: 04c49f[redacted]hf4xm
Machine Token Secret: ZcA[redacted]sk1

I save and put aside the Token ID and Token Secret values for deployment later.

I now have a machine called slackin-bb8r2d2, which is a member of the slackin team, but the slackin team has no access yet.

I need to tell the team what it can access using torus allow:

$ torus allow rl /skywalker/slackin/production/default/*/*/* slackin
Policy generated and attached to the slackin team.
Effect: allow
Action(s): read, list
Resource: /skywalker/slackin/production/default/*/*/*
Necessary permissions (read, list) have also been granted.

I’ve given the machine rl (Read, List from the possible Create Read Updated Delete List) access to all secrets in my production environment, as it’s best practice not to give write access to deployments.

This will allow slackin-bb8r2d2 to read the values it needs to run SlackIn, but won’t allow anyone who may gain access to that server to overwrite or read anything outside of its scope.

Deployment

Now that our Torus environment is bootstrapped, we can go ahead and deploy our application.

Once my application server’s environment is set up, and I’ve deployed the code, I can set up Torus to run the process. We’ll talk about more complex deployments in a future post.

I have a Machine Token ID and a Machine Token Secret.

In deployment to platforms like Heroku or running in CI/CD systems, where you are trusting the security of the operating environment, Torus accepts these values through environment variables.

$ export TORUS_TOKEN_ID=04c49f[redacted]hf4xm
$ export TORUS_TOKEN_SECRET=ZcA[redacted]sk1
Note: We will cover more complex deployment options which properly segregate your token ID and Secret from your application user in a future post.

I also need to specify which Torus environment this machine belongs in:

$ export TORUS_ENVIRONMENT=production

Now the environment is setup to authenticate as our machine named slackin-bb8r2d2.

When I start my process, for example with PM2, the secrets are automatically read into the process environment:

$ pm2 start torus -- run ./bin/slackin
[PM2] Applying action restartProcessId on app [torus](ids: 0)
[PM2] [torus](0) ✓
[PM2] Process successfully started

I’m now running SlackIn for two environments: development and production using Torus to manage the secret configuration.