pig-monkey.com

Cold Storage

This past spring I mentioned my cold storage setup: a number of encrypted 2.5” drives in external enclosures, stored inside a Pelican 1200 case, secured with Abloy Protec2 321 locks. Offline, secure, and infrequently accessed storage is an important component of any strategy for resilient data. The ease with which this can be managed with git-annex only increases my infatuation with the software.

Data Data Data Data Data

I’ve been happy with the Seagate ST2000LM003 drives for this application. Unfortunately the enclosures I first purchased did not work out so well. I had two die within a few weeks. They’ve been replaced with the SIG JU-SA0Q12-S1. These claim to be compatible with drives up to 8TB (someday I’ll be able to buy 8TB 2.5” drives) and support USB 3.1. They’re also a bit thinner than the previous enclosures, so I can easily fit five in my box. The Seagate drives offer about 1.7 terabytes of usable space, giving this setup a total capacity of 8.5 terabytes.

Setting up git-annex to support this type of cold storage is fairly straightforward, but does necessitate some familiarity with how the program works. Personally, I prefer to do all my setup manually. I’m happy to let the assistant watch my repositories and manage them after the setup, and I’ll occasionally fire up the web app to see what the assistant daemon is doing, but I like the control and understanding provided by a manual setup. The power and flexibility of git-annex is deceptive. Using it solely through the simplified interface of the web app greatly limits what can be accomplished with it.

Encryption

Before even getting into git-annex, the drive should be encrypted with LUKS/dm-crypt. The need for this could be avoided by using something like gcrypt, but LUKS/dm-crypt is an ingrained habit and part of my workflow for all external drives. Assuming the drive is /dev/sdc, pass cryptsetup some sane defaults:

$ sudo cryptsetup --cipher aes-xts-plain64 --key-size 512 --hash sha512 luksFormat /dev/sdc

With the drive encrypted, it can then be opened and formatted. I’ll give the drive a human-friendly label of themisto.

$ sudo cryptsetup luksOpen /dev/sdc themisto_crypt
$ sudo mkfs.ext4 -L themisto /dev/mapper/themisto_crypt

At this point the drive is ready. I close it and then mount it with udiskie to make sure everything is working. How the drive is mounted doesn’t matter, but I like udiskie because it can integrate with my password manager to get the drive passphrase.

$ sudo cryptsetup luksClose /dev/mapper/themisto_crypt
$ udiskie-mount -r /dev/sdc

Git-Annex

With the encryption handled, the drive should now be mounted at /media/themisto. For the first few steps, we’ll basically follow the git-annex walkthrough. Let’s assume that we are setting up this drive to be a repository of the annex ~/video. The first step is to go to the drive, clone the repository, and initialize the annex. When initializing the annex I prepend the name of the remote with satellite :. My cold storage drives are all named after satellites, and doing this allows me to easily identify them when looking at a list of remotes.

$ cd /media/themisto
$ git clone ~/video
$ cd video
$ git annex init "satellite : themisto"

Disk Reserve

Whenever dealing with a repository that is bigger (or may become bigger) than the drive it is being stored on, it is important to set a disk reserve. This tells git-annex to always keep some free space around. I generally like to set this to 1 GB, which is way larger than it needs to be.

$ git config annex.diskreserve "1 gb"

Adding Remotes

I’ll then tell this new repository where the original repository is located. In this case I’ll refer to the original using the name of my computer, nous.

$ git remote add nous ~/video

If other remotes already exist, now is a good time to add them. These could be special remotes or normal ones. For this example, let’s say that we have already completed this whole process for another cold storage drive called sinope, and that we have an s3 remote creatively named s3.

$ git remote add sinope /media/sinope/video
$ export AWS_ACCESS_KEY_ID="..."
$ export AWS_SECRET_ACCESS_KEY="..."
$ git annex enableremote s3

Trust

Trust is a critical component of how git-annex works. Any new annex will default to being semi-trusted, which means that when running operations within the annex on the main computer — say, dropping a file — git-annex will want to confirm that themisto has the files that it is supposed to have. In the case of themisto being a USB drive that is rarely connected, this is not very useful. I tell git-annex to trust my cold storage drives, which means that if git-annex has a record of a certain file being on the drive, it will be satisfied with that. This increases the risk for potential data-loss, but for this application I feel it is appropriate.

$ git annex trust .

Preferred Content

The final step that needs to be taken on the new repository is to tell it what files it should want. This is done using preferred content. The standard groups that git-annex ships with cover most of the bases. Of interest for this application is the archive group, which wants all content except that which has already found its way to another archive. This is the behaviour I want, but I will duplicate it into a custom group called satellite. This keeps my cold storage drives as standalone things that do not influence any other remotes where I may want to use the default archive.

$ git annex groupwanted satellite "(not copies=satellite:1) or approxlackingcopies=1"
$ git annex group . satellite
$ git annex wanted . groupwanted

For other repositories, I may want to store the data on multiple cold storage drives. In that case I would create a redundantsatellite group that wants all content which is not already present in two other members of the group.

$ git annex groupwanted redundantsatellite "(not copies=redundantsatellite:2) or approxlackingcopies=1"
$ git annex group . redundantsatellite
$ git annex wanted . groupwanted

Syncing

With everything setup, the new repository is ready to sync and to start to ingest content from the remotes it knows about!

$ git annex sync --content

However, the original repository also needs to know about the new remote.

$ cd ~/video
$ git remote add themisto /media/themisto/video
$ git annex sync

The same is the case for any other previously existing repository, such as sinope.

Currently reading The New Spymasters by Stephen Grey.

The book begins with an overview of espionage immediately before, during, and shortly after the Cold War, before moving on to the role played by Western intelligence agencies in the current millenium. Grey contrasts the earlier focus on human intelligence with the growing dependency on signals intelligence and assassination programs, and makes a compelling case for the need to return to a balanced approach with a focus on traditional spy running.

The dichotomy is reminiscent between that of the longer-term, unconventional warfare practiced by US Special Forces and the direct action focus of other Special Operations Forces as discussed by Tony Schwalm.

This post was published on . It was tagged with books, micro.

Redundant File Storage

As I’ve mentioned previously, I store just about everything that matters in git-annex (the only exception is code, which is stored directly in regular git). One of git-annex’s many killer features is special remotes. They make tenable this whole “cloud storage” thing that we do now.

A special remote allows me to store my files with a large number of service providers. It makes this easy to do by abstracting away the particulars of the provider, allowing me to interact with all of them in the same way. It makes this safe to do by providing encryption. These factors encourage redundancy, reducing my reliance on any one provider.

Recently I began playing with rclone. Rclone is a program that supports file syncing for a handful of cloud storage providers. That’s semi-interesting by itself but, more significantly, there is a git-annex special remote wrapper. That means any of the providers supported by rclone can be used as a special remote. I looked through all of rclone’s supported providers and decided there were a few that I had no reason not to use.

Hubic

Hubic is a storage provider from OVH with a data center in France. Their pricing is attractive. I’d happily pay €50 per year for 10TB of storage. Unfortunately they limit connections to 10 Mbit/s. In my experience they ended up being even slower than this. Slow enough that I don’t want to give them money, but there’s still no reason not to take advantage of their free 25 GB plan.

After signing up, I setup a new remote in rclone.

$ rclone config
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
name> hubic-annex
Type of storage to configure.
Choose a number from below, or type in your own value
 1 / Amazon Drive
   \ "amazon cloud drive"
 2 / Amazon S3 (also Dreamhost, Ceph)
   \ "s3"
 3 / Backblaze B2
   \ "b2"
 4 / Dropbox
   \ "dropbox"
 5 / Google Cloud Storage (this is not Google Drive)
   \ "google cloud storage"
 6 / Google Drive
   \ "drive"
 7 / Hubic
   \ "hubic"
 8 / Local Disk
   \ "local"
 9 / Microsoft OneDrive
   \ "onedrive"
10 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
   \ "swift"
11 / Yandex Disk
   \ "yandex"
Storage> 7
Hubic Client Id - leave blank normally.
client_id> 
Hubic Client Secret - leave blank normally.
client_secret> 
Remote config
Use auto config?
 * Say Y if not sure
 * Say N if you are working on a remote or headless machine
y) Yes
n) No
y/n> y
If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
Log in and authorize rclone for access
Waiting for code...
Got code
--------------------
[remote]
client_id = 
client_secret = 
token = {"access_token":"XXXXXX"}
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y

With that setup, I went into my ~/documents annex and added the remote.

$ git annex initremote hubic type=external externaltype=rclone target=hubic-annex prefix=annex-documents chunk=50MiB encryption=shared rclone_layout=lower mac=HMACSHA512

I want git-annex to automatically send everything to Hubic, so I took advantage of standard groups and put the repository in the backup group.

$ git annex wanted hubic standard
$ git annex group hubic backup

Given Hubic’s slow speed, I don’t really want to download files from it unless I need to. This can be configured in git-annex by setting the cost of the remote. Local repositories default to 100 and remote repositories default to 200. I gave the Hubic remote a high cost so that it will only be used if no other remotes are available.

$ git config remote.hubic.annex-cost 500

If you would like to try Hubic, I have a referral code which gives us both an extra 5GB for free.

Backblaze B2

B2 is the cloud storage offering from backup company Backblaze. I don’t know anything about them, but at $0.005 per GB I like their pricing. A quick search of reviews shows that the main complaint about the service is that they offer no geographic redundancy, which is entirely irrelevant to me since I build my own redundancy with my half-dozen or so remotes per repository.

Signing up with Backblaze took a bit longer. They wanted a phone number for 2-factor authentication, I wanted to give them a credit card so that I could use more than the 10GB they offer for free, and I had to generate an application key to use with rclone. After that, the rclone setup was simple.

$ rclone config
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
name> b2-annex
Type of storage to configure.
Choose a number from below, or type in your own value
 1 / Amazon Drive
   \ "amazon cloud drive"
 2 / Amazon S3 (also Dreamhost, Ceph)
   \ "s3"
 3 / Backblaze B2
   \ "b2"
 4 / Dropbox
   \ "dropbox"
 5 / Google Cloud Storage (this is not Google Drive)
   \ "google cloud storage"
 6 / Google Drive
   \ "drive"
 7 / Hubic
   \ "hubic"
 8 / Local Disk
   \ "local"
 9 / Microsoft OneDrive
   \ "onedrive"
10 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
   \ "swift"
11 / Yandex Disk
   \ "yandex"
Storage> 3
Account ID
account> 123456789abc
Application Key
key> 0123456789abcdef0123456789abcdef0123456789
Endpoint for the service - leave blank normally.
endpoint> 
Remote config
--------------------
[remote]
account = 123456789abc
key = 0123456789abcdef0123456789abcdef0123456789
endpoint = 
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y

With that, it was back to ~/documents to initialize the remote and send it all the things

$ git annex initremote b2 type=external externaltype=rclone target=b2-annex prefix=annex-documents chunk=50MiB encryption=shared rclone_layout=lower mac=HMACSHA512
$ git annex wanted b2 standard
$ git annex group b2 backup

While I did not measure the speed with B2, it feels as fast as my S3 or rsync.net remotes, so I didn’t bother setting the cost.

Google Drive

While I do not regularly use Google services for personal things, I do have a Google account for Android stuff. Google Drive offers 15 GB of storage for free and rclone supports it, so why not take advantage?

$ rclone config
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
name> gdrive-annex
Type of storage to configure.
Choose a number from below, or type in your own value
 1 / Amazon Drive
   \ "amazon cloud drive"
 2 / Amazon S3 (also Dreamhost, Ceph)
   \ "s3"
 3 / Backblaze B2
   \ "b2"
 4 / Dropbox
   \ "dropbox"
 5 / Google Cloud Storage (this is not Google Drive)
   \ "google cloud storage"
 6 / Google Drive
   \ "drive"
 7 / Hubic
   \ "hubic"
 8 / Local Disk
   \ "local"
 9 / Microsoft OneDrive
   \ "onedrive"
10 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
   \ "swift"
11 / Yandex Disk
   \ "yandex"
Storage> 6
Google Application Client Id - leave blank normally.
client_id> 
Google Application Client Secret - leave blank normally.
client_secret> 
Remote config
Use auto config?
 * Say Y if not sure
 * Say N if you are working on a remote or headless machine or Y didn't work
y) Yes
n) No
y/n> y
If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
Log in and authorize rclone for access
Waiting for code...
Got code
--------------------
[remote]
client_id = 
client_secret = 
token = {"AccessToken":"xxxx.x.xxxxx_xxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","RefreshToken":"1/xxxxxxxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxx","Expiry":"2014-03-16T13:57:58.955387075Z","Extra":null}
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y

And again, to ~/documents.

$ git annex initremote gdrive type=external externaltype=rclone target=gdrive-annex prefix=annex-documents chunk=50MiB encryption=shared rclone_layout=lower mac=HMACSHA512
$ git annex wanted gdrive standard
$ git annex group gdrive backup

Rinse and repeat the process for other annexes. Revel in having simple, secure, and redundant storage.

A Place for the SWAT-T

My dislike of the SWAT Tourniquet stems from its difficulty to self-apply one-handed. That eliminated it from the running when evaluating pocket tourniquets, but the PHLster Flatpack has made that category of tool less relevant. Now that I can easily and comfortably carry a primary tourniquet (specifically, a SOFTT-W) on-body, I’ve rethought what I should be carrying in my pack.

In the past I’ve carried a SOFTT-W as part of a small blow-out kit. The kit is in a Triple Seven Gear Micro Kit pouch, which fits easily into whatever pack I’m using. If I’m already carrying a SOFTT-W on my belt, is carrying a second one the best use of the available weight and space? The SWAT tourniquet does have a few things going for it. It works well as a tourniquet, as long as you have two hands to apply it. The width of the SWAT-T allows it to occlude blood flow at a relatively low pressure, and its elasticity can help it to compensate for muscle relaxation. It can function as part of a pressure dressing, or be used to improve an improvised splint, swathe, or sling. And it burns well.

SWAT-T

I decided to replace the SOFTT-W in my blow-out kit with a SWAT-T. With a SOFTT-W in the PHLster Flatpack on my belt, I’m confident in my ability to quickly administer self-aid. Having the SWAT-T in my bag gives me additional options, whether I need to use it as a second tourniquet or as something else.

This post was published on . It was tagged with gear, medical, edc.

I put a Raven Pocket Clip on my Elzetta Alpha.

The Elzetta Alpha A323 has been part of my EDC for 2 years now. For all but a few weeks of that time I’ve been carrying it on my belt with a Prometheus Lights Titanium Pocket Clip, which works great on the Alpha. I changed over to the Raven Concealment Systems Pocket Clip to get the finger ring, which is just a large rubber O-ring that allows you to use your hands for something else without dropping the light.

Elzetta Alpha w/ RCS Pocket Clip

This post was published on . It was tagged with gear, micro, edc.

Rudy Project Rydon

I’ve used a pair of Rudy Project Rydon Stealth glasses as sun and safety glasses for about five years now. They’re a great eye protection system for active wear, and I think are especially attractive for those who require prescription eyewear. The Rydon offers an adaptable system with interchangeable lenses and full coverage, in a lower profile compared to popular tactical eyewear systems like the Revision Sawfly, Oakley M Frame or ESS ICE.

Rudy Project Rydon System

The temples and nosepads of the Rydon are made from an pliable rubber material that lets the user adjust them however they want. You can have straight temples or hook them down behind your ears. You can move the nosepads to get the right height and clearance on your face. I find that both adjustments largely stay in place after being set. If they do move, it is simple to reset them.

Safety & Durability

The only difference between Rudy’s “Stealth” and non-“Stealth” line is the ANSI Z87.1 rating applied to the frame. Z87.1 is the standard for eyewear protection that will be familiar to anyone who has worn safety glasses or so-called “tactical” eyewear. It describes, among other things, impact resistance. The Rydon lenses are interchangeable between Stealth and non-Stealth variants, and certain lens selections have their own Z87.1 ratings. The Stealth frames are made from a different material than non-Stealth. The non-Stealth Rydons fail to meet Z87.1 standards due to how they shatter1.

Unfortunately, Rudy has not certified the Rydon for MIL-PRF-32432, the military specification for ballistic eyewear.

As you might expect, the Rydon have proven to be very durable. I’ve used them regularly over the past five years. I shoot in them, I crash bikes in them, and I’ve been punched in the face more times than I can count in them. They’re none the worse for wear. Certain lenses have minor scratches, but none that I notice when I’m actually wearing them. The frames themselves are like new. The rubber material does not absorb sweat and odors, which is a complaint I’ve heard of Oakley’s “unobtainium” rubber temples.

ImpactX

Rudy is well known for their ImpactX lenses. They describe it as being a “bullet-proof, transparent, and light-weight material capable of providing superior protection, reliability and longer lasting performance than polycarbonate”. It is what Apache windshield panels are made out of, which makes me feel good about myself2.

ImpactX is actually just Rudy-branded NXT. NXT is a variant of Trivex. I’m not sure what the difference is between NXT and Trivex. It may be that NXT is just a specific branding of Trivex. I do know that the Z87.1 impact protection that NXT/ImpactX claims is a property of the Trivex. Trivex gives equal protection.

Trivex is a polymer that was introduced as an alternative to polycarbonate. Traditionally, most safety glasses are made out of polycarbonate. When you get a prescription insert from Revision or ESS and have them fill the prescription, the lenses they’re putting in are polycarbonate. “Plutonite” is Oakley’s proprietary brand of polycarbonate. Trivex offers equal impact protection, but has a lower Abbe number than polycarbonate, which translates to superior optical quality.

Trivex is slightly more expensive (polycarbonate costs ~$40, Trivex ~$50). Trivex also has a slightly lower refractive index, which translates to Trivex prescription lenses being slightly thicker than polycarbonate. But Trivex has a lower specific gravity, so the Trivex lenses will be slightly lighter than the equivalent polycarbonate, despite the added thickness.

There’s nothing rare or special about Trivex. Everyone does it. You can bring any set of frames into any optometry office and tell them you want to put Trivex lenses in it. As long as your prescription fits the frame, they can do it. The resulting lenses will meet or exceed Z87.1, even though your optometrist likely isn’t going to get them certified for the Z87.1 stamp.

Photochromic

The ImpactX lenses, in addition to offering impact protection, are also photochromic. Photochromic lenses darken when exposed to ultraviolet radiation. Having a clear lens in a pair of safety glasses is critical for indoor work, and being able to use the same lenses as sunglasses outdoors keeps the overall price down. It is also helpful for transitions. If you start out in the sun and then go inside, or the clouds roll in or the sun sets, your optics quickly respond without you needing to take time to remove the glasses or swap lenses.

When I first purchased the Rydon system, one of the lenses I included was the ImpactX photochromic clear-to-black. This is a neutral lens that offered 18-78% light transmission. A few months ago, I purchased Rudy’s ImpactX-2 photochromic clear-to-black lenses. These offer 9-74% light transmission. In addition to the change in light transmission, ImpactX-2 also reacts faster and is supposed to respond slightly better to non-UV light3.

Other Lenses

While the ImpactX lenses are the only offerings from Rudy with the Z87.1 stamp, the company does offer some other lens options.

Rudy’s Polar 3FX is their polarized lens solution. Polarized lenses reduce glare, which is useful on water and snow. I have a pair of Polar 3FX brown lenses. At 15% light transmission, these offer about the same protection as my old Julbo Micropore glacier glasses. They’re a great supplement to the ImpactX photochromic lenses, and are light enough that I am happy to carry them as a secondary option on backpacking trips for use above the tree line.

The third lens I went with is what Rudy calls Racing Red. These are a high contrast red lens with 28% light transmission. A contrast lens in something like red or yellow is a great option for hazy days when it isn’t bright out and you don’t want much light reduction, but you find yourself squinting from the glare4.

Of the three lenses (or four, since I now have two of the photochromic clear-to-black lenses), I use the Racing Red least of all. They’re great in certain conditions, but the ImpactX(-2) lenses work well in all conditions, so I find that I can just leave them in the frame all the time and never think about it. Plus, Z87.1.

Prescription

Rudy has a few options for people who need prescription lenses. The option that I’ve gone for is the Optical Insert. I used a similar setup back in the day with Revision Sawfly eyewear and their insert, but I think Rudy does it better. While this kind of dual lens system does result in a slight degradation of optical quality, it means that you only pay for the prescription once. Being able to purchase multiple non-prescription lenses that sit in front of the prescription lens is the only thing that makes this kind of multi-lens system tenable.

Rudy offers two different carrier styles: a full metal frame and a “rimless” option.

When I first purchased the Rydon system I went with the full metal frame. It has served well over the years. Like the Rydon frame, it still functions like new. Occasionally a punch in the face will cause the insert to be knocked out of the Rydon, but that hasn’t caused any damage to the carrier, and it only takes a second to pop it back in.

This past spring, when I bought the new ImpactX-2 lenses, I also wanted to purchase another carrier to have my new prescription put in it. This time around I tried the “rimless” model. The “rimless” carrier is not actually rimless: the lenses are held in place by a thin wire that goes around the circumference of each lens. This can result in a thicker lens. With a rimless carrier, the lens need a groove cut into it to accept the wire. Prescription lenses are thinnest on the inside (near the nose) and thickest on the outside. If your prescription is weak enough that the outside of the lens is not already thick enough for the groove to be cut, the thickness will need to be increased.

I’ve not found that the lenses in my rimless carrier appear any thicker than the ones in my old carrier, but that’s something that you may want to keep in mind when deciding between the two carrier options.

When you’ve received the carrier, you can take it to any optometrist to have your prescription lenses filled. Finding an optometrist who has some experience with this kind of setup — whether they are actually a Rudy dealer or offer some other sport brand with an insert system — will like behoove you. With my first pair, I used a Rudy dealer. The optometrist that I used when filling the new carriers this year was not a Rudy dealer, nor had they ever dealt with something like this before. However, they actually grind their lenses in house, which is something I’ve never seen before. When placing the order I was able to talk with the lab manager who would actually be making the lenses, which gave me confidence that I wouldn’t be wasting my money5.

For the prescription lenses, I went with polycarbonate. With a dual lens system like this, both weight and thickness are a concern. While Trivex would have been lighter, we decided that the thinner polycarbonate would be better suited for my prescription. The polycarbonate prescription lenses would be behind the Rydon’s ImpactX(/NXT/Trivex) lenses, so I don’t have much concern about the lower impact resistance.

This time around I went with an anti-fog coating, which I did not have previously. While it was rarely ever an issue, I did occasionally experience fogging with the old lenses. It usually happens when walking into a sweaty gym when it is cold and dry outside. It’s not really the time of year for me to be able to evaluate if the anti-fog coating is doing anything on the new lenses.

I think an anti-glare coating on the prescription lens is unnecessary. The tinted Rydon lenses should take care of that problem whenever necessary.

I did also go with an edge polish on the new prescription lenses. Previously I opted to forgo that option with the old metal carriers, but with the rimless carrier I thought that it would help maintain a lower profile look — both from the outside looking in and the inside looking out. I think it was a smart decision.

Notes

  1. Personally, I think that wearing sunglasses which are not Z87.1 rated is stupid and a waste of money, outside of certain specialty requirements. Eyes are important. You can’t fight what you can’t see, unless you are a time travelling samurai.
  2. When you get down to it, why wouldn’t you protect your eyeballs with the same technology used in gunships?
  3. Traditionally, one of the shortcomings of photochromic lenses is that they don’t work well in cars. Windshields filter UV light. ImpactX-2 is supposed to handle that a little better. I can’t comment on that — I spend very little time in cars.
  4. I also believe that having a contrast lens that you can wear when it isn’t necessarily bright enough for normal sunglasses is useful to combat color-based advertising intended to condition us to better suit our extraterrestrial overlords.
  5. With most optometrists you end up dealing with the people who sell lenses. Talking to someone who actually makes the things lets you dip into a different knowledge base.