pig-monkey.com

You are currently viewing all posts tagged with shell.

Undertime

Undertime is a simple program that assists in coordinating events across time zones. It prints a table of your system’s local time zone, along with other any other specified zones. The output is colorized based on the start and end hour of the working day. If you want to talk to someone in Paris tomorrow, and you want the conversation to happen at an hour that is reasonable for both parties, Undertime can help.

Undertime Paris Meeting Example

I often find myself converting between local time and UTC. Usually this happens when working with system logs. If I have a specific date and time I want to translate, I’ll use date.

# Convert a time from PDT to UTC:
$ env TZ="UTC" date -d "2016-03-25T11:33 PDT"
# Convert a time from UTC to local:
$ date -d '2016-03-24T12:00 UTC'

If I’m not looking to convert an exact time, but just want to answer a more generalized question like “Approximately when was 14:00 UTC?” without doing the mental math, I find that Undertime is the quickest solution.

$ undertime UTC
╔═══════╦═══════╗
║  PDTUTC  ║
╠═══════╬═══════╣
║ 00:0007:00 ║
║ 01:0008:00 ║
║ 02:0009:00 ║
║ 03:0010:00 ║
║ 04:0011:00 ║
║ 05:0012:00 ║
║ 06:0013:00 ║
║ 07:0014:00 ║
║ 08:0015:00 ║
║ 09:0016:00 ║
║ 10:0017:00 ║
║ 11:0018:00 ║
║ 12:0019:00 ║
║ 13:0020:00 ║
║ 14:0021:00 ║
║ 15:0022:00 ║
║ 16:0023:00 ║
║ 17:0000:00 ║
║ 18:0001:00 ║
║ 19:0002:00 ║
║ 19:0402:04 ║
║ 20:0003:00 ║
║ 21:0004:00 ║
║ 22:0005:00 ║
║ 23:0006:00 ║
╚═══════╩═══════╝
Table generated for time: 2019-07-23 19:04:00-07:00

Music Organization with Beets

I organize my music with Beets.

Beets imports music into my library, warns me if I’m missing tracks, identifies tracks based on their accoustic fingerprint, scrubs extraneous metadata, fetches and stores album art, cleans genres, fetches lyrics, and – most importantly – fetches metadata from MusicBrainz. After some basic configuration, all of this happens automatically when I import new files into my library.

After the files have been imported, beets makes it easy to query my library based on any of the clean, consistent, high quality, crowd-sourced metadata.

$ beet stats genre:ambient
Tracks: 649
Total time: 2.7 days
Approximate total size: 22.4 GiB
Artists: 76
Albums: 53
Album artists: 34

$ beet ls -a 'added:2019-07-01..'
Deathcount in Silicon Valley - Acheron
Dlareme - Compass
The Higher Intelligence Agency & Biosphere - Polar Sequences
JK/47 - Tokyo Empires
Matt Morton - Apollo 11 Soundtrack

$ beet ls -ap albumartist:joplin
/home/pigmonkey/library/audio/music/Janis Joplin/Full Tilt Boogie
/home/pigmonkey/library/audio/music/Janis Joplin/I Got Dem Ol' Kozmic Blues Again Mama!

As regular readers will have surmised, the files themselves are stored in git-annex.

Terminal Countdown

Termdown is a program that provides a countdown timer and stopwatch in the terminal. It uses FIGlet for its display. Its most attractive feature, I think, is the ability to support arbitrary script execution.

I use it most often as a countdown timer. One of my frequent applications is as a meditation timer. For this I want a 11 minute timer, with an alert at 10.5 minutes, 60 seconds, and 1 second. This gives me a 10 minute session with 30 seconds preparation and 30 seconds to return. Termdown makes this easy.

$ termdown --exec-cmd "case {0} in 630|30) mpv ~/library/audio/sounds/bell.mp3;; 1) mpv ~/library/audio/sounds/ring.mp3;; esac" 11m

An Offline Lexicon

dictd is a dictionary database server and client. It can be used to lookup word definitions over a network. I don’t use it for that. I use the program to provide an offline dictionary. Depending on a network connection, web browser and third-party websites just to define a word strikes me as dumb.

To make this go, dictionary files must be installed. I use the GNU Collaborative International Dictionary of English (GCIDE), WordNet, and the Moby Thesaurus. The GCIDE is derived from Noah Webster’s famous American dictionary. WordNet is a more modern (one might say “dry”) resource. The Moby Thesaurus is a public domain thesaurus originally built by Grady Ward. Between these three sources I can have a pretty good grasp on the English language. No network connectivity required.

I use a shell alias to always pipe the definitions through less.

def () {
    dict $1 | less
}

GOESImage

GOESImage is a bash script which downloads the latest imagery from the NOAA Geostationary Operational Environment Satellites and sets it as the desktop background via feh. If you don’t use feh, it should be easy to plug GOESImage into any desktop background control program.

GOESImage Example

I wrote GOESImage after using himawaripy for a few years, which is a program that provides imagery of the Asia-Pacific region from the Himawari 8 Japanese weather satellite. I like seeing the Earth, and I’ve found that real time imagery of my location is actually useful for identifying the approach of large-scale weather systems. NOAA’s nighttime multispectral infrared coloring is pretty neat, too.

This post was published on . It was tagged with code, shell, linux.

Delta Drone

Last year BoingBoing linked to a video featuring delta waves produced by the idling engine of an ice breaker in the arctic. I found it to be a useful tool, so downloaded it for offline access. Later, I decided I wanted the audio on my phone. The video is a 10 hour loop, resulting in too large a file for mobile storage. To turn it into something reasonable for a phone, I used ffmpeg to extract the audio, chop it down to 3 hours, and add a 10 second fade on either end.

$ ffmpeg \
    -i ~/library/video/web/White\ Noise\ Sounds\ of\ Frozen\ Arctic\ Ocean\ with\ Polar\ Icebreaker\ Idling\ -\ Creating\ Delta\ Waves-gpW7iYfuGDU.webm \
    -vn \
    -ss 00:00:00 \
    -t 03:00:00 \
    -af afade=in:st=0:d=10,afade=out:st=10790:d=10 \
    ~/library/audio/misc/soundscape/arctic_white_noise.mp3

I then added ID3 tags from the metadata of the original video.

$ id3tag \
    --artist="Relax Sleep ASMR" \
    --song="White Noise Sounds of Frozen Arctic Ocean with Polar Icebreaker Idling - Creating Delta Waves" \
    --year=2017 \
    ~/library/audio/misc/soundscape/arctic_white_noise.mp3

The result is a 165 MB file of loopable delta waves, perfect for drowning out the world.

The original video has since been deleted (a reminder to download any data that you find to be useful), but is available at the Internet Archive.

Delta Waves

The above spectrogram of the file is produced by Spek.

This post was published on . It was tagged with audio, sleep, shell.

Sending Documents Like It's 1988

Once every year or two I need to send a fax. Never receive, just send. Usually for something involving the finance industry. Twilio makes this about as painless as it can be in the 21st century.

Unfortunately the Twilio Fax API doesn’t allow you to post the document to it directly, so the first step is to get the PDF online somewhere. After that, it can be faxed via curl.

$ curl https://fax.twilio.com/v1/Faxes \
    -X POST \
    -d 'To=%2B15408684391'  \
    -d 'From=%2B14158675309'  \
    -d 'MediaUrl=https://example.com/document.pdf' \
    -u $TWILIO_ACCOUNT_ID:$TWILIO_AUTH_TOKEN

This queues up the document to be sent, which usually takes a couple minutes. Somewhere in the response will be a URL that looks like https://fax.twilio.com/v1/Faxes/$GIBBERISH. After a few minutes, this URL can be used to check the status.

$ curl https://fax.twilio.com/v1/Faxes/$GIBBERISH \
    -X GET \
    -u $TWILIO_ACCOUNT_ID:$TWILIO_AUTH_TOKEN | python -m json.tool

If the status is queued, processing or sending, check back in a few minutes. If it is delivered, you’re all done and can delete the uploaded PDF. If the status is something else, you probably need to try again. Perhaps ask the recipient to sign out of AOL and hang-up their modem so that their fax machine can accept your call.

This post was published on . It was tagged with shell.

Password Management with Vim and GnuPG

The first password manager I ever used was a simple text file encrypted with GnuPG. When I needed a password I would decrypt the file, read it in Vim, and copy the required entry to the system clipboard. This system didn’t last. At the time I wasn’t using GnuPG for much else, and this was in the very beginning of my Vim days, when the program seemed cumbersome and daunting. I shortly moved to other, purpose-built password managers.

After some experimentation I landed on KeePassX, which I used for a number of years. Some time ago I decided that I wanted to move to a command-line solution. KeePassX and a web browser were the only graphical applications that I was using with any regularity. I could see no need for a password manager to have a graphical interface, and the GUI’s dependency on a mouse decreased my productivity. After a cursory look at the available choices I landed right back where I started all those years ago: Vim and GnuPG.

These days Vim is my most used program outside of a web browser and I use GnuPG daily for handling the majority of my encryption needs. My greater familiarity with both of these tools is one of the reasons I’ve been successful with the system this time around. I believe the other reason is my more systematic approach.

Structure

The power of this system comes from its simplicity: passwords are stored in plain text files that have been encrypted with GnuPG. Every platform out there has some implementation of the PGP protocol, so the files can easily be decrypted anywhere. After they’ve been decrypted, there’s no fancy file formats to deal with. It’s all just text, which can be manipulated with a plethora of powerful tools. I favor reading the text in Vim, but any text editor will do the job.

All passwords are stored within a directory called ~/pw. Within this directory are multiple files. Each of these files can be thought of as a separate password database. I store bank information in financial.gpg. Login information for various shopping websites are in ecommerce.gpg. My email credentials are in email.gpg. All of these entries could very well be stored in a single file, but breaking it out into multiple files allows me some measure of access control.

Access

I regularly use two computers: my laptop at home and a desktop machine at work. I trust my laptop. It has my GnuPG key on it and it should have access to all password database files. I do not place complete trust in my machine at work. I don’t trust it enough to give it access to my GnuPG key, and as such I have a different GnuPG key on that machine that I use for encryption at work.

Having passwords segregated into multiple database files allows me to encrypt the different files to different keys. Every file is encrypted to my primary GnuPG key, but only some are encrypted with my work key. Login credentials needed for work are encrypted to the work key. I have no need to login to my bank accounts at work, and it wouldn’t be prudent to do so on a machine that I do not fully trust, so the financial.gpg file is not encrypted to my work key. If someone compromises my work computer, they still will be no closer to accessing my banking credentials.

Git

The ~/pw directory is a git repository. This gives me version control on all of my passwords. If I accidentally delete an entry I can always get it back. It also provides syncing and redundant storage without depending on a third-party like Dropbox.

Keys

An advantage of using a directory full of encrypted files as my password manager is that I’m not limited to only storing usernames and passwords. Any file can be added to the repository. I keep keys for backups, SSH keys, and SSL keys (all of which have been encrypted with my GnuPG key) in the directory. This gives me one location for all of my authentication credentials, which simplifies the locating and backing up of these important files.

Markup

Each file is structured with Vim folds and indentation. There are various ways for Vim to fold text. I use markers, sticking with the default {{{/}}} characters. A typical password entry will look like this:

1
2
3
4
5
Amazon{{{
    user:   foo@bar.com
    pass:   supers3cr3t
    url:    https://amazon.com
}}}

Each file is full of entries like this. Certain entries are grouped together within other folds for organization. Certain entries may have comments so that I have a record of the false personally identifiable information the service requested when I registered.

1
2
3
4
5
6
7
8
Super Ecommerce{{{
    user:   foobar
    pass:   g0d
    Comments{{{
        birthday:   1/1/1911
        first car:  delorean
    }}}
}}}

Following a consistent structure like this makes the file easier to navigate and allows for the possibility of the file being parsed by a script. The fold markers come into play with my Vim configuration.

Vim

I use Vim with the vim-gnupg plugin. This makes editing of encrypted files seamless. When opening existing files, the contents are decrypted. When opening new files, the plugin asks which recipients the file should be encrypted to. When a file is open, leaking the clear text is avoided by disabling viminfo, swapfile, and undofile. I run gpg-agent so that my passphrase is remembered for a short period of time after I use it. This makes it easy and secure to work with (and create) the encrypted files with Vim. I define a few extra options in my vimrc to facilitate working with passwords.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
""""""""""""""""""""
" GnuPG Extensions "
""""""""""""""""""""

" Tell the GnuPG plugin to armor new files.
let g:GPGPreferArmor=1

" Tell the GnuPG plugin to sign new files.
let g:GPGPreferSign=1

augroup GnuPGExtra
" Set extra file options.
    autocmd BufReadCmd,FileReadCmd *.\(gpg\|asc\|pgp\) call SetGPGOptions()
" Automatically close unmodified files after inactivity.
    autocmd CursorHold *.\(gpg\|asc\|pgp\) quit
augroup END

function SetGPGOptions()
" Set updatetime to 1 minute.
    set updatetime=60000
" Fold at markers.
    set foldmethod=marker
" Automatically close all folds.
    set foldclose=all
" Only open folds with insert commands.
    set foldopen=insert
endfunction

The first two options simply tell vim-gnupg to always ASCII-armor and sign new files. These have nothing particular to do with password management, but are good practices for all encrypted files.

The first autocmd calls a function which holds the options that I wanted applied to my password files. I have these options apply to all encrypted files, although they’re intended primarily for use when Vim is acting as my password manager.

Folding

The primary shortcoming with using an encrypted text file as a password database is the lack of protection against shoulder-surfing. After the file has been decrypted and opened, anyone standing behind you can look over your shoulder and view all the entries. This is solved with folds and is what most of these extra options address.

I set foldmethod to marker so that Vim knows to look for all the {{{/}}} characters and use them to build the folds. Then I set foldclose to all. This closes all folds unless the cursor is in them. This way only one fold can be open at a time – or, to put it another way, only one password entry is ever visible at once.

The final fold option instructs Vim when it is allowed to open folds. Folds can always be opened manually, but by default Vim will also open them for many other cases: if you navigate to a fold, jump to a mark within a fold or search for a pattern within a fold, they will open. By setting foldopen to insert I instruct Vim that the only time it should automatically open a fold is if my cursor is in a fold and I change to insert mode. The effect of this is that when I open a file, all folds are closed by default. I can navigate through the file, search and jump through matches, all without opening any of the folds and inadvertently exposing the passwords on my screen. The fold will open if I change to insert mode within it, but it is difficult to do that by mistake.

I have my spacebar setup to toggle folds within Vim. After I have navigated to the desired entry, I can simply whack the spacebar to open it and copy the credential that I need to the system clipboard. At that point I can whack the spacebar again to close the fold, or I can quit Vim. Or I can simply wait.

Locking

The other special option I set is updatetime. Vim uses this option to determine when it should write swap files for crash recovery. Since vim-gnupg disables swap files for decrypted files, this has no effect. I use it for something else.

In the second autocmd I tell Vim to close itself on CursorHold. CursorHold is triggered whenever no key has been pressed for the time specified by updatetime. So the effect of this is that my password files are automatically closed after 1 minute of inactivity. This is similar to KeePassX’s behaviour of “locking the workspace” after a set period of inactivity.

Clipboard

To easily copy a credential to the system clipboard from Vim I have two shortcuts mapped.

1
2
3
4
5
" Yank WORD to system clipboard in normal mode
nmap <leader>y "+yE

" Yank selection to system clipboard in visual mode
vmap <leader>y "+y

Vim can access the system clipboard using both the * and + registers. I opt to use + because X treats it as a selection rather than a cut-buffer. As the Vim documentation explains:

Selections are “owned” by an application, and disappear when that application (e.g., Vim) exits, thus losing the data, whereas cut-buffers, are stored within the X-server itself and remain until written over or the X-server exits (e.g., upon logging out).

The result is that I can copy a username or password by placing the cursor on its first character and hitting <leader>y. I can paste the credential wherever it is needed. After I close Vim, or after Vim closes itself after 1 minute of inactivity, the credential is removed from the clipboard. This replicates KeePassX’s behaviour of clearing the clipboard so many seconds after a username or password has been copied.

Generation

Passwords should be long and unique. To satisfy this any password manager needs some sort of password generator. Vim provides this with its ability to call and read external commands I can tell Vim to call the standard-issue pwgen program to generate a secure 24-character password utilizing special characters and insert the output at the cursor, like this:

1
:r!pwgen -sy 24 1

Backups

The ~/pw directory is backed up in the same way as most other things on my hard drive: to Tarsnap via Tarsnapper, to an external drive via rsnapshot and cryptshot, rsync to a mirror drive. The issue with these standard backups is that they’re all encrypted and the keys to decrypt them are stored in the password manager. If I loose ~/pw I’ll have plenty of backups around, but none that I can actually access. I address this problem with regular backups to optical media.

At the beginning of every month I burn the password directory to two CDs. One copy is stored at home and the other at an off-site location. I began these optical media backups in December, so I currently have two sets consisting of five discs each. Any one of these discs will provide me with the keys I need to access a backup made with one of the more frequent methods.

Of course, all the files being burned to these discs are still encrypted with my GnuPG key. If I loose that key or passphrase I will have no way to decrypt any of these files. Protecting one’s GnuPG key is another problem entirely. I’ve taken steps that make me feel confident in my ability to always be able to recover a copy of my key, but none that I’m comfortable discussing publicly.

Shell

I’ve defined a shell function, pw(), that operates exactly like the function I use for notes on Unix.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Set the password database directory.
PASSDIR=~/pw

# Create or edit password databases.
pw() {
    cd "$PASSDIR"
    if [ ! -z "$1" ]; then
        $EDITOR $(buildfile "$1")
        cd "$OLDPWD"
    fi
}

This allows me to easily open any password file from wherever I am in the filesystem without specifying the full path. These two commands are equivalent, but the one utilizing pw() requires fewer keystrokes:

1
2
$ vim ~/pw/financial.gpg
$ pw financial

The function changes to the password directory before opening the file so that while I’m in Vim I can drop down to a shell with :sh and already be in the proper directory to manipulate the files. After I close Vim the function returns me to the previous working directory.

This still required a few more keystrokes than I like, so I configured my shell to perform autocompletion in the directory. If financial.gpg is the only file in the directory beginning with an “f”, typing pw f<tab> is all that is required to open the file.

Simplicity

This setup provides simplicity, power, and portability. It uses the same tools that I already employ in my daily life, and does not require the use of the mouse or any graphical windows. I’ve been happily utilizing it for about 6 months now.

Initially I had thought I would supplement the setup with a script that would search the databases for a desired entry, using some combination of grep, awk and cut, and then copy it to my clipboard via xsel. As it turns out, I haven’t felt the desire to do this. Simply opening the file in Vim, searching for the desired entry, opening the fold and copying the credential to the system clipboard is quick enough. The whole process, absent of typing in my passphrase, takes me only a couple of seconds.

Resources

I’m certainly not the first to come up with the idea of managing password with Vim. These resources were particularly useful to me when I was researching the possibilities:

If you’re interesting in other ideas for password management, password-store and KeePassC are both neat projects that I follow.

2013 June 30: larsks has hacked together a Python script to convert KeepassX XML exports to the plain-text markup format that I use.