Quick Time Machine Tip

Tip first, because the backstory ended up being longer to write out than I thought:

I’ve been using Time Machine for a few months now and recently ran into an issue where my backups were filling up too quickly. At first I just disconnected my external HDD and made a mental note to investigate the issue (haha free time), but ultimately only “made time” when it gave me a message like “hey, just wanted to let you know I’m deleting some of your older backups”.

Like hell you are, Time Machine. Like hell you are. We haven’t been in a relationship long enough for you to start telling me that I need to get rid of my things because there’s too much clutter in the house. I mean drive. I meant drive.

Aaaaanyway. It turns out there is a Quick and Easy way to get Time Machine to exclude things it really, really has no business backing up. Like, say, 50 GB worth of virtual machines. Stuff like that.

It’s actually pretty straightforward to do this and I think I can successfully capture it in two screenshots.

And now for some backstory:

As is probably obvious by now I do most of my development on a Mac. I was previously working on a 2012 Macbook Pro from work, but then when it started to have some weird issues it was supposed to be sent off to Apple, repaired, and rehomed while I got to work on a new 2015 Macbook Pro.

As with all things technology this didn’t go as smoothly as anyone would have desired. After going in over the weekend to diagnose the heat issue that wasn’t entirely the fault of the fans, I went to pick up my new 2015 Macbook Pro. While I was there asked if I could bring in the 2012 the next day, instead of dropping it off then, after making sure I could get everything on the new Macbook. Apple Genius bar said it was no problem, so hurray.

Except no hurray, because after I arrived home and turned on the new, shiny, 2015 Macbook Pro it kernel panicked on the first try. Yikes. So I restarted it, was eventually brought through the prompts that all Mac users are familiar with by now to sync all the things. Yay. And things seemed fine, for a time. Until bam. Panic. Shut down.

Ooookaaaay… so then I tried resetting the NVRAM. Rebooted. Seemed fine. Until, again, kernel panic and shut down. This happened a total of four times in the span of … maybe an hour IIRC. After that, I tried just letting the 2015 Mac idle instead of transferring my data onto it, but alas it still panicked just by being on – even when I wasn’t doing anything to it.

So I went back to the Genius Bar with the new 2015 Mac. Explained that it wasn’t the 2012 Mac they thought I’d be dropping off, but that new 2015 Mac had been kernel panicking even without me touching it. So then they tried to boot it and it didn’t boot at all, haha. They ran some hardware tests and everything passed so they tried booting it again and it still didn’t boot. So they handed me a new 2015 Macbook Pro no-(other)-questions asked. In their words “a brand new machine shouldn’t behave this way, we’ll just exchange it”.

Apple Genius bar = lovely people to work with.

Of course when the 2012 Macbook Pro started having heat issues I started to become more attentive with my backups. I didn’t use Time Machine, I just backed up my files to my Synology NAS. I figured that if and when I set up a new machine it would be the perfect time to “declutter” my environment and only install new things and set them up as I need them.

That was before I tried setting up two new environments in a week while also worrying about what would happen if my New New Macbook ended up with similar behavior problems and then I’d have to do it again.

So that’s when I decided something needed to happen and I started using Time Machine. Which brings us full circle to the Quick Tip at the top.

By the way, as for my New New Macbook Pro it is mostly problem free with the exception of some pesky networking issues…

New languages, new woes: tonight’s Golang roadblock

Right now I’m working on learning Go by writing a small app that will, eventually, take a Markdown file and convert it to an HTML presentation. When I started working on my program today, my code wouldn’t compile. It was referring to missing packages/etc. I thought, well this can’t be good.

First step, check $GOPATH and $GOROOT. In my case, $GOROOT was null and $GOPATH was returning its old value. (Now that I’ve figured out what’s going on, I’m pretty sure that those more experienced than I already know where this is going…) I fixed $GOPATH to include the new path and $GOROOT to, IIRC, /usr/local/go. Then I checked my $PATH variable and thankfully that was still fine.

I tried running my program again, only to encounter this lovely gem*:

/usr/local/go/pkg/tool/darwin_amd64/6g: unknown flag -trimpath

I had NO idea what this error meant, so I googled for it. It took a little time for me to find something useful, but thankfully Stack Overflow came to my rescue (…again). I learned that I needed to reinstall Go after first making sure I removed the current version. Lovely.

I head over to the Go project page and check how to uninstall Go. Short and sweet, everything seemed fine. Then I download the current installer for Mac OS X, which as of this writing is for 1.3.1, and off I go.

I re-open terminal and try to run my little app. Unfortunately, I still have import problems (I wish I had saved those errors to paste in here, sorry guys). I try to reinstall blackfriday, which is a Markdown processor in Go:

$ go get github.com/russross/blackfriday
# github.com/russross/blackfriday
../../russross/blackfriday/sanitize.go:6: import /Users/ladyivangrey/Development/Go/pkg/darwin_amd64/code.google.com/p/go.net/html.a: object is [darwin amd64 go1.3 X:precisestack] expected [darwin amd64 go1.3.1 X:precisestack]

Great. I have NO idea what this means as it appears that the expected and fond objects are the same. So I “phone a friend”.

His first question? “Are you running more than one version of Go?”

So I tell him: No. Well, I hope not. Maybe. I’m not trying to. So I tell him the story: when I first downloaded Go I just wanted to put it somewhere, so I had it on my Desktop. Then I decided that was a bad idea. So I moved it to ~/Development which is where I have all my other code projects. Then I tried updating the various environmental variables and it all seemed fine. Until I reopened terminal. So I removed the old, I think, to install the new.

He had me check the version using the explicit path to the command. I then tried running it without the path, to see if the output differed:

$ go version
go version go1.3 darwin/amd64

$ /usr/local/go/bin/go version
go version go1.3.1 darwin/amd64

Yeah. Two versions of Go. To see where the other version of Go was installed, I ran “which go”:

$ which go
/usr/local/bin/go

Ah. See. There it is then. Great.

Before removing the “old” version of Go I updated my ~/.bash_profile to the current path I am using for Go: ~/Development/Go/. Then I closed and re-opened Terminal and everything was fine:

$ go run main.go -inputFile=~/Development/temp/myfile.md -log=info
2014-09-18 20:33:23.509 INFO Starting gopress
2014-09-18 20:33:23.511 WARNING Output directory unspecified or does not exist, creating new directory: /Users/ladyivangrey/Development/temp/myfile/
2014-09-18 20:33:23.514 INFO Successfully created new directory.
2014-09-18 20:33:23.517 INFO Successfully copied files.
2014-09-18 20:33:23.517 INFO Exited with no errors.

I even ran “go get” successfully, which after all this was a relief.

Then the magic happened, removing the old version:

$ rm /usr/local/bin/go

And everything was right again in my world. Good grief.

*I suppose I should use the term “gem” with caution since I may run into Ruby again. Don’t want to give it any ideas.
** Also posted this article on my employer’s blog.

And now… BOSH Lite (and Cloud Foundry)

After successfully running the routes script it was finally time to get to starting up vagrant – then onto BOSH Lite and Cloud Foundry.

Since both AWS and VMWare Fusion would need to be purchased for their options to work, I choose the free option (Virtual Box). To make sure that the Virtual Box option was the only option ever executed, I went into the Vagrantfile and commented out the VMWare Fusion portion of the file – leaving only Virtual Box.

Note: Vagrantfiles do not support multiline comments, so you have to comment unwanted lines out one at a time using #.

This means that for my setup, the following two lines are the same:

> vagrant up –provider virtualbox

> vagrant up

I suppose you could argue it’s not really necessary, but I don’t want to specify virtualbox every time when it’s the only provider I intend to use.

So to sync up with the directions I’m following on the repo page, thus far I have:

• Installed vagrant
• Created and booted the VM (with the “Using the Virtualbox Provider”) instructions
• Skipped restarting the director, as that wasn’t necessary in my page

So now I’m at the Stemcell portion. I worked on this with someone and, thank goodness, because this part actually went quite quickly.

My initial assumption was that I needed to get the stemcell while logged into the VM (the instructions use the wget command, which is usually indicative of Linux, and this setup is configured to use Ubuntu in the Vagrant VM); however, apparently this is incorrect: the wget is executed in terminal before you go into the VM (using some variable of “vagrant up” as needed).

Since I don’t have wget installed on my Mac, I grab the command using brew. Thankfully, unlike what happened with nokogiri, brew did not laugh in my face this time:

> brew install wget
[Bunch of output.]
> wget http://bosh-jenkins-gems-warden.s3.amazonaws.com/stemcells/latest-bosh-stemcell-warden.tgz
[Bunch more ouput.]
> bosh upload stemcell latest-bosh-stemcell-warden.tgz
[Output for a successful install]

Success! After completing all of that, I decide that I would try getting the stemcell through their other method, the public stemcells. To see the current listing:

> bosh public stemcells

Although the directions indicate that I should be looking for bosh-stemcell-24-warden-boshlite-ubuntu.tgz, I’m reasonably certain this is an older version. That said, I don’t see anything remotely similar in the output. So I try it anyway:

> bosh download public stemcell bosh-stemcell-24-warden-boshlite-ubuntu.tgz
‘bosh-stemcell-24-warden-boshlite-ubuntu.tgz’ not found.

Well… no surprises there I suppose. My mentor suggests that I look at the Bosh Artifacts page in case the output was truncated in some way. Unfortunately, there’s nothing of the format bosh-stemcell-<##>-warden-boshlite-ubuntu.tgz there either. Or anything that even says warden at all.

To see my current stemcell version, I enter the following:

bosh stemcells

Right now, I’m on version 60, so I tried this:

bosh download public stemcell bosh-stemcell-60-warden-boshlite-ubuntu.tgz

And…. this also failed (for the same reason). I think it’s safe to say that the second method they show on the instructions is no longer available, alas.

Although, I think I prefer the first method anyway. The second requires actually figuring out the current version and then manually putting that in, whereas the former method just grabs the latest release of that type.

Cloud Foundry

I have finally hit what all of this set up is for: to run Cloud Foundry locally. There appear to be two sets of instructions. One that I can actually read and the next that my Twitter-generation brain checks out of a little bit. Let’s try the “easy” one liner first and see if that will run without heartache:

./scripts/provision_cf
[Whole bunch of output that looks good]
./scripts/provision_cf: line 32: cd: /Users/ladyivangrey/Development/cf-release: No such file or directory

Ah, damn it. That’s what I get for being hopeful.

Let’s try it the manual way.

Wait a minute, looks like I need to start by cloning the CF repo. Well, maybe there’s hope for the script after all!

> git clone https://github.com/cloudfoundry/cf-release
> cd ~/cf-release
> ./update
[Shit ton of output, this may take a few minutes depending on your connection.]

My only complaint with the update is, while tons of information was filling my screen, there was no way to track how far the update had actually progressed. I mean, sure, it told me what it was doing at each second, but  not (e.g.) if it was on step 7/10 or 7/100,000. This is important information!

Anyway, once all that finished I did see a nice “CF Release has been updated” message, and there weren’t any errors, so I think I should just count my blessings.

According to the cf release page, the latest release as of this writing is 173, so:

> git checkout v173
> bosh upload release releases/cf-173.yml

It may come as no surprise, but that last one is another one of those “go get a cup of coffee” commands.

Since I’m still waiting on it ~20 minutes later, I’d like to humbly suggest you may want to get dinner while you wait as well.

Now that all of that is taken care of, I will try to provision_cf again:

> ./scripts/provision_cf
[Whole bunch of output that looks good]
./scripts/provision_cf: line 32: cd: /Users/ladyivangrey/Development/cf-release: No such file or directory

Of course. Ok, spiff it is.

> brew tap xoebus/homebrew-cloudfoundry
> brew install spiff

Now if I just run “spiff” it gives me the help output – how to use the application. Good! Progress… now:

> ./scripts/make_manifest_spiff
Cannot find cf-release at /Users/ladyivangrey/workspace/cf-release; override with $CF_RELEASE_DIR variable

Right on… simple text edit (FYI: my directory is /Users/ladyivangrey/Development/bosh-lite/cf-release), so the message makes sense). And again:

> ./scripts/make_manifest_spiff

Good! Successful execution (I’ve stopped putting bracketed output placeholders for now).

Now for bosh deploy:

> bosh deploy

This is going to take awhile… meal break!

Annnnnd we’re back. Since I already have Go, I could head straight to the acceptance tests:

> go get github.com/cloudfoundry/cf-acceptance-tests
> cd $GOPATH/src/github.com/cloudfoundry/cf-acceptance-tests

Although I have successfully set that up, I don’t really have anything to test yet, so I’m going to finish actually running the tests at this point. I think it is now time for me to actually progress to using the software so I can get started on the tasks that this was all for!

Thank goodness.

Adding Routes on Mac OS 10.9.x

While working with the BOSH-lite set up, I found I could not add routes. Specifically, this happened:

> vagrant up
> bosh target 192.168.50.4 lite
Your username: [admin username]
Your password: [admin password]
> scripts/add-route
Adding the following route entry to your local route table to enable direct warden container access. Your sudo password may be required.
Password: [sudo password]

sudo: route: command not found

First step to figuring out what is going on: figuring out what the add-route script is trying to execute. So I go less’ed the file and saw this:

#!/bin/bash

echo “Adding the following route entry to your local route table to enable direct warden container access. Your sudo password may be required.”

echo ”  – net 10.244.0.0/19 via 192.168.50.4″

if [ `uname` = “Darwin” ]; then
sudo route delete -net 10.244.0.0/19 192.168.50.4 > /dev/null 2>&1
sudo route add -net 10.244.0.0/19 192.168.50.4
elif [ `uname` = “Linux” ]; then
sudo route add -net 10.244.0.0/19 gw 192.168.50.4
fi

Next step: simply execute the Mac lines myself:

> sudo route delete -net 10.244.0.0/19 192.168.50.4 > /dev/null 2>&1
> sudo route add -net 10.244.0.0/19 192.168.50.4
sudo: route: command not found

Ok. So, specifically the problem appears to be “route add”. Quick sanity check: what is the syntax for route add in Mac OS 10.9? Well, according to at least one source the syntax is slightly different. In the add-route file the syntax for the Mac and Linux commands is the same for this line… I wonder if that’s the issue?

> sudo route -n add -net 10.244.0.0/19 192.168.50.4
sudo: route: command not found

Great.

Fishing for more information, or at least more helpful error messages, I tried this:

> sudo which route
> sudo which ip

You may notice I did not include any output above – because there wasn’t any. Not even an error message. On the bright side: there wasn’t an error message! On the downside: there wasn’t an error message…

Next try: figuring out what parameters are actually being used with the route command. According to the man file, the following syntax is valid:

route [-n] command [-net | -host] [-ifscope boundif] destination gateway [netmask]

Well, I need to add (add) the route and I need it to be interpreted as a network (-net), but I don’t necessarily need to “bypass attempts to print host and network names … when reporting actions”. Unfortunately, dropping “-n” does not affect the result.

After some additional research, I come across netstat. Thinking it couldn’t hurt to view the routing tables, I view the man page and discover I need to use the -r option:

> netstat -r
-bash: netstat: command not found
> sudo netstat -r
sudo: netstat: command not found

Clearly terminal does NOT love me today. It actually hurts my feelings a little bit as I have given terminal nothing but love since … forever, really. It must still be angry after the whole bundler/nokogiri thing yesterday… maybe I should send flowers.

After surfing googling for a bit longer, I came across some interesting advice: check your PATH variable.

My first thought: why would anything be wrong with my PATH variable? I’ve always treated it nicely.

My second thought: my Air has the moods of a cat and would probably seek retribution at the smallest slight. See yesterday’s entry.

Wouldn’t you know: I view the PATH variable and although there are various bin/sbin paths in there, I do NOT actually see the path to where these commands actually are.

Well, damn it.

> export PATH=$PATH:/sbin
> sudo route -n add -net 10.244.0.0/19 192.168.50.4
add net 10.244.0.0: gateway 192.168.50.4
> sudo which route
/sbin/route

Success! Maybe? After having issues with netstat, I decided to test that out again. I know I’ll be needing those tools at some point in the project, so it’s probably good to have them all working… right?

> netstat -i
-bash: netstat: command not found

> sudo lsof -i :8080
sudo: lsof: command not found

Awesome. Clearly I am missing more than just sbin. Mhm. Maybe chocolates and flowers…

Of course, as with all things, wherever a problem has been encountered, it is highly likely that someone else has had the same problem (or at least a similar one). It’s probably best if I see what is in the default PATH rather than picking through what I do and do not have (between ruby, Go, and misc. projects my PATH variable has gotten a tad bulky). According to a user on Stack Overflow, this is what their (virgin?) PATH variable looks like:

> echo $PATH
/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin

So I’m just going to add ALL these things:

> export PATH = $PATH:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin

Now to see if the two missing commands are back:

> netstat -i
[Whole bunch of output]
> lsof -i :8080
[No output, but no error]
> lsof -i -u ladyivangrey
[Whole shit ton of output]

I have my missing commands again! Booyah! \o/

Disclaimer: Using the export method is temporary in the sense that it will only change the PATH variable for the current session. If you close/re-open terminal (or iTerm, whatever your app of choice is), the changes will not be there. I will post about permanent changes to the PATH variable at a different time.