Discovering SmartOS

Posted on by
Revolutionize the datacenter: ZFS, DTrace, Zones, KVM

What 22TB looks like.

It has been a long and interesting weekend of fixing computers.
Adopt the pose: sit cross-legged on the floor surrounded by 9 hard-drives – wait, I need another one, make it 10 hard-drives – and the attendant spaghetti of SATA cables and plastic housings and fragments of case.
Funnily enough the need for screwdrivers has reduced over the years, albeit more than compensated by the cost of a case alone. I’m sure it never used to make for such a sore back either…
Anyway. Amidst the turmoil of fixing my main archive/work/backup server, I discovered a new OS.
For a few years now, I’ve been fond of ZFS – reliable as a brick, convenient as anything to use; I choose my OSes based on their ability to support ZFS, amongst other things. Just a quick
zpool create data /dev/ada1 /dev/ada2
zfs create data/Pictures
and that’s it, a new pool and filesystem created, another 1-liner command to add NFS sharing… Not a format or a mount in sight.
Of course, Linux has not been able to include ZFS in the kernel due to licensing considerations, so the various implementations (custom kernel; user-space FUSE module) have been less than desirable. So I’ve been using FreeBSD as server operating-system of choice. The most convenient way to control a plethora of virtual machines on a FreeBSD host seems to be to use VirtualBox – rather large and clunky nowadays.
However, a couple of weeks ago I stumbled across SmartOS, a new-to-me OS combining ZFS, DTrace and a Solaris/Illumos kernel, with both its own native Zones and Linux’s KVM virtualization.
There have been a few steps in this direction previously – most memorably was Nexenta, an opensolaris/illumos kernel with Debian packaging and GNU toolchain. That was a nice idea, but it lacked virtualization.
So, this weekend, with a storage server box rebuilt (staying with FreeBSD) and a whole new machine on which to experiment, I installed SmartOS.
Overall, it’s the perfect feature blend for running one’s own little cloud server. ZFS remains the filesystem of choice, DTrace has yet to be experimented with, and KVM is a breeze, mostly since Joyent have provided their own OS semi-installed images to work from (think: Docker, but without the Linux-specificity). The vmadm command shares a high-level succinctness with the zfs tools. Just import an image, make a JSON config file describing the guest VM and create an instance and it’s away and running with a VNC interface before you know it.
There’s one quirk that deserves special note so far. If you wish to use a guest VM as a gateway, e.g. via VPN to another network, you have to enable spoofing of IPs and IP forwarding on the private netblocks, in the VM config file.
      "allow_dhcp_spoofing": "true",
      "allow_ip_spoofing": "true",
      "allowed_ips": [ "192.168.99.0/24" ]
[root@78-24-af-39-19-7a ~]# imgadm avail | grep centos-7 
5e164fac-286d-11e4-9cf7-b3f73eefcd01 centos-7 20140820 linux 2014-08-20T13:24:52Z 
553da8ba-499e-11e4-8bee-5f8dadc234ce centos-7 20141001 linux 2014-10-01T19:08:31Z 
1f061f26-6aa9-11e4-941b-ff1a9c437feb centos-7 20141112 linux 2014-11-12T20:18:53Z 
b1df4936-7a5c-11e4-98ed-dfe1fa3a813a centos-7 20141202 linux 2014-12-02T19:52:06Z 
02dbab66-a70a-11e4-819b-b3dc41b361d6 centos-7 20150128 linux 2015-01-28T16:23:36Z 
3269b9fa-d22e-11e4-afcc-2b4d49a11805 centos-7 20150324 linux 2015-03-24T14:00:58Z 
c41bf236-dc75-11e4-88e5-038814c07c11 centos-7 20150406 linux 2015-04-06T15:58:28Z 
d8e65ea2-1f3e-11e5-8557-6b43e0a88b38 centos-7 20150630 linux 2015-06-30T15:44:09Z 

[root@78-24-af-39-19-7a ~]# imgadm import d8e65ea2-1f3e-11e5-8557-6b43e0a88b38 Importing d8e65ea2-1f3e-11e5-8557-6b43e0a88b38 (centos-7@20150630) from "https://images.joyent.com" 
Gather image d8e65ea2-1f3e-11e5-8557-6b43e0a88b38 ancestry 
Must download and install 1 image (514.3 MiB) 
Download 1 image [=====================================================>] 100% 514.39MB 564.58KB/s 15m32s 
Downloaded image d8e65ea2-1f3e-11e5-8557-6b43e0a88b38 (514.3 MiB) ...1f3e-11e5-8557-6b43e0a88b38 [=====================================================>] 100% 514.39MB 38.13MB/s 13s 
Imported image d8e65ea2-1f3e-11e5-8557-6b43e0a88b38 (centos-7@20150630) 
[root@78-24-af-39-19-7a ~]# 

[root@78-24-af-39-19-7a ~]# cat newbox.config 
{
  "brand": "kvm",
  "resolvers": [
    "8.8.8.8",
    "8.8.4.4"
  ],
  "ram": "256",
  "vcpus": "2",
  "nics": [
    {
      "nic_tag": "admin",
      "ip": "192.168.5.48",
      "netmask": "255.255.255.0",
      "gateway": "192.168.5.1",
      "model": "virtio",
      "primary": true,
      "allow_dhcp_spoofing": "true",
      "allow_ip_spoofing": "true",
      "allowed_ips": [ "192.168.99.0/24" ]
    }
  ],
  "disks": [
    {
      "image_uuid": "d8e65ea2-1f3e-11e5-8557-6b43e0a88b38",
      "boot": true,
      "model": "virtio"
    }
  ],
"customer_metadata": {
    "root_authorized_keys":
"ssh-rsa AAAAB3NzaC1y[...]"
  }

}
[root@78-24-af-39-19-7a ~]# vmadm create -f newbox.config 
Successfully created VM d7b00fa6-8aa5-466b-aba4-664913e80a2e 
[root@78-24-af-39-19-7a ~]# ping -s 192.168.5.48 
PING 192.168.5.48: 56 data bytes 
64 bytes from 192.168.5.48: icmp_seq=0. time=0.377 ms 
64 bytes from 192.168.5.48: icmp_seq=1. time=0.519 ms 
64 bytes from 192.168.5.48: icmp_seq=2. time=0.525 ms ... 

zsh, basalt% ssh root@192.168.5.48 
Warning: Permanently added '192.168.5.48' (ECDSA) to the list of known hosts.
Last login: Mon Aug  3 16:49:24 2015 from 192.168.5.47
   __        .                   .
 _|  |_      | .-. .  . .-. :--. |-
|_    _|     ;|   ||  |(.-' |  | |
  |__|   `--'  `-' `;-| `-' '  ' `-'
                   /  ;  Instance (CentOS 7.1 (1503) 20150630)
                   `-'   https://docs.joyent.com/images/linux/centos

[root@d7b00fa6-8aa5-466b-aba4-664913e80a2e ~]#

And there we have a new guest VM up and running in less than a minute’s effort.

Infrastructure and development environments recreated from scratch (partly thanks to storing my ~/etc/ in git) in under an hour.

I’m still looking for the perfect distributed filesystem, however…

Fields

Posted on by

Fields run deep in golden swards
Hot summer winds blow through the corn

Ripening corn/barley, Auchterarder.

Dunnottar Castle

Posted on by

Dunnottar Castle is a ruined medieval fortress located upon a rocky headland on the north-east coast of Scotland, about 2 miles south of Stonehaven. The current ruins date from the 15th and 16th centuries, but there is believed to have been fortification on the site since the Early Middle Ages.

The ruins of the castle are surrounded by steep cliffs that drop to the North Sea, 50 metres (160 ft) below. A narrow strip of land joins the headland to the mainland, with a steep path leading up to the gatehouse.

I made these photos over the course of a couple of hours after the eclipse in March, partly because I know the place well, partly because I was reminded of it by a photo in the local photo-club, and partly because I wanted to reshoot it at greater quality with newer processing techniques. It’s a pity the path down to the shore is so muddy – perhaps I should revisit either in winter or early spring instead.

For the record, the workflow for these is:

  • tripod, SRB ND1000 filter, multiple frames around 8s shutter-speed at source
  • RAW conversion in Photivo
  • HDR panorama in Hugin + enfuse
  • tonemapping in LuminanceHDR
  • post-processing in darktable
  • further post-processing in Gimp:
    • colour toning
    • film emulation (vintage, Ilford Pan-F or Rollei black and white film emulation)
    • wavelet sharpening
  • organization (tagging + metadata) in digiKam
  • bulk resizing with ImageMagick.

Moody Eclipse Photos

Posted on by

I was up early on the morning of March 20th to get to Stonehaven on the coast in time for the solar eclipse.

It’s funny how there was so much discussion as to what filters one should use when shooting the sun: on the one hand, a direct view of the sun’s disc requires special Baader solar filter (approx 23 stops’ filtration); however, when I arrived to see the extent of the clouds, only conventional photographic filters were needed (a mixture of ND1000 and circular polarizer 2-stop filters). And I think the results were all the more dramatic for it, too.

The first of these photos was made using a Centon 500mm mirror lens over 20 years old – from when I bought my first film SLR (a Canon EOS500n – that dates it) It even shows sunspot N 2303 pretty clearly.

The others are with the Sony 55-210mm lens at full stretch instead.

Each image is an HDR of 3 source frames bracketed +/-1EV, converted in photivo, blended in enfuse and worked in darktable.

Glen Clunie: Landscape Vistas

Posted on by

I must admit to not having found the landscape in Glen Clunie particularly inspiring – good for covering distance whilst hiking but not many trees to catch one’s interest. However, the resultant photos have some merit – hopefully the convey a sense of the expansive topography of the post-glacial floodplain through which the Baddoch Burn runs.

The particularly dark photo is an experiment inspired by my twitter friend Neil Mansfield‘s work with Dark Landscapes.

As I was returning back along the glen, three dogs in the garden of the stone house started shouting and running around me; the owners invited me in and plied me with tea. Next thing you know there’s three dogs all clambering over my knee on the sofa. Highland hospitality at its best.

In the Woods

Posted on by

It’s been a while since I made photos of closeups in the woods – and for the most part, last time around I avoided contrasty light for the purpose too. Last night, I took a single prime lens (my favourite Pentacon 50mm f/1.8 of old) and one of my favoured strolls over Craigie Hill around the golf course, seeing what there was to be seen under the trees…

Falls of Bruar: Flow

Posted on by

One Saturday lunchtime several years ago, I spent a happy hour bugging the assistants in my local favourite camera shop, trying to find the ideal tripod.

Having visited the Falls of Bruar the weekend previously, I had a particular photo of the waterfalls flowing around the rocks in mind.

As usual, Manfrotto was the most recommended make. I tried to believe in them, honestly, but with no combination of legs, invertible centre-column and 3-dimensional head being sturdy enough for the camera of the time, I emerged with a Slik. (This process has been repeated with the same outcome a few times since.)

That afternoon, I went back to Bruar with my new tripod and totally failed to get the photo I wanted, but by dint of pointing the camera the other way staring down the gorge after sunset had happened and the light was fading – what’s come to be known as the blue hour – I wound up with a photo that would be my No.1 most-popular on Flickr for about 5 years.

That was “Raw“.

The first of these is “Flow”, the photo I intended to make in the first place.

Water close-ups

Posted on by

A small series of closeup studies in flowing water, taken on a stroll around the Falls of Bruar.

I’ve admired the striation lines  in the psammite riverbed below the lower bridge at the Falls many times – yet every visit they’re still fascinating every time.

Noctilucent Clouds, Perth, 20150623

Posted on by

A classic location for long-exposure night-time photography: standing on the bridge over the M90 at Rhynd, with the road snaking away into the distance… and a clear display of noctilucent clouds above Kinnoull Hill.

From wikipedia:

Night clouds or noctilucent clouds are tenuous cloud-like phenomena that are the “ragged edge” of a much brighter and pervasive polar cloud layer called polar mesospheric clouds in the upper atmosphere, visible in a deep twilight. They are made of crystals of water ice. Noctilucent roughly means night shining in Latin. They are most commonly observed in the summer months at latitudes between 50° and 70° north and south of the equator. They can be observed only when the Sun is below the horizon.

They are the highest clouds in Earth’s atmosphere, located in the mesosphere at altitudes of around 76 to 85 kilometres (47 to 53 mi). They are normally too faint to be seen, and are visible only when illuminated by sunlight from below the horizon while the lower layers of the atmosphere are in the Earth’s shadow. Noctilucent clouds are not fully understood and are a recently discovered meteorological phenomenon; there is no record of their observation before 1885.

Dramatic Sunset

Posted on by

In the evening of Feb 7 2015, I was driving back along the A90 from Dundee to Perth, right into this sunset. It lasted a while, so I pulled over beside the road in the Carse of Gowrie to shoot it, including a reflection in the car roof (as one does!).

Stunning light.

A Favourite Walk

Posted on by

Around January I discovered a new walk near Dunkeld that quickly became a favourite way to spend a weekend afternoon. Starting from the Pass of Birnam, head up the track from Bee Cottage and turn left to go around the south side of Duncan’s Hill then rejoin the path up to Stair Bridge Viewpoint and the top of Birnam Hill.

As routes go, it gives a mixture of sheltered woodland tracks and sweeping landscape views, complete with my favourite feature – you can watch the rocks changing from till to slate to psammite and semi-pelite as you cross the Highland Boundary Fault. Small wonder I’ve done it half a dozen times dragging various folks along with me, gradually exploring further each time as the winter receded.

These photos are from an experiment with a Prakticar 24mm lens (M42 fit) – acquired for cheap from ebay and stuck on a wonky adapter which might explain some focussing issues. Several of them depict the line of the HBF through the landscape, with hills on one side in the Highlands and on the other in the Lowlands.