Glen Turret is an obvious glacial valley. At the north end of the loch the land is covered with undulating mounds – morraine, formed by the receding glacier.
Monthly Archives: August 2015
Glen Turret: Dark
Two of my twitter friends have developed particular styles – extreme dark low-key black+white rendition and negative inversion, respectively. It’s intriguing how scenes come out – a very different mapping from the usual realism.
Glen Turret: Wet Stuff
A short study in water – taken on a stroll along the side of Loch Turret.
Too-Big Data? That don’t impress me much
On a whim, I spent the evening in Edinburgh at a meetup presentation/meeting concerning Big Data, the talk given by a “Big Data hero” (IBM’s term), employed by a huge multinational corporation with a lot of fingers in a lot of pies (including the UK Welfare system).
I think I was supposed to be awed by the scale of data under discussion, but mostly what I heard was all immodest massive business-speak and buzzwords and acronyms. A few scattered examples to claim “we did that”, “look at the size of our supercomputer”, but the only technical word he uttered all evening was “Hadoop”.
In the absence of a clear directed message, I’ve come away with my own thoughts instead.
So the idea of Big Data is altogether a source of disappointment and concern.
There seems to be a discrepancy: on the one hand, one’s fitbit and phone are rich sources of data; the thought of analyzing it all thoroughly sets my data-geek senses twitching in excitement. However, the Internet of Things experience relies on huge companies doing the analysis – outsourced to the cloud – which forms a disjoint as they proceed to do inter-company business based on one’s personal data (read: sell it, however aggregated it might be – the presenter this evening scoffed at the idea of “anonymized”), above one’s head and outwith one’s control. The power flows upwards.
To people such as this evening’s speaker, privacy and ethics are just more buzzwords to bolt on to a “data value pipeline” to tout the profit optimizations of “data-driven companies”. So are the terms data, information, knowledge and even wisdom.
But I think he’s lost direction in the process. We’ve come a long way from sitting on the sofa making choices how to spend the evening pushing buttons on the mobile.
And that is where I break contact with The Matrix.
I believe in appreciating the value of little things. In people, humanity and compassion more than companies. In substance. In the genuine kind of Quality sought by Pirsig, not as “defined” by ISO code 9000. Value may arise from people taking care in their craft: one might put a price on a carved wooden bowl in order to sell it, but the brain that contains the skill required to make it is precious beyond the scope of the dollar.
Data is data and insights are a way to lead to knowledge, but real wisdom is not just knowing how to guide analysis – it’s understanding that human intervention is sometimes required, and knowing when to deploy it, awareness, critical thinking to see and choose.
The story goes that a salesman once approached a pianist, offering a new keyboard “with eight nuances”. The response came back: “but my playing requires nine”.
Portknockie (3/3): Bow-Fiddle
I’ve left the usual photos to last, seeing as how everyone else has shot this scene before.
It wasn’t particularly easy; the tripod was struggling to stay steady in the breeze and the course of a few seconds between adjusting the camera, leaving it to stop vibrating and pushing the shutter remote release, the light was changing radically from dull shade to bright sunlight on the foreground rocks. Still, a moderately long exposure worked, eventually.
Herewith, four different ways of processing the same images.
Perth Highland Games
I spent much of this afternoon at the Perth Highland Games held on the North Inch. Amongst other things, there were many stalls selling leather and jewellery products, cyclists, heavy-weight sports including the shot-put and hammer, lots of pipe & drum bands (including St Andrew’s Pipe Band from Brisbane, Australia) and several cuddly dogs (never met a Pharoah Hound before!).
Portknockie (2/3): colourful rocks
The coast at Portknockie features an intermingling of Cullen quartzite (dating from Lower Dalradian times, 650 million years ago during which time they’ve transformed from sedimentary sandstone through partial volcanic metamorphosis) and the usual Highland psammite and semi-pelite.
The colours in these photos are more or less natural; it was totally stunning to be in the shady cave with the daylight behind and beyond, with these huge colourful boulders to play with.
For a sense of scale: the photos featuring a distant patch of light playing on the sandy pebble floor, well that gap is large enough to walk right through. A veritable cathedral of colour.
Portknockie (1/3): an unusual view
This is by way of a little tease. The Interwebs are full of photos of this lump of rock taken from a few yards to the right; I thought it would be more interesting to see it through the mouth of an adjacent cave instead.
Discovering SmartOS
What 22TB looks like.
It has been a long and interesting weekend of fixing computers.
Adopt the pose: sit cross-legged on the floor surrounded by 9 hard-drives – wait, I need another one, make it 10 hard-drives – and the attendant spaghetti of SATA cables and plastic housings and fragments of case.
Funnily enough the need for screwdrivers has reduced over the years, albeit more than compensated by the cost of a case alone. I’m sure it never used to make for such a sore back either…
Anyway. Amidst the turmoil of fixing my main archive/work/backup server, I discovered a new OS.
For a few years now, I’ve been fond of ZFS – reliable as a brick, convenient as anything to use; I choose my OSes based on their ability to support ZFS, amongst other things. Just a quick
zpool create data /dev/ada1 /dev/ada2 zfs create data/Pictures
and that’s it, a new pool and filesystem created, another 1-liner command to add NFS sharing… Not a format or a mount in sight.
Of course, Linux has not been able to include ZFS in the kernel due to licensing considerations, so the various implementations (custom kernel; user-space FUSE module) have been less than desirable. So I’ve been using FreeBSD as server operating-system of choice. The most convenient way to control a plethora of virtual machines on a FreeBSD host seems to be to use VirtualBox – rather large and clunky nowadays.
However, a couple of weeks ago I stumbled across SmartOS, a new-to-me OS combining ZFS, DTrace and a Solaris/Illumos kernel, with both its own native Zones and Linux’s KVM virtualization.
There have been a few steps in this direction previously – most memorably was Nexenta, an opensolaris/illumos kernel with Debian packaging and GNU toolchain. That was a nice idea, but it lacked virtualization.
So, this weekend, with a storage server box rebuilt (staying with FreeBSD) and a whole new machine on which to experiment, I installed SmartOS.
Overall, it’s the perfect feature blend for running one’s own little cloud server. ZFS remains the filesystem of choice, DTrace has yet to be experimented with, and KVM is a breeze, mostly since Joyent have provided their own OS semi-installed images to work from (think: Docker, but without the Linux-specificity). The vmadm command shares a high-level succinctness with the zfs tools. Just import an image, make a JSON config file describing the guest VM and create an instance and it’s away and running with a VNC interface before you know it.
There’s one quirk that deserves special note so far. If you wish to use a guest VM as a gateway, e.g. via VPN to another network, you have to enable spoofing of IPs and IP forwarding on the private netblocks, in the VM config file.
"allow_dhcp_spoofing": "true",
"allow_ip_spoofing": "true",
"allowed_ips": [ "192.168.99.0/24" ]
[root@78-24-af-39-19-7a ~]# imgadm avail | grep centos-7 5e164fac-286d-11e4-9cf7-b3f73eefcd01 centos-7 20140820 linux 2014-08-20T13:24:52Z 553da8ba-499e-11e4-8bee-5f8dadc234ce centos-7 20141001 linux 2014-10-01T19:08:31Z 1f061f26-6aa9-11e4-941b-ff1a9c437feb centos-7 20141112 linux 2014-11-12T20:18:53Z b1df4936-7a5c-11e4-98ed-dfe1fa3a813a centos-7 20141202 linux 2014-12-02T19:52:06Z 02dbab66-a70a-11e4-819b-b3dc41b361d6 centos-7 20150128 linux 2015-01-28T16:23:36Z 3269b9fa-d22e-11e4-afcc-2b4d49a11805 centos-7 20150324 linux 2015-03-24T14:00:58Z c41bf236-dc75-11e4-88e5-038814c07c11 centos-7 20150406 linux 2015-04-06T15:58:28Z d8e65ea2-1f3e-11e5-8557-6b43e0a88b38 centos-7 20150630 linux 2015-06-30T15:44:09Z [root@78-24-af-39-19-7a ~]# imgadm import d8e65ea2-1f3e-11e5-8557-6b43e0a88b38 Importing d8e65ea2-1f3e-11e5-8557-6b43e0a88b38 (centos-7@20150630) from "https://images.joyent.com" Gather image d8e65ea2-1f3e-11e5-8557-6b43e0a88b38 ancestry Must download and install 1 image (514.3 MiB) Download 1 image [=====================================================>] 100% 514.39MB 564.58KB/s 15m32s Downloaded image d8e65ea2-1f3e-11e5-8557-6b43e0a88b38 (514.3 MiB) ...1f3e-11e5-8557-6b43e0a88b38 [=====================================================>] 100% 514.39MB 38.13MB/s 13s Imported image d8e65ea2-1f3e-11e5-8557-6b43e0a88b38 (centos-7@20150630) [root@78-24-af-39-19-7a ~]# [root@78-24-af-39-19-7a ~]# cat newbox.config { "brand": "kvm", "resolvers": [ "8.8.8.8", "8.8.4.4" ], "ram": "256", "vcpus": "2", "nics": [ { "nic_tag": "admin", "ip": "192.168.5.48", "netmask": "255.255.255.0", "gateway": "192.168.5.1", "model": "virtio", "primary": true, "allow_dhcp_spoofing": "true", "allow_ip_spoofing": "true", "allowed_ips": [ "192.168.99.0/24" ] } ], "disks": [ { "image_uuid": "d8e65ea2-1f3e-11e5-8557-6b43e0a88b38", "boot": true, "model": "virtio" } ], "customer_metadata": { "root_authorized_keys": "ssh-rsa AAAAB3NzaC1y[...]" } } [root@78-24-af-39-19-7a ~]# vmadm create -f newbox.config Successfully created VM d7b00fa6-8aa5-466b-aba4-664913e80a2e [root@78-24-af-39-19-7a ~]# ping -s 192.168.5.48 PING 192.168.5.48: 56 data bytes 64 bytes from 192.168.5.48: icmp_seq=0. time=0.377 ms 64 bytes from 192.168.5.48: icmp_seq=1. time=0.519 ms 64 bytes from 192.168.5.48: icmp_seq=2. time=0.525 ms ... zsh, basalt% ssh root@192.168.5.48 Warning: Permanently added '192.168.5.48' (ECDSA) to the list of known hosts. Last login: Mon Aug 3 16:49:24 2015 from 192.168.5.47 __ . . _| |_ | .-. . . .-. :--. |- |_ _| ;| || |(.-' | | | |__| `--' `-' `;-| `-' ' ' `-' / ; Instance (CentOS 7.1 (1503) 20150630) `-' https://docs.joyent.com/images/linux/centos [root@d7b00fa6-8aa5-466b-aba4-664913e80a2e ~]#
And there we have a new guest VM up and running in less than a minute’s effort.
Infrastructure and development environments recreated from scratch (partly thanks to storing my ~/etc/ in git) in under an hour.
I’m still looking for the perfect distributed filesystem, however…