Random Musings

It’s 2012. When I started this blog, I never imagined that it would live so long. I created the blog one bored afternoon in 2005, back when I was in law school. The reason you don’t see posts from 2005 is simple—I lost the database while moving the site. Now, of course, I backup. Multiple places.

When I started writing here, I didn’t know what I would write about. Now, seven years later, I still don’t know! I’ve written about math and technology and sport and books and friends and family and dogs and bikes and cars.. And it’s always been crazy fun.

Early on, I decided that I wouldn’t be editing my posts. Sure, I’d correct egregious spelling mistakes, but I would not go over what I’d written, trying to fix it. Practically all my posts are written in one session; I don’t think I’ve ever used the drafts feature. While this does not make for the most compelling prose, it does keep me honest. I always looked at this blog as an odd mashup of a private journal and a public blog; I write about stuff that I care about, but not in too much detail, just in case.

And I’m glad I did. More than anything else, this blog has become a way for me to remember my own life. Even though I may have forgotten things, reading these posts brings it all back. Even the most innocuous post is full of meaning for me, sparking associations in my mind and reminding me of what I was thinking about when I wrote it. Through law school, through moving to Bangalore and working for PC World, through moving to the US… it’s been here, waiting for me to fill the white box. Some of my posts have been excellent, others not so much.

Onwards then, to seven, nay, seventy! more years.

I’m immersed in Big Data these days. The company that I work for, EMC, is one of the big players in this space, and I can’t help but see lots of new stuff that flows across the company. Now, I’m not going to talk about prerelease hardware or software, so don’t get your hopes up.

My own definition of Big Data is quite simple. It’s the collection, capture, and analysis of billions of data points about any subject. Sure, databases have been doing this kind of thing for ages; the difference is that Big Data is very often unstructured, and we have tools and software (Hadoop etc) that allow us to analyse this huge mass of data and make sensible inferences.

I’ve been asked if there is a clear delineation between ‘Data’ and ‘Big Data’. The best way to answer that is to paraphrase Justice Potter Stewart from the US Supreme Court: “I shall not today attempt further to define the kinds of material I understand to be embraced . . . but I know it when I see it . . . “. He made this remark in a completely different context (trying to define obscenity in the case of Jacobellis v. Ohio, 378 U.S. 184, 197 (1964)) but it’s just as relevant.

There’s no clearly defined line past which data becomes Big; no place where we can say “below this limit it’s just data, and above that it’s Big Data”. You know it when you see it is the best answer there is, and I personally believe that today’s understanding of what makes Big Data Big will change over the years. When we all have access to Petabytes of storage, Big Data will mean something else entirely.

However, that’s neither here nor there. I’m more concerned about certain aspects of big data. A major concern is the drawing of incorrect inferences. When people have access to such large amounts of data, and the tools to slice and dice the data as they see fit, the temptation to confuse correlation and causation is almost irresistible. As an example: suppose we are able to correlate instances of diabetes with school dropout rates. If there is a strong positive correlation, we can then state that education leads to less diabetes.

You do see the problem, right? A correlation of this nature does not lead to an understanding of root causes. At best, it gives as an avenue of investigation, and smart researchers will know that and dig deeper. The problem becomes more pronounced when people start to blindly confuse correlation and causation. Can’t you just see a manager saying “the data shows that X was caused by Y, so we need to fix Y. We have xxMillion data points, and the data never lies.” Yes, data never lies, but numbers, by themselves, mean nothing (or very little).

It has been said that 86% of statistics are just made up (including this one), and students of stats and data analysis will know that increasing the size of the data pool does not automatically lead to better results.

I do think that Big Data has the potential to change many things, but I’m also sure that we need new skills to properly interpret what that data is telling us. The math boffins are already coming up with new ways to analyse the data, and the hugely buzzy field of data science and data scientists is just about taking off.

Does that mean we all have to be math whizzes or data scientists to effectively use big data? Absolutely not. We just need to be careful not to become too enamored by big data. Critical thinking, reasoning, and root cause analysis aren’t going anywhere, and when applied to big data, will lead to many new breakthroughs.

I just hope that those breakthroughs are not used solely by Facebook et. al. to show us more tailored ads.

Full Disclosure & Disclaimer- I work for EMC, but everything here is my personal opinion, and should not be construed as being a corporate message.

I’ve been reading technology blogs for a long time (way before they were called blogs, in fact, but that’s a story for another time), and there’s one thing that always bugged me: Watching a blog sell out to a big company. Does anyone remember what happened to Tom’s Hardware? When every third post is a blatent shill for the company, it doesn’t take too long to figure out what’s going on.

I worked for a big tech magazine for many years, and I found out first hand how some of this stuff works. I was lucky; we had a clear separation between Editorial and Marketing, but that didn’t stop the Country Manager of asking to meet me to “discuss the discrepancy between my review and our lab tests”. They claimed an output of 300 lumens, but my tests showed that it was only about 200 lumens. That’s a pretty big difference.

So the guy came down to our labs with one of his engineers, and we ran through all the tests again. The test methodolgy proved to be correct, and my numbers were accurate. The engineer and I figured out what was going on, but the manager was not happy. Later on, I learnt that there was talk of reducing the ad spends. They didn’t do that in the end, but it was an interesting time.

The point is, a company that’s paying a lot of money to a publication expects something in return. Nothing wrong with that, but where do you draw the line? What do you guys think? If you went to a blog, what would make you question its credibility?

What’s the best way to clearly show a sponsored relationship while maintaining editorial integrity?

Personally, I think that keeping it simple is best. Clearly identify sponsors, show that they do not have editorial control, talk about your sponsor as analytically and dispassionately as you speak about the competition. The sponsor has to understand that sponsorship does not equate to purchasing content. As long as both sides keep it clear, transparent, and honest, things should work out fine.

But that’s just my opinion. What do you think?

I spent an instructive evening setting up Google Authenticator for my web server. It was surprisingly easy, and now I have two factor authentication enabled for SSH logins. In fact, it was so easy, I went ahead and setup 2 factor authentication for my blog as well.

The idea came to me this afternoon, as I used my SecurID key fob to log into my corporate network. “Hmmm”, I thought. “My server is as valuable to me (more, probably) as this network to the company”. And so it began. I googled it at work, and then came home and set it up.

I looked at Duo Security, but eventually settled on Google authenticator. Duo Security looked very nice, but I just like the idea of having all my data on my own server.

And of course, to make things even better, I added CloudFlare!. So I have 2 Factor Authentication and the power of the Cloud. Life is cloudy and nice.. at least till the next 0-day exploit arrives.

I’ve been reading a few interesting articles about the changes in large networks, and how today’s networks don’t function well with old fashioned network topologies. When I finished reading, I sat back, and thought about how all that was being said applies almost exactly to corporate structures. It’s an interesting (and geeky) analogy, so let’s see if I can coalsce my thoughts here.

The articles that I mentioned (they’re linked at the end), had some common themes:

1. Traditional network topologies worked on a north-south pattern; all data flowed up and down a tree like network, from edge to core and back again. Networks were designed to efficiently move data, and STP helped avoid loops and made sure stuff kept moving.

2. Applications were welded to the servers. A server got set up, connected to the network, and then largely forgotten about (till something went wrong, anyway).

3. Storage used to be server specific. Even after the advent of SANs, storage was still quite tightly linked to a particular physical location and a particular physical server.

All this hummed happliy along for quite a long time. Life was good, stock valuations were up, and people wore shorts to work on Fridays. Of course, like all idyllic times, it couldn’t last. Virtualization poked its head into the picture, and messed everything up. Suddenly data was flowing in every direction; servers were jumping around from machine to machine, applications were moving from server to server.. heck no one even knew where the damn data was stored anymore. The happy little switches weren’t happy no more, oh no.

Things had to change, and indeed they are. Network fabrics are flattening out, and networks are not designed to assume data flows in fixed directions. Networks these days have to be dynamic, adapting rapidly to the changing data patterns.

So what’s all this got to do with a corporation? Naturally, you’ve spotted the parallels as well.

1. Companies worked on a siloed north south pattern. Different departments communicated up the network and then back down again. Yes, there were east west links, since people make friends (I know, weird, eh?), but the official channels of communication were up and down, not side to side.

2. Business fuctions were welded to departments/divisions/functional practices/whatever-you-want-to-call-em. A division got set up, connected to the rest of the company, and then largely forgotten about (till revenues were down, anyway).

3. Information used to be division specific. Each division knew its own area very well, but had little knowledge of the other areas.

Now, of course, things are changing. Very few companies (especially in IT) sell only products. Everyone pretty much has to talk to everyone else to figure stuff out, and the traditional corporate model is finding it hard to cope. People find workarounds to get stuff done, and many companies are realizing that they have to change how information flows inside the company. Some are doing a good job, some are wondering where the good times went..

The reason for the similarities between networks and companies is pretty simple. After all, we’re talking about transferring information in both cases. The nature of information (the content, the scope, and the amount) changes constantly, and causes everything else to change. There’s nothing new under the sun; I bet this has been happening since the first human bashed an animal on the head with a rock and went “hmmmm…”. It’s powerful too.. political systems, religions, and entire countries have come into being based on the effective dispersal of information.

May you live in interesting times. I know it’s supposed to be a Chinese curse, but aren’t all times interesting times for the people who live through them?

http://etherealmind.com/layer-2-multipath-east-west-bandwidth-switch-designs/

http://www.theregister.co.uk/2011/09/16/flattening_ethernet_expert_panel/

http://blog.ioshints.info/2011/03/data-center-fabric-architectures.html

It’s the title for Neal Stephenson’s new book. I got it yesterday, and it’s shaping up well.. though I’m not sure about the direction he’s taken now. Anathem, the Baroque Cycle, Cryptonomicon; these were cerebral thrillers, with generous lashings of metaphysics, math, physics, history, politics, science fiction, pirates (Malabar pirates too!), cryptography… Reading these books called for some effort on the part of the reader. You couldn’t just read them like you watch Glee, you had to think about what the damn thing was trying to say. I loved all the math in Anathem, even though the ending was a little too deus ex machina, frankly. Fraa Judd could probably solve the current economic crises, though getting legislation through Congress might be too much, even for him.

Reamde is more of a straight up thriller in the Tom Clancy style. Still good, but I miss the old Stephenson a little. A book where you have no idea what the author is saying but still can’t put it down is a Good Thing™.

I was just thinking about how I started reading Stephenson. It was a dark and stormy night a warm summers afternoon seven years ago. I was in New Delhi for the weekend, and looking for new books to read. At that time, I was reading a lot (and I mean a LOT) of fantasy and sci-fi. Stuff like the Wheel of Time, Ursula LeGuin, The Riddle Master’s Game, the Dark Elf stuff blah blah blah. These books were not exactly flying off the shelves, and finding them in my home town was an exercise in frustration. Delhi, though, being the capital of the country, was slightly better off.

So there I was, wandering about at PVR Saket (which is kinda like a cross between a mall and Quincy Market, for those of you who have no idea what I’m talking about). There’s a nice bookstore there (there used to be, at least), and I walked in and headed straight for the fantasy and sci-fi section. I do NOT like the term speculative fiction. That’s what I use when I read the newspaper these days. But I digress. Anyway, up in the corner, I spotted three gleaming books. Literally gleaming—the Baroque Cycle paperbacks were clad in shiney gold, silver and bronze covers, and they gleamed in a stray sunbeam.

I took them off the shelf and read the blurb at the back. ‘Hmmm,’ said I, ‘this looks interesting’. And so I bought all three. Quicksilver, Confusion, and the System of the World.

It took me a month to read them all. This is rather a long time, but good god! those books were complicated. Half the time I was struggling to grasp the cast of characters, and the other half I was cursing the author. But I couldn’t put the books down. They were a combination of the Pickwick Papers, Issac Asimov, P.G Wodehouse, Jerome. K. Jerome, Mark Twain, Dostoyevsky, Sartre, and many, many more. I loved it. Those were good times, those were.

I always love discovering a great new author. I’ve discovered some more since then, but that’s a story for another day. I’ve taken my twenty minute break and need to get back to work. If you’ve read, or are reading, Reamde, gentle reader, leave a comment.

Yes, yes, I know, it’s been ages. Lots of stuff has happened. Briefly:

I graduated with an MBA. And hey, I did pretty well, graduating in the top 5% of my class, with a 3.89 GPA. Not too shabby, eh? Apparently, I can study, occasionally.

I’m working for EMC now. Never heard of EMC? Well, that means you’re not the kind of person who finds the idea of visiting a data center cool. Jokes aside, EMC is a huge company that does storage, big data, virtualization (they own most of VMware) and a bunch of other stuff. What am I doing there? I’m working in Global Services Strategy and Marketing, doing a little bit of this, and a little bit of that. You know how it is.

Lots of other stuff too, but this will have to suffice for now. I’ll be posting more often now, keeping this blog going. You don’t have to read, but if you do, you’re just going to get cooler.

I kid, I kid…

Plenipotentiary
Parcheesi
Pathos

Random strings of jumbleness
hidden away toys of summer
remember when 21 years was old

going out to the beach with salty salty sand
wondering what to do now

course, things are zipping along,
they always do

quiet and uptight, and quite uptight
let’s go to the top of the hub

feeling stupid about writing
writing stupid about feeling

streams, and counters, and deep sea bluefish
one stroke to the edge of the shelf;
the continental shelf.

warm blue waters with droopy headed beagles

it will all work out, now won’t it
it will
it will

will it?

That was the sound of February passing by. In the blink of an eye. I didn’t write anything on this journal, not because I had nothing to write, but because I was lazy/busy. There was too much to do.. work and school, and plucking pigeons. I just tossed the pigeons in there to wake you up. It’s all about Writing with Power™.

The snow is finally receding. It was fun while it lasted—I learnt how to snowboard. Of course, I’m still falling down, but I now it’s just a matter of time. I get the board now, and now I have to make it mine. Besides that, what else is up? Hmm, nothing much at all. Life is quite placid, with no eddies or swirls. There was some brief unpleasantness with some friends of mine, but that is over now, and they are only acquaintances. Sometimes, you just have to walk away, especially when you’re getting taken for a ride. (But be careful, cause if you’re being taken for a ride, and you ry to walk, it can be painful when you hit the ground.. talk about mixed metaphors. On the other hand, I did make another friend, and I’m more sure that this time we’re going to stay friends. So it’s all in a day’s work

Speaking of work, it’s time to start looking. My goal is to be able to snowboard down a black diamond, and for that, I have to be where the snow is. So, gotta get a job here. It’s not hard, and it’s not easy, it just is.

Anyway, it’s Saturday night, and you know what that means! It means it’s time to go.

The air is like something out of a George R.R. Martin book. Now I know why all the great fantasy authors are from cold, cold places. When it’s this cold outside, it fires the imagination, and you can almost see dragons and battle scarred men walking the frozen wastelands through the mists of your breath.

This new year, I decided to change many many things. Alas for new year resolutions. Though I am happy to report that I have managed to change some things. I have wrinkles on my forehead now.. a sure sign that I’m (much) closer to 30 than I am to 20. I knew that, of course, but it’s still something that comes home when I look in the mirror. I begin to understand why people like to cling to when the sky was blue and the skin was flawless. I guess I’m thankful I’m a guy; it’s easier for us, and hey, apparently guys look better as they grow older. I need all the help I can get!

The year looms ahead of me. Unlike most years, this one fills me with some apprehension. I wonder if it’s going to be like this from now on. I’m going to be travelling again this year, to places I’ve never been before. Meeting new people, eating new food, breathing new air. All of that.

Must say though, life is never boring. Every time I start getting slightly bored, events conspire to keep me on my toes. Twinkle-toes, they call me (only when they’re inebriated, you know I do not dance).

Here’s to an interesting 2011. I know when the Chinese say “May you live in interesting times”, they don’t quite mean it the way you think they do, but pooh! to them, say I, and continue to live in interesting times.

I shall step out presently, and brave the elements. Watch my breath freeze as it leaves my mouth. Hear the icicles falling down. Feel my sense of smell get destroyed as millions of tiny ice particles collide in my nose..

Brrrrrr….