Antifragile and implications for software

AntifragileI just finished reading Nassim Nicholas Taleb’s book Antifragile: Things That Gain from Disorder. This is a fascinating book, particularly for those interested in statistics and critical thinking and better understanding the world we currently deal with. He is clearly very bright, and makes a lot of good points. However I don’t agree with many of his arguments. I would not like us to go back to MS-DOS and Windows 3.1 or Java 1.3, just because they are old. I tend to take a more careful thinking and evaluation before moving forward toward new technology. For example products like NodeJs. I don’t want to replace nginx or apache webservers with javascript code running in NodeJs, I think NodeJs, the community and a lot of the libraries are far to immature, like things were with Java 1.2 or Microsoft’s first C++ compiler, it takes more time before things develop. I Like Mongodb, but decided to wait before building applications depending on it. Taleb talks about Black Swan events, which remind me of Hurricane Sandy and the damage it left, and several of the Amazon Aws outages. I think companies need to be careful of putting all their operations with one provider or getting too tightly coupled to platforms that, can have outage events, or problems with availability and developers need to build in to their applications handling to deal with problems like availability and alternative schemes that can be switched, for instance local databases in one’s data center. Companies need to anticipates big swings in demand, and assume that’s something you will deal with, not I’ll deal with it when it comes

Great visualizations with D3

I’ve been seeing a lot of amazing infographics and visualizations. I was at a conference presented by Actuate the BI company. They presented a talk on visualizations in part because of purchasing a company to help compete with products like Tableau whose product helps visualizing data. In the discussion was d3 a javascript html5 library, which sites like The New York Times uses to do some of the wonderful graphics they do. You can see from the samples gallery some of the amazing things you can do with it. If you have good skills with Css and Javascript, you can create very dynamic graphics for projects you are working on.

You can see to the left a clip from a project I have been working on in my free time. During the period leading up to the elections I was working on a project to capture twitter data during the debates and later build hadoop jobs to crunch the data and reduce it down to data. The sample to the left from the town hall debate is from the source data, top 100 sources, which represents the twitter clients that people were using with the larget text representing the most used clients. These used a word cloud type visualization, which is hard to draw conclusions from it, though you can pick out the important information. The data needed to be scaled from a range of approximately 30:58000, so I scaled using log10(n/500).