Twitter4j + twitter + hadoop

I have been doing a bit of work with hadoop of late in my work life, mainly using streaming map reduce and pig working to extract additional data out of weblogs, which is a powerful paradigm. Before the election I wanted to develop a way to look at data during the election period. Twitter is a powerful communication tool often trivialized, but is a powerful way to promote and for mass sentiment to be made known.

Twitter has a powerful streaming api, that allows twitter to push to the client the data in a large mass. PHP is often a tool that I have used as a rapid development tool, but usually lacks a multi-threaded model and libraries that implement features like twitter’s streaming api. Twitter4j is a good library for java and also works with android, which works well with twitter. This allowed me to capture a significant amount of data for analysis. the code had matured significantly by the time the town hall debate took place, which led to capturing a good quality of data. This run used a the Query Stream, which allowed to filter from the global data set that twitter is, and limit it to the united states and topics relating to the debate and presidential election. Wanting to do more work with hadoop’s java libraries and features, I wrote the hadoop map reduce jobs in java and setup a single pseudo distributed node to process the data. These are the results imported into Google spreadsheets.

Barnes and Noble, You Could Do Better

Let’s give Barnes and Noble some credit. The Nook color is a nice device as a reader with some tablet stuff. At the time they came out with it, the android honeycomb code base was not available, so one can see why they went with the older code base when they designed their device. The Honeycomb source has been available for a while now. Why not upgrade your device to the new Android codebase so you can run better more modern apps which can be those optimized for a tablet versus a phone. Now Barnes and Noble comes out with the Nook Tablet and they are still in the old android code base. What gives, the honeycomb code has been around for a while now. Why use an aging code base that isn’t even intended for tablet use?

Kindle Fire, Lame Decision

Amazon made a lame decision to build the kindle fire with an android 2.2 tablet and not working with the android Honeycomb class codebase, which are typically 3.2 version. The device has a limited amount of memory as well (8GB) and not 16GB or 32GB which is standard with the honeycomb tablets. the User Interface also looks like it was  borrowed from Barnes and Noble’s Nook Color device, also a android 2.2 device. One wonders what manufacturer is building these devices for them. When the honeycomb devices started to come out, Google clamped down on the rights to the code base so only well established manufacturers could get the honeycomb code base.

Built new Celebrity Astrology Tablet App

I just built a new version of my Celebrity Astrology App for Honeycomb tablets with a new design. It turned out very nice, much better than the standard phone modal designs. Using the new Fragment architecture allows multiple views to be placed on screen, each with separate code, so user isn’t switching as often to new Activities and the user flow is far more natural and easy to work with. This is a real aspect where the 2.2 android devices fall short. I am sure there are those who will try to emulate them for devices like Kindle Fire, but Honeycomb and later devices are far superior and are designed for tablets.

Problems with Early Android Tablets

A full range of tablets have been developed using the Android operating system. The new 3.x Honeycomb versions such as the Motorola Xoom (below) represented a major change for tablets. This is an operating system with many features to help provide a base for a powerful and highly interactive solutions. Initially Android was not built for tablets, only smaller devices such as phones. Eventually support for large screens was added but it was really minor changes and the Ui was not built for tablets. I have several android applications, which later I will have honeycomb specific versions which will make use of the new user interface features. In the mean time I was working on support for the older larger screen devices like the Samsung Galaxy tab (left), which is a non-standard aspect ratio 600 x 1024 dpi, people have reported problems with applications on this device. A developer can rework a layout for specific Android software versions or device features such as aspect ratio. One still is relying on the operating system scaling the layout to fill the space. Android has predefined screen sizes, which one can use and the emulator knows how to use these predefined sizes, but when device manufacturers built tablets from the older, the sizes didn’t exist. A developer dealing with this situation may say, skip the older tablets and spend the energy on the newer tablets, which may be the best solution.

More about the limitations of Mobile devices.

I had been working on an Android app that would work with a popular professional photography site Smug Mug which I have been using to promote my photography. Smug Mug has a nice REST layer with a number of different formats including json, XML, php, and other formats. It works well and is reliable. One of the biggest challenge is the limited bandwidth and power of these devices. The app is nice, allowing to browse from an account. Later I had been planing a photo uploader feature, which is definitely a good idea and could work. My account, which I was using for testing has a lot of photos, and surfaced real challenges in terms of performance. Building a persistence layer with sqlite has promise, along with background threads loading resources, but with a heavily loaded account, the devices don’t keep up. Given some of the newer devices available for instance the Droid Bionic, which will have dual cores, and making available 4G capabilities such as Verizon’s LTE and Sprint’s Wimax, and HSPA+ on T-Mobile and later on AT&T will definitely help.

Published Celebrity Astrology Android App

I have been working on an android application to work with my site. It is now in alpha and can be downloaded from the android marketplace. Built a json service into the celebrity astrology site, after doing mobile/webkit optimized version. Application works with this service to display lists and show content. I needed to downgrade the chartwheel activity to a webview until the image viewer can be perfected.

Motorola Droid Review

Having been a Blackberry user before, I had waited for Verizon to do the Droid. I have been happy with their service and didn’t want to move to another carrier. I had been considering a better Blackberry, but made the leap to the Droid. It is easy to get started with and user interface good and it is enjoyable to use. It’s a little heavy and clunky. Ordering a case is a good thing, the finish is not as durable as many would like. The battery life span is not the best but given the features it is worth it. The camera sensitivity could be better. Other reviewers have noted issues on the keyboard, and there are with the pull out, working with the top row on the keyboard takes practice. On the other hand the on screen keyboard is good enough that you don’t really need the pull out keyboard much. The standard usable SD Card that comes with the device is more than adequate and works well with the usb cable. Voice quality is also very good. The screen resolution is also very good compared to other phones, and the photo and video quality is far superior to other phones I have used.