The data set was published by the Audubon Society Field Guide and describes the characteristics of mushrooms found in North America.  The version I’m using was compiled by the UCI Machine Learning Repository.  It contains 8124 records (one record per line) with its classification and each of the 22 mushroom characteristics represented by a character value in a comma separated list.  The first value describes the poisonous or edible classification.

p,x,s,n,t,p,f,c,n,k,e,e,s,s,w,w,p,w,o,p,k,s,u
e,x,s,y,t,a,f,c,b,k,e,c,s,s,w,w,p,w,o,p,n,n,g

JOONE requires semicolon separated numerical values for input, so I replaced each character value with its alphabetical position and changed the commas to semicolons.  Missing values were given the value 27.

16;24;19;14;20;16;6;3;14;11;5;5;19;19;23;23;16;23;15;16;11;19;21
5;24;19;25;20;1;6;3;2;11;5;3;19;19;23;23;16;23;15;16;14;14;7

The network has three layers: 22 input nodes, 10 hidden nodes, and a single output node.  If the output node is 16 (p), the mushroom is classified as poisonous.  If this node is 5 (e), it is classified as edible.  The hidden nodes and output node have a sigmoidal activation function.  The network is trained on the first 3000 elements of the data set using JOON’s built in back propagation functions and a Root Mean Squared Error (RMSE) function.  The remaining ~5124 nodes can be used in verifying the application.  Running in training batches of 10,000 iterations (epochs) and storing a serialized representation of the network to disk every 100 iterations allowed fine grained monitor the progress of the application and ensure net trained network could be recovered in the even of a crash.

Serialization is a mechanism where an object in memory is converted into a portable form (XML in this case) so it can be later retrieved and the object restored to memory exactly as it once was.  In this case, we are using the ’serializeable’ java interface to store a neural network that contains the network diagram, weighted synapses, and trainer (error).

The error after the first 100 iterations was ~5%, and decreased to 4.25% after 50,000 iterations.  While this is rather slow, the error is still decreasing and could be within acceptable levels with a few million iterations.

Further research should be done on the design of the network and its training.  Adding another layer or changing the number of hidden nodes could converge more quickly.  The serialization mechanism could produce an easy way to distribute and parallelize the training.  If the current RMSE of the network were stored along with serialized net, a node could determine if its error is less than the current “best” for a group of nodes.  The node with the lowest error would write a new serialized net and global error file and nodes with greater error would use the least error net to continue training.

Here are the files required to continue developing this network:

MushroomFFNN.java

mushroom_numerical.data

After June 30, 2009, AvantGo will no longer be providing mobile Web content for sync or online access, and you will not be able to access or update your AvantGo content or account.  Your account information will continue to be protected by our privacy policy, and we plan to delete any personally identifiable information you’ve provided (e.g., your e-mail address) as soon as reasonably possible.

If you are an 8MB account subscriber you may be entitled to a refund for a portion of your subscription fees that are unused. To request a refund, please click here and submit the refund request form. You will need to reference your AvantGo User ID (included in this email).

To continue receiving news and information from your favorite content providers, you should visit that content provider’s channel before June 30 for details on how to obtain their content other than through AvantGo.  Also, AvantGo recommends the Snac mobile widget application - a new, fun way to get your favorite content on your mobile device. You can find out more about Snac at: http://www.mysnacs.com/landing?token=avantgo0609

Best wishes,
The AvantGo Team

The original AvantGo sync service was a pioneering effort.  It brought the web to your handheld device and allowed users to take any piece of the web they wished along for the ride.  Web applications were supported with an offline form submission system and advanced javascript hooks allowed mobile web developers to interact with their users in a useful way while they were offline.  The “HandheldFriendly” meta tag was originally developed by AvantGo to differentiate websites designed for desktop or mobile view and is now used by many browsers including Opera Mini, Opera Mobile and some BlackBerry devices.

AvantGo taught its approximately 10 million recorded users what to expect of the mobile web and defined a market for the mobile device browsers followed.  Aspects of the service’s design are sure to remain in the mobile computing field.  Many former AvantGo engineers went on to help develop Yahoo! Go, Google Gears, and other mobile content services.

If HTML 5 ever gets adopted, most of the offline browsing features will be available on a large number of browsers.  Safari 4 beta already supports the site manifest file aspects of HTML 5 for pre-downloading content (a feature that has caused some controversy over wasted bandwidth) which allows offline view of pages.

According to parent company Sybase, AvantGo will transition from a mobile web service, to an SMS advertising and content delivery system.

Two years ago I began building a series of base classes in Java that could be used to create neural networks.  I managed to get it to a somewhat useable state, but it needed a lot of cleaning up before I could release it or expand it to apply to a larger set of problems.  I found JOONE while writing my libraries, but shelved it because my goal was to learn more about machine learning.  Just using an existing toolset hides most the important “educational” bits.  I returned to JOONE a few months ago, and discovered it shared many of the design elements had built into my own library along with a bunch of really great features I hadn’t even thought of.

It comes pre-packaged with useful example codes to get you started, and an extensive PDF manual (which could use some copy-editing).  I ordered a book (that’s also available online) that discusses the basics of neural nets in the context of JOONE.  While I’ve found the book useful, I think the examples are written for a slightly older version of JOONE.  Some of the method calls suggested in the book are listed as @deprecated in the actual JOONE source.  Fortunately, the examples included with the JOONE source code make this easy enough to modify the book’s samples and use the more modern methods.

I like the example based format of the book.  There are a series of problems to solve and the JOONE way of solving them is presented.  The pace is good and they increase in complexity as more advanced topics are covered.  There is an element I dislike about the book: some of the topics covered are not in a JOONE context.  Heaton Research, which published the book, seems to have its own basic NN library and some sections of the book use this library instead of JOONE.  The lack of consistancy could be a problem for someone trying to use JOONE as an engine and apply it to an actual problem.  For simply a learning experience, the subject matter is well described no matter which library is used.

As I continue playing with JOONE, I may post my example codes or describe the process of getting JOONE to play nice with Eclipse.

When developing an app on a new platform I take one of my existing apps and port it to the platform.  Since I already know the bulk of the code, I can learn the new features of the platform while not balking a the task of a totally new project.  A good number of my apps are open source or use libraries that are protected by various open source licenses.  The GPL is by far the most restrictive, but these licenses all generally require (or at lease encourage) the user to release extensions to these libraries back to the public.  If Apple’s iPhone SDK requires me to be silent, and the OSI license requires me to publish my changes, I’m stuck and can’t produce anything.

The obvious answer is to just use the tools that Apple has offered and build from there.  As great as that sounds, it’s really not going to fly for an open source developer.  If I write an app for the iPhone and list it for free in the App Store, I still can’t release my code.  This is the very core of “Free as in Freedom, not Free as in Beer”.  Developing for the iPhone means Apple has control of my source code.

I know that the jailbroken iPhone community has solved this and boast some rediculous penetration rates in the iPhone market.  This is not a sustainable solution.  Eventually Apple and AT&T will learn to hate each other and we’ll have iPhones on other networks.  I predict this will happen in 4 years.  This will remove the major reason for jailbreaking the iPhone for the average user and jailbroken iPhones will as useless as iPod Linux.

I really like the iPhone and iPod Touch.  Like most Apple hardware, they’re really well designed products.  I’m not sure if this was just an oversight on the part of Apple’s upper management or just an over-zelous general counsel, but until I can have control over my own code I won’t be developing for Apple.

This last point is important: users are actively scanning local airwaves and sending this information to SkyHook. Cellular network operators have long been able to track movements of individuals, but they only have visibility of the people connected to their own network. SkyHook is retaining information from an active scan.  Now you not only get information about a users’ current location, but also the location of ajacent users not connected to your network.

It may not be obvious how this data can be used for behavioral analysis, so I’ll present this scenario.  Sally takes the train to work every day and has no knowledge of the SkyHook service.  She reads her email on the platform by picking up a local wifi hotspot.  This same train platform is periodically populated by tourists that are using the SkyHook service to map out their day.  SkyHook is now receiving active scans of this train station and seeing Sally’s wifi radio talking to the hotspot. When Sally goes to get coffee later in the afternoon, her phone is detected by another SkyHook user and her location again entered into the SkyHook database.  It’s now possible for SkyHook to see that Sally stops at the same train platform at around 7:30am each day and gets coffee on 4th St. on Tuesdays, even though Sally has never used or heard of the SkyHook service.

If we presume that connecting to a network and using a service constitutes an agreement between user and provider, then the provider has some reason to know your location.  If the person sitting next to me on the train belongs to a different network, my network doesn’t know anything about and has no agreement with that individual.  If my phone is running an active scan and finds the person next to me and sends this information to my network operator, who owns this information?  Do I have the right to send someone else’s location to a third party?  Is my location information public domain?

I see SkyHook as a potential disruptive technology that has yet to be challenged in court.  How long will it take before this information is considered valuable in a court case and Skyhook subpoenaed for probable suspects?

There are better examples elsewhere, but the idea is simple.  If your result looks better than the next guys, users are more likely to choose your link to click on.  Altering the presentation of a search result seems like a massive security hole at first.  I certainly wouldn’t want to give the world access to change how my search results appear.  However, these enhanced are only available on an opt-in basis so a user can only change the presentation of their own search results.  The Presentation Applications can be selected and enabled in their preferences.

I’m happy with the developer tools and speed with which a new presentation can be created, but only enabling these apps on an opt-in basis means that the vast majority of users won’t ever see them.  As a small site owner, it would be nice is Yahoo would allow the modified search results to be the default based on my preferences (after proper authentication with Site Explorer or similar tool).  Perhaps a simple flag similar to SafeSearch called something like “Use Enchanced Search Results” could be turned on in a users’ preferences enable these features.  No one is going to install a presentation application for my tiny little blog.  (Besides, most of my traffic comes from google.)

I’m worried that Yahoo is ignoring the Long Tail with SearchMonkey and only catering to large web properties.  Given their new “open” motto, this seems rather out of character.  The project would make a good way to separate them from Google and Live search, although it does share some usability features with Ask’s Site Preview.

I plan on participating in their developer feedback program, so perhaps this is still possible with SearchMonkey.

A friend of mine made a really cool bookmarklet that lets the user select a bounding box on an image, then sends an AJAX call to his server which downloads the image, crops it to the bounding box, and pipes it through GOCR (GNU Optical Character Recognition). The result is then dropped in a div exactly positioned over the originally selected bounding box approximating text size and color. The goal is to make it possible to copy and paste text out of an image. It works quite well and it’s clean interface makes it a beautiful thing to watch.

Given the complete failure by the web development community to accurately populate image alt attributes, I thought it would be slick to grab all images on a page, get any embedded text and automatically populate the alt text in much the same way as my friend’s bookmarklet draws a div. I ran a few tests by piping web comics through GOCR with horrible results. Accuracy couldn’t have been above 10%. I told my friend about my results hoping he could give me some insight as to why GOCR was failing me so badly. As I probably should have expected, the simple bounding box he uses is really important. GOCR doesn’t have any layout detection.

Google open sourced a big OCR package called Tesseract originally shelved by HP in the 1980s and is (I believe) using it to scan books and make them available on the net. This is one of Google’s many efforts to make all the world’s information available. I was hoping it’s touted increased accuracy would help overcome the lack of a bounding box in my application. It compiles and installs without errors and it runs just fine on the test images provided, but producing images it can read is a challenge. I’ve tried converting saving gif files as tiff from Pixelmator and Tesseract gave me errors. I tried using ImageMagick and a little bash script I found with the same results. Tesseract complains about minute differences in tiff header information (datetime format, bpp info, etc), so some care is needed. I’m not sure if this has something to do with my version of libtiff (v3.6.1) that Tesseract is using, or if there’s some parsing code in Tesseract that isn’t happy.

I finally did manage to get things to work by creating a very basic bash script and using the simplest settings for ImageMagick. The goal was to have the bash script work the same way GOCR’s command line utility works.

#!/bin/bash

tmpid=$$
convert -compress none $1 /tmp/img.${tmpid}.tif

tesseract /tmp/img.${tmpid}.tif /tmp/tout.${tmpid} 2> /dev/null
cat /tmp/tout.${tmpid}.txt | perl -e ‘while(<>) { $_ =~ s/\|//g; $_ =~ s/\^~R//g; print $_ } ‘
rm /tmp/tout.${tmpid}.*
rm /tmp/img.${tmpid}.tif

Now that I have two tools who’s interface is the same, I can write a wrapper around them to use in PHP and compare their performance. I’ll handle that in another post.

I checked back few weeks later expecting to see the completed entry available for download.  I wanted to see how the finished version compared with my approach.  Instead I found this:

Unfortunately - and unexpectedly - we haven’t seen any reasonable applications for the bounty process so we’ll most likely be contracting for the development of the WordPress plugin.

They hadn’t seen any reasonable applications?  Was I wrong about them getting hundreds of applications from the unwashed masses of slashdot?  Did I still have a chance at getting the prize?  The deadline or the submissions was the next day (Saturday) and all I had was a little bit of very raw code.  I spent my Saturday morning building my submission and writing up my proposal for what I would do with my entry.  A demo plugin was not mentioned in the proposal request, but I included my basic plugin anyway.  I thought it may let me stand out of the crowd a little.  So, I submitted a proposal to the OpenCalais bounty project a few hours before the deadline.

The winner was to be announced in 10 days, so I waited.  And waited.  There was nothing on the boards and not even an email response confirming my submission had been received.  Finally, I emailed them and they replied with a link to a new post on their boards.

Bounty Update

Over the last week or so we’ve been thinking hard about what to do with the WordPress bounty. Here’s the situation:

We received a number of proposals of varying quality at the very last minute - from three sentences long to reasonably well articulated. We’ve read each one carefully and evaluated them for how innovative they were and how experienced the proposer was in developing production strength WordPress plugins. While we appreciate the effort the individuals made in putting these proposals together, the fact of the matter is that none of them had the combination of great ideas and great experience that we were looking for.

So, we’ve decided to to down another path. We feel badly about it - but we feel strongly that its the right thing to do.

So, our apologies if you had your hopes up. Our thanks for taking the time to apply. There will be contest opportunities in the near future that will be contests - not bounties.

Comments, criticism and suggestions welcome,

Regards

I was horribly disappointed.  Not because my proposal wasn’t chosen, but because they just shot themselves in the foot.  The point of a bounty is to draw attention to your project and get people interested in using your service and building add-ons.  The initial proposal is not the finished product.  All the people that submitted proposals could have been an army of developers combining their ideas into a fantastic product in true OpenSource fashion, but instead they’ve been alienated.  All OpenCalais needed to do was choose the person with the most extensible architecture and allow them to wrangle the rest of the developers as the benevolent dictator.  In a few months, they’d have a product they could be proud of.

Now, I have a bit of code without a home and I really hate that.  There isn’t a project somewhere that I can check out and maybe add some of my ideas to.  I’ve found some of the other OpenCalais submitters on the web, but without a single point of contact or the endorsement of the parent project I doubt they’ll go very far.

OpenCalais just joined Duke Nukem Forever.  Cheers!

“We’re Sorry… … but your query looks similar to automated requests from a computer virus or spyware application. To protect our users, we can’t process your request right now.

We’ll restore your access as quickly as possible, so try again soon. In the meantime, if you suspect that your computer or network has been infected, you might want to run a virus checker or spyware remover to make sure that your systems are free of viruses and other spurious software.

If you’re continually receiving this error, you may be able to resolve the problem by deleting your Google cookie and revisiting Google. For browser-specific instructions, please consult your browser’s online support center.

If your entire network is affected, more information is available in the Google Web Search Help Center.

We apologize for the inconvenience, and hope we’ll see you again on Google.
To continue searching, please type the characters you see below:”

Of course, considering the date this could just be a well crafted joke.

More importantly, do the previous flips effect the outcome of the next?

This is the concept of conditional independence. Clearly, the probability of any coin flip is 1/2, but the probability of eight flips all being the same is 1/256 (0.5^8). So, if the future depends on the past, can the future be determined within some finite probability?

To take the gambling metafore a bit further, slot machines are perfect examples of discrete random number generators. If you watch someone else play a slot machine for a while, the more they play and don’t win, the higher the probability that their text turn will be a jackpot. Ideally, a winning slot player should observe the other players in the casino and try to take over a ‘cold’ machine that has been played frequently but never produced a jackpot. Despite the social aspects of a slot machine or table ‘going cold’, these machines actually have the highest probability of making you rich.

Being able to predict the future under these very controlled situations doesn’t sound like a very useful super power, but it has many real world applications. Physicists use Markov Chain Monty Carlo (MCMC) models in determining the movement of subatomic particles, and it’s becoming popular with information theorists for behavioral analytics of internet click-streams.