I think there must be literally thousands of guys with the name Jason Adams in the English speaking world.  There are many who share my middle initial (Jason M. Adams) and I’ve encountered several with the same middle name:  Michael.  Really, my parents couldn’t have given me a more common name, and Jason was a very popular boy name at the time.

I think I need to change my name to stand out.  I want a googlewhackblatt.  This needs to be a unique identifier, after all.  Any suggestions?

Mind42 is a web-based mind mapping app that I used about six months ago and forgot about.  They just recently ended their so-called beta stage, which reminded me that they existed.  After giving them another spin, I think I’m going to start using mind maps more often.  I also used FreeMind for a while, but the collaborative features of mind42 are more appealing to me right now.

A mind map is a graphical representation of the relationship between concepts.  You start with a central concept and build outwards.  The way you organize the concepts is up to you and good mind mapping software will let you completely restructure the graph easily.  For example, I created a mind map to keep track of all the places I had applied for jobs.  First, I organized it by city and state.  This is good for keeping track of where we might be moving, but it doesn’t really capture what I think about the jobs I’m looking at.  Some jobs I’m looking at deal with summarization, some are for government contractors, some are linguistics-oriented, etc.  By arranging them by some other sort of topic, I can visualize the options available to me.  Also, the act of organization helps me think through the choices I am facing.  The more you think about the issues, the better equipped you will be to make a decision.  That doesn’t necessarily mean you will make the best decision..

I contend that mind maps are also good for paper writing.  They offer ways of teasing out information and the relationship between the ideas you are trying to convey.  Of course, your mileage may vary.  Not everyone is helped by mind maps.  But if you are, try using them the next time you have a paper to write.

One drawback to mind maps are the time involved.  You can spend a lot of time constructing a mind map and reorganizing the ideas.  There is a trade off between the time spent organizing your thoughts and the value or length of the task.  If it is very important for you to have thought through something in detail, a mind map is probably the right choice.  If you are just jotting down thoughts in the form of a blog post, they may be too time-consuming.  I doubt your readers will mind if you spend that kind of time, though.

And no, I did not take my own advice in using them for this blog post.  Maybe next time.

Here’s an exciting potential computational linguistics task:  construct a mind map automatically from text.  I can think of a few ways to get started…

I have just joined a reading group related to prediction markets and crowd wisdom. One topic that has come up is range voting.  Our current election system in the US is known as plurality voting:  the person with the most votes wins.  This is fine in elections with two candidates, but when there are more candidates it is very bad.  More on that in a second.  Range voting takes a different approach and lets you assign a value on a scale to each candidate.  Let’s suppose the scale is 1 to 10, with 10 being your favorite choice.  The person with the most points at the end of the day wins.  This is much, much better.  So much better.

Bayesian regret is very simply the difference between the best case scenario and what actually happened measured in terms of utility.  The obvious question is “what does utility mean in a voting system?”  It’s not exactly something that’s easy to quantify.  Will I gain financially if Obama is elected?  Will I gain satisfaction if Nader is elected?  Will I lose my mind if McCain is elected?

Warren Smith has this solution:  generate a multitude of random simulations and see which method comes out on top.  It’s impossible to generate every possible permutation of voters in an election of any significant size, so he uses Monte Carlo methods to randomly sample the space.  This has the added advantage of alleviating the need to quantify utility for real people.  Just assign them a random value according to some model of voter mentality.  Smith points out that’s not perfect since these simplified models cannot exactly match real human behavior.  But if you run the simulation enough, it all comes out in the wash.

The result is that range voting beats everything every time in his simulations.  Even when considering strategic voters, range voting performs better against other voting styles with strategic voters.  Strategic voters are those who choose the lesser of the evils that they think might win.  Plurality voting is actually one of the worst methods when it comes to elections with more than two candidates.  So our voting system reinforces the two-party system we have in America.  Go figure.  The high numbers our method gets on Bayesian regret plays out in everyday life.  There are all these rallies to get people out to vote.  Most won’t, since they are disillusioned with the whole process.  Why shouldn’t they be?  It blows.  Read the paper or check out RangeVoting.org for details.

Instead of rocking the vote, let’s change it altogether.  Less Bayesian regret.  I should sell T-shirts.

Range voting: less bayesian regret

So I am on the market after getting my masters.  I’ve posted my resume to Dice and Monster and a couple others.  Monster gets the most unsolicited calls.  I’m finding that recruiters are an odd lot.  There are some who are pleasant, though to a man (or woman) they’ve never heard of NLP or computational linguistics and have no idea how to help me (with the exception of the one or two recruiters I’ve contacted for NLP jobs).  For the most part, they seem to not even read my resume.  Oh, you have Java skills?  How about this Java grunt job that only requires a bachelor’s degree? Waste of time.  The best are the ones who contact me in broken English with a multitude of typos.  Yeah, right.

I have been told that with my CMU degree, I should be looking exclusively at the big corporations:  Google, Microsoft, Amazon, Yahoo, etc.  If I do my time there, I can get a job anywhere and have a good career.  That’s true, I’m sure.  Something about startups is really attractive to me, though, so I’ve been looking at a lot of them.  What if the only job I can get at a Googlosoftazonahoo is not NLP-related?  Everything is so rushed.  I have a September 1st exit date for CMU and I want to be in the city of my chosen job by then.  Add lease problems.  The problem is my decision to abandon academia didn’t come at the right time:  back in the winter.  I am, however, more confident than ever that it was the right decision.

While travelling to York, PA yesterday to visit my wife’s family, I spotted a large moth under a pine tree.  Daedalus was sniffing around the tree and this bright orange thing caught my eye.  It was very lethargic, so I imagine it was either sleepy (since moths tend to be nocturnal) or about to die.  My guess is the latter.  I was able to snap a few shots and one actually turned out.  Autofocus kept slipping off of it.  Were the patterns in the wings maybe confusing it?

This particular specimen wasn’t as large as these Giant Silkworm moths (aka Cecropia moth or Robin moth) can get, which is about a six inch wingspan.  But it was big.  What a beautiful creature.  I love the fern-like antennae and the speckled wings.

Cecropia moth (giant silkworm moth, robin moth) with black and white speckled wings and fern-like antennae found under a pine tree in central Pennsylvania

This is research I did a while ago and presented Monday to fulfill the requirements of my Masters degree.  The presentation only needed to be about 20 minutes, so it was a very short intro.  We have moved on since then, so when I say future work, I really mean future work.  The post is rather lengthy, so I have moved the main content below the jump.

Recommender System Types

Recommender systems come in two main flavors: content-based and collaborative filtering. Content-based recommenders use information about the items themselves to recommend new items to a user.  The intuition behind this approach is that users tend to like or have a need for similar things.  Of course, this breaks down when I buy Great Aunt Edna a puppy-embroidered sweater for Christmas and Amazon starts recommending me ugly sweaters left and right.  The alternative is collaborative filtering, which uses similarity between users to recommend new items.  Whereas content-based recommenders are limited to only finding items similar to those you “enjoyed” in the past, collaborative filtering can present novel things that users like you also liked.  Collaborative filtering suffers from the problem that you have to provide the system enough data to find other users similar enough to you that it can produce reasonable predictions.  This is the user cold start problem.  Likewise, if a new item is added to the system and no one has rated it, collaborative filtering can’t recommend it either.  This is the item cold start problem.  Content-based recommenders are more robust to this.  However, the user must rate at least one item for a content-based recommender to get started (otherwise, you just recommend popular items or base it on the user demographics).  If there are no similar items, content-based fails as well.  That is less of a problem for collaborative filtering.

In collaborative filtering, the users and items can be thought of as a matrix.  The rows represent users and the columns represent items.  The value in each cell is either zero, if no rating has been given, or the value of the rating assigned by the user.  It is typically asserted that items in the matrix are not missing at random.  This means there is some function that maps a user to the items they will rate.  There will be items they will never rate because those items lie outside of the area of interests of the user.  This is fairly intuitive.  I will never read a Harlequin romance, and so I would never rate a book of that type in a book recommender system.  Also, a system can never present every item a user could rate to the user.  Users get bored entering ratings and will stop after a while.  If these two facts weren’t true, we wouldn’t need a recommender system in the first place.  This means that there are items a user could still rate if given the opportunity.

The ratio of the total items in the system to the items a user will rate is usually very large.  There are many times more missing values than present for the average user.  So the collaborative filtering matrix is very sparse.  This has the advantage of allowing us to use sparse matrix techniques to speed up some operations on the matrix, but it has the disadvantage normally associated with lack of data:  it is more difficult to build reliable models with less information.  The more data you have, the better off your machine learning algorithm is.

The Stacked Agents Model

Our idea was to supplement the user-provided information in the collaborative filtering matrix with machine-generated predictions.  In this way, we can turn a sparse matrix into a full one.  To generate the predictions, we construct a content-based model for each user.  Each item the user has rated is assigned a feature vector .  The features depend on the domain, but if we’re recommending songs they might be artist, genre, record sales, year released, etc.  The label is the rating the user gave that item.   The feature weights can be chosen in a number of ways.  We used tf-idf to weight the features we collected (see the paper for details). A content-based agent for each user constructs a model and learns the user’s preferences.  It then predicts a rating for each and every item the user did not rate.

Returning to the idea of missing at random, we also would like an idea of how confident the system is that the user would rate the item at all.  For this we construct a confidence model for the user.  Again, we can use the content-based feature vector , but instead of the label being the rating, it is positive or negative (binary) to indicate whether the user has rated it or not.  The confidence model was the weakest part of our work and improving that is important moving forward.  The problem with this approach is that we tell the learner that an unrated item is a negative example.  This is true of some items, but not for the cases where the user would rate the item but hasn’t taken the time or had the chance.  We automatically bias our learner against the very examples we are seeking to learn.  So like I said, this part needs work.  There was a track at last year’s SIGKDD on predicting whether a user would rate an item, if you’re interested in more about this topic.

We constructed confidence agents for each user to predict the confidence the system has that the user would rate the item.  These confidence predictions are real values in the interval [0,1] (not confident, confident).  Once we have generated a prediction and confidence value for an unrated item, we can combine those into a new rating score:  , where is the predicted rating for user and item and is the confidence score.  These form the basis of the collaborative filtering matrix in the stacked agents model.  Stacking is the process of using predictions from a previous machine learning method as the training input for another.  The combined observed data (user ratings) and machine predicted data are used to train each user’s stacked agent.

For details about the implementation, please see the paper.

Conclusions

While the results weren’t state of the art overall, we did manage to show that this method could improve plain content-based recommendations.  I see two main paths for future research on this method.  First of all, the confidence model was very weak and needs to be improved.  Secondly, we are combining collaborative filtering with content-based recommenders, to form a hybrid recommender system.  In theory, we should overcome some of the problems with cold start recommendations.  Further experiments would be needed to verify that.

Reference

Jason M. Adams, Paul N. Bennett, Anthony Tomasic. Combining Personalized Agents to Improve Content-Based Recommendations. Technical Report CMU-LTI-07-015, Carnegie Mellon University, December 2007. [pdf]

I just completed the final requirements of my Masters degree today (the details of which I will save for a future post).  It has been a difficult road, and I’m glad it’s done.  I didn’t attend any sort of graduation ceremonies, because I don’t go for that sort of thing — at all.  Until today, it didn’t feel like the weight was off my shoulders.  Now I actually feel like celebrating!  But I won’t, because I’m a nerd.  I’m currently celebrating by working on a programming puzzle.  And surfing the blagoblag.

I still have a couple months of servitude to complete the requirements of my fellowship, but the degree is mine.

I’ve taken to calling her kitten. I was messing around with obscuring the flash. This shot had the flash obscured by an opaque object. I liked the look on her face, but the lighting was too low and led to noise in the capture. So I hit it with the sepia and film grain. The pictures are rather large, so I’m hiding them below the jump.

This shot was taken by holding my finger over the flash. The red tint is the flash through human flesh. Thinking about how bright that is, I wonder if I should be worried about finger cancer.

Wordle is a totally awesome tag cloud creator.  You can create a tag cloud from text or some del.icio.us username.  My rendition of my del.icio.us tags is below.

[via SMRB]

Peter Turney posted recently on the logic of attributional and relational similarity. Attributes are features or characteristics of a single entity. Relations describe some connection between two entities, such as a comparison. We’ll denote a relation between two entities A and B as A:B. A relational similarity between two groups A, B and C,D will be denoted as A:B::C:D. This is the standard SAT-style proportional analogy: A is to B as C is to D. An attributional similarity indicates that two entities share the same attribute (this could be to varying degrees, but in the boolean case, it’s either shared or it isn’t). An attributional similarity between A and B will be denoted as A~B. This is like saying Z, A:Z::B:Z. I’m just giving a brief introduction here, but this is all in Peter’s post to greater detail, so I recommend reading that for more information.

This got me thinking about collaborative filtering (because, well, I’ve been thinking about it all the time for the past two years). Collaborative filtering exploits similarities between users to predict preferences for items the user has not seen. In the case of movie recommendations, like with Netflix, this means that users can recommend movies they have seen to similar users who have not seen those movies. There are many ways of doing this. At the heart of it, however, is this notion of relational and attributional similarity.

A: base user
B: movies rated by A
C: some set of other users
D: movies rated by C

We can’t just say that A:B::C:D, since A and C may be nothing like each other. If we constrain it to users with attributional similarity, then we arrive at the definition of collaborative filtering: A~C & A:B::C:D. Logically, it follows that B~D also holds. See Peter’s post for some properties of proportional analogies that make this more clear.

In the non-binary case, we can choose C to be a set of users whose similarity varies with A. Also, our measure of what exactly constitutes similarity can be any number of different metrics. From here, it seems pretty clear that the limit of collaborative filtering is bounded by the attributional similarity A~C. If (A~C) & (A = C) (complete similarity) then it follows that B = D or else A C. If A C then does it logically follow that B D? I guess it depends on the similarity metric and how we are defining the differences in the sets of movies and the differences in the sets of users.

I wonder if there has been any work done in this area? I wasn’t able to find anything, but maybe I’m just not searching for the right thing. Is it even worth pursuing?

While browsing my various photo albums (all of them digital) for a proper head shot of myself, I was struck by my complete absence. I make appearances so rarely, it’s like I don’t exist in the photo history of my life. I’m always the cameraman. Really, I have so few pictures of myself throughout my entire life, I am basically a ghost. There was a period of time in the late 90’s when I had long hair. I don’t have a single picture of that (this post reminded me of that time). But I do have pictures of my babies. I found this while hunting.

Willow gets nervous when Donna leaves.  She watches her like a hawk through the window until she disappears from sight.  Today I decided to take a picture of this, which meant opening the screen as well as the window.  However, Willow started getting a little too “jumpy” and I had to tug her away from the window before she tried to learn how to fly.

The berries on the ground are mulberries.  We have several trees in our yard, so they are falling everywhere right now.  Daedalus is addicted to them, the little blighter.  It’s a constant tug-of-war when we’re outside.

I need these all in one place:

• Distractions make you dumb
• The Myth of Multitasking
• Research stamina
• Genius, Sustained Effort, and Passion
• How to become smarter
• How to be a genius
• Does one have to be a genius to do maths?
• Google makes us stupid [Added 2008-06-24]

It’s no coincidence a lot of that originated from Daniel Lemire. Posts like the ones above are a great reason to subscribe to his blog, even if you’re not into computer science.

If you know of any links that belong here, please let me know in the comments.

The North American Computational Linguistics Olympiad is an annual competition open to US high school students that introduces kids to computational linguistics at a much younger age than people normally hear about it. I didn’t hear about CL until I was three years into my undergrad program. The instant I did hear about it, I knew I wanted to do it. Most people I talk to about it, look like I’ve just uttered a phrase of Klingon. I suspect most people don’t hear about it at all, or if they do, it’s sometime during their undergrad program and not at the beginning, when they might be better able to plan their educational career path. Also, CL is pretty much a graduate program and rarely taught before then. Granted, a lot of the maths involved are beyond what’s taught to high school students and early undergrads, but the linguistics is not. And thinking about linguistics computationally is not. So NACLO is doing an extremely valuable service which I support completely. And not just because one of my professors is one of the General Chairs of the organizing committee for it. She no longer can affect my grade and I have no need to suck up — so this is genuine. How’s that for full disclosure?

One of my google alerts popped up a post on a spam blog I tracked down to this original post, which talks about a lot of young kids doing some great things in science. In the post is an interview with last year’s winner, Adam Hesterberg. He said, “I’d never studied linguistics, and ‘computation’ sounded like boring calculation.” That reminded me of the fact that computation might mean a different thing for most people than it does for scientists. I’m no corpus linguist, so I’m not gonna try to find out right here. What I suspect is that computation has a more “hard work” connotation for people outside of science: it’s the “plugging and chugging” meaning. Inside science, it’s tacked onto the beginning of some other field to mean anything in that field that can be computed. Computational linguistics deals with the computable aspects of linguistic theories. A very quick search on wikipedia finds at least a dozen other computational fields:

• Computational biology
• Computational chemistry
• Computational economics
• Computational electromagnetics
• Computational engineering
• Computational finance
• Computational fluid dynamics
• Computational mathematics
• Computational mechanics
• Computational particle physics
• Computational physics
• Computational statistics

Is it a good idea to use this name when approaching high school students? What about language technologies? Well, the competition isn’t about language technologies, it’s about critical problem solving in a linguistics setting. And trying to fit that into a competition name isn’t going to work, either. North American Critical Problem Solving about Linguistics Olympiad (NACPSLO)? It makes me think of narcolepsy.

So my proposal is North American Logic and Language Olympiad (NALLO). It’s easy to say (rhymes with hallow) and accurately describes the subject matter. Plus, I think it has broader appeal. A lot of kids are interested in logic, language, or both. It shakes free of the negative connotation of computation and draws kids where they can be introduced to it a little more easily. The downside is that it doesn’t mention linguistics directly, so that might trouble some people who are a little more traditional about their outreach.

What do you think?