The Mendicant Bug

There are quite a few well-known libraries for doing various NLP tasks in Java and Python, such as the Stanford Parser (Java) and the Natural Language Toolkit (Python).  For Ruby, there are a few resources out there, but they are usually derivative or not as mature.  By derivative, I mean they are ports from other languages or extensions using code from another language.  And I’m responsible for two of them! :)

• Ruby Linguistics – this is one of the more ambitious projects, but is not as mature as NLTK
  • interface for WordNet
  • Link grammar parser
  • some inflection stuff
• Stanford Parser interface – uses a Java bridge to access the Stanford Parser library
• Mark Watson has a part of speech tagger [zip], a text categorizer [zip], and some text extraction utilities [zip], but I haven’t tried to use them yet
• LDA Ruby Gem – Ruby port of David Blei’s lda-c library by yours truly
  • Uses Blei’s c-code for the actual LDA but I include some wrappers to make using it a bit easier
• UEA Stemmer – Ruby port (again by yours truly) of a conservative stemmer based on Jenkins and Smith’s UEA Stemmer
• Stemmer gem – Porter stemmer
• Lingua Stemmer - another stemming library, Porter stemmer
• Ruby WordNet - basically what’s included in Ruby Linguistics
• Raspell – Ruby interface to Aspell spell checker

There are also a number of fledgling or orphaned projects out there purporting to be ports or interfaces for various other libraries like Stanford POS Tagger and Named Entity Recognizer.  Ruby (straight Ruby, not just JRuby) can interface just about any Java library using the Ruby Java Bridge (RJB).  RJB can be a pain, and I could only initialize it once per run (a second attempt never succeeds), so there are some limitations.  But using it, I was able to easily interface with the Stanford POS tagger.

So while there aren’t terribly many libraries for NLP tasks in Ruby, the availability of interfacing with Java directly widens the scope quite a bit.  You can also incorporate a c library using extensions.

Naturally, if I missed anything, no matter how small, please let me know.

Update: Here is a great list of AI-related ruby libraries from Dustin Smith.