text categorization example<\/a> given by Joachims contains 2000 weighted example vectors. The precision of the resultant model, as estimated by svm_learn, is 93.07%.<\/p>\nMy first training set used a search for “cars” for positive examples and a search for “film -cars” for negative examples. It contains 63 binary example vectors. The estimated precision is 12.90%.<\/p>\n
My second training set used a search for “basketball” for positive examples and a search for “racing -basketball” for negative examples. It contains 61 binary example vectors. The estimated precision is 9.09%. Furthermore, I turned the sentence “Michael Jordan is out to shoot some hoops on the court this week.” into a test vector. It was categorized incorrectly.<\/p>\n
Analysis<\/strong><\/p>\nThere aren’t the sort of statistics I was hoping to see. There are a number of reasons why I might be getting these subpar results.<\/p>\n
\n- Quantity<\/strong>
\nSixty-some example vectors simply aren’t going to stand up to the example set of 2000. Of course, the internet is a big place, so there’s no reason (other than Google’s API limitations) that I shouldn’t be generating my own large training sets.<\/li>\n- Quality (Part A)<\/strong>
\nThe example set uses weighted vectors while my sets use only binary vectors. In short, I’m not including information about word frequency, only about word appearance.<\/li>\n- Quality (Part B)<\/strong>
\nI don’t know how counterexamples were selected for the example set, but I’ll admit that my current strategy for finding negative examples is flawed. The selection of a counterexample search term was arbitrary, and using a single search term probably produces an undesirably uniform counterexample set.<\/li>\n- Quality (Part C)<\/strong>
\nThe example set was generated by a system trained to ignore trivial words and to reduce complex words to word-parts for consistency. My system currently has no such bells and whistles. I had hoped that the equal presence of elements like markup in positive and negative examples would lead the vector machine to ignore those elements, but the results say otherwise.<\/li>\n<\/ol>\nWhat remains to be seen is whether these factors can account for an 80% difference in accuracy. Next steps:<\/p>\n
\n- Quantity<\/strong>
\nTime to switch to Yahoo’s API and start pulling down large result sets.<\/li>\n- Quality (Part A)<\/strong>
\nI can try switching to using word frequency within a document, but I’ll need to modify my shared dictionary class to use the same weight calculation that the example set does.<\/li>\n- Quality (Part B)<\/strong>
\nI’ll either generate counterexamples using a set of searches over other category keywords, or just an OR search. One counter-keyword is not enough.<\/li>\n- Quality (Part C)<\/strong>
\nI’ll start a word filter list to ignore low-content words like “the.”<\/li>\n- Persistence<\/strong>
\nEverything is runtime right now. I need to rebuild some things and include a mechanism for saving and reloading a common dictionary at the very least. I also need to be able to consult the dictionary to get a feel for which words it’s picking up.<\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"I ran my first couple of training sets today. I must confess, the results are not pretty. Let’s start with the summary: Summary The training set for the text categorization example given by Joachims contains 2000 weighted example vectors. The precision of the resultant model, as estimated by svm_learn, is 93.07%. My first training set…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8],"tags":[41,344],"class_list":["post-929","post","type-post","status-publish","format-standard","hentry","category-programmer","tag-svm-trainer","tag-svmlight"],"_links":{"self":[{"href":"https:\/\/www.bradleycbuchanan.com\/b\/wp-json\/wp\/v2\/posts\/929","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.bradleycbuchanan.com\/b\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.bradleycbuchanan.com\/b\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.bradleycbuchanan.com\/b\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bradleycbuchanan.com\/b\/wp-json\/wp\/v2\/comments?post=929"}],"version-history":[{"count":1,"href":"https:\/\/www.bradleycbuchanan.com\/b\/wp-json\/wp\/v2\/posts\/929\/revisions"}],"predecessor-version":[{"id":1221,"href":"https:\/\/www.bradleycbuchanan.com\/b\/wp-json\/wp\/v2\/posts\/929\/revisions\/1221"}],"wp:attachment":[{"href":"https:\/\/www.bradleycbuchanan.com\/b\/wp-json\/wp\/v2\/media?parent=929"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.bradleycbuchanan.com\/b\/wp-json\/wp\/v2\/categories?post=929"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.bradleycbuchanan.com\/b\/wp-json\/wp\/v2\/tags?post=929"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}