Quokka vs capybara classifier, deployed

FastAI lesson 2 notes and thoughts

I was honestly going to skip this lesson because it didn’t “technically” feature new AI concepts, but boy am I glad I didn’t!

For starters, I can now enjoy my quokka vs capybara classification demo live, which isn’t something I expected to be able to do so quickly! Here’s a link. Pleasingly, the classifier correctly thinks my DALL-E generated profile picture is a quokka with 100% accuracy:

It even managed to correctly classify quokka cartoon art (source), albeit with lower confidence!

There are definitely clear limitations. For starters, we get some funny results when we throw something unexpected at the classifier:

Apparently pumpkins are 99% capybara. Hmmm. :)

To be fair, this is to be expected considering the fact that I don’t have a “neither” category, which would be the natural next thing to add to this classifier!

A “neither” category raises its own host of interesting ML questions, such as:

• How does one go about obtaining a “neither” set? Perhaps a sample of some more general-purpose image classification dataset suffice?
• What would be the ideal ratio between the three classes of capybara, quokka, and neither?
• Do we need to adjust sample weights for our three classes?

My gut feeling tells me that the classes need to be balanced, just as in a binary classification setting. In terms of sample weights, we might set sample weights to account for the true prevalence of each class if it were somehow important (for example, if we were Twitter and we were scanning each image to compute P(quokka) or P(capybara) for business critical reasons, haha). But for a demo? I’m not so sure it matters, especially considering most people would probably throw a quokka or a capybara image at the classifier. Honestly, I just don’t know, and that’s why I’m trying to learn all this. But that’s my intuition, in any case!

Some other thoughts:

• Getting a working Gradio demo in my Kaggle notebook was super easy, thanks to this awesome guide, but getting said demo into HuggingFace was suprisingly annoying!
• My issue was that I was getting weird AttributeError(name) errors when the code was accessing the PIL/Image.py file. Learn from my mistakes, anyone who’s reading this: you shouldn’t call img = PILImage.create(img) in the predict function for Gradio. Just comment out that line in the guide entirely, and things will work. Don’t ask how long it took for me to figure this out :)
• In terms of debugging the above error: I was probably going to be stuck on this forever, until I searched for “fastai” in HuggingFace and found a similar looking project with perfectly running code. Judging by the contents, that user was starting out as well, but becuase they put their code out there, I was able to benefit. Putting my code and thoughts out there goes against all my privacy-inclined instincts, but that’s why I’m now doing the same (but under a flimsy alias because honestly, it’s still kind of embarassing).
• It kind of blows my mind that we can use a free online notebook service (Kaggle) to fine tune models really quickly, and then we can use other free services (Gradio, HuggingFace) to deploy these models into working demos. That can be used on a phone or computer. By other people. In minutes. For free. This is completely insane and I love it!! And this is why I’m glad I didn’t end up skipping FastAI lesson 2.

My n00b read on the deep learning landscape right now, based on what I’ve just learned, is that it really feels like the Wild West. It’s gotten so ridiculously easy to train and deploy these models that surely people are using deep learning as a hammer for anything and everything. It’s almost feels like the dot com craze, but for AI, if that makes any sense. And sure, there was a dot com bubble and all, but hey, web pages are still around decades later and not going anywhere any time soon. Likewise, I don’t think deep learning is going anywhere, so may as well get with the program!

(substack mirror)