Our Journey Building the Background Burner

After more than two years development in our labs, we've decided it is time to see what can be built out in the wild with the Bonanza Background Burner. Want to use the burner for your project? Check out the quickstart guide, the API reference, or, for website embedding, the burn widget reference. Otherwise, read on to learn the backstory, technical details, and future plans for the Background Burner.

Quest for Conversion

Ever since we founded Bonanza back in 2008, one of our guiding principles has been to make more sales for our merchants by maximizing their products' visibility, traffic and sell-through. We’ve created a huge amount of great tools and tech to help achieve those goals, and its been working: our conversion rates have steadily increased every year as a result of our continued efforts.

But when we looked at the differences between our top-selling merchants and those who were struggling, we noticed a clear trend popping up again and again: The top-selling merchants had great product photos (often shot in expensive photo studios), while the others had product photos that were, to put it gently, “meh.”

There are many studies showing that attractive product photos can increase sell-through significantly. Nothing new there. But it’s actually quite difficult for most merchants to take good product photos because the products they're listing are in their garage, or their yard, or in the middle of a messy kids’ room. In other words, the products might look great - but the backdrops, not so much. And a bad backdrop makes for a very bad product photo. 

We felt the best way to help our merchants improve their photos was to replace the background with a white backdrop. It's the result that top-selling merchants pay thousands of dollars to attain (via studio & equipment), and it's what marketplaces like Amazon already require. So we began searching for any existing software that could automatically process product images, and change a cluttered back yard into a clean white background. How hard could it be?

What we discovered was more than just discouraging... it was a little bit shocking.

A Fools' Errand

With the power of the Internet at our fingertips, we assumed that finding software to remove the background from a photo wouldn't take long. But searching for "background image removal" yielded a wasteland of companies that wanted to charge $1 per picture to manually remove the background - presumably outsourcing to the cheapest possible country - with as much as a 24-hour delay. And the software tools available ranged from slow, to inaccurate, to just plain terrible. Neither option would be a good experience for our merchants.

It was clear that we needed an expert. So I visited a University of Washington image processing professor who delivered some surprising news: To his knowledge, the problem of how to remove an arbitrary background was a "computer vision" problem that had remained unsolved since at least the 1980s (when the still-ubiquitous canny edge detector was developed). He told me that he would be "shocked" if we could solve a problem that has stumped image scientists for decades. He warned us not to try it. It was a fools' errand.

But I just couldn’t stop thinking about how completely awesome it would be for our sellers to have a technology that would make their images look great, and increase conversions accordingly. Maybe the problem was unsolved because past attempts had been academic exercises by students (or professors) who didn't have as much time or programming experience as we did? I hoped so, because I’d already made up my mind: We were going for it.

Baby Steps

When we started digging into the problem ourselves, our first observation was that pictures of items are often centered, with a blank or neutral background behind the item, like this:

We downloaded OpenCV and built an algorithm that would loop around the edge of the image and remove everything within a given color threshold. It worked surprisingly well. More than 30% of our test images could suddenly be processed into something much cleaner-looking (and higher-converting) for buyers. Progress!

The bad news was that we quickly learned this 30% was a local maxima. Tweaking the color threshold to be more sensitive didn't cut out enough, and training it to be less sensitive would remove the subject. We spent a couple weeks trying different tricks to salvage the idea, but it was clear that the approach would never yield more than 50% accuracy in its results. We ended up scrapping 2 months’ worth of development effort.

We started a second version from scratch, then scrapped that one about 6 months later. Maybe that UW professor was right. What had we gotten ourselves into?

 

The Part of the Story Where You Enter in a Cape

Getting the foreground identified in 70% of pictures (including the hard ones) is a great start. It is a significant leap forward over any other background recognition technology that we know to exist. But it isn't enough, because we know that with our current debugging suite, we can push our accuracy rate to 90% by the end of 2014. We should also be able to speed up its processing by at least 50% this year.

The problem? We've still got this top-rated marketplace to grow. That marketplace earns money, whereas the Background Burner doesn't, yet. And so our sellers have taken precedence over perfecting the Background Burner, which, for the past year, has been mostly "back burnered" in our development schedule. In order to drive the Background Burner up to the 95% accuracy we can ultimately achieve, we need some killer apps to be built with it so we can financially justify hiring a dedicated developer (maybe you? [2]) to work alongside our successful and growing marketplace team.

Take Our Ideas, Please!

We've been using and improving the burner on Bonanza for a couple years now. It is a battle-tested platform [3] upon which you can build apps that feel like magic. We're hoping that you will build these apps and discover great success:

• A tool that can take a picture of an item, recognize what it is, and post it to Bonanza from mobile platforms (no typing necessary!)
• A mobile app that lets people burn backgrounds and put the photo subjects into greeting cards
• A mobile app that can use the burner to recognize a product at a store (using its color and shape properties)
• A tool that burns away background from Facebook/Twitter pictures and replaces with something funnier/campier
• Customizing avatars or graphics in video games
• Make graphic design (brochure layout, Powerpoints, etc.) easier
• Google Glass object identification

The more interesting ways that the burner gets used, the more we can justify perfecting its core technology.

Roadmap

Assuming we see solid adoption, here's where we plan to be by the end of 2014, given a dedicated developer:

1. 90% accuracy identifying image subject given arbitrary input image
2. All burns complete in 5 seconds or less
3. Burn editing tool loads and switches all pages in < 2 seconds
4. Create (and make available via API) v1 of our technology to match one image to another (combines with burning technology to identify arbitrary images taken from mobile phones, or wherever)  

We already have the debugging suite built out to make goal #1 a straightforward-enough matter. We've also built a performance analysis framework that lets us see where the bottlenecks currently exist in our image processing, putting us on the road to goals #2 and #3. And we already have a beta version of our image matching product, but it needs more polish before it can see the light of day.  

Getting started with the API

We've put together the following docs to help you get your project underway with minimum headache:

• Burn quickstart guide. Provides easy-to-follow specific examples on using the Background Burner.
• Burn reference docs. Full explanation of the various API calls available in our restful API.
• Burn website integration docs. Want to integrate the Background Burner on your website? Here's the reference to be able to do it with Javascript. And here are some examples of that to get you started.
• Burner API pricing. tl;dr about one cent per image burned.

Drop us a line!

Much of the rationale for releasing the Background Burner (at the price of "almost-free") is that we want to spark the creativity of developers; we believe that the Background Burner opens new doors for interactive image processing applications. If you've got an idea for a world-changing app, we'd love to hear about it. If you have an idea for a boring novelty app, tell us about that, too. Drop us a line at api -at- bonanza.com and let us hear your grandiose visions!

Or, you know, just start hacking on this and get back to us later. 

 

[1] To the extent that complex results aren't perfect, we provide our burn touchup tool at no cost to users. It generally can wrangle any burn into the correct result in seconds. You can experience it at our Background Burner landing page if you click "touch up" below your burn result.
[2] Does the challenge of building something once thought impossible get you excited? Do you live in the Seattle area? We'd like to find someone to help us continue improving the Background Burner and build out related image technologies. Drop us a line at jobs -at- bonanza.com. We've got a pretty sweet benefits package (i.e., work every February from the tropics), as you can see in our still-active Craigslist posting for a Devops Engineer.
[3] Though as of publishing time, the API is still brand spanking new, so it may have some hiccups in its first few days. Don't despair, we fix things fast.