Cool post. I thought this was the most interesting point:
"As an outside agency, we can’t (and shouldn’t) make this decision for Zidisha. However, it IS our job to inform them well enough to make it. The ROC curve is abstract and hard to interpret for this purpose, so we translated its information into a plot that directly measures the trade-off at every possible threshold value."
So many of the standard techniques in machine learning assume an objective function that's different from what the end-user actually cares about. Good practitioners like Everett know to translate between the algorithm's loss function (e.g., squared error) and the end goal. I'm surprised there's not more research to let ML algorithms optimize the ultimate loss function directly!
I'm the founder of Zidisha, and I can't thank the Bayes Impact team enough for this project. Like most small nonprofits, we don't have the resources to hire in-house data scientists. Bayes Impact brought data science expertise within our reach for the first time, and they've had a transformative impact on Zidisha.
I was really surprised to see that you're actually disintermediating microfinance. Other crowdfunded/donor-choice orgs create profiles after a borrower has received a loan, and pay any loans made to that profile into a pool for future borrowers. I thought this was a scammy but necessary part of the business model until I saw your new loan growth - very exciting development in microfinance.
I'd be curious to see how BI can inform your strategy in areas like market selection, which can be especially crucial in microfinance. Which region should you enter next, and what type of borrower should you focus on first? There's plenty of data out there for analysis, but maybe others' repayment rates for models like lending circles may not correlate too closely with yours.
That's an interesting question. Thus far our market selection has been driven more by practical constraints than sophisticated analysis. We've focused on countries that have well-developed electronic payment systems (such as M-PESA in Kenya). We also try to target places where average incomes are low enough that loan amounts that seem small in wealthy countries can have life-changing impact.
Wondering this as well, I only recently learned that all these Kiva loans I'm participating in, helping to get it funded, had already happened! It kind of makes me wonder everytime my charity-team sends me a message 'only 5 more hours to get this loan passed!'. Not that I particularly mind, but my curiosity was activated as to how this works, why, if a different model can be better and what the benefits/drawbacks are.
So I'd love to learn if people know more about how all this works!
Clicking through on the article I ended up browsing through several of the projects on Zidisha and reading the founding story. Very impressive project!
Having lived the last year and a half in South America I really believe in the power of micro finance as a tool against poverty. It's really eye-opening to see from up close how difficult/impossible it is to escape poverty without access to credit.
Great job with Bayes Impact and thanks for the interesting post.
"So moving from a 20% fraud reduction to 50% will block about 200 additional fraudulent loans and also 200 honest loans. Is that okay?"
From the accompanying chart it looks more like ~700 additional honest loans blocked?
Any information about which threshold they picked? Looks like a difficult decision for a non-profit, from the chart it looks like about 15% of applications have been fraudsters but model accuracy is obviously limited given the available features.
I'm a fan of what Bayes Impact, but the article opening seems really naive about what the financial situation is for a pretty big chunk of US residents.
>> As a Westerner, getting a credit card is only slightly more complicated than tying my shoes. My world is raining with opportunities to borrow money to go to school, open a store, consolidate loans, or buy an iPhone 6.
A little under 10% in the US are without a bank. And the "Underbanked", people have some type of banking but still use Money Orders and stuff like Payday loans, is over 20%. I wouldn't say "Raining with opportunities" lines up too closely with those people.
Like I said, it's great to see what Bayes Impact is trying to help, but just needed to clarify a bit about the US.
Edit: Updated to clarify that this was directed to the tone of the article opening, not the author. I had previously used the word "author" instead of "article"
It is true financial access is a problem even in developed countries like the US -- though it is nothing compared to the situation of people in developing countries (and I would say calling the author naive is an unnecessary stretch), you are right in that it's important to raise awareness about the underbanked in the US as well.
You'd be glad to hear we are actually working with other financial institutions in the US like Opportunity Fund that provides microloans to Californians. At Bayes Impact, we have a commitment to building repeatable processes -- the good thing with using data to tackle problems is that it allows us to benefit from economies of scale when working with different actors that are facing similar problems.
I didn't mean to come across as saying the author was unaware, partly because I'm positive the author is extremely well versed in the financial industries within the US. I meant to relay that it "seemed" that way from what was written to open that article.
I should have clarified more explicitly that I was discussing the way the article appeared and not trying to make a statement about the authors aptitude or understanding of the US situation. My apologies to the author.
"As an outside agency, we can’t (and shouldn’t) make this decision for Zidisha. However, it IS our job to inform them well enough to make it. The ROC curve is abstract and hard to interpret for this purpose, so we translated its information into a plot that directly measures the trade-off at every possible threshold value."
So many of the standard techniques in machine learning assume an objective function that's different from what the end-user actually cares about. Good practitioners like Everett know to translate between the algorithm's loss function (e.g., squared error) and the end goal. I'm surprised there's not more research to let ML algorithms optimize the ultimate loss function directly!