This is part 5 of a series of guest posts by British Bondora p2p lending investor ‘ParisinGOC’. Please read part 1, part 2, part 3 and part 4 first.
The Management of Change
As mentioned in my earlier article on the construction of the decision Trees, my responsibilities when employed (yes, dear reader, I am now retired) included the successful proposal to create new teams to conduct Data Mining and produce and disseminate Metrics relating to the research activities. As on many other occasions, I was then charged with making my assertions real by staffing and then running said teams to realise the benefits I had stated should arise.
As part of my (rapid) learning in these activities, I came to understand the need to maintain processes until solid analysis could isolate and support changes. So in this review period, for those elements under my control, I have maintained certain actions within set parameters until I felt I could justify a change and then have maintained that changed process until the next time the data supported a further change.
Changes I Controlled
Given that my need to change my selection process was as a direct of seeing my money rapidly disappear (!) I limited my ongoing expenditure to the minimum purchase (5 Euros) allowed by Bondora and only made 1 purchase per selected Loan Application.
This continued throughout October 2014, when I felt that the downward trend in parts falling behind with payments was established and likely to continue. From the beginning of November 2014 onwards I increased the number of parts of any single loan application I would buy to 2, still of 5 Euros each. Note that for some application types with, for example, a higher (between 5% to 7%) indicated historical failure rate, or a very high (above 45%) interest rate; I still limited my purchasing to 1 part of 5 Euros.
This Purchasing policy remained in place until the beginning of April 2015 when my increasing confidence in the selection process, my increasing cash reserve and other factors described below, meant I felt able to increase the value of purchases (to include 10 Euro parts if I felt an application was sufficiently strong) and increased the number parts purchased of any particular loan. This latter element in particular allowed me to take advantage of events outside of my control that offered opportunities that had not previously existed, explained later in this article.
Errors in my Process
In the period October 2014 to the end of the year, I was updating the Trees twice a month. There was no detailed timetable, but the Trees did exhibit a greater degree of change in this time than was later the case. It was during the first update in December, week 51 of 2014, I noticed that the previous Tree had been built using corrupted data. It was only later in the review period that I noticed that this period – from weeks 48 to 50 inclusive – exhibited the last “spike†in defaults.
From the next update onwards (31st December 2014) I implemented a more rigorous update procedure and restricted the updates to 1 at the end of each month. I felt that this may enable changes in the Tree Structures to be more visible and so attract my attention to these changes and validate the process that had generate them, thus avoiding process errors. The fact that the datasets provided by Bondora were subject change without notice (and did so often) was an additional factor in the decision to have fewer, more rigorous build events.
I worried that fewer updates to the Trees would lead to out-of-date trees and more In Debt and Defaulting loan parts, but this has not become apparent either in daily use or this review process.
I have noticed that the Decision Trees are not static and do change over time. Sometimes – rarely – these changes occur at a high level and are very noticeable. However, the Trees have changed in a subtle way at lower, more compartmentalised levels. This is discussed later in this article.
Changes I could not Control
Whilst I have tried to maintain a tight control over my activity since starting to use the Decision Trees to guide my loan selection, there is the overall Bondora environment over which I have no control. As noted in the previous article (see part 1-3), Bondora is a dynamic environment and changes, whilst usually signalled in advance, cannot usually be planned for and just have to be accommodated when the reality of the change becomes apparent. Where possible I have noted the changes that have occurred. As part of this review, I have gone back over the last 9 months activity to try and relate these changes and how I believe they have, or may have, affected my results.
Portfolio Manager
The Portfolio Manager in place up to the end of 2014 was an automated, parameter-driven mechanism to allow investors to automatically invest in loans that meet the criteria set by the investor. From the start of 2015, Bondora made major changes to the Portfolio Manager, preceded by allocating a “Risk Segment†(running from low to high risk) to each Loan Application.
Whilst a Loan Application retained the previous Credit Score and associated Credit Group (essentially an income-related grading), these no longer played a part in the new Portfolio Manager, which no longer allowed Loan Selection by any criteria other than the new “Risk Segmentâ€. Probably the most contentious element of the new Portfolio Manager was the loss of selection by Country. The use of Country was a critical element in the previous automated selection process for most ( if not all) investors, and its loss was not well received on the official forum.
In terms of my process of Decision Tree analysis, this changed nothing. All the previous data was still present and some new data was added about the New Risk Segment and the process associated with it. I have considered adding the new Risk Segment data to the Decision Tree analysis, but decided against this primarily as its introduction, occurring as it did some 3 months into my experiment, had the potential to dramatically alter the structure of the Decision Trees, creating a possible disconnect at this point.
A secondary reason in my decision was the fact that this data was itself the result of an analysis conducted by Bondora and for which there is no detailed discussion or publication showing how it has been arrived at. Whilst I am not surprised at the decision not to publish what is, after all, company confidential data, the output – a legend consisting of a 1- or 2-letter classification – is not an independently verifiable fact, it is merely the output from an analysis and shares this feature with my own Decision Trees.
The major difference between this and the Decision Tree output I have is the context that is provided by a full Decision Tree to those who wish to use it. IMHO, the discerning viewer can decide from the context of a complete Decision Tree whether the end point of a particular branching of the tree indeed describes a trend or is just a convenient mathematical activity that segregates the data, but reveals no trend. I offer the snapshot of Self Employment from the Decision Tree for Estonia as an example of this added value.
To me, the bigger picture describes a trend suggesting that the longer the applicant has been in the same employment, the less likely a default will occur. It also shows that the Decision Tree has found that those in the same employment for over 5 years can be further segregated by age, with all defaults occurring in a single age range (45 to 51). Furthermore, the sample size of the >5 years employment is 51 and the defaults, which all occur in the noted age group, amount to just 2 examples – a 4% default rate on the set of 51 as a whole. Is this further segregation a guide to investment or just a “Clump†in a larger data set? In the words of the immortal Clint Eastwood “You’ve gotta ask yourself one question: “Do I feel lucky?” Well, do ya, punk?â€.
Application Process
In last half of February 2015, Bondora introduced changes to the application process designed to allow applications to be assessed by Investors before all data had been collected and, where applicable, validated.
This had no immediate effect on the Decision Tree analysis, but did require minor amendments to the process. Many applications were taking up to 5 or even 6 attempts before they became fully acceptable and finally funded. Many of these rejections took place after funding was in place. They were then cancelled and re-submitted with updated data. It was important that such applications did not get counted as “Previous Applicationsâ€. This field does appear in some lower levels in a Decision Tree and therefore new data cleaning activities (explained in the previous article) had to be introduced into the process.
Server Capacity Issues at Bondora
Around the 2nd week in March, 2015, the servers at Bondora ran into capacity issues. This affected both the ability of the applicant to apply for loans and for investors to lend.
Aggregated effect of Bondora changes
Concurrent with the introduction of the changed application process and the server capacity problems, it is apparent from a chart provided by Peerlan that the new Portfolio Manager’s ability to fund loans collapsed, effectively to zero.
When Bondora fixed their capacity problems, the mix of Loan Applications becoming available to manual investors had changed dramatically. Whilst this had no effect on the use of Decision Trees to select loans, it meant that many more loans became available to manual bidders. Many of these loans were Estonian, historically considered to be of higher quality.
This availability of more loans of potentially higher quality is reflected in my activity by the highest level of loan part purchases seen since the start of my use of Decision Trees. This higher number of purchases occurred even with the restrictions I had placed on myself regarding the level of purchases per Loan Application, mentioned earlier.
As I write this review, the new Portfolio Manager process has again changed, this time to run more often, with a target of running effectively all the time. This new process appears to have a dramatic effect during the 16th July, reducing opportunities for manual bidding on new Loan Applications essentially to zero, as the new Portfolio Manager process swept up all new listings.
New Loan Applications have appeared again the next day and a close reading of the Bondora “Guide to Investing†FAQ suggests that Loans that fail to be filled immediately should appear out of the back of the new process and become available to manual investing and this appears to be the case. This occurrence and the availability of loans on the Secondary Market (at a premium in most cases), leaves me feeling that my work to date has not been in vain. Time will tell!
Flip forward to the final part 6.
