This is part 5 of a series of guest posts by British Bondora p2p lending investor ‘ParisinGOC’. Please read part 1, part 2,  part 3 and part 4 first.

The Management of Change

As mentioned in my earlier article on the construction of the decision Trees, my responsibilities when employed (yes, dear reader, I am now retired) included the successful proposal to create new teams to conduct Data Mining and produce and disseminate Metrics relating to the research activities. As on many other occasions, I was then charged with making my assertions real by staffing and then running said teams to realise the benefits I had stated should arise.

As part of my (rapid) learning in these activities, I came to understand the need to maintain processes until solid analysis could isolate and support changes. So in this review period, for those elements under my control, I have maintained certain actions within set parameters until I felt I could justify a change and then have maintained that changed process until the next time the data supported a further change.

Changes I Controlled

Given that my need to change my selection process was as a direct of seeing my money rapidly disappear (!) I limited my ongoing expenditure to the minimum purchase (5 Euros) allowed by Bondora and only made 1 purchase per selected Loan Application.

This continued throughout October 2014, when I felt that the downward trend in parts falling behind with payments was established and likely to continue. From the beginning of November 2014 onwards I increased the number of parts of any single loan application I would buy to 2, still of 5 Euros each. Note that for some application types with, for example, a higher (between 5% to 7%) indicated historical failure rate, or a very high (above 45%) interest rate; I still limited my purchasing to 1 part of 5 Euros.

This Purchasing policy remained in place until the beginning of April 2015 when my increasing confidence in the selection process, my increasing cash reserve and other factors described below, meant I felt able to increase the value of purchases (to include 10 Euro parts if I felt an application was sufficiently strong) and increased the number parts purchased of any particular loan. This latter element in particular allowed me to take advantage of events outside of my control that offered opportunities that had not previously existed, explained later in this article.

Errors in my Process
In the period October 2014 to the end of the year, I was updating the Trees twice a month. There was no detailed timetable, but the Trees did exhibit a greater degree of change in this time than was later the case. It was during the first update in December, week 51 of 2014, I noticed that the previous Tree had been built using corrupted data. It was only later in the review period that I noticed that this period – from weeks 48 to 50 inclusive – exhibited the last “spike” in defaults.

From the next update onwards (31st December 2014) I implemented a more rigorous update procedure and restricted the updates to 1 at the end of each month. I felt that this may enable changes in the Tree Structures to be more visible and so attract my attention to these changes and validate the process that had generate them, thus avoiding process errors. The fact that the datasets provided by Bondora were subject change without notice (and did so often) was an additional factor in the decision to have fewer, more rigorous build events.

I worried that fewer updates to the Trees would lead to out-of-date trees and more In Debt and Defaulting loan parts, but this has not become apparent either in daily use or this review process.
I have noticed that the Decision Trees are not static and do change over time. Sometimes – rarely – these changes occur at a high level and are very noticeable. However, the Trees have changed in a subtle way at lower, more compartmentalised levels. This is discussed later in this article.

Changes I could not Control

Whilst I have tried to maintain a tight control over my activity since starting to use the Decision Trees to guide my loan selection, there is the overall Bondora environment over which I have no control. As noted in the previous article (see part 1-3), Bondora is a dynamic environment and changes, whilst usually signalled in advance, cannot usually be planned for and just have to be accommodated when the reality of the change becomes apparent. Where possible I have noted the changes that have occurred. As part of this review, I have gone back over the last 9 months activity to try and relate these changes and how I believe they have, or may have, affected my results.

Portfolio Manager
The Portfolio Manager in place up to the end of 2014 was an automated, parameter-driven mechanism to allow investors to automatically invest in loans that meet the criteria set by the investor. From the start of 2015, Bondora made major changes to the Portfolio Manager, preceded by allocating a “Risk Segment” (running from low to high risk) to each Loan Application.

Whilst a Loan Application retained the previous Credit Score and associated Credit Group (essentially an income-related grading), these no longer played a part in the new Portfolio Manager, which no longer allowed Loan Selection by any criteria other than the new “Risk Segment”. Probably the most contentious element of the new Portfolio Manager was the loss of selection by Country. The use of Country was a critical element in the previous automated selection process for most ( if not all) investors, and its loss was not well received on the official forum.

In terms of my process of Decision Tree analysis, this changed nothing. All the previous data was still present and some new data was added about the New Risk Segment and the process associated with it. I have considered adding the new Risk Segment data to the Decision Tree analysis, but decided against this primarily as its introduction, occurring as it did some 3 months into my experiment, had the potential to dramatically alter the structure of the Decision Trees, creating a possible disconnect at this point.

A secondary reason in my decision was the fact that this data was itself the result of an analysis conducted by Bondora and for which there is no detailed discussion or publication showing how it has been arrived at. Whilst I am not surprised at the decision not to publish what is, after all, company confidential data, the output – a legend consisting of a 1- or 2-letter classification – is not an independently verifiable fact, it is merely the output from an analysis and shares this feature with my own Decision Trees.

The major difference between this and the Decision Tree output I have is the context that is provided by a full Decision Tree to those who wish to use it. IMHO, the discerning viewer can decide from the context of a complete Decision Tree whether the end point of a particular branching of the tree indeed describes a trend or is just a convenient mathematical activity that segregates the data, but reveals no trend. I offer the snapshot of Self Employment from the Decision Tree for Estonia as an example of this added value.

To me, the bigger picture describes a trend suggesting that the longer the applicant has been in the same employment, the less likely a default will occur. It also shows that the Decision Tree has found that those in the same employment for over 5 years can be further segregated by age, with all defaults occurring in a single age range (45 to 51). Furthermore, the sample size of the >5 years employment is 51 and the defaults, which all occur in the noted age group, amount to just 2 examples – a 4% default rate on the set of 51 as a whole. Is this further segregation a guide to investment or just a “Clump” in a larger data set? In the words of the immortal Clint Eastwood “You’ve gotta ask yourself one question: “Do I feel lucky?” Well, do ya, punk?”.

Application Process

In last half of February 2015, Bondora introduced changes to the application process designed to allow applications to be assessed by Investors before all data had been collected and, where applicable, validated.

This had no immediate effect on the Decision Tree analysis, but did require minor amendments to the process. Many applications were taking up to 5 or even 6 attempts before they became fully acceptable and finally funded. Many of these rejections took place after funding was in place. They were then cancelled and re-submitted with updated data. It was important that such applications did not get counted as “Previous Applications”. This field does appear in some lower levels in a Decision Tree and therefore new data cleaning activities (explained in the previous article) had to be introduced into the process.

Server Capacity Issues at Bondora

Around the 2nd week in March, 2015, the servers at Bondora ran into capacity issues. This affected both the ability of the applicant to apply for loans and for investors to lend.

Aggregated effect of Bondora changes

Concurrent with the introduction of the changed application process and the server capacity problems, it is apparent from a chart provided by Peerlan that the new Portfolio Manager’s ability to fund loans collapsed, effectively to zero.

When Bondora fixed their capacity problems, the mix of Loan Applications becoming available to manual investors had changed dramatically. Whilst this had no effect on the use of Decision Trees to select loans, it meant that many more loans became available to manual bidders. Many of these loans were Estonian, historically considered to be of higher quality.

This availability of more loans of potentially higher quality is reflected in my activity by the highest level of loan part purchases seen since the start of my use of Decision Trees. This higher number of purchases occurred even with the restrictions I had placed on myself regarding the level of purchases per Loan Application, mentioned earlier.

As I write this review, the new Portfolio Manager process has again changed, this time to run more often, with a target of running effectively all the time. This new process appears to have a dramatic effect during the 16th July, reducing opportunities for manual bidding on new Loan Applications essentially to zero, as the new Portfolio Manager process swept up all new listings.

New Loan Applications have appeared again the next day and a close reading of the Bondora “Guide to Investing” FAQ suggests that Loans that fail to be filled immediately should appear out of the back of the new process and become available to manual investing and this appears to be the case. This occurrence and the availability of loans on the Secondary Market (at a premium in most cases), leaves me feeling that my work to date has not been in vain. Time will tell!

Flip forward to the final part 6.

This is part 4 of a series of guest posts by British Bondora p2p lending investor ‘ParisinGOC’. In part 1, part 2 and part 3 published in December 2014 you could read how he used the data to built decision trees to identify lending opportunities. Now you can read how that strategy worked out.

Introduction

In August 2014, I realised my portfolio of P2P loans at Bondora was not performing as I would wish. There was an urgent need to change the way I selected loans in which to invest the money I had at my disposal. My search for a better way of selecting loans lead me to use Decision Trees to analyse the loan data available from Bondora using “RapidMiner” – software available to download for free.

It is now over 6 months since I described my original work to construct the Trees. This follow-up article chronicles what I believe is the success of my efforts to date whilst also describing the multiple factors, both within and beyond my control, that mean that, whilst I feel very comfortable with the progress made to date, others may feel that I have just been lucky!

The journey since I created my first Decision Tree and started to make purchasing decisions based almost totally on their outputs has been one of constant change. Detailing the changes to elements over which I have no control has shown me how they contribute to what I believe is success as much as my own efforts to improve the selection processes. Describing the change in the Decision Trees as well as their use in the dynamic Bondora environment has left me feeling that, without constant monitoring and review of both the process of creating the Trees as well as their use, it may still be very easy to snatch defeat from the jaws of victory.

Key to ensuring the veracity of my protestations of success has been the maintenance of a consistent approach to my selection and lending process. To this end, I will describe those changes to my process that I can control and explain how and why such changes have taken place. In short, I have maintained a restricted buying policy, investing only the minimum amount (5 Euros) at any one time and, latterly, only buying a maximum of 2 loan parts (of 5 Euros each) in any one loan, depending on the outputs from the Decision Trees and my own mood at the moment of purchase.
I realise that this last phrase is not at all scientific, but the fact that my Portfolio of c.12000 Euros was not performing as expected was for me, a non-trivial affair and some emotional response has to be accommodated.

I have already stated that I believe my efforts have been successful. This is based on the fact that the rate of default (Once a loan principal has been overdue for 60+ days, it is labelled as “defaulted” – Bondora FAQ) in my portfolio has returned to historical, pre-2014 levels. Up to this time, even though I had come to realise that I needed to actively manage my portfolio, my selection of loans was done almost entirely using the “Portfolio Manager” – an automated, parameter-driven purchasing function provided by Bondora and supplemented by instinctual analysis of the descriptions of the Loan Applications available to invest in.

Looking at the simple chart of Held Parts/Defaults, the number of defaults in held loans rose significantly over the summer of 2014, coinciding with a big increase in both the number and value of investments on my part. Referring to the same chart, it can be seen that, even though the number of investments remains close to summer 2014 levels, my defaults have fallen to the numbers experienced earlier, at much lower volumes.

With my new-found confidence that I have a process for selection and management that appears to be sound, I have started to increase the volume of Loan Parts purchased so that the value is now approaching Summer 2014 levels of investment.

Progress to date

Graphical representation of Progress

I will use a more detailed graph showing the volume of Loan Parts purchased, those subsequently sold, those “Overdue” and those in default (still held by me as well as sold) to hopefully illustrate the performance of my selection and management processes. Continue reading →