Artificial intelligence (AI) is a two-edged sword. While AI and machine learning (ML) models are powerful, they can be known to make egregious mistakes. For example, the integration of AI and automated systems into the criminal justice system is creating challenges for fairness and due process. Technologies such as the COMPAS risk assessment tool and TrueAllele DNA analysis software are used to make critical decisions but are often kept behind closed doors, preventing defendants and their lawyers from examining or challenging the methodologies. This lack of transparency can and has led to wrongful denials of parole and unfair convictions. In the case of Glenn Rodríguez, an inmate at the Eastern Correctional Facility in upstate New York, a black box AI model ‘Compas risk assessment’ was used to deny his parole despite a nearly perfect record of rehabilitation because the model predicted a high risk of recidivism. This was due to an error in a single input.
Another black box error came in the form of air quality. Smoke from Northern California wildfires brought unhealthy air conditions to Sacramento, prompting a weeklong air quality warning. However, discrepancies in online air quality data caused confusion, with a source, BreezoMeter, inaccurately reporting higher air quality despite other sources showing significantly higher AQI levels. This caused Google to list the AQI as below 50, qualifying it as “good,” while other maps showed levels in the mid-to-high 200s. These examples show the potential risks of relying solely on black-box models, which can provide misleading information, potentially impacting the public during critical moments.
“Egregious errors such as these are a result of un-interpretable machine learning models,” says Varun Nakra, an expert in developing interpretable machine learning models that are in line with human intuition, as well as business and regulatory requirements. This was the subject of his expert talk at a major data science event in New York attended by many data science entrepreneurs, quantitative professionals, and experts working in AI and machine learning. Varun passionately emphasized the need to build interpretable machine learning models to avoid making egregious mistakes for high-stakes decisions such as judicial, medical assessments, and finance, which negatively impact end users. Instead of using black box models, he proposed a novel approach, ‘functional Analysis of Variance (fANOVA) methodology,’ which adds intuitive constraints to off-the-shelf machine learning models to make them interpretable to human intuition, justifiable to stakeholders and auditable to scrutiny, thus helping the end users achieve better outcomes. “We apply the principles of ‘additivity,’ which combines variables linearly; ‘sparsity,’ which uses just enough variables; and ‘monotonicity,’ which ensures that the signals are monotonic if needed. Thereafter, we perform constrained optimization. This is the general recipe to make an ML model interpretable and avoid pitfalls that lead to harmful decisions caused by black box models,” says Varun, whose achievements in machine learning and data analytics have been integral to the technological revolution in banking and finance and have contributed greatly to industry innovation and implementation.
As with any large leap in innovation, implementing AI and ML into these fields hasn’t been met with unanimous support. The use of this kind of AI tech requires a great deal of experience and knowledge in statistics, something that is currently lacking in the workforce. In Nakra’s own words, “One of the main challenges that I see is a lack of a PhD in statistics. Whilst around 70% of my teams and peer group are PhDs, I overcame this impediment by learning on the job, studying relevant academic material, doing research work, and spending time on practically relevant concepts that are impactful in the industry.”
The other major challenge throughout this evolutionary phase has been the movement across programming languages. Nakra points out just how quickly things happen within this field and emphasizes how important it is to keep up with the constant changes in AI and ML, explaining, “When I started my career, knowing languages such as C++, SQL, and SAS was great; however, things evolved, and I had to move from those languages to Python and R. Therefore, keeping up with the programming and technological advancements in the field can be a challenge.”
Nakra started as a data analytics and predictive modeling consultant and has provided consulting services to important insurance clients such as American Insurance Group, Marsh McLennan, The Guardian Life Insurance Company of America, and United Airlines. During this time, he acquired a great deal of experience working as a consultant, which gave him critical insight into the field and all its possible pitfalls and gave him a strong foundation in analytics and data science, which would go on to become an important part of his later work when he became a full-fledged member of “credit risk model development” teams across major financial institutions such as Standard Chartered bank in Singapore, National Australia Bank in Australia and Deutsche bank in the US.
One of Nakra’s greatest accomplishments is the development of the Downturn Loss Given Default (LGD) model. Of this groundbreaking program, he says, “I led the development of this model that comprised end-to-end model development (project management, methodology development, work allocation to the team, team leadership, stakeholder management, dealing with the external regulators). The model was about predicting losses on residential mortgages in a downturn event.”
The primary challenge Nakra and his team faced was the lack of relevant downturn data for the model. To compensate for this, Nakra formulated multiple expert views of the impact of an economic downturn on residential mortgages and converted them into constraints applied to the ratio of probabilities of a multinomial logistic regression model, thus creating relevant downturn data for modeling.
“I used Bayesian analysis which has not been used widely in credit risk models. The methodology consisted of Markov Chain Monte Carlo sampling strategies such as Importance, Rejection, and Gibbs sampling to sample from the posterior distribution and non-linear constrained optimization to solve for polynomial regression parameters. Since this was a model of national economic importance (residential mortgage loans constitute a systemic risk in Sydney and Melbourne), the regulator responded with 100+ challenging questions, which I answered with further data analysis and strong rationale,” Nakra explains.
As a result of this innovative work, the model was not only approved but also received special recognition and appreciation from the regulator, senior bank executives, the Chief Credit Officer, and the Chief Risk Officer.
With a vision to advance interpretable machine learning models by continuing to implement innovative technology and educating those both within and outside of these fields about its uses and benefits, Nakra’s ongoing contributions in the form of research papers and conference presentations have played a significant role in helping to illustrate just how beneficial this machine-learning technology can be. He has written scholarly articles in prestigious journals such as Professional Risk Managers’ International Association and websites such as Hackernoon and Towards AI. He also regularly speaks at AI/ML/data science conferences, reviews data science books, peer reviews scientific papers, and judges multiple competition/award ceremonies like Hackathons and the MIT100K.
Nakra continues to lead significant, well-funded, high-impact projects and has interest from multiple stakeholders both within and outside Australia. He hopes to continue tapping into the undiscovered potential of technological tools like AI and ML.
As Varun Nakra says, “Show respect and obedience to your superiors and be kind and empathetic to people who report to you or whom you’re leading, and rewards and recognition will automatically follow.”
The post Varun Nakra: Mastering the Delicate Art of Interpretable Machine Learning Models appeared first on LA Weekly.
(Except for the headline, this story has not been edited by PostX News and is published from a syndicated feed.)