Select Page

Avoid the Black Box: The Need for Explainable Machine Learning

by | Nov 3, 2021

Explainable Machine Learning

This blog post is taken from the InRule report, Smarter Predictions = Smarter Decisions How Machine Learning Is Revolutionizing Automated Decision-Making. Download your copy today.


Machine learning dramatically enhances the power of decision platforms by analyzing business data and making predictions—and then (as the name indicates) learning over time to make even better predictions in the future. But, while there is a lot of excitement about how machine learning can transform business operations, many business leaders share concerns about handing over too much to “intelligent” software.

What are the potential risks to my business? What are the consequences? How would I even know if my machine learning models are delivering biased outcomes? 

A widely reported recent example of this issue relates to Amazon’s automated recruiting engine introducing gender biases into its calculations:

Amazon’s experimental recruiting engine…[learned] to penalize resumes including the word “women’s,” until the company discovered the problem. In effect, Amazon’s system taught itself that male candidates were preferable. It penalized resumes that included the word “women’s,” as in “women’s chess club captain.” And it downgraded graduates of two all-women’s colleges, according to people familiar with the matter.

Despite its position as one of the world’s biggest tech companies, even Amazon wasn’t immune to issues with its machine learning algorithms. So, while machine learning has the potential to transform business operations, actionable explainability is a necessary component for the next era of decision platforms—and that is one of the core philosophies driving our product roadmap.

Explainable AI (XAI) are systems that provide transparency into the specifics of their decisioning processes. This provides several key benefits and gives business leaders the confidence to deploy machine learning:

Explainability makes it easy to ensure things are working correctly

AI technologies are gaining momentum in today’s enterprises. In fact, research from Forrester revealed that 67% of IT decision-makers said they expect their AI/ML use cases to increase at least slightly over the next 18-24 months. However, there needs to be an insight layer for humans to monitor and optimally utilize their machine colleagues.

At any moment, your team should be able to assess how and why a machine learning model is returning a certain prediction and why your decision automation returned a specific outcome. Gaining this insight needs to be straightforward and seamless.

Explainability makes your company smarter

The power of machine learning is that it can efficiently recognize non-intuitive patterns and trends in giant datasets that often go unnoticed by even the best data scientists. One of the unique benefits of explainable machine learning is that your teams can understand exactly which data points impacted predictions and the weight of each factor on the prediction. This information isn’t just for data scientists, but for anyone looking to enhance processes and outcomes.

Data Points

For example, a global telecom provider may know that one of the leading indicators of customer churn may simply be the type of contract in place. Perhaps customers with month-to-month contracts churn at a higher rate than ones with annual agreements. Without a detailed understanding of the problem, a company might conclude the best option is to discontinue month-to-month contracts, costing some customers but holding the rest tighter.

A more nuanced response to this problem requires a more nuanced understanding of how this effect is different for customers with different product combinations in different locations. But teasing out those details can be quite difficult, requiring lots of math and computation. And even with that, having an accurate black box Machine Learning classification prediction of who is likely to churn doesn’t tell you why, and won’t be enough to inform how to effectively intervene and convert at-risk customers to longer-term contracts or otherwise retain their business.

Explainability enables you to improve your logic

As your machine learning models make predictions, your teams can make more informed choices and build automated decisions that are based not only on those predictions but the specific factors that went into each one.

Maybe the next step after predicting which month-to-month customers have the highest possibility to churn, rules are created to proactively approach high-risk month-to-month customers with enticing promotions to renew their contracts. When you consider what it takes to manage a business at scale with potentially tens of millions of customers and prospects, even small enhancements to your automated logic could equate to significant gains to the bottom line.

Explainability helps you manage compliance—especially in highly regulated industries

In no other use case is explainable AI more important than in pharmaceuticals, financial services, and other highly regulated industries.

These industries face ongoing compliance requirements, and the consequences of being out of compliance—even accidentally—can be disastrous. Although decision automation and machine learning can be valuable tools to speed up and make better decisions, the risk that they may stray from regulations and requirements is real.

According to a recent report from Deloitte:

The cost of compliance and risk mitigation over the last eight years has jettisoned almost all discretionary funding available to firms. Compared to pre-financial crisis spending levels, operating costs spent on compliance have increased by over 60 percent for retail and corporate banks. Policymakers, regulators, and shareholders are looking for firms to not only meet new regulatory requirements but to ensure the effectiveness of all that has already been built.

Decision platforms provide explainability to ensure compliance with regulations or adherence to corporate guidelines.

End-to-End Explainability: The InRule Advantage

InRule Technology has always focused on explainability and transparency to make decision automation easier and more accessible across the enterprise. Our point of view is that one should not need advanced technical skills to automate decisions and manage a best-in-class decision practice.

This key differentiator has made our technology one of the leading decision platforms in the market since 2002.

For more information about InRule, our decision platform, and our end-to-end explainability, visit our platform overview.

For more content related to this blog post, download our report, Smarter Predictions =  Smarter Decisions How Machine Learning Is Revolutionizing Automated Decision-Making.




We’d love to share company and product updates with you! Please enter your email address to subscribe to monthly updates from InRule. 

If at any time you want to unsubscribe, you can easily do so by clicking “unsubscribe” at the bottom of every message we send or visiting this page.