A few weeks ago, the tech world was upended with public disclosure of the Meltdown (https://meltdownattack.com/) and Spectre (https://spectreattack.com/) CPU hardware vulnerabilities. The nature of these vulnerabilities is such that any type of resolution for them is guaranteed to add additional CPU clock cycles to any type of operation performed by the computer. The degree and extent of the performance impact of the Meltdown and Spectre patches is now starting to become evident. Here’s a breakdown of why this is happening, what it might mean for customers of InRule, and recommendations on how to look at assessing performance.
Analysis: Implications of fixing
We’ve been living in a long, golden summer of cheap, high-performance CPU architectures for a long time now. Not all the gains in CPU speed come from cramming additional or smaller transitors on a chip. Many improvements lie in optimizing how a CPU does its work so it can get more out of each computational cycle. Meltdown and Spectre exploit these microarchitectural hardware optimizations in software. This (and other reasons) are why server, OS, and Cloud service providers and vendors have been nervously assessing fixes for these vulnerabilities – the two attack vectors are even more dangerous in a shared VM (Cloud) environment than they are on a person’s laptop or smartphone.
To patch the holes used by these exploits, engineers needed to disable the vulnerable hardware optimizations when performing certain types of operations involving kernel level access, meaning that any benefit from those optimizations is not only lost but an additional burden of the logic needed to detect and execute the alternative instructions adds salt to an already painful wound. In the research I’ve done on the topic, I’ve seen a lot of anecdotal figures tossed around, but there are a few more rigorously controlled assessments that are useful indicators of “real world” impact. One of the best is from Terry Myerson, EVP of Microsoft’s Windows and Devices Group. He sums up the vulnerabilities as well as the various different patches available to mitigate them in a Jan-9 blog post (https://cloudblogs.microsoft.com/microsoftsecure/2018/01/09/understanding-the-performance-impact-of-spectre-and-meltdown-mitigations-on-windows-systems/). While it doesn’t appear that he has any specific performance figures or benchmarks referenced, that doesn’t mean we aren’t able to reach some conclusions of our own from the information provided.
Here are a few items that stuck out to me as being most relevant for InRule customers:
- The older the OS and hardware generally, the more noticeable the impact will be
- Windows 7 and 8 particularly are affected regardless of hardware
- Newer Intel platforms (Skylake) will still see some performance decrease, but not nearly as great as in older chips
- The Windows Server line of OS will see the most significant performance impacts, especially in extensive I/O scenarios
From these points, I feel confident in making some projections and recommendations for customers using InRule who are wondering what this all means.
Projections
Users of irAuthor running Windows 7 or Windows 8 are likely to notice degraded performance, especially when launching/running irVerify. Transitioning between Business Language rules, especially those containing multiple or complex Vocabulary, is an area where users on computers with specs closer to the minimum for irAuthor will likely observe degraded UI responsiveness.
InRule services hosted on-premise will see the greatest relative reduction in performance when hosted on Windows Server 2012/R2 than if hosted on Windows Server 2016. On-premise servers with older processors will also see greater performance degradation than those running newer affected CPU’s. Rule execution services (either custom or out-of-the-box) running in the cloud are likely to observe a moderate to substantial change in performance.
In general, the degree and amount of change in system performance customers might are experience is highly dependent upon the complexity, types, and actions taken in rules:
- Rule Applications that use and call Endpoints – Assembly, Database, Web Service, etc. will see the greatest performance hit in execution times since they involve disk or network I/O and/or dynamically generate/load executable code at runtime
- UDF (user defined functions) are also likely to incur slightly longer execution times
- The “cold-start” compilation when a rule application is first loaded by an executing AppDomain will likely see an increase in clock time. For larger/complex ruleapps, this could add non-negligible time to application startup.
With that said, in a large percentage of situations, it’s important to note that it’s easily possible that any impact from these patches could simply be lost in the noise of a high-latency operation (e.g., database query or high-latency service call). On that note, talking and looking at performance can make it easy to lose sight of what’s relevant and what’s not when in the middle of developing a project. Usually, it’s more important that rules provide the expected, correct answers than it is to save some number of milliseconds. It’s important to remember the oft-quoted developer’s aphorism: premature optimization is the root of all evil
Recommendations
What can be done to mitigate these potential areas of concern? The greatest value-to-effort in almost any case is going to be optimizing the rules for better performance. There are a lot of things that can be done (cautiously!) to achieve this, but here is a small sample of some actions that you can take to improve performance:
- Remove unused fields and entities from the rule application’s schema
- Disable and/or remove unused or unneeded rulesets from the rule app
- Cut down on unneeded entity instances by running rules in irVerify and then comparing the entities listed with expected counts and instances. Sometimes rules are written that unintentionally create new entities when the intent is to work with already-existing ones.
- Examine calculated fields and auto-rulesets to ensure they are not being redundantly invoked
- When using UDFs or any type of Endpoint, check and set appropriately the entity state refresh options to prevent the engine from having to perform full entity state scans after any invocation
Performance tuning and optimization is a complex topic, and care must be taken not to allow the goal of lower execution times outshine the goal of having a useful and maintainable business solution. Take performance measurements both before and after making a single, discrete change (this is important!). Make sure test runs are as consistent as possible, taking multiple samples to rule out transient or unreproducible conditions. Define ahead of time your key performance metrics (KPI)’s and then identify what you’ll use to measure them.
Finally, don’t be afraid to ask for help! The InRule support team (reachable at [email protected]) is here to assist you with questions or issues around your product installation, configuration, and usage. If you need more personalized and detailed assistance, our ROAD services team is happy to help! Get started with ROAD by talking to your InRule account manager or by emailing [email protected] to request pricing or a quote.
References:
