What Was Happening
Some Portal users were experiencing random page load delays. Pages that normally load instantly would occasionally take 4-5 seconds, and in some cases up to 30 seconds or more. The delays were unpredictable — they could happen on any page at any time, and then everything would go back to normal on the next click.
This issue has been present intermittently for years across various Portal installations, making it extremely difficult to reproduce and diagnose.
What Caused It
After extensive investigation, we identified the root cause as an outdated database connection driver that was being used on some Portal installations. This driver, originally released over 20 years ago, has a known issue where database connections intermittently stall for 4-5 seconds during normal operation.
Importantly, this issue was not seen on all installations. Newer on-premise installs and recently provisioned environments were already using the updated driver by default, so they never experienced these delays. Only older installations that were set up before the driver update was included in the installer were affected.
The issue was particularly difficult to diagnose because:
- The database server itself was completely healthy — no errors, no performance problems
- The delays were random and affected different pages unpredictably
- Server CPU, memory, and disk performance were all normal during the stalls
- The delay occurred in the connection layer between the web server and database, which is invisible to standard monitoring tools
- Not all customers experienced the issue, making it hard to identify a common pattern
How It Was Fixed
The fix was straightforward — we switched the Portal to use a modern database connection driver that is actively maintained and supported by Microsoft. This is the same driver already used by newer Portal installations, which is why some customers never experienced this issue.
This fix is part of a broader effort as we continue building toward a fully Dot Net Core-driven UI by the end of this year. While the legacy driver would have been retired as part of that migration, we wanted to ensure customers benefit from the improvement now rather than waiting for the full rollout.
Result: After applying the fix, page load times returned to a consistent 140-180ms with zero intermittent stalls.
Who Is Affected
- Cloud Portal customers: The fix has been applied to cloud instances.
- On-premise customers (newer installs): Already using the updated driver. No action needed.
- On-premise customers (older installs): The fix will be applied automatically during the next Portal upgrade.
What You Need To Do
For most customers, no action is required. The fix is being rolled out automatically.
If you are an on-premise customer and have been experiencing intermittent page delays, running the Portal upgrade will resolve the issue. If you need the fix applied urgently before your next scheduled upgrade, please contact support.
Additional Details
- This update does not change any Portal features or functionality
- No data is affected by this change
- No user action or configuration changes are required after the update
- The fix is a one-time change that persists across future upgrades
If you have any questions about this update, please contact support.
