How to Manage User Experience for Silverlight and Flash Applications
We sat down with www.timecockpit.com, which made the strategic decision to go with Silverlight when implementing their SaaS-based Time Tracking solution. In this blog we explain why they chose to use Silverlight and discuss their 4 key requirements for successful UEM for Silverlight/Flash-based applications:
- User Experience for Every User
- True End-to-End Visibility (from Silverlight to the Database),
- Automatic Client-Side Error Detection and Capture
- Client and Server-Side Performance Analytics
Requirement #1: User Experience for Every User
To gain a better understanding of customer experience, key questions to be answered are:
- Where do my users come from? What are the browsers they are using? What is their bandwidth?
- What is the user experience by geographical region? How does performance differ across browsers?
The following screenshots visualize User Experience data captured from Silverlight users. It helps www.timecockpit.com to identify who is really using their software and, in case there are any problems, whether these problems are related to the user’s environment (geographical region, browser, operating system, network connectivity, etc.), the rich client implementation (e.g. problems in the Silverlight code) or the server-side application implementation.
Looking at these high level metrics is a great way to figure out if there are any general problems that affect everybody, that are bound to a geographical region or to a specific browser. To identify the actual problem the requirement then becomes getting visibility into every user and every interaction this user had with the application to answer the following questions:
- How does the user navigate through the application? Is it the intended way? Can we learn from their flow?
- Does the user encounter functional problems in the application workflow? Or are these performance related problems?
- Where do these errors happen? On the user’s device or in the server-side implementation?
The following screenshot shows a list of real visits in a specific timeframe. The highlighted visitor used IE8 in Redmond, WA. The individual Page Actions show how the user navigated through the application. Starting with several attempts to login through the HTML login mechanism (with one failed attempt indicated by the orange light bulb) the user eventually ended up in the Silverlight application where the user viewed data using an application specific filter setting. These two actions in Silverlight took a rather long time (40+ and 100+ seconds–a starting point to identify the root cause, but definitely proof that this user had a serious problem.
Requirement #2: True End-to-End Visibility
Knowing where your users come from is one part of the story. Knowing where the application has problems in delivering the right user experience is the other essential part that allows easy identification and fixing of problems. The questions that come up have to do with End-to-End Visibility:
- Do we have a problem in the Client-side implementation? Is it related to a combination of Silverlight/Flash and a particular Browser/OS?
- Is there a problem in the Web, App or Database tier?
- Do we have an architectural problem with too many roundtrips from the end-user’s device to our application server?
The following screenshots shows where along the application delivery chain (End User – Browser – Web Server – App Server – Database Server) we impact end user experience. This might be related to functional problems on an application server (e.g.: an exception thrown in a web service), a performance problem (e.g: an inefficient algorithm or slow SQL statement) or an architectural problem (caused by too many roundtrips between browser and server or app server and database):
After looking at the high-level view to identify the deliver hotspots one has to go deeper and identify the root cause. The following screenshot shows incoming transactions from the End User’s Rich Client Application. Following these requests all the way back to the database allows us to see what causes performance or functional problems. In this case, an exception is thrown when the app server tries to put a message in the Windows Azure Service Bus which results in an HTTP 500 error for the end user that in turn negatively impacts end user experience:
Requirement #3: Automatic Client-Side Error Detection and Capture
It can be challenging when application code that runs on the end user’s device has problems that happen locally on the device itself. Getting the stack trace of a client-side exception would help diagnosing the problem – or additional context information such as the action the user triggered in the client. But – how do you capture these problem details to make diagnostics easier? Do you use log files and periodically send them to your system? The worst that can happen is that you roll-out a new version of your application and suddenly requests to your servers stop. Is it because nobody likes the updated version? Or is there a problem at the client? Do we need to wait until a customer complains or sends in log files? This leads us to the requirement that we need visibility into problems that happen on the end user’s device. We need to answer questions such as:
- What type of exceptions are thrown in the client application? Which interactions result in exceptions/errors?
- Are these errors impacting the end user? Are they only happening on some types of devices/browsers/OSs?
I want these errors reported automatically and not wait for somebody to call! The following shows a screenshot of errors captured within the Silverlight application. These errors get caught in the client application and automatically sent to the user experience management system. Not only do we see the full error details but also know which end user was impacted. Looking at a list of errors, which can also be grouped by browser, device or OS, , enables one to identify whether these problems are bound to a certain environment.
From one of these errors it is important to understand more about the end user that is impacted. Is it an error that was caused by a user error, e.g.: invalid user input? Or is it really something that is wrong in the client-side code? The following screenshot shows more context of one of these errors. It shows the actual end users action, performance information as well as additional errors that happened in the same interaction:
Requirement #4: Client and Server-Side Performance Analytics
What’s the next step when we know that we have a performance-related user experience problem? The questions that come up are:
- Is the problem in the rich client or on the server side?
- What did the user do prior to encountering that performance problem? What feature of the application was used?
- Which developer to contact to look into this in more detail?
The following screenshot shows the performance breakdown of a user interaction in the Silverlight application that took a little over ten seconds. The PurePath tree at the bottom shows which performance-relevant methods were called in the Silverlight client and for how long these methods were executing. The highlighted rows are methods that developers should focus on – they all took more than 1s to execute resulting in the overall long execution time. We can also see that it is a clear client-side problem as the three server-side requests to the .js and .ss files were rather fast and don’t have a significant performance contribution:
In case we see requests to the server side that take long and impact end user performance it is time to look at the server-side hotspots. The following screenshot shows the performance hotspots split by application tier and application layer (aka API). Apart from a very active web server we see that the Time Cockpit layer is the next major performance contributor spending about 50% of its time in Garbage Collection (= runtime suspension). So – clearly an area to drill in further and identify why this layer is causing so many GC runs and with that impacting overall response time.
There is a lot going on in rich client applications implemented with Silverlight or Flash. These applications need the same user experience management attention as your “traditional” web applications. Besides just investing in plain-old web analytics make sure you also cover these listed requirements as it will make your life easier when it comes to managing user experience. If you’re a Compuware dynaTrace customer check out the latest version of dynaTrace User Experience Management and our updated support for Silverlight/Flash applications on our Community Portal.