Silverlight modal window is not defined by Telerik

I apologize for the late response.

Are you certain that the problem is found within a Silverlight app? Step 7 from the screenshot looks like its clicking on an HTML element. If the pop-up window is found within an HTML-based app, that would make things much easier.

We'll need to reproduce this locally in order to troubleshoot it. With HTML-based applications, we can usually reproduce these kinds of problems with a Fiddler trace:
http://tv.telerik.com/watch/automated-testing-tools/test-studio-creating-fiddler-trace

Go ahead and send us that along with the actual test. If you don't want to share this data on a public forum, please go ahead and open a support ticket on this. As a paying customer, you're entitled to access to your support system. This will also guarantee a much quicker response.