Performance Feedback with YSlow & Selenium

29 10 2009

Lately I’ve been experimenting with integrating YSlow into our existing Selenium test suites. For those not familar with YSlow, its a Firefox add-on developed by Yahoo which provides page performance analysis. An excellent tool if web testing is a part of your repertoire.

Example: YSlow Statistics

The Selenium / YSlow integration concept itself isn’t new. There’s a great post on Adam Goucher’s blog discussing the idea and while I wasn’t keen on tying it to our CI system, the idea of getting page performance data at the completion of our automated tests sounded great.
In terms of setup, its relatively painless. The solution is based around the free Yslow firefox addon and the (opensource) ShowSlow reporting server. Given that there isn’t a lot of information available on the setup process, here’s a brief overview:

ShowSlow Server Setup

1. Download and install Apache, PHP and MySQL (essentials will do).
2. Once all of the above are setup, checkout the ShowSlow project source from here and host it via Apache.
3. You’ll also need to create a new database in MySQL and run in tables.sql (from the ShowSlow directory) to create all the relevant tables for ShowSlow.
4. Edit config.php and supply your database credentials.
5. Load the ShowSlow site in your browser on your webserver and click on the “configuring your yslow” link – you’ll need the values from this page later.

 YSlow Client Setup

1. Create a custom Firefox profile on your Selenium RC machines (using firefox.exe -ProfileManager). The profile needs to be setup with the following attributes (at minimum):

  • YSlow add-on installed and setup to run perf analysis on page load (set in Yslow options).
  • In about:config, set “extensions.yslow.optinBeacon” to true and “extensions.yslow.beaconUrl” to the value from your “configuring your yslow” page.

2. On each Selenium RC box (the machine you’ll use to run automated tests), you’ll need to setup Selenium to use your new custom profile (using DseleniumArgs=”-firefoxProfileTemplate ” argument). In order to achieve this, modify the Selenium RC launch command to include the path to the new Firefox profile you just created. For example:

java -jar selenium-server.jar -firefoxProfileTemplate “C:/SELENI~2/SELENI~1”

NOTE: You need to provide the profile path using the DOS short name format. For more info on finding your firefox profile directory, see: http://support.mozilla.com/en-US/kb/profiles

Once you’ve completed the setup, running your automated tests should result in some YSlow performance statistics being sent to your ShowSlow server.

It’s worth noting that there are some downsides to this setup. Firstly, its Firefox only (currently) so any tests run against other browsers wont record any results with ShowSlow. And finally, it records page performance data for every page that your browser loads. This means that you’ll get performance data for URL’s that you don’t necessarily care about like those used by Selenium and those with variable query strings. While you can’t explicitly tell YSlow to not record performance data for these URL’s, you can modify the code within ShowSlow to prevent these URL’s from contaminating your results (eg./ I’m currently stripping some variable data out of certain URL’s as well as filtering the ShowSlow ouput).

If you’re already using Selenium as the basis for your automated testing, I’d definitely recommend tying it in with YSlow. With a little extra effort up front, it’ll add ongoing value to your automated tests.

UPDATE: It seem that 2.0b6 is the last that works correctly with the current version of ShowSlow. Version 2.0.1 of Yslow seems to rely on POST’s to send the full set of performance data to the beacon server. Version 2.0.1 will still work, you’ll just notice that the details page has a “0” value for all of the Yslow Breakdown results.

Advertisements

Actions

Information

8 responses

29 10 2009
David Henderson

We’ve been doing some similar work over the past year, and recently did a Talk at GTAC (http://www.gtac.biz)

– Slides: http://www.gtac.biz/abstract-bios/presentations/gtac-slides
– Reporting Demo: http://performance.smartfocusdigital.com
– Blog Post http://tech.smartfocusdigital.com/2009/10/automated-performance-collection-and.html

29 10 2009
Twitter Trackbacks for Performance Feedback with YSlow & Selenium « STANDARD DeViATiON [stdev.wordpress.com] on Topsy.com

[…] Performance Feedback with YSlow & Selenium « STANDARD DeViATiON stdev.wordpress.com/2009/10/29/yslow-selenium – view page – cached Lately I’ve been experimenting with integrating YSlow into our existing Selenium test suites. For those not familar with YSlow, its a Firefox add-on developed by Yahoo which provides page… (Read more)Lately I’ve been experimenting with integrating YSlow into our existing Selenium test suites. For those not familar with YSlow, its a Firefox add-on developed by Yahoo which provides page performance analysis. An excellent tool if web testing is a part of your repertoire. (Read less) — From the page […]

16 05 2010
O melhor da semana 09/05 a 15/05 « QualidadeBR

[…] Performance Feedback with YSlow & Selenium – Steven Stojanovski (STANDARD DeViATiON); […]

4 08 2010
Julien

Typo : DseleniumArgs=”-firefoxProfileTempate “). <— missing l … L

DseleniumArgs=”-firefoxProfileTemplate “).

Except the typo , that saves me a lot of times, thanks (was wondering why hudson build was nt reporting to our showslow)

4 08 2010
stdev

Thanks Julien, nice spot!
All fixed.

9 08 2010
Guido

Dear,

First of all; thanks for the manual Yslow & Selenium, looks great!

I would like to connect Showslow (Pagespeed and Yslow included) with Selenium. Unfortanetly I do not understand the following statement:

”2. On each Selenium RC box, you’ll need to setup Selenium to use your new custom profile (using DseleniumArgs=”-firefoxProfileTemplate “).”

Can you please help me out? Do you have a detailled description or installation manual?

If you have further questions, please feel free to contact me.
Thanks in advance!

Guido.

10 08 2010
stdev

Thanks for your comment.
I’ve updated the post with a little more information relating to the configuration of Selenium RC.
Hopefully this will point you in the right direction.

Cheers.

13 08 2010
Guido Ploum

Ok thanks! Keep in touch

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s




%d bloggers like this: