Comments by Patrick Lightbody
Reply | Original | Permalink | TweetIf the Java requirement is your main beef, you might be interested to know that Selenium 2.0 is the product of a merger between WebDriver and Selenium. The result will be no intermediate “server” component unless you truly need true remote control capabilities. Perhaps you’d like to join us as we build it out?
Reply | Original | Permalink | TweetGiven that Joe Walker is working on Bespin with you guys, I would assume DWR is high on the list of frameworks to work with. It certainly is my first choice when it comes to AJAX and Comet.
Reply | Original | Permalink | TweetI love TechCrunch. They even let my company (BrowserMob) present at their recent cloud computing roundtable.
BUT…
The folks at TechCrunch need to get educated about the “cloud”. It was evident by the moderators at the roundtable, and it’s evident now. As the other commentators have pointed out, website != platform != infrastructure. Comparing SF.com or Ning or Facebook to Amazon EC2 is absurd.
Both groups are definitely changing the game, but please understand that they are doing it in VERY different ways. Doing so would make your reporting that much more valuable to many of us
That said, keep up the otherwise great work!
Reply | Original | Permalink | Tweetmotomlins - I absolutely agree that change is part of the problem. It’s also getting tougher for non-engineers (those who didn’t _write_ Ajax apps) to understand how to test these Ajax apps. There’s definitely a growing gap in education.
But I disagree that traditional load testing support for Ajax has been good. That’s not because they are bad products. I think it’s just because they are the _wrong_ products for this problem.
At the end of the day, LoadRunner can never provide great Ajax support because it doesn’t run a real browser in real time. Today’s Ajax apps are just too complex to get in to the game of simulating the protocol-level traffic. Sure, it can be done with some level of work, but I just don’t believe it’s worth it.
Reply | Original | Permalink | TweetJohn - got it. Yes, if you're going for a basic JS framework in which Core can run on top of, that's fantastic! I just wanted to make sure no one thought that the JS libs themselves had to build _with_ Selenium for Selenium to work :)
Over time we might see Selenium expanding outside the JS sandbox. For example, we've found a more robust way to type values in to text is to issue OS-level automation commands and rather than trying to simulate a type event from JS.
Do you imagine that over time TestSwam might be able to support some of those types of APIs? Obviously being that it's a shared env it might be a bit sensitive, but if we can get some sort of standard around a small set of useful automation commands that sit outside the JS sandbox, that'd be really nice!
Reply | Original | Permalink | TweetI'm also one of the Selenium developers and I'd love to see us work together on this. I think there may be some confusion about how Selenium works and what it does. It definitely doesn't require any JS library to "use" Selenium. It works very well with any system - I have tests running against apps build with YUI, Dojo, jQuery, prototype, DWR, etc.
It also works reasonably well with most browsers. While there are launches for specific ones, you can always use the "custom" launcher, which will work with most browser executables, including brand new ones we don't have a specific browser launch for.
I love the idea of a community-supported pool of browsers and environments for open source projects to build on top of. I could imagine this "open source farm" being useful for all the JS libraries, but also for any open source/free/non-profit/whatever project that has Selenium tests.
Reply | Original | Permalink | Tweetvote: IntelliJ IDEA
Reply | Original | Permalink | TweetVery funny! Glad to see you’re making it up to Portland, it’s a great place to live (I moved there from the SF bay almost 5 years ago). I hope to try to swing by your talk tomorrow, though I have some plans that may get in the way.
Reply | Original | Permalink | TweetEric,
Good tip - that change got me the behavior I was expecting. I swear I’ve never changed this though, and I never copy over my settings (I like starting from a clean environment). In all my years of being an IDEA user, I’d never changed this setting, so something is up.Either way, IMO (and probably yours) “First letter” is a sucky default
Reply | Original | Permalink | TweetNum,
Thanks for leaving a comment. I agree that testing AJAX sites with traditional load testing tools such as JMeter, HTTPPERF, Apache ab, LoadRunner, etc is certainly possible.However, I disagree that it’s “easy” for most people. Based on your blog post, you have very deep technical knowledge, going so far as to patch a C library to achieve your goals. That’s fantastic that you have those skills, but I’m not sure the average developers wants to dedicate that level of effort for someone as mundane as testing :)
While it certainly is possible to skip the browser, doing so introduces additional complexity that could have been avoided if a real browser was used in the first place. What I’ve found is that people who are responsible for performance testing (web developers, QA engineers, etc) often have a very hard time simulating browser session traffic, especially when that traffic has complex state embedded in the browser. This article outlines those cases, which a tool like HTTPPERF would still need complex scripting to achieve.
It becomes especially difficult, as I show in the Google Suggest scenario, when there is a need to provide random/parameterized data in to each simulated virtual user. If you aren’t running a browser (or at least a browser emulator, such as HtmlUnit), then your load test scripts will need to duplicate much of the same AJAX logic that effects state.
For those that don’t have the system resources necessary to run real browsers and aren’t interested in my service, HtmlUnit (http://htmlunit.sourceforge.net/) is a decent alternative. It doesn’t consume as many resources, so it can be run in a larger quantity on a single machine. However, it also isn’t a perfect emulator and often chokes on complex JavaScript such as Dojo and jQuery. The team is always working to improve this though, so check with them for the latest status.
Thanks again for checking out the article!
Patrick
Reply | Original | Permalink | TweetShams - Great idea! I’ll get around to it eventually, but if you beat me to it please share your modified code. Thanks!
Reply | Original | Permalink | TweetStephane - feel free to take it and do what you want with it. If you do use it or modify it, I’d love to hear about it though. Thanks!
Ignacio - Yes, that’s my feeling as well. It doesn’t provide a 100% unique identification, but it’s usually “good enough”
Reply | Original | Permalink | TweetSteven,
Great article! I’ll be sharing this with a bunch of folks. I’ve already posted to it on my new startup’s blog at http://blog.browsermob.com.I’d love to hear your thoughts on our new business. We built the world’s first load testing service that actually uses real browser instances to play back load. Our theory is that as web apps become more advanced, testing externally and using real browsers to drive load is going to become much more important.
Plus, it’s cool to be able to command 5000 Firefox instances at will :)
Patrick
Reply | Original | Permalink | TweetI find that having a good blogging tool helps blog more frequently. I use ecto for OS X, which has decent browser integration and can handle images fairly well. The other thing I do is make notes of blog ideas inside of my GTD app of choice (CulcturedCode’s Things) so that I’ve always got a list to draw from.
PS: Feel free to email me thoughts on http://browsermob.com. Thanks for checking it out!
Reply | Original | Permalink | TweetGreat interview Andy!
