Drupal Web Application Development Company

call us//
773-525-8255

blog

/Drupal and Mobile web development thoughts
from our Open Source application team

Don't Get Run Over By Traffic! Perform Tests!

Mon, 2014-02-17 16:07 -- Greg
Promet's SysAdmin, Greg Palmier reviews a case in LoadStorm. Now, you can measure how your site handles traffic in real time.  If you learn nothing else, remember: Don't just perform tests on your front page!
 

Assessing the performance of your site will typically be one of your tasks as a system administrator, project manager or developer before delivering your final product.  Establishing a goal for performance such as load time and concurrent users is common for evaluating the load a server can continuously deliver without flinching.  However, not all sites are created equal.  The overall performance will be impacted the most by the most recurring visit profile, i.e. how the site is actually used the most.  

 

Establish a Load Test Profile

Storing and serving precompiled dynamic content to serve as-fast-as-static content is a common trick for pretty much any content management system (CMS).  Apart from the CMS internal caching algorithms that include CSS and Javascript aggregation, there are external services that provide additional caching layers as well as modules developed for the CMS to generate HTML "copies" of dynamic content to serve until a change in the content is detected.  This would give the end user the impression of real time data but realistically be serving a precompiled page.  This is good in a sense that you can offload asset delivery to different local network resources or even a Content Delivery Network (CDN) ultimately.  The problem there lies in what exactly it is you wish to assess.  If you want to test the speed of the same brochure page refreshed five thousand times you are essentially only testing the server's ability to serve one cached piece of content.  This type of load test may be applicable to the profile of most traffic but if your typical visit centers around writing data like automatically generating a user or recording a generic set of data, you aren't doing yourself a service by loading the front page over and over.  That being the case, you need to scrutinize the server with load tests that assess write capabilities.
 

Understanding the Use Case

Using Loadstorm and precompiled datasets makes it possible to click through a multi-step process and fill in formed data to make each visit unique.  Remember: Not all websites are created equal.  Perhaps you have a client with an "obscene" amount of editing or content being added that bogs down the server.  Maybe the client came to you to solve this problem because slowness is blocking their editors.  In these instances, a "Front Page" load test would be of very little use to you in validating performance.  During a recent project, one of the biggest pain points from a newly onboarded client was the insufferable slowness involved in adding content.  The nature of the site involved constant content editing which made read performance reasonable to the end user, but a huge bottleneck for the large team of editors.  A key requirement for the new deployment was to provide significant improvement in the addition of new content.  We established some ways to test this using LoadStorm which would help us write some basic data by automatically logging into the site and uploading a file.  This is a simple Drupal core function, but doing so involves writing to the disk for both the image and the associated database record in addition to soaking up some bandwidth.
 

Building the Test

There isn’t a lot of complexity involved in setting up LoadStorm tests.  Most of the complexity will reside on knowledge of the steps needed to successfully navigate the CMS.  The following steps will cover only the essentials for providing authentication and form data sets.  Getting there is pretty easy otherwise.
 
First, you need to provide a way for LoadStorm to be able to make each form input unique.  The run list of the test will be executed sequentially similar to a user opening the page, logging in, and submitting the image to upload.  LoadStorm will create hundreds of these that run concurrently, so making each one unique is essential.  Create a CSV with the minimum amount of details required to make each test pass unique.  
 

Figure #1 shows two basic columns that the LoadStorm will leverage when filling out forms later.

 

Figure #1: Example CSV Data
 
Next, when building the test, instruct LoadStorm to utilize the custom data set.  In this example, I uploaded a file named 22-addfile-test.csv and the form data is now available from a dropdown menu when building the test.
 

 
Figure #2: Leverage the Custom Data Set

When creating the test steps, logging in as a user will be somewhat necessary for accomplishing the follow-up steps.  
 

Figure #3 shows how to fill in user credentials when the test surfs to the user login page.  In this example, the test will use static data, i.e.the same user login for every visit.

 

 

Figure #3: Submit Credentials
 
When saving the previous step, LoadStorm will run through all previous steps and validate the steps.  If there is a white screen in the preview window or any obvious error, there may be a firewall or htaccess pop-up blocking the remote visitor.  During the test, you may need to open up the test site to the public to allow traffic from any potential LoadStorm ip address.
 
At this point, instruct the run list to browse to the file/add/web uri for the Drupal site and save that as a step.  The next page will open with what LoadStorm sees as a form to fill out.  In this example, the test will grab a remote image from Wikipedia of a considerable size to upload for each visit.  Even though the same image will be created for every test, the form data created earlier will alias the content to a fresh new entry each pass.
 
 

 

Figure #4: Upload Form Data Set
 
Upon saving the Figure #4 form data input, LoadStorm will continue to another form-capable page in the next step that’s a little more involved.  To make these entries unique, the test will need to leverage the form data for the “filename” and “alias” input per visit.  Select the form data button which will instruct the test to make available a set of data sourcing the CSV provided earlier.  The form data to input should correspond to the column heading provided in the CSV.
 

 In Figure #5, the “NAME” and “ALIAS” are selected.

 

 

 
Figure #5: From Data Usage
 
After submitting the form data in Figure #5, a successful entry should be completed showing the addition of the content!  When completed, the “Run Steps” will show basic statistics of one complete pass of the test.

See Figure #6.

 

 

 

Figure #6: Overall Process Steps

 

The upload test should now provide a way to spam the Drupal site with some decent network traffic and provide some insight on how well authenticated users can create some basic content.
 

Notes

While this is seemingly easy, the test is really only possible due to the form data being a unique visit in each instance.  Instances where AJAX is used to provide pop-ups doesn’t work very well because LoadStorm doesn’t have the ability to input data into an already processed node submission.  Additionally, you’ll need to clear out any submitted data during the test and content added when creating the test because the end load test will start at the beginning of your form again.  Keep a snapshot of your content handy to easily refresh the site to a state you wish to begin with.  At any rate, the test works and only requires basic knowledge of some Drupal features.
 

Conclusion

Throwing some visits over the fence at the main page isn’t indicative of how the site will perform when in the wild.  Understand the customizations and integrations that could be heavier hitting on performance and how they will be used. Have a conversation with the client to identify pain points currently dragging down any current workflow.  Creating a test that best encapsulates the most often used sections of a website will ensure resiliency when fully deployed.
 
Google AdWords: 

Comments

That is an exceptional blog post. I love your opening premise:

"Don't just perform tests on your front page!"

To elaborate, the performance of your site should certainly be tuned to optimize the highest priority business processes. Speed = Money. That's a truth in any web-related business. But when performance testing, you should carry that to the details of how your site makes money. I bet your front page isn't where your site makes money. It's probably in a purchase or subscription or simply collecting registrations. If you aren't testing the money transactions, you are missing the primary objective.

Greg, I appreciate your insight into testing a Login Form. The performance burden of credentialing is usually 10-100x that of hitting a home page with static resources. If a developer only load tests the front page, then that developer may think they have a high performance system, when in reality, they are probably going to be greatly disappointed when the first successful marketing campaign (or Reddit link) crashes their site.

As for AJAX, we have a new version of LoadStorm that is in private beta. We welcome the opportunity to show it to you and let you participate. It overcomes the limitations around AJAX and Javascript processing. It also includes much better reporting that allows filtering to see results by script, server, pages, error code, HTML vs. images, etc. We would love for you to try it out. Please contact me at your convenience. We expect to fully release it in March 2014.

Thanks again.

Submitted by Pharmb160 (not verified) on

Wonderful work! This is the kind of info that are meant to be shared across the internet. Disgrace on the search engines for not positioning this post higher! Come on over and consult with my website . Thank you kdckgbd

Submitted by Pharme328 (not verified) on

I think this is a real great blog post.Much thanks again. daabdef

Submitted by Pharmd826 (not verified) on

You must indulge in a contest for among the greatest blogs over the internet. Ill suggest this web website! caedeee

Add new comment