Blitz and Red Hat’s OpenShift PaaS Partnership Program

Posted on by
Reply

Here at Blitz, we’re all about making things simple, effective and fun. It’s the driving force behind our load-testing tool, and it’s why we have the partners we have.

That’s why we’re excited about our involvement with Red Hat’s OpenShift PaaS Partnership Program, as they announce the extension of their program to include more partners in line with the launch of their on-premise OpenShift Enterprise PaaS product.

What does this unique partnership mean?

It means that existing OpenShift users can deploy Blitz onto their platforms for capacity planning, scalability, optimization, and performance monitoring. Blitz brings a simple form to OpenShift users so they can better understand the performance of their apps. At any given time, OpenShift users can see what they need to do to optimize their website or application.

And on the other side, this partnership enables users who don’t yet have an application or website to load test can see how the end-user experience of Blitz works. Essentially, we’ve worked with OpenShift to pre-deploy a number of different apps so that people can run load tests against them, even though those apps don’t belong to them. They get to experience Blitz firsthand without needing a product to test.

So, as you can see, this partnership makes a lot of sense, for both Red Hat’s users and for those who want to use Blitz. We’re happy that we can continue to provide a well-rounded, complete user experience for our partners’ users and especially for our own Blitz community.

Announcing Blitz Credits

Posted on by
14

Blitz creditsLast week, we talked a lot about the freemium model and the lessons we have learned so far in our journey with Blitz. We also asked that you stay tuned for some exciting news that demonstrates our biggest lesson learned: be flexible.

The number-one complaint we’ve heard from our customers doesn’t have to do with the Blitz product, but rather, our pricing model. Currently, users purchase by the hour to test on our system – this is what we call a wall-clock system – basically, time starts at the swipe of a credit card. Our users found this inconvenient for two reasons: 1.) they have to be ready to start testing as soon as they swipe their credit cards, and 2.) they were never able to test for the full hour. If a mistake occurred at the beginning of the test, they would spend their time fixing it, instead of running the test.

Our customers started asking to pay by testing runs instead of by time. And since everything in testing is project-based – whether new code is being introduced for validation or a website or app is launched – we realized that it made sense to change.

So we’re introducing our new pricing model – a Credit-based system that will allow users to pay by test, which allows customers to fine-tune their apps in-between tests without worrying about the clock.  Moving from a wall-clock system to a credit-based system gives the customer control of when they want to do their testing, and ensures they get what they pay for.

Here is how our credit based system works:

  • Pay by using credits; each credit is a 1-minute, 1,000-concurrent-user rush for $1
  • Customers can take advantage of bundles that offer savings for buying in bulk
  • Credits don’t expire and are available whenever customers are ready to start testing
  • Existing customer accounts will be converted to this Credit-based plan automagically.

You can also earn FREE credits:

  • Blogging earns 100 credits!
  • Referring friends earns you a credit per friend—are you popular?
  • Monthly top-up; everyone starts the month with at least 10 credits!

We think that this pricing model will take performance testing to the next level. Starting October 30th, everyone will be able to do their load testing based on this new plan. We’re so excited to help our customers reach their goals and succeed with Blitz. As always, we’re here for your questions, concerns, and comments. Email us at hello@blitz.io, or give us a shout on Twitter @Blitz_io.

Blitz and New Relic Connect

Posted on by
Reply

Today, our partner New Relic announced the launch of their New Relic Connect program. New Relic provides a SaaS-based application and performance management tool for Ruby, PHP, Python, Java and .NET web apps.

Connect is designed to make our users’ – and admittedly, our own – lives a little bit easier by letting partners with complimentary technologies easily combine New Relic’s rich app performance data with their own solutions and services. It’s the latest addition to New Relic’s world-class partner program and we’re excited to be among the first to participate.

We’ve actually been integrated with New Relic for a little while, but today we’re making the partnership official.  The integration is now fully supported and, as an additional perk, all Blitz customers who sign up with New Relic will get New Relic Standard free of charge!

So how exactly does it work? New Relic gives you insight how your Web application is performing in real time.  When you integrate your Blitz account with New Relic, you can monitor the performance of your application from the inside while running a load test.  New Relic metrics will be displayed along with the Blitz results so that it’s easier to find and fix your performance bottlenecks.

So, it’s easy to see how complimentary this partnership is. And coupled with some exciting news we’ll be announcing on Monday that will make Blitz easier and more effective for our customers, this integration doesn’t just make perfect sense, but it also is extremely beneficial for both Blitz and New Relic users.

Connect New Relic to your Blitz account your Plugins page.

Sign up for a New Relic account and get New Relic Standard free of charge!

Lessons Learned and Looking Forward with Blitz

Posted on by
Reply

We’ve been in business for a year now, and what a year it’s been. We’ve grown and learned so much in such a short amount of time that we felt it wouldn’t be fair if we kept all of those lessons to ourselves. So we wanted to pass on to anyone who is trying to start their own freemium-based products some of the things we’ve learned along the way.

When we first started, we launched Blitz as a load tester in the cloud. Blitz takes advantage of two major trends that are becoming more and more prevalent within software and application development. The first has to do with “where” – development is moving into the cloud. This means that the “where” development done is no longer tied to specific organizations or dedicated environments, opening access to software development in an unprecedented way.

The second trend in development deals with the “who.” The actual process of development testing is now done by a new group of folks: the developers. What used to be two disciplines – performance engineering and development – the ever-so-popular DevOps movement means that things are blending together, and developers are now the ones doing the testing. Which also means they are on the hunt for tools that help them do this.

Another key aspect of Blitz, outside of the unique needs that it addresses for developers, is the “freemium” model on which it is based, which allows developers to apply load tests, otherwise known as “rushes,” of up to 250 concurrent users for up to 1 minute, for an unlimited number of times, at this level.

Keep in mind that freemium isn’t for everyone. You really have to run the math and see if it makes sense for your application. But, if you’ve done your homework and you think that the freemium-model path makes sense for you, we at Blitz have done some of the dirty work for you, and here are some of the things we’ve discovered throughout the process:

Invest in the Correct Outreach Efforts

You may have noticed, we don’t do traditional media and marketing. Our users know our product, and they don’t need a cheerleader bombarding them about how great it is. What we do care about is our community, and what they are telling us – not vice versa. That’s why we’ve put our resources into social media, which allows us to actually communicate with our customers rather than communicate at them.

Think Big Picture

In a world of instant results – when a delay of two seconds puts you behind 90 percent of the population – it’s naturally difficult to think longer-term, especially at the expense of short-term.  And of course, it’s great to give all customers everything they want, but that’s pretty close to impossible. For example, we’ve had users who have requested that we add some functions like specific encryption or testing that would pull us away from making things simple and easy for them to use, and we’ve unfortunately had to say no. Again, it’s not like it gives us pleasure to not be able to offer our customers the moon and beyond, it’s just not realistic, and in the end, these requests didn’t serve our Blitz community as a whole. You have to always look at the bigger picture and trends – it’s not about short-term gains, but about long-term success.

Referral Programs are Your Best Friends

We implemented some successful referral programs in our pricing model, which ties back to our first lesson about investing in appropriate outreach methods. The best thing about social media is that it gets the word out about your product, and encourages customers to talk about you, which leads us to creating referral plans that have kept our existing users happy, while simultaneously helping us grow that user base. For instance, if you refer a friend to Blitz, we give you an extra 25 users for life for your rushing.  In this way, our customers are spreading our name through word-of-mouth, and we get to help those loyal customers keep testing at an even higher level.

Be There

It seems like this point would be obvious, but seriously, having a product or solution that is based on do-it-yourself doesn’t mean that your customers should do it by themselves….and only by themselves. The DIY method means that you as a provider need to have good documentation, good content, and at the end of the day, have a live person available to help your customers when they need it. Be responsive and let them know that you’re there when they have questions, concerns, or need your assistance.

KISS

No, we’re not suggesting you do anything inappropriate to garner customers for your freemium model. We’re suggesting the principle: Keep It Simple, Stupid. We want to urge you to have a clear problem for which your product or service is answering a clear solution – and stick to it.  It’s this simplicity on which we’ve based our product – and our freemium model – that has given us our success.

So given all of those hard-learned lessons we’ve discovered over the past year, we have to say we’re pretty happy with where we are now. Blitz has emulated 1 billion virtual users from the cloud, generated from seven different locations around the world. We’ve run 250,000 load tests for almost 30,000 users, testing 17,000 domains. Our convergence from freemium users to paid users is around 2 percent. Not bad for one year!

But despite those lessons, and despite our successes, the most important lesson we learned through our journey so far is to be agile and responsive. And it’s this lesson that we’ve applied to the top feedback we’ve gathered from our users.

So looking forward, we’ve got some great new changes down the pipeline that are going to help our users become more efficient and more productive. We’re really excited to announce these changes – stay tuned for more!

How we use DynamoDB and what we’ve learned along the way

Posted on by
Reply

amazon datacenter

Recently, we started to store a history of tests run by our Blitz users.  With over a billion hits so far, we’re growing quickly and we want to make sure that we’ll continue to be able to keep up with our users’ needs.  Eventually we chose Amazon’s DynamoDB as our storage database for this task because it frees us from tedious operations like backups, monitoring, and increasing capacity.  You can see how it performs by going to our http://blitz.io/dashboard.  After running a few tests, they will all show up in the dashboard for later reference.  We had a few surprises as we were bringing up our DynamoDB, so we thought we’d share some lessons learned. Read on…

Lesson 1. DynamoDB doesn’t have JSON types

DynamoDB is a schema-less NoSQL database, with the data stored as key-value pairs. Unlike other NoSQL databases that we’ve used, such as MongoDB or CouchDB, which operate on JSON documents, DynamoDB only accepts Number, String or Binary values.  So, to store Hash or Array objects, we must handle serialization ourselves.  We convert these to JSON strings before sending to DynamoDB, then back from JSON on each fetch. This works well, but you need to be careful not to exceed the 64KB maximum item size.

Lesson 2. Break up your data into fine-grained attributes

To use DynamoDB we need to provision read/write throughput capacity.  Amazon charges a flat hourly rate based on the capacity we reserve.  To keep costs low, we want to retrieve only the data we need on each query.

The configuration and results of our tests have a good amount of data.  And in the dashboard page, we only show the most relevant information, such as response time, error rate and hit rate.  So, before we send the data to store in DynamoDB, we extract or compute those fields and put them into their own key-value pairs.  Then when we query for data to show in the dashboard page, we just need to specify those keys to retrieve, which greatly reduces the overall size of each query result.

Lesson 3. Use the :select option to boost query performance

We designed our tests table with a composite primary key.  The :hash_value is a UUID of the user, the :range_value is the timestamp which is the start time of the test.  For such tables in DynamoDB, we can use the query API to query the test items.  It takes in some options and the :hash_value is required.  We can either specify different kinds of range values or use the :limit option to filter down the tests we want to retrieve from the table.

In our initial implementation, we were doing something like the following:

tests = items.query(
  :hash_value => "user UUID",
  :scan_index_forward => false,
  :limit => 20)

tests.each do |test|
  render test.attributes.to_h
end

This will retrieve the last 20 tests run by a user.  It provides the desired results, but it was unexpectedly slow… really slow.  By default, the aws-sdk gem will lazy-load the attributes for each item.  So, calling `test.attributes.to_h` will trigger a new request to DynamoDB to retrieve each attribute.  In total, it will make 1 + 20 * num_attributes requests to retrieve our data!

To remedy this, we want to use the :select option to specify the attributes we want, such as:

 
tests = items.query(
  :hash_value => "user UUID",
  :scan_index_forward => false,
  :limit => 20,
  :select => ['timestamp', 'url', 'response_time'])

tests.each do |test|
  render test.attributes['timestamp'], 
    test.attributes['url'], 
    test.attributes['response_time']
end

Now, this query will retrieve all the attributes we want in a single request.  With this change, we saw the total load time reduce from as much as 7 seconds to a few hundred milliseconds.

We’ve been running happily with thousands of test results in our DynamoDB for some time now.  We’ll be sure to share any additional lessons to be learned as our datastore grows into the millions, so stay tuned…

Happy Blitzing!

Try Blitz : A whole new meaning to the word TRY

Posted on by
Reply
TRY BLITZ - A winning demo

Do or Do Not – There is no try

Why is it when trying something you always have to give something in return (your email, a phone number, go through annoying sales guy, and more importantly your time? What turned trying into such a effort? What ever happened to “Do or Do Not, There is No Try” – Yoda.

So today, we are revisiting the meaning of try by giving EVERYONE the chance to do and experience Blitz for themselves. Don’t have an App to test? Not deployed in the Cloud? Not a current Blitz user? It doesn’t matter, we have you covered!

Working with the OpenShift team, we have created a bunch of sample apps in the cloud for you to load test to experience first hand what our customers have been enjoying since we set out to make performance testing a fun sport.

Check it out for yourself –> http://blitz.io/try 

 

Blitz is now available on the Engine Yard Add-On Program

Posted on by
Reply

Engine YardIf you’re looking for a fun way to spend the afternoon, web load testing is probably not the first thing that comes to mind. In fact, it can be a pain. Although there are some free plans out there, testing can be expensive and a commitment, in more ways than one.

For Engine Yard users, all of that changes starting right now.

We’re proud to announce that Blitz is part of the Engine Yard Add-on program, providing a great way for development and operations teams to take the sting out of web and API load testing.

The Blitz add-on for Engine Yard allows you to perform unlimited unit tests (sprints) and unlimited 1-minute load tests (rushes) with up to 250 concurrent users.

You heard right – 250 concurrent users.  Did I mention it’s free for all Engine Yard accounts?

So, no more excuses for not load testing your app. And who knows, it might actually be fun.

Blitz.io: Making load and performance testing a fun sport!

If you’re already an Engine Yard customer, then it’s easy to get started:

  1. Go to https://cloud.engineyard.com/addons (login required) or navigate to “Add-ons” in Engine Yard Cloud and click on Blitz.io
  2. Sign up and click the “Blitz Dashboard” link to begin rushing

Webinar: Tools to improve the performance of your Drupal site

Posted on by
Reply

If you missed it, we got links to the webinar recording (Thanks to Acquia TV!).

Come listen to our very own Mike Smith and a panel of partners talk about way to improve the performance of your Drupal site.

Tools featured in this webinar are :

  • Acquia Insight for code and configuration testing
  • Blitz.io for load testing
  • New Relic for application performance analytics
Questions?

 

Introducing the Blitz plugin for Jenkins

Posted on by
Reply

We’re big fans of automation, and we know you are too.  Today, we’re introducing the Blitz plugin for Jenkins.

The plugin makes it easy to integrate load testing into your existing continuous integration jobs.  You can run sprints or rushes from Blitz without any custom scripting or code.  Jenkins will capture the results so that you can track your performance trends over time.

Here’s a quick overview.  For a full walkthrough, check out the screencast.

Installation

The “Blitz.io Plugin” is hosted on the Jenkins community repository.  You can install it directly from your Jenkins plugin manager.

blitz.io plugin installed

Enter your Blitz account information

On your Jenkins configuration page, find the Blitz.io section and enter your Blitz User ID & API key (found on the Blitz settings page).

Run Sprint or Rush tests

Blitz Sprints or Rushes are configured as Post-build actions for each Job.

configure the blitz.io tests

Note: Blitz runs from the cloud, so your URL needs to be reachable from the public web.

Interpret Test Results

When the Job completes, it will have a Blitz.io link on the left hand side.

blitz.io build finishes

Here, you can see the Sprint results

blitz.io sprint result
and/or the Rush results.

blitz.io rush result

 

 

Blitz Plugin for Jenkins:

Happy Blitzing!

Introducing the Blitz API for PHP

Posted on by
Reply

Here it is, the code that will enable you to integrate Blitz directly into your PHP app. It’s a simple library available on Github.  Download it and get started with a simple php require. All you need is a free Blitz account and your API key.

Here’s a gist to get you started on Blitz.io, PHP style: