What is it?
Rtmpload — a load testing tool for RTMP servers About. The tool spawns threads (with scriptable flow) which consume RTMP streams, parse them (only containers), collect some metrics (e.g. Frames per second) and sumbit metrics to Mistress stat server for aggregation, storage and visualization. Rough performance numbers for 10k threads on one host (Xeon E5-2640, Debian 6 3.2.0-0.bpo): 20-50%. Load testing tools (also called performance testing tools—one in the same depending upon which side of tech you’re coming from) are an important part of ensuring that your website, web application, API or other web device is able to handle the load of users and function properly under that load. StreamTest load testing supports RTMP. For further details, see our full FAQ. What do I need to start my load test? To start a LoadTest you will need to provide a compatible streaming link, the number of test users required, the bitrate of your stream, and the locations to test from. For more information, see the full FAQ. Will the load testing. LoadRunner is a software testing tool from Micro Focus.It is used to test applications, measuring system behaviour and performance under load.LoadRunner can simulate thousands of users concurrently using application software, recording and later analyzing the performance of key components of the application. RTMP Load Testing Tools Resources All the load testing tools that can help a website or intranet application developer to ensure good performance and no crash under stress.
StreamTest's Load Testing service is 'not your grandaddy's' load testing service. It is not the standard generic web 'objects' or html based server load testing service.
The service provides the online broadcaster, the ability to simulate massive streaming audiences on their distribution network in seconds. Without complicated user clients, server applications or expensive set up procedures. Using the latest in cloud and streaming technologies StreamTest.net is able to do load tests big and small, simulating audiences from an individual user to millions of users. Equally important, Streamtest's load testing service also samples and measures the stream under load to determine the quality of experience. After the load test has been completed every user experience that connected can be reviewed for problems if they score low on the test.
How does it work?
Streamtest.net's has custom designed and built a highly powerful, streaming specific Content Delivery Network (CDN) distribution network load testing service. This service uses StreamTest's customized virtual streaming users (VSU's). Through Streamtest.net service portal the online broadcaster selects a couple test options which activates the VSU's to simultaneously connect to the CDN distributed stream. This instant connection creates the load test. However the VSU's also sample and measure the streaming channel. Which are then collated to render results that can be viewed on the Streamtest Monitoring Dashboard. All in real time. Plus each VSU connection and measurement can be drill down into a detailed, active, real time, running graph of the streaming channel under test and after. The data results obtained by the VSU's are displayed in a simple, easy to read and understand format.
Benefits
- Streaming specific - not your grandaddy's load test. Streaming specific, built by streaming tech's for streaming techs
- Huge user scale user simulations 1,000,000+
- Anywhere in the world Load Test where your audience is (see geo locations)
- More than just a load test. Get all the other important streaming test measurements you need and want to provide the best streaming service possible.
- Scheduling option - test when you want not when you have to
- Test as many times as you want
- Easy to read graphical performance report 4 Easy to share performance report - test published to unique URL - easy to share. Better than large excel
- Get StreamTest Certified - receive a load test certificate confirming test results so you can prove and share that your set up is ready for your audience
- Compatible with almost every streaming format (see below)
Features
- Real Time Test Monitoring (see video – coming soon)
- API access to launch tests and receive test result data (see API docs here)
- Generates thumbnails from each simulated stream
- Each test user has a unique identity that you can follow from our result data through your own log files.
- Drill down performance metrics for each simulated user
- User data geographically present
- Client data collection
- Scheduled load test
- Easy to read graphical performance report
- Aggregate and individual metrics for latency, throughput, bytes received
- Waypoint data taken at 10 second intervals
Compatibility
Our StreamTest simulated client is compatible with many different stream types, both live and on-demand. Below we’ve listed the known compatible/ supported stream types and formats.
What Is Rtmp
Media Formats:
.mp3, .m4v, .mov, .wmv, .wma, .ra, .rmv, .aif, .aifc, .aiff, .m4a, m3u, .3gp, .flv, .m1v, .avi, .mpg, .mp4, .ogg, .m2v, .mpeg, .mp2, .m3u8, .f4m
.mp3, .m4v, .mov, .wmv, .wma, .ra, .rmv, .aif, .aifc, .aiff, .m4a, m3u, .3gp, .flv, .m1v, .avi, .mpg, .mp4, .ogg, .m2v, .mpeg, .mp2, .m3u8, .f4m
Servers:
Wowza Media Server, Adobe Flash Media, Evostream Media Server, NGINX-RTMP, IIS, apache…
Wowza Media Server, Adobe Flash Media, Evostream Media Server, NGINX-RTMP, IIS, apache…
Stream Providers:
Akamai, limelight, CloudFront, Azure BLOB, Amazon s3, (most standard http delivery services should work)
Akamai, limelight, CloudFront, Azure BLOB, Amazon s3, (most standard http delivery services should work)
Delivery Types:
Live, On-Demand, Cached, HLS (HTTP Live Streaming), Progressive, FTRT (Faster than real-time)
Live, On-Demand, Cached, HLS (HTTP Live Streaming), Progressive, FTRT (Faster than real-time)
Delivery Protocols:
RTMP, HTTP, HTTPS
RTMP, HTTP, HTTPS
Maximum Bit-rates:
tested between 10k and 10mbps
tested between 10k and 10mbps
Maximum Resolution:
tested between 128X128 and full 1080p at 1920X1080
tested between 128X128 and full 1080p at 1920X1080
FAQs
What are the load test supported formats?
StreamTest load testing supports RTMP. For further details, see our full FAQ.What do I need to start my load test?
To start a LoadTest you will need to provide a compatible streaming link, the number of test users required, the bitrate of your stream, and the locations to test from. For more information, see the full FAQ.![Rtmp Rtmp](/uploads/1/1/3/6/113618411/670246302.png)
Will the load testing service work on secure streams?
LoadTest can be used with streams that are secured with a Token. See the Full FAQ for further detail.© StreamTest.netContact Us | About Us | Privacy Policy | Terms of Use
There are two types of test I’ll describe below. First of all using Apple HLS streams, which is HTTP Live Streaming via port 80, supported by iOS and Safari, and also by Android (apps and browser). Then we have Adobe’s RTMP over port 1935, mostly used by Flash players on desktop, this covers browsers like Internet Explorer and Chrome on desktop. These tests apply to Wowza server but I think it’ll also cover Adobe Media Server.
All links to files and software mentioned are duplicated at the end of this post.
It’s worth noting that you can stick to HLS entirely by using an HLS plugin for Flash video players such as this one, and that is what we’re doing in order to make good use of Amazon’s CloudFront CDN.
For the purpose of testing you may also wish to simulate some live camera streams from static video files, see further down this post for info on how to do that on your computer, server or EC2.
Testing RTMP Live Streaming with Flazr
In this test we want to load test a Wowza origin server itself to see the direct effect of a lot of users on CPU load and RAM usage. This test is performed with Flazr, via RTMP on port 1935.
Assuming you’ve set up your Wowza or Adobe Media server already, for example by using a pre-built Wowza Amazon EC2 AMI. We’re using an m3.xlarge instance for this test as it has high network availabilty and a tonne of RAM – and we’re streaming 4 unique 720p ~4Mbit streams to it, transcoded to multiple SD and HD outputs (CPU use from this alone is up to 80%).
Installing flazr
First up, for the instance size and test configuration I’m using I modified flazr’s client.sh to increase the Java heap size to 8GB, otherwise you run out of RAM. Next up FTP and upload (or wget) flazr to a directory on your server/EC2 instance. Then SSH in and:
The order of parameters does seem to matter in later versions of flazr, but either way this test runs for 60 seconds, with a load of 1000 viewers. Given all the transcoding our CPU was already feeling the pain, but there was no sign of trouble. We managed 4500 before anything started to stutter in our test player from another m3.xlarge instance.
Of course this only matters if you are not using a CDN, but it’s good to know this EC2 instance can handle a lot of HD viewers.
![Tool Tool](/uploads/1/1/3/6/113618411/611674117.png)
Testing HLS Live Streaming (or a CDN such as Amazon CloudFront) with hlsprobe
Onto HLS streaming, the standard for mobile apps and sites. We have used Wowza CloudFront Formations to set up HLS caching for content delivery, so that we can handle a very large number of viewers without impacting on the CPU load or network throughput of the origin server, and to giver us greater redundancy. Given CloudFront works with HLS streams we are not using RTMP for this test, so we cannot use Flazr again. To test HLS consumption –that being the continuous download of .m3u8 files and their linked .ts video chunks– we can use a tool called hlsprobe, which written in python.
Rtmp Load Testing Tool Kit
If you’re on a Mac and don’t have python I recommend you install it via brew to get up and running quickly. If you don’t have brew, get it here.
hlsprobe also relies on there being an SMTP server running, not that you need a fully functional one but:
Then to install hlsprobe’s dependencies and hlsprobe itself:
A sample config is linked at the end of the post.
Running hls probe is as simple as this (note the -v verbose mode, you can turn that off once you have it working).
Now if you fire up the Wowza Engine Manager admin interface, you should still see the connection count and network traffic, but the traffic. If you’re testing your CDN such as with CloudFront, you should note that your CPU usage does not increase substantially as you add thousands of clients.
Simulating cameras to Wowza via nodeJS
It’s good to be able to simulate live streams at any time, either from your computer or in my case, from some EC2 instances. To do this I’ve written a simple nodejs script which loops a video, optionally transcoding as you go. I recommend against that due to high CPU use and therefore frame-loss; in my sample script I am passing through video and audio directly, the video is already using the correct codecs, frame size and bitrate via Handbrake.
Rtmp Download
The script runs ffmpeg, so you’ll need to install that first:
Edit the js script to point to your server, port, and video file, the run the script with:
If the video completes, it’ll restart the stream but there will be a second of downtime, some video players automatically retry, but make sure your video is long enough for the test to be safe.
Download Rtmp Stream
These are just a couple of ways of load testing a live streaming server, there are 3rd parties out there but we’ve not had great success so far, and this way you have a lot more control over the test environments.
Rtmp Load Testing Tool Harbor Freight
Links
Rtmp Viewer
fakestream.js – NodeJS script to simulate live streams
config.yaml – Sample config for hlsprobe
hlsprobe – Tool for testing HLS streams
Flazr – Tool for testing RTMP streams
OSMF-HLS – OSMF HLS Plugin to support HLS in Flash video players
config.yaml – Sample config for hlsprobe
hlsprobe – Tool for testing HLS streams
Flazr – Tool for testing RTMP streams
OSMF-HLS – OSMF HLS Plugin to support HLS in Flash video players