MANUAL STEPS - publish JMeter results to AWS CloudWatch and get ready for performance test automation.

This is a child post from this article that describes how to export JMeter test results into CloudWatch. The parent blog post uses a CloudFormation template to setup JMeter, the CloudWatch Logs agent and metric filters, but here I am showing the manual steps. I will assume you know how to launch an EC2 instance, you are familiar with EC2 instance profiles, you know how to SSH to your instance, you have the AWS CLI installed and you have some basic Linux command knowledge.

JMeter

In this section we will install JMeter, install the Flexible File Writer plugin, configure a simple JMeter test plan and run it.

1. Create IAM Role

The first thing you need to do is launch an EC2 instance and assign to it an EC2 Instance Profile. An EC2 Instance Profile is a way to assign permissions to an EC2 instance, so it can execute operations defined in an IAM Role. First you create an IAM role, then link the IAM role to the EC2 instance via the EC2 Instance Profile. The CloudWatch Logs agent needs the EC2 Instance Profile in order to execute CloudWatch Logs APIs.

Create a file (for example, jmeter-trust-policy.json ) with the following content:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": "ec2.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Run the AWS IAM CLI command create-role

aws iam create-role --role-name jmeter-cw-logs --assume-role-policy-document file://jmeter-trust.policy.json

Now we only need to attach a policy to this IAM Role. I chose the Managed Policy CloudWatchLogsFullAccess:

aws iam attach-role-policy --role-name jmeter-cw-logs --policy-arn arn:aws:iam::aws:policy/CloudWatchLogsFullAccess

Next, we’ll create an EC2 Instance Profile:

aws iam create-instance-profile --instance-profile-name jmeter-cw-logs

and attach the instance profile to the IAM Role we created earlier:

aws iam add-role-to-instance-profile --instance-profile-name jmeter-cw-logs --role-name jmeter-cw-logs

2. Launch EC2 instance

I have published an EC2 AMI in the US East region: ami-af7359c5 . This AMI is based on Amazon Linux and has JMeter installed as well as the Flexible File Writer plugin, which we’ll need for this example. Feel free to use this AMI, this way you won’t have to install JMeter.

When you launch the EC2 instance, consider the following:

  • You can use any instance type, but for this exercise a t2.nano or t2.micro would work (I actually did the whole exercise in a t2.nano).
  • AMI is based on Amazon Linux (or use ami-af7359c5)
  • Security Group allows incoming traffic on port 22 (SSH) and outbound traffic at least on port 80 (HTTP)
  • The Instance Profile created earlier shows up in the “IAM Role” drop-down in “Step-3 Configure Instance Details”. Select it. This is required by the CloudWatch Logs agent, so it has permissions to write data into CloudWatch Logs.
  • You have an existing EC2 key pair for this instance (or create a new one). You’ll need it for SSHing to the instance and launching JMeter tests.

3. Install JMeter

If you want to install JMeter manually, the first thing you need to do is download and unzip the JMeter installation file into the directory you want to run JMeter from.

You can run the command directly from your EC2 instance, for example:

wget http://www.eu.apache.org/dist/jmeter/binaries/apache-jmeter-2.13.zip

and unzip the file:

unzip apache-jmeter-2.13.zip -d <your destination folder>

Let’s not forget to give execute permissions to the current user:

chmod u+x <jmeter installation root>/bin/jmeter
chmod u+x <jmeter installation root>/bin/shutdown.sh

4. Install Flexible File Writer

The Flexible File Writer plugin allows you to write test results in any format of your choice. This is important because CloudWatch Logs only parses log records with a certain format (we’ll get to that later). JMeter alone does not write log entries in the format CW Logs expects, therefore the need to install the Flexible File Writer plugin.

Go to the JMeter plugins page, find the location of the installation file and download it. For example:

wget http://jmeter-plugins.org/downloads/file/JMeterPlugins-Standard-1.3.1.zip

Make sure you unzip the file into the root location of your JMeter installation:

unzip JMeterPlugins-Standard-1.3.1.zip -d {root location of your Jmeter installation}

To run a sample JMeter test plan you can follow the steps in this section of the parent blog post.

5. Configure the CloudWatch Logs agent

The next step is to install the CloudWatch Logs agent. You can find more details in the CloudWatch Logs documentation.

The steps can be summarized as follows:

  • Update your EC2 Amazon Linux instance sudo yum update -y

  • Install the AWS Logs package sudo yum install -y awslogs

  • Edit /etc/awslogs/awslogs.conf For this section it is important to understand the difference between Log Groups (high level logical containers of log streams, such as an application name) and Log Streams (log sources, typically specific servers that generate logs)

[jmeter-results]
datetime_format = %Y/%m/%d %H:%M:%S.%f %Z
file = <location and name of your JMeter results log file (make sure the path exists), for example /home/ec2-user/jmeter/test-results/results.log>
buffer_duration = 5000
log_stream_name = jmeter-server
initial_position = start_of_file
log_group_name = <the name of your log group, i.e. 'jmeter'>

Note that datetime_format = %Y/%m/%d %H:%M:%S.%f %Z must follow the same format as configured in JMeter’s Flexible File Writer, otherwise CloudWatch Logs will not be able to parse the timestamp in the log entries.

Also note that you don’t have to configure credentials anywhere in the instance. This is thanks to the EC2 Instance Profile, which is linked to an IAM Role that defines the IAM permissions assigned to this instance. This is a much better option than configuring credentials in a text file inside the instance.

  • Restart the CW Logs agent sudo service awslogs restart

You can go back to the parent blog post and validate that JMeter test results are published in CloudWatch Logs

6. Create CloudWatch metric filters

Here are the AWS CLI commands to create the metric filters:

Requests

aws logs put-metric-filter --log-group-name jmeter --filter-name Requests --filter-pattern '[]' --metric-transformations metricName=AllRequests,metricNamespace=jmeter,metricValue=1

HTTP 200 Responses

aws logs put-metric-filter --log-group-name jmeter --filter-name HTTP200Responses --filter-pattern '[...,responseCode=200,responseMessage,isSuccsessful]' --metric-transformations metricName=AllHTTP_200,metricNamespace=jmeter,metricValue=1

HTTP 400 Responses

aws logs put-metric-filter --log-group-name jmeter --filter-name HTTP400Responses --filter-pattern '[...,responseCode=400,responseMessage,isSuccsessful]' --metric-transformations metricName=AllHTTP_400,metricNamespace=jmeter,metricValue=1

HTTP 500 Responses

aws logs put-metric-filter --log-group-name jmeter --filter-name HTTP500Responses --filter-pattern '[...,responseCode=500,responseMessage,isSuccsessful]' --metric-transformations metricName=AllHTTP_500,metricNamespace=jmeter,metricValue=1

Response Time

aws logs put-metric-filter --log-group-name jmeter --filter-name ResponseTimeMs --filter-pattern '[timeStamp,sampleLabel,threadName,responseTime,...]' --metric-transformations metricName=AllResponseTime_ms,metricNamespace=jmeter,metricValue='$responseTime'

Connect Time

aws logs put-metric-filter --log-group-name jmeter --filter-name ConnectTimeMs --filter-pattern '[timeStamp,sampleLabel,threadName,responseTime,connectTime,...]' --metric-transformations metricName=AllConnectTime_ms,metricNamespace=jmeter,metricValue='$connectTime'

Latency

aws logs put-metric-filter --log-group-name jmeter --filter-name LatencyMs --filter-pattern '[timeStamp,sampleLabel,threadName,responseTime,connectTime,latency,...]' --metric-transformations metricName=AllLatency_ms,metricNamespace=jmeter,metricValue='$latency'

Sent Bytes

aws logs put-metric-filter --log-group-name jmeter --filter-name SentBytes --filter-pattern '[timeStamp,sampleLabel,threadName,responseTime,connectTime,latency,sentBytes,...]' --metric-transformations metricName=AllSentBytes,metricNamespace=jmeter,metricValue='$sentBytes'

Received Bytes

aws logs put-metric-filter --log-group-name jmeter --filter-name ReceivedBytes --filter-pattern '[timeStamp,sampleLabel,threadName,responseTime,connectTime,latency,sentBytes,receivedBytes,...]' --metric-transformations metricName=AllReceivedBytes,metricNamespace=jmeter,metricValue='$receivedBytes'

This CloudFormation template automates all the manual steps I just described in this page:

Launch Stack

Now you can go back to the parent post and take a look at some test results. Have fun!

Ernesto Marquez

ErnestoMarquezProfilePic

I am the Project Director at Concurrency Labs Ltd, ex-Amazon (AWS), Certified AWS Solutions Architect and I want to help you run AWS optimally, so your applications reliably generate revenue for your business.

Running an optimal AWS infrastructure is complicated - that's why I follow a methodology that makes it simpler to run applications that will support your business growth.

Do you want to learn more? Do you have other questions related to AWS? Click on the button below to schedule a free 30-minute consultation.

Do you have any comments or questions about this post, or my services?