Menu ▾ ▴

Home

christian liesch

INTRO

httest is a script based tool for testing and benchmarking web applications, web servers, proxy servers and web browsers. httest can emulate clients and servers in the same test script, very useful for testing proxys.

GETTING STARTED


To get a quick start here a simple example. Copy past the following script into a file "simple.htt" 

CLIENT
_REQ www.google.ch 80
__GET / HTTP/1.1
__Host: www.google.ch
__
_EXPECT headers "200 OK"
_WAIT
END

and run httest with this script with the following command:

httest simple.htt

The test script gets googles root and test if we get a 200 OK. If not the script will fail else it will terminate with a OK and exit code 0.

Useful

To list all httest script commands run

httest -L

To get help for a specific script command run for example

httest -C_REQ

EXAMPLES

Automatic cookie handling. But do not handle path or stuff like that, it just sends a received cookie on a given connection. If need more features do it with _MATCH or ask for new feature ;) 

CLIENT
### _AUTO_COOKIE on 
_REQ foo.bar.com 80 
__GET / HTTP/1.1 
__Host: foo.bar.com 
__Cookie: ### AUTO 
__ 
_WAIT 

__GET /bla HTTP/1.1 
__Host: foo.bar.com 
__Cookie: ### AUTO 
__ 
_WAIT
END

The header Cookie: AUTO will be filled with a cookie if there is any, else this header will be skipped automatically. Safest way is to set the Cooke: AUTO for every request.

Bandwidth test

Restrict the bandwidth while httest is sending.

CLIENT
_BPS 100 20
_REQ localhost 8080
__POST / HTTP/1.1
__Host: $YOUR_HOST
__User-Agent: mozilla
__Content-Length: AUTO
__
__.............................................................................
_WAIT
_CLOSE
_END BPS
_EXIT OK
END

Explanation:

This test will send the HTTP request with 100 byte/s to http://localhost:8080/ for about 20 seconds.
Do not send too much date in the BPS body with a low bandwidth, cause the test will delay next request until it reaches the given byte/s. If you like to send with a very high bandwidth do it also parallel with i.e CLIENT 10 and bandwith of 1000 byte/s per client

Client SSL Example

Assume there is a server with a CA loaded and does require client certificate. Let further assume there is a client pem cert and key signed by the CA loaded from the server.
The server is on your.domain:443

CLIENT
_REQ your.domain SSL:443 client.cert.pem client.key.pem
__GET / HTTP/1.1
__Host: your.domain
__
_EXPECT . "HTTP/1.1 200 OK"
_WAIT
END

Explanation:

As in the SSL Example we open the connection on SSL:443 and we tell httest which is our cert and key. Attention must be taken with the key, there is no password support, so free your key from any password.

Cut Stuff from Stdout

Many time you need to call an external script and match its output.

CLIENT
_MATCH exec "Foo(.*)" BAR
_EXEC echo Foobar
END

Explanation

In the variable $BAR "bar" ist stored in this example. This script will fail if noc Foo.* is read on the stdout.

Note

Need newline for every line else will not appear, will be fixed with the new httest version 1.3.1

Cut Session and Use It

Many times you have to login and test with a valid session. Below is a very simple example how to cut session infos out of the socket stream.

CLIENT
  _REQ localhost 8080
  __GET / HTTP/1.1
  __Host: localhost:8080
  __
  _MATCH headers "Set-Cookie: SessionId=([^;]*);" SESSION
  _WAIT

  _REQ localhost 8080
  __GET / HTTP/1.1
  __Host: localhost:8080
  __Cookie: SessionId=$SESSION
  __
  _WAIT
END

SERVER 8080
  _RES
  _WAIT
  __HTTP/1.1 200 OK
  __Host: localhost
  __Set-Cookie: SessionId=foobar path=/
  __Content-Length: AUTO
  __
  __Data

  _RES
  _WAIT
  __HTTP/1.1 200 OK
  __Host: localhost
  __Content-Length: AUTO
  __
  __Data 
END

Explanation:

_MATCH do match with a regex every line (and fail if no hits) and cut the stuff in () out and store it in the given variable name.
_MATCH has two scope for performance impact "headers" and "body". Most test will only need the scope "headers".

Httest Script Debugging

This is self contain example and should run out of the box.

With _DBG:BP you can set a breakpoint. On break point you can
* type "cont" or "c" <enter> to continue script
* type "list" or "l" to show script around breakpoint
* type "get <variable-name>" or "g <variable-name>" to inspect the value of a variable
* type "set <variable-name>=<value>" or "s <variable-name>=<value> to overwrite or set new variable with a value.
* type "quit" or "q" to abord test</value></variable-name></value></variable-name></variable-name></variable-name></enter>

CLIENT
_REQ localhost 8080
__GET /foo HTTP/1.1
__Host: localhost:8080
__
_WAIT

_DBG:BP

_REQ localhost 8080
__GET /foo HTTP/1.1
__Host: localhost:8080
__
_WAIT
END

SERVER 8080
_RES
_WAIT
__HTTP/1.1 200 OK
__Content-Length: AUTO
__
__foo

_RES
_WAIT
__HTTP/1.1 200 OK
__Content-Length: AUTO
__
__foo
END

Deflate a gzipped stream

CLIENT
_REQ localhost 8080
__GET /your/path HTTP/1.1
__Host: localhost
__User-Agent: mozilla
__
_EXEC< gunzip
_WAIT
END

Explanation:

With the command _EXEC< the received stream can be piped in and the output of the executed shell command ist piped back to the _WAIT command

Test Duration

There are use cases where you want to print the test duration. Need ### version 1.9.0 or higher.

CLIENT
_RES yourhost yourport
__GET /foo HTTP/1.1
__Host: localhost
__
_WAIT
END

BLOCK FINALLY
_IF "$__STATUS" MATCH "0"
_TIMER GET TIME
_EXEC echo $TIME >> file
END

Explanation:

This script measures the duration time of the test script and print this to file only if test status is "0" mean no error.

Embedded Scripts

To hold everything in a single test script, you could also embedd a shell script right in the httest script.

CLIENT
_MATCH exec "hello (.*)" WORLD
_SH #!/bin/bash
_SH echo hello world
_SH END
_DEBUG $WORLD
END

Explanation:

The embedded shell script will be written in a temporary file with a random name. On _SH END the script will be called like any ohter external written script, therefore you can also use _MATCH exec.

Execute External Commands

It is very useful to call external commands in a test script.

EXEC ./your_external_program start

CLIENT
...
# Optional match to cut some values from the stdout of the called shell 
# command. I.e. the shell would return "foobar", the value stored in VAL
# would be "bar"
_MATCH EXEC "foo(.*)" VAL
# call command
_EXEC ./your_external_program stop
END

Explanation:

There is a global and local exec command. First the external program is called with the parameter "start" and on clients end the external command is called with the parameter "stop".
_MATCH, _GREP and _EXPECT knows the body EXEC and operate on the commands output on stdout.
Attention, one could doing this task with two global EXEC commands but this will not work. Because the CLIENT body is running in the background, the last exec command would not wait for the clients end.

=## Pipe Socket Stream to External Command=
This is useful to validate for example content of a server.

CLIENT
_REQ localhost 8080
__GET / HTTP/1.1
__Host: localhost
__
_EXEC| cat > yourfile
_WAIT
END

SERVER 8080
_RES
_WAIT
__HTTP/1.1 200 OK
__Host: localhost
__Content-Length: AUTO
__
__Stream me to yourfile :)
END

Explanation:

Stupid example to show how to pipe socket stream to a external command like "cat". yourfile contains after running this test script: "Stream me to yourfile :)".

HEAD Request

The head request has as a response a Content-Length but not content.

CLIENT
_REQ localhost 8080
__HEAD /your/path HTTP/1.1
__Host: $YOUR_HOST
__User-Agent: mozilla
__
_WAIT 0
END

Explanation:

To cope witht the a Content-Length header but no content in the response, we told httest to wait for the headers and zero bytes in the body.

Load Test Example

Start on your host you want to use to generate load the following command:

htremote -r -p 8080 -e "/path/to/httest -Ssn"

httest -Ssn runs the script absolut silent. The -r option tels htremote to restart after httest has terminated, -p defines on which port this agent is listening and -e specifies the command to exectute in our case it is httest.

Your script may look like that

PERF:RAMPUP 100 60000
PERF:DISTRIBUTE your.host:8080
PERF:DISTRIBUTE your.other.host:8080
PERF:DISTRIBUTE your.last.host:8080

CLIENT 2000
_AUTO_CLOSE on
_LOOP 600000 &#91;ms&#93;
  _REQ your.web.host 80
  __GET /index.html HTTP/1.1
  __Host: your.web.host
  __User-Agent: httest-2.2.5
  __
  _WAIT
_END
END

Explanation:

The 2000 CLIENTs will be distribute on the host the script is running and on your.host, your.other.host and your.last.host. In this case on every host there are 500 CLIENTs running. The script will wait until all host are finished with their job.

With the new PERF:RAMPUP you can define how many Clients per interval should be started. Even _LOOP can now handle time instead of count. In this example a client do run vor 10 minutes. Every minute we start a 100 clients.

Lua Block

Httest 2.1.6 do have a Lua module. You need Lua library installed on your System or download and build it directly from [http://www.lua.org/]. And Httest 2.1.8 do now also have the possiblity to call httest commands in a Lua block.

Build Httest with Lua

Enable Lua Support in httest do as follow

configure --enable-lua-module
make all

If you have the Lua library build by your self do 

configure --enable-lua-module --with-lua=/your/lua/src
make all

Hello World

We first start with simplest possible 

BLOCK:LUA myTestLuaBlock
  print("hello world")
END

CLIENT
  myTestLuaBlock
END

_CALL is optional and not needed any more.

Parameters

To bring Variables from httest to a Lua block and to get results from Lua, you can use the wellknown block signature technics in httest.

BLOCK:LUA myTestLuaBlock param fooparam anyparm : myRet fooRet
  for i = 1,10 do
    print(param .." "..fooparam..";"..anyparm.." end")
  end
  return "any string", "another string"
END

CLIENT
  myTestLuaBlock "hello" world "bla bla bla" stuffFromLua moreReturnStuff
  _DEBUG $stuffFromLua $moreReturnStuff
END

Call httest

Also with httest 2.1.8 it is possible to call a httest snipplet within a Lua block

BLOCK:LUA myTestLuaBlock
  print(htt.version())
  htt.interpret([[
    _REQ localhost 8080
    __GET / HTTP/1.1
    __Host: localhost
    __
    _MATCH "foo=(.*)" FOO
    _WAIT
  ]])
  myFooInLua = htt.getvar("FOO");
END

CLIENT
  myTestLuaBlock
END

Use Transport Object

With httest 2.1.11 you can get transport object from the current connection, to read/write in Lua.

BLOCK:LUA MyTestLuaBlock
  t = htt.get_transport()
  print()
  buf = t:read(8192)
  print("\n--------------")
  print(buf.."--------------")
  t:write("GET / HTTP/1.1\r\n");
  t:write("\r\n");
  print()
END

CLIENT
  _REQ localhost 8080
  __GET / HTTP/1.1
  __Host: httest
  __
  _FLUSH
  MyTestLuaBlock
END

Define Macros

There is a primitiv macro mechanisme implemented to handle complicated stuff outside the test.

BLOCK foo
_REQ localhost 8080
__POST $1 HTTP/1.1
__Content-Length: AUTO
__
__This blocks name is $0
_WAIT
END

CLIENT
_CALL foo /path/to/your/file
END

Explanation:

The macro can handle parameter. It is same like in shell commands. The parameter $0 is the block name, $1 ... $x are the parameters.

Mutual authentication

Do mutual authentication.

CLIENT
_REQ localhost SSL:8080 client.cert.pem client.key.pem ca.cert.pem
_VERIFY_PEER
__GET / HTTP/1.1
__Host: localhost
__
_WAIT
END

SERVER 8080
_CERT server.cert.pem server.key.pem ca.cert.pem
_RES
_WAIT
_VERIFY_PEER
__HTTP/1.1 200 OK
__Content-Length: AUTO
__
__Hello World
END

Explanation:

The key point is the command _VERIFY_PEER, which is integrated in the version 0.12.1 and higher.
The client do verify the server certificate and the server do request and validate the client certificate.
The certificates must all be from the same CA in this example.

Pipe Output of External Command into Socket Stream

It could be useful to call an external program to tranform a String, for example a base64 transformation and pipe the output back to the socket stream.

CLIENT
_REQ localhost 8080
_-GET
_PIPE
_EXEC echo /foo/bar
__ HTTP/1.1
__Host: localhost
__
_EXPECT . "HTTP/1.1 200 OK"
_WAIT
END

SERVER 8080
_RES
_EXPECT . "/foo/bar"
_WAIT
__HTTP/1.1 200 OK
__Host: localhost
__Content-Length: AUTO
__
__Answer
END

Explanation:

This is somehow a stupid example you could short write the GET request with a simple

__GET /foo/bar HTTP/1.1

But it demonstrate how to pipe a external commands out put, i.e. "echo /foo/bar", into an open socket stream.

Pipe Output of External Command into Socket Stream 2

Mostly we want to stream a file output to the caller.

CLIENT
_REQ localhost 8080
__GET / HTTP/1.1
__Host: localhost
__
_WAIT
END

SERVER 8080
_RES
_WAIT
__HTTP/1.1 200 OK
__Host: localhost
__Transfer-Encoding: chunked
_FLUSH
_PIPE CHUNKED 30
_EXEC echo blabla bla bla bla bla bla bla bla bla bla bla
_CHUNKED
__
END

Explanation:

The EXEC echo blabla is piped into socket stream with chunks of 30 bytes.

POST chunked

There are many Fat Clients which do post with transfer-encoding: chunked. A simple example will demonstrat the chunked support with in httest.

CLIENT
_REQ localhost 8080
__POST / HTTP/1.1
__Host: localhost
__Transfer-Encoding: chunked
_FLUSH
__Some data
_CHUNKED
__More data
_CHUNKED
__Last data
_CHUNKED
_CHUNKED
__
_EXPECT . "HTTP/1.1 200 OK"
_WAIT
END

SERVER 8080
_RES
_EXPECT . "Some data"
_EXPECT . "More data"
_EXPECT . "Last data"
_WAIT
__HTTP/1.1 200 OK
__Host: localhost
__Transfer-Encoding: chunked
_FLUSH
__Answer
_CHUNKED
_CHUNKED
__
END

Explanation:

The command _CHUNKED to \r\n<chunk_size_in_hex>\r\n. Every command do store the data in a line cache. The _FLUSH command do send all lines in the line cache, in this case all headers. The command _CHUNKED to calculate the size in the line cache, with all lines not allready flushed.
The empty line between the headers and the body is done with the first _CHUNKED command. The Last 0 Chunkded could also be done with CHUNKED with a following newline __.</chunk_size_in_hex>

Read a line

There could be a need to read line by line manually. This could be done with the following construct:

CLIENT
_REQ localhost 8080
__GET /your/path HTTP/1.1
__Host: localhost
__User-Agent: mozilla
__
_SOCKET
_EXPECT . "HTTP/1.1 200 OK"
_READLINE
_EXPECT . "Content-Type: text/plain
_READLINE
_MATCH body "Content-Length: (.*)" CONTENT_LEN
_END SOCKET
END

Explanation:

The _SOCKET ... _END SOCKET do open a buffered socket for various _READLINE and _RECV commands. Every _READLINE can handle one or more expect and match commands. In our example we first expect "HTTP/1.1 200 OK", next line must contain the Content-Type and the third line must containt the Content-Length which is then stored in the variable $CONTENT_LEN

Time Format

SERVER
_RES
_WAIT
_TIME TIME
_STRFTIME $TIME "S GMT" DATE
__HTTP/1.1 200 OK
__Date: $DATE
__
END

Explanation:

The Variable TIME is formated with the format string"S GMT" and stored in the variable DATE. See man page for strftime for format details.

URL encoding

CLIENT
_URLENC "foo bar&blafasel" VAR
_REQ localhost 8080
__GET /your/path?param=$VAR HTTP/1.1
__Host: localhost
__User-Agent: mozilla
__
_WAIT
END

Explanation:

The string foo bar&blafasel will be stored URL encoded in the variable VAR as "foo+bar%26blafasel"

Use XPath

Httest 2.2.1 do integrate the power of gnoms libxml2 XPath support.

CLIENT
_REQ www.google.com 80
__GET / HTTP/1.1
__Host: www.google.com
__
_WAIT BUF
_HTML:PARSE VAR(BUF)
_HTML:XPATH /html/body/a&#91;1&#93; result
_DEBUG $result
END

PROJECT INFOS

Project Members: