Module molly.runner

Module with main functions that runs tests.

Design Overview

A Molly test runs as a Lua program on a control node. That program may use remote access to log into a bunch of DB nodes, where it sets up the distributed system you're going to test or use local DB instances.

``

         +-------------+
+------- | controller  | -------+
|        +-------------+        |
|          |    |    |          |
|     +----+    |    |          |
v     v         |    |          v

+----+----+----+ | | +----+----+ | n1 | n2 | n3 | <+ +> | n4 | n5 | +----+----+----+ +----+----+ `

Once the system is running, the control node spins up a set of logically single-threaded processes (see molly.thread), each with its own client for the distributed system.

A generator (see molly.gen) generates new operations (see molly.op) for each process to perform. Processes then apply those operations to the system using their clients, see molly.client. The start and end of each operation is recorded in a history, see molly.history. While performing operations, a special nemesis process (see molly.nemesis) introduces faults into the system - also scheduled by the generator.

Finally, the DB is turn down. Molly uses a checker (see molly.checker) to analyze the test's history for correctness, and to generate reports, graphs, etc. The test, history, analysis, and any supplementary results are written to the filesystem for later review.

Performance Tips

Disable debug mode: by default tests enables type checking, see statements with dev_checks() in source code, and code coverage gathering. This requires using Lua debug module, that can significantly slowdown of execution (about 1.6 times). You can control it with environment variable DEV, to run tests with disabled type checking and code coverage: DEV=OFF make test.

Disable verbose mode: see description of verbose` mode.

Use fibers: TODO: performance of fibers vs coroutines.

Functions

run_test (workload, opts) Create test and run.


Functions

run_test (workload, opts)
Create test and run.

Parameters:

  • workload Table with workload options.
    • client Workload client. Learn more about creating clients in molly.client .
    • generator table Generator of operations used in test workload. Generator must be a table with unwrap() method that returns an iterator triplet. You can make generator youself or use molly.gen module.
    • checker Function for checking history in workload.
  • opts Table with test options.
    • create_reports boolean

      Option to control creating reports, disabled by default. When enabled a number of files created:

      • history.txt with plain history;
      • history.json with history encoded to JSON;
    • threads number Number of threads in a test workload, default value is 1.
    • verbose boolean shows details about the results and progress of running test. This can be especially useful when the results might not be obvious. For example, if you want to see the progress of testing as it setup, teardown or invokes operations, you can use the 'verbose' option. In the beginning, you may find it useful to use 'verbose' at all times; when you are more accustomed to molly , you will likely want to use it at certain times but not at others. Disabled by default. Take into account that logging to standart output is a slow operation and with enabled verbose mode molly logs status of every operation before and after it's invocation and this may slowdown overall testing performance significantly. It is recommended to disable verbose mode in a final testing and use it only for debugging.
    • thread_type string Type of threads used in a test workload. Possible values are 'fiber' (see molly.thread_fiber ) and 'coroutine' (see molly.thread_coroutine ), default value is 'fiber' on Tarantool and 'coroutine' on LuaJIT. Learn more about possible thread types in molly.thread .
    • time_limit number Number of seconds to limit time of testing. By default testing time is endless and limited by a number of operations produced by generator.
    • nodes table A table that contains IP addresses of nodes participated in testing.

Returns:

    true on success or nil with error

See also:

Usage:

    
 local test_options = {
     create_reports = true,
     thread_type = 'fiber',
     threads = 5,
     nodes = {
         '127.0.0.1'
     }
 }
 local ok, err = runner.run_test({
     client = client.new(),
     generator = gen_lib.cycle(gen_lib.iter({ r, w })):take(1000)
 }, test_options)
generated by LDoc 1.4.2