/* Copyright (c) 2015, Google Inc. * * Permission to use, copy, modify, and/or distribute this software for any * purpose with or without fee is hereby granted, provided that the above * copyright notice and this permission notice appear in all copies. * * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY * SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION * OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN * CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ #ifndef OPENSSL_HEADER_CRYPTO_TEST_FILE_TEST_H #define OPENSSL_HEADER_CRYPTO_TEST_FILE_TEST_H #include <openssl/base.h> #include <stdint.h> OPENSSL_MSVC_PRAGMA(…) OPENSSL_MSVC_PRAGMA(…) #include <functional> #include <map> #include <memory> #include <set> #include <string> #include <vector> OPENSSL_MSVC_PRAGMA(…) // File-based test framework. // // This module provides a file-based test framework. The file format is based on // that of OpenSSL upstream's evp_test and BoringSSL's aead_test. NIST CAVP test // vector files are also supported. Each input file is a sequence of attributes, // instructions and blank lines. // // Each attribute has the form: // // Name = Value // // Instructions are enclosed in square brackets and may appear without a value: // // [Name = Value] // // or // // [Name] // // Commas in instruction lines are treated as separate instructions. Thus this: // // [Name1,Name2] // // is the same as: // // [Name1] // [Name2] // // Either '=' or ':' may be used to delimit the name from the value. Both the // name and value have leading and trailing spaces stripped. // // Each file contains a number of instruction blocks and test cases. // // An instruction block is a sequence of instructions followed by a blank line. // Instructions apply to all test cases following its appearance, until the next // instruction block. Instructions are unordered. // // A test is a sequence of one or more attributes followed by a blank line. For // tests that process multiple kinds of test cases, the first attribute is // parsed out as the test's type and parameter. Otherwise, attributes are // unordered. The first attribute is also included in the set of attributes, so // tests which do not dispatch may ignore this mechanism. // // Additional blank lines and lines beginning with # are ignored. // // Functions in this module freely output to |stderr| on failure. Tests should // also do so, and it is recommended they include the corresponding test's line // number in any output. |PrintLine| does this automatically. // // Each attribute in a test and all instructions applying to it must be // consumed. When a test completes, if any attributes or insturctions haven't // been processed, the framework reports an error. class FileTest; FileTestFunc; class FileTest { … }; // FileTestMain runs a file-based test out of |path| and returns an exit code // suitable to return out of |main|. |run_test| should return true on pass and // false on failure. FileTestMain also implements common handling of the 'Error' // attribute. A test with that attribute is expected to fail. The value of the // attribute is the reason string of the expected OpenSSL error code. // // Tests are guaranteed to run serially and may affect global state if need be. // It is legal to use "tests" which, for example, import a private key into a // list of keys. This may be used to initialize a shared set of keys for many // tests. However, if one test fails, the framework will continue to run // subsequent tests. int FileTestMain(FileTestFunc run_test, void *arg, const char *path); // FileTestMain accepts a larger number of options via a struct. int FileTestMain(const FileTest::Options &opts); // FileTestGTest behaves like FileTestMain, but for GTest. |path| must be the // name of a test file embedded in the test binary. void FileTestGTest(const char *path, std::function<void(FileTest *)> run_test); #endif // OPENSSL_HEADER_CRYPTO_TEST_FILE_TEST_H