Bioinformatics Benchmark System is a software tool designed to measure the performance of bioinformatics platforms. It is used for benchmarking and performance evaluation.
The Bioinformatics Benchmark System is a testing framework that aims to provide end users and vendors with the ability to evaluate the performance of their systems. It includes a core set of tests that can be downloaded and used to measure specific elements of performance. The source code for these tests is available under GPL and is hosted on Bioinformatics.org as well as Scalable Informatics LLC.
The main goal is to make it easy for users, consumers, system developers, and others to create and utilize meaningful tests for measurement and tuning purposes. The idea was originally conceived by Joe Landman and the codes were written by him. They are currently accepting suggestions for additional benchmark codes, tests, and data sets.
Currently, the baseline tests include NCBI BLAST runs, HMMer runs, and others. More tests such as ClustalW, X!Tandem, and various chemistry and dynamics tests are planned for inclusion in the future. It was found that tests like LINPACK or HPL do not provide useful performance indicators or predictive models for high-performance informatics. These additional tests fail to consider elements such as disk I/O, memory latency and other factors that contribute to performance issues.
It is best to use tests that closely resemble the codes that will run on the system. The tests themselves should be reasonable approximations of a real execution of the code using actual data. Realistic run times can be achieved by adjusting the data size for testing.
It is important to remember that a single test does not reveal how the system will scale, and that the performance needs to be tested with increasing workloads. One should try to run the tests in as realistic an operating environment as possible since a quiescent system is not often achievable in a cluster. A baseline test in a quiescent system may set unrealistic expectations.
The main goal is to make it easy for users, consumers, system developers, and others to create and utilize meaningful tests for measurement and tuning purposes. The idea was originally conceived by Joe Landman and the codes were written by him. They are currently accepting suggestions for additional benchmark codes, tests, and data sets.
Currently, the baseline tests include NCBI BLAST runs, HMMer runs, and others. More tests such as ClustalW, X!Tandem, and various chemistry and dynamics tests are planned for inclusion in the future. It was found that tests like LINPACK or HPL do not provide useful performance indicators or predictive models for high-performance informatics. These additional tests fail to consider elements such as disk I/O, memory latency and other factors that contribute to performance issues.
It is best to use tests that closely resemble the codes that will run on the system. The tests themselves should be reasonable approximations of a real execution of the code using actual data. Realistic run times can be achieved by adjusting the data size for testing.
It is important to remember that a single test does not reveal how the system will scale, and that the performance needs to be tested with increasing workloads. One should try to run the tests in as realistic an operating environment as possible since a quiescent system is not often achievable in a cluster. A baseline test in a quiescent system may set unrealistic expectations.
What's New
Version 3: N/A