The BPS suite is a collection of many tests. You should have minimal or no problems with the single machine tests. As more machines become involved with the tests, there is room for more configuration errors to arise. If a test does not run, check the test_name.log file in the log directory. In the case of the NAS tests, the results are in the form npb.COMPILER.MPI.CLASS.PROCESSORS.
In general, if you have problems with a test it may be best to run it from the command line. In the case of the NAS suite, the -k option will keep the npb directory in the log directory so you can run the tests more directly by using the run_suite script in the npb directory. Also the README.plogic file in the npb directly should provide more information on how the tests are run and how to resolve possible problems.