This file describes the recommended format for submissions to the CMU-based collection of learning benchmarks for neural networks. Each benchmark problem will be described by a separate ASCII text file in the directory "/afs/cs.cmu.edu/project/connect/bench". These will be called the "benchmark description" files. There may additionally be one or more "data" files as described below. (If we find that a lot of the benchmarks require multiple files, we may move to a scheme in which each benchmark is in its own unix subdirectory, but that makes it much harder for some researchers to grab the files via FTP; for now, we'll try to stick with a single-level directory.) Administrative files, such as this one, will have names in ALL-CAPITALS so that they will stand out in directory listings, at least for users with case-sensitive file systems. The benchmark files and data sets should not be copyrighted. In rare cases we might accept a copyrighted data set if the owner gives blanket permission for free distribution and non-commercial use, but we would prefer to keep that to a minimum. Once a benchmark has been installed, we will assign someone at CMU as "maintainer". This person will add new results to the benchmark description file and perform any other housekeeping that is required. Obviously, we would try to avoid changing the problem itself once people have started to work on it; it is better to release any substantive revision as a distinct benchmark. As a courtesy to people who must read this stuff on old-fashioned terminals and text editors, lines of text in the benchmark description files should be no longer than 79 characters unless there is a compelling reason to include a longer line. The proposed format for a benchmark description file follows the dashed line. Text within <> is descriptive; everything else is meant to be verbatim. One last comment: There will occasionally be judgment calls about what goes into the collection (or into one of these files) and what does not. In general, we will try to resolve these issues by discussion and consensus-building on the nn-bench mailing list. However, if consensus cannot be reached and a decision has to be made, the final decision will rest with the maintainers of this collection. --------------------------------------------------------------------------- NAME: <> SUMMARY: <> SOURCE: <> MAINTAINER: <> PROBLEM DESCRIPTION: < 0 01 => 1 10 => 1 11 => 0 3. As a separate data file whose format is clearly specified here. If the benchmark description file is named "foo", the associated data file would be "foo.data". 4. At present, we have rather limited storage space for this project at CMU. For very large benchmark problems, we may keep a benchmark description file here, but not the data itself. In such cases, this section would describe how to obtain the data set via FTP, tape, or whatever.>> METHODOLOGY: <> VARIATIONS: <> REFERENCES: <> COMMENTS: <>