Operations Research

Logo

A site for hosting software and data repositories associated with papers appearing in the journal _Operations Research_

View source on GitHub

Procedures for submission of accompanying software and data

[Note: This page is a supplement to the Code and Data Disclosure Policy here]

All authors submitting a paper to Operations Research are expected to make the data and code necessary to reproduce the results in their paper available (unless an exemption has been explicitly asked for and granted).

Submission of software and data can be done in two different ways. The traditional process is to submit a tarball along with the paper, but it is also possible to create an archive on Github, hosted in the ORJournal Github organization. The latter is typically the mode of submission when volunteering to participate in a replication study. If you have not been contacted about that, you will most like be following the traditional method of submission.

If you already host the software and/or data on Gtihub (or elsewhere), you may reference that hosting location in the paper, but we still ask authors to either prepare a tarball (or an archive on Github) to ensure the exact data and code used to produce the results in the paper is preserved along with the paper. In the below, we provide suggestions for how to organize your tarball that are based on the instructions given farther down for what to do in order to prepare for a replication study.

Submitting a Tarball/ZIP archive

To make it easier for reviewers and readers who wish to look at your code and data to access it, please try to organize the files and directories according to the following recommendations. Of course, deviations are fine and there are no hard rules, just try to organize things in a way that makes it easy to understand. The first three files described should almost always be there.

You may wish to have an additional README in any of the subdirectories to provide additional information.

Preparing for a replication study

If you have volunteered for a replication study, then first of all, thank you! Note that there are two goals for this process. One is to allow editors to attempt to replicate the experiments and reproduce the empirical data and results associated with submitted papers. The other is to archive the code in a way that will be useful to readers and allow the experiments in the paper to be replicated in the future. For this reason, we would like to include the full source code needed to perform the experiments in this repository, which is a fixed snapshot we can guarantee will be available in the future. This is the case whether or not the code is available elsewhere already (such as in a Github repo or on Pypi. See the FAQs below for more information.

Accessing your repository

When requesting your software/data, an editor will give you read access to to a private repository created by the editorial staff. This is the repository that should be populated and that will be reviewed and eventually archived with the paper. The name of the repository associated with your paper will be XXXX.YYYY and is derived from the manuscript number in manuscript central. Typically, XXXX will be the year in which the paper is submitted, whereas YYYY is a four digit sequence number that is part of the manuscript number.

Populating the repository

To populate the repository with your own materials, fork it to make a copy that will live in your own Github account. Once you populate and customize the repository to your liking, open a Pull Request to begin the review.

The contents of the repository should be organized to make it as easy as possible for the associate editor to do the replication. The layout should resemble that of the template here to the extent possible, although some variations may occur and there are no inviolable rules. In general, the repository contents should be as follows.

You may wish to have an additional README.md in any of the subdirectories to provide additional information.

Things to keep in mind

When you are submitting your code for a replication study, there are some additional things to keep in mind. To make it as easy as possible for the editors to replicate the results in your paper, please provide the following information in your README.

Please also provide

In case replicating all experiments would be a long process, please divide the experiments into subsets that can be run in a reasonable amount of time and can verify an easily identifiable part of the reported results.

Review process

The review process begins as soon as you submit your pull request. As part of the review process, additional changes may be requested. If so, the area editor will communicate with you by putting comments into the pull request. You can make the requested changes by updating your fork and these will be immediately reflected in the pull request. Once everything is finalized, then the pull request will be accepted. Upon publication of the paper, the repository will be made public.

Please ensure that all files contain proper copyright and licensing statements and that the copyright holders have been notified of the submission. The copyright holder may or may not be you, depending on your employment contract and who funded the work.

Archive and DOI

Once accepted by the editor, a snapshot of the contents of the repo will be archived by creating a tag (known as a release on Github) with the name vXXXX.YYYY, where the repo’s name is XXXX.YYYY. This paper and the snapshot of the repo will be given their own separate DOIs, also derived from the manuscript number. If the repo is named XXXX.YYYY, then the DOIs will be

https://doi.org/10.1287/opre.XXXX.YYYY

https://doi.org/10.1287/opre.XXXX.YYYY.cd

Citing the repo in your paper

The repo must be cited in your paper, as a regular reference, and appear in the list of references as follows (if using BibTex). Notice that you must use both the DOI and the Note lines to make this reference appear correctly.

@misc{CacheTest,
  author =        {Put the authors' names here},
  publisher =     {Operations Research},
  title =         {Put the title of your paper here},
  year =          {put the year here - formal like 2023},
  doi =           {Put your DOI here - see above - example: 10.1287/opre.2019.0000.cd},
  note =          {Again, use your DOI here at the end of the link.  example: Available for download at https://github.com/ORJournal/2024.0000},
}

FAQs