Response to the OMB Data Quality Notice from the National Academy of Science (9 August 2001)
August 9, 2001
Ms. Brooke Dickson
Office of Information and Regulatory Affairs
Office of Management and Budget
Washington, D.C. 20503
Dear Ms. Dickson:
These comments are being submitted on behalf of the National Academy of
Sciences in response to the notice that appeared in the Federal Register
for June 28, 2001, pages 34489-34493.
As explained in the Federal Register notice, Pub. L. 106-554,
Section 515 directs the Office of Management and Budget (OMB) to issue
government-wide guidelines that "provide policy and procedural guidance
to Federal agencies for ensuring and maximizing the quality, objectivity,
utility, and integrity of information (including statistical information)
disseminated by Federal agencies. Within one year after OMB issues these
guidelines, agencies must issue their own implementing guidelines that
include administrative mechanisms allowing affected persons to seek
and obtain correction of information maintained and disseminated by the
agency that does not comply with the OMB guidelines."
The National Academies applauds the government's efforts to ensure
that information disseminated by Federal agencies meets a basic level
of quality, objectivity, utility, and integrity. However, The National
Academies has significant concerns with the following statement in the
proposed guidelines,
"With respect to scientific research information, the
results must be substantially reproducible upon independent analysis of
the underlying data."
This statement establishes a new and unreasonable standard for the
dissemination of scientific information by the government. Agencies
frequently rely on peer reviewed, published information. This proposed
requirement would severely restrict agency use of the best and most
current scientific information available.
The peer review process and the criteria for scientific publication,
which represent "quality, objectivity, utility, and integrity" of
information for the scientific community, do not include demonstration
a priori of independent reproduction of the underlying data. Requiring
the results to be "substantially reproducible upon independent analysis
of the underlying data" establishes a standard that is not the norm for
the scientific community. The peer review process typically evaluates the
methods and procedures--the soundness of the approach and results--it does
not include independent reproduction of the research results. As I noted
previously (Alberts letter to Charney, April 5, 1999), "publication of
research results in peer reviewed scientific journals is one of the most
critical elements of the research process. It is the means by which new
discoveries are communicated to others in the scientific community and
to the public at large." One of the objectives of publishing results
is to expose the research findings to a larger scientific audience to
critique and build upon, which does frequently result in other researchers
attempting to reproduce and check research results.
We strongly encourage OMB to seek an extension to the September
30 deadline, so that the issues raised above can be more thoroughly
vetted and discussed. The National Academies welcomes the opportunity
to participate in further discussion of these issues.
Sincerely,
Bruce Alberts
President
