ee

‘ine

REVIEW AND APPROVAL OF OPERATIONAL GRANTS

Background

These procedures were developed after extensive consideration of:
(1) the philosophy and purposes of Title IX; (2) the initial experience in
reviewing the planning grant applications awarded under Section 903;
(3) consideration of the first operational grant proposals, including site
visits to the regions involving members of the National Advisory Council on
Regional Medical Programs and the Regional Medical Programs Review Committee;
(4) preliminary discussion of the issues involved in the review of
operational applications by the National Advisory Council on Regional Medical
Programs at its November 1966 meeting; and (5) extensive discussion with
both the Review Committee and the National Advisory Council concerning the
effectiveness of these procedures during the actual review of the first
operational. applications. As a result of these considerations, the resulting
review and approval process is to the greatest possible extent keyed to

the anticipated nature of operational grant requests and to the policy

issues inherent in the Regional Medical Programs concept.

Characteristics of Operational Grants
The review process recognizes the following characteristics of
Regional Medical Program grants: (1) complexity of the proposals with many

discrete but interrelated activities involving different medical fields;
@)

(2) the diversity and variety of grant proposals resulting from encourage-
ment of initiative and determination at the regional level under the flexible ©
parameters provided in the law, regulations, and guidelines; (3) the many
different attributes of the overall operational proposal which are
appropriate concerns of the review process, including factors affecting

the organization and delivery of health services, the degree of effective
cooperation and commitment of the major medical resources and extending to
“the evaluation and the merit of highly technical medical activities in | c
the fields of heart disease, cancer, stroke, and related diseases; (4) the
relationship of the proposal to the responsibilities of many other components
of the Public Health Service and other Federal programs requiring a review

process that allows adequate opportunity for input from these other sources;

 

(5) the characteristics of these initial proposals as the first steps in the ws
more complete development of the regional medical program, guided by a |
continuing planning process.
Objectives of Review Process

The objectives sought in the development of this review process
are based on a careful assessment of the goals of the Regional Medical
_ Programs and how the achievement of those goals can be most effectively
furthered by the process used in making decisions on the award of grant
funds. Consideration of these basic policy issues led to delineation of

the following objectives of the review process:

 
 

Saco?

1. The operational grant application mist be viewed as a totality
rather than as a collection of discrete and separate projects.

2. The decision-making process for the review and approval of
operational grants must be developed in a way that stimulates and preserves
the essential goal setting, priority determination, decision making and
evaluation at the regional level.

3. The review process must be concerned with the probability of
effective implementation of the proposed activities in addition to the
inherent technical merit of the specific proposals.

4, The review process must assure a basic level of quality and
feasibility of the individual activities that will make an investment of
grant funds worthwhile.

5. The review process must: have sufficient flexibility to cope with
the variety of operational proposals submitted, allowing for the tailoring
of the review to the needs of the particular proposal.

6. The review process should view a regional medical program as a
continuing activity, not a discrete project with time limits; therefore,
the review process will have continuity during the grant activity and will
have the opportunity to judge the development of Regional Medical Programs
on the basis of results and evaluation of progress, in addition to the

evaluation of the probable effectiveness of initial proposals.
6

Criteria
The basic criteria for the review of Regional Medical Program grant
requests are set forth in the regulations as follows:

"54.406 Award,

Upon recommendation of the National Advisory Council on Regional
Medical Programs, and within the limits of available funds, the

approved programs will in his judgment best promote the purposes
of Title IX. In awarding grants, the Surgeon General shall take
into consideration, among other relevant factors the following:

"(a) Generally, the extent to which the proposed program will
carry out, through regional cooperation, the purposes of Title Ix,
within a geographic area. ,

"(b) The capacity of the institutions or agencies within the
program, individually and collectively, for research, training,
and demonstration activities with respect to Title IX,

"(c) The extent to which the applicant or the participants in
the program plan to coordinate or have coordinated the regional
medical program with other activities supported pursuant to the
authority contained in the Public Health Service Act and other
Acts of Congress including those relating to planning and use
of facilities, personnel, equipment, and training of manpower,

"(d) The population to be served by the regional medical program
and relationships to adjacent or other regional medical programs,

"(e) The extent to which all ‘the health resources of the region
have been taken into consideration in the planning and/or
establishment of the program,

"(£) The extent to which the participating institutions will
utilize existing resources and will continue to seek additional
nonfederal resources for carrying out the objectives of the
regional medical program, 7

"(g) The geographic distribution of grants throughout the Nation,"

 
nm,

‘specific aspects of the proposal unless this first ess

In utilizing these eriteria in the review process, it was determined
that the sequence of consideration of the various attributes of the
proposal would be important if the objectives of the review process listed
above were to be achieved. The review process, therefore, must focus
on three general characteristics of the total proposal which separately

and yet collectively determine its nature as a comprehensive and potentially

_effective regional medical program:

(1) The first focus must be on those elements of the proposal which

identify it as truly representing the concept of a regional medical program.

Our review groups have determined that it is not fruitful to consider

ential determination

concerning the core of the program is positive. In making this determination

considerations include such questions as: "Ig there a unifying conceptual
strategy which will be the basis for initial priorities of action, +
evaluation, and future decision making?" "Tg there an administrative

and coordinating mechanism involving the health resources of the regions

which can make effective decisions, relate those decisions to regional

needs, and stimulate the essential cooperative effort among the major

health interests?" "Will the key leadership of the overall regional
medical program provide the necessary guidance and coordination for the
development of the program?" "What is the relationship of the planning

already undertaken and the ongoing planning process to the inittal

operational proposal?"

 
(2) After having made a positive determination about this core
activity, the next step widens the focus to include both the nature
and the effectiveness of the proposed cooperative arrangements, In
evaluating the effectiveness of these arrangements attention is given to
the degree of involvement and commitment of the major health resources, the
role of the Regional Advisory Group, and the effectiveness of the proposed
activities in strengthening cooperation, Only after. the determination
has been made that the proposal reflects a regional medical program concept
and that it will stimulate and strengthen cooperative efforts will a more
detailed evaluation of the specific operational activities be made,

. (3) If both of the two previous evaluations are favorable, the
operational activities can then be reviewed, individually and collectively.
Each activity is judged for its ow intrinsic merit, for its contribution J
to the cooperative arrangements, and for the degree .to which it includes
the core concept of the Regional Medical Programs, It should also fit
as an integral part of the total‘ operational activities, and contribute
to the overall objectives of the Regional Medical Programs. .

Review Procedures

| Attached is a chart which describes the various steps in the review
process which will be applied to initial operational grant proposals from
each region, The first four operational grant proposals were subject to

the various steps of this process. Those steps were not carried out in
precisely the order and sequence provided in this chart-since the first
four applications were used as a test situation for the development of this
operational procedure. It is auee likely that further experience will lead
to appropriate modification of these procedures. The following comments
may help to explain this review process, which has been agreed to by the
Regional Medical Programs Review Committee and the National Advisory Council
on Regional Medical Programs. The complexity of ‘these grant requests and
the steps in the review process which seems appropriate for their review
will require as much as six months for the completion of the total review
process in most cases.

(1) Initial Consideration by Review Committee--The first steps of
the review process involve preparation for the site visit which will be
conducted for each operational grant application. The first consideration
of the application by the Review Committee will be for the purposes of
providing information and comments for the guidance of the gite visit team,
utilizing ‘staff analyses of the planning grant experience, considerations
of gross technical validity, policy issues raised by the particular
application, and initial input on relationships to other, Federal programs.

' (2) Site Visit--Initial experience has indicated that a site visit
the extent to which the regional medical

is essential for the assessment of

program has developed an overall guiding concept for the regional medical

program which has been the base of assignment of priorities of action, It

also provides the opportunity to assess the probable effectiveness of
6)
cooperative arrangements and degree of commitment of the many elements
which will be essential to the success of a regional medical program. , As
the discussion above points out, it is necessary to reach favorable
conclusion on these aspects of the regional medical program before it is
justifiable to begin the major investment of the time of the Division staff,
technical reviewers in other parts of the PHS, technical consultants, and
the DRMP review groups, which is necessary for the assessment of the various
components of the application, The site visit is not a substitute for

the investment of this effort but provides the opportunity to evaluate the
cooperative framework of the regional medical program and the overall
probability of the success of the proposed program.

(3) Intensive Analysis and Technical Reviews~--If the site visit

report justifies the investment of additional effort in the review of the )
application, the review process proceeds with the undertaking of an intensive
staff analysis of the application which is the basis for obtaining

specific comments from other components of the PHS and other Federal health
agencies with related programs, detailed comments from the various

components of the DRMP staff, technical site visits on specific projects
within the overall application when considered necessary, and the assimi-
lation of additional information from the applicant as a result of the

site visit. The technical review of specific projects should not be simply

from the standpoint of national merit criteria but should identify specific

problems with any project which might prevent that project from making a
meaningful contribution to the objectives of the regional medical program.
Technical reviews also consider the justification for the particular
project budget as presented, This aspect of the review process also
presents the opportunity to consider possible overlaps and duplications
with other PHS programs which can be a factor in determining how much support
should be provided for the particular activity from the regional medical
program grant. The opportunity to raise these questions is not limited
to DRMP staff initiative since copies of all applications are distributed
te all bureaus of the PHS at the time of receipt and representation. from
all bureaus is invited to all DRMP review meetings.

(4) Second Review by Review Committee and Recommendation for Action-~<
The Review Committee considers all of the information available concerning
the application. In addition to the application itself and the site visit
report, a summary of all available information is presented to the Committee
in a staff presentation, The Review Committee then makes its recommendation
concerning the application. Because of the complex nature of the applications,
the Review Committee can divide its recommendation into several parts
relating to different parts of the application. If there is an overall
favorable recommendation, the Review Committee recommends an overall grant
amount based on a discussion of the specifics of the application. This .
amount takes into consideration problems raised by technical reviewers,
overlap with other programs, feasibility of the proposals, and .other

relevant considerations raised by the review process. While the overall
10

amount recommended is based nn discussion of the specific components of the

total application, it does not in most cases involve a specific approval
ox rejection of individual projects within the application, although the
attitude of the review group toward a particular project may be a major
determinant in the calculation of the overall amount recommended.

(5) Review by National Advisory Council on Regional Medical Programs~~

The National Advisory Council then considers the Review Committee recom~

 

mendations. It has available to it the full array of material presented
to the Review Committee and a staff summary of that material, Further
information obtained by the staff on the instructions of the Review
Committee may also be presented, The National Advisory Council then

makes its required legal recommendation concerning the application. The

1)

Advisory Council also recommends an overall grant amount, The Council
may delegate to the staff the authority to negotiate the final grant amount:
within set limits. .

(6) Meeting with applicant--Following the National Advisory Council
meeting, the staff of the Division meets with representatives of the applicant
and presents to the applicant the recommendation and comments of the
Council. If that recommendation is favorable, the staff also presents
to the applicant the recommended overall budget ceiling for the grant
along with a summation of all the comments derived from the review process
concerning particular activities contained within the application, including

negative comments about specific projects and the budget levels proposed

for specific projects. The review process does not disapprove specific
*)

ll

projects even if there may have been considerable doubts raised by the
review process except where the project is infeasible, outside the scope
of a regional medical program, raises serious overlap questions, or is
technically useless.
(7) Submission of Revised Proposal--On the basis of this meeting,
the applicant submits a revised proposal within the recommended overall
budget ceiling, utilizing in the revision the comments and criticisms
resulting from the review process. This step of the process requires
the applicant to reconsider their priorities and to assume the basic
responsibility for making the final decisions as to the constituent
elements of their operational activity after having the benefit of technical
comments and advice. The applicant may choose to undertake an activity even
if serious doubts about the activity were raised by the review process with
the understanding that such an activity will be followed with interest by
the review groups and will be judged in the future on the basis of results.
(8) Final Award Decision--Following staff review of the revised
proposal, the final award decision is made resulting in the award of a grant.

At this stage some final negotiations with the applicant may take place.

February 1967
fnitial Staf£ Information re

a.
b..
Cs
d,

Planning grant experience
Gross technical validity
Policy issues

FLOW CHART

Division of Regional Medical Programs

OPERATIONAL GRANT REVIEW AND APPROVAL PROCESS

 

Operational Grant
‘ Application Received

 

 

 

 

 

 

Review Committee

 

Staff
Members
i rs
v \ ¥.

 

 

Review Committee Meeting
for
Information & Comment

 

 

Relationship to other Federal programs

Review Committee Guidance

 

 

SZ

 

Site Visit
(Two days)

 

 

(Prepared 2nd day by site team)

 

 

Site

Visit
Report

 

 

4

February 1967

Guidance for Site Visit Team

Judgments re:

1,

2.
3.

4.

5

Concept of RMP.
Cooperative Arrangements
Relationship of projects,
one to another and to the
total

Approximate magnitude of
support warranted
Quality of projects where
appropriate
In addition to application and

site visit report poets cee

1.

2.

Additional information from
applicant,

Expert review re quality of project
from outside DRMP, where indicated,
including comments from other com-
ponents of the PHS; may have neces-
sitated technical site visit on
specific project(s)

Further Staff information

Discussion by site visitor(s) of additional
information obtained subsequent to site visit,

In addition to above;

1.

2.

Review Committee recom-
mendations

Further Staff information
per Committee instructions

Provided to Applicant:

1,

Recommendation and comments of
Council; if overall approval
proceed to 2.

Recommend overall budget celling
for grant

we eee} for

 

MW
Review Committee Meeting
“di for
Consideration & Action

 

 

‘

Natiomal Council Meeting

Consideration & Action

 

r

a

Meeting between DRMP
Staff and Applicant
Representatives

 

 

Summation of all comments derived from the
review process about particular activities

py be

yl.

Actions:

' Recommendations

a. Approval

b. Approval with conditions

ce. Deferral

d. Return for revision

e.  Disapproval

2. Instructions to Staff

3. Recommendation of an overall
grant amount based on discussion
of specifics of the application

Actions:

Recommendations

a. Approval

b. Approval with conditions

ec. Deferral

d. Return for revision

e. Disapproval

2. estavetions to Staff .

3. Recommendation of an overall
grant amount

\

Applicant action:

Submission of revised proposal
within recommended overall budget
ceiling utilizing the comments and
criticism resulting from the
review process, '
Staff review of
revised Proposal

 

weap] Final Award Decision Action:

&. Award of Grant or
b. Further negotiation
with applicant

1 seta aseney,

 
it.

ISSUES IN REVIEW OF OPERATIONAL APPLICATIONS

Characteristics of first operational proposals

A. Many projects contained in each complex proposal

B. Sizable budget requests; including large hardware requests

Cc.

Commitment of effort

Relevant factors for judgment: Regional Medical Program vs Collection
of projects “

A.

Overall leadership and guiding philosophy

1,

Is there a unifying conceptual strategy which will be
the basis for initial priorities of action, evaluation,
and future decision making? Are there sufficient
feedback loops in the strategy?

Is there an administrative mechanism which can:

a. take decisions
b, relate to regional needs :
c. stimulate cooperative effort among major health interests

Are the key leadership persons identified? Can they work
with the major health interests? Do they have experience
and skills appropriate for providing leadership to a
complex endeavor? “

Is there involvement and commitment of the major health interests

Medical schools
Practicing physicians
Hospitals

Public health agencies

m& Oo Oo om

Will the ongoing planning process interact with the first
operational steps in the development of a program that meets
the broader needs of the entire region?

Nature and adequacy of cooperative arrangements

Relationship of projects to each other and to the total regional
medical program
iV.

¥.

Quality standards

A.

Highly specialized nature of some projects
Variety of projects included in each proposal
Regional vs. National standards

Block grant support

Degree of detailed justification expected
Type of evaluation necessary for final action

Emphasis on grantees own evaluation mechanisms as quality
uplifting factor at regional level

Criteria for judging appropriateness of support

A.

B.

Cc.

Scope and limitations of Regional Medical Programs legislative
authority

Availability of other sources of support

Priority on innovative and leverage effects

Criteria for judging level of support

A.

Geographic distribution - Should consideration of availability
of funds for later proposals be a part of decision on amount
awarded to first applicants

Partial or phased support as mechanism for:

1. Allowing fuller development of plans before proceeding
to fuller implementation

2. Permitting better decisions on distribution of funds
3. Early review of progress

Need to support “critical mass" of activity which will have a
sufficient impact to permit evaluation of results

Support of costly activities as national or interregional resources
when justified by the involvement of unique capabilities in a
specific Regional Medical Program

Extent of need for support of operational activities as necessary
for further development, extension, and solidification of regional
cooperative arrangements
Vt.

Vil.

VIIt.

Ik.

Length of commitment
A. Degree of emphasis to be placed on self-limiting nature of projects

B. Need for long range commitment for "core" activities which are
essential investment for conduct of specific projects

Relationship of operational proposals to ongoing planning activities
A. Need for documentation of reiationship

B. Extent of prior planning and its relationship to proposed operations
and continued planning

C. Extent to which needs of periphery of the region need to be
documented as basis for undertaking operational activities

Need to spell out relationship with adjacent regions

Adequacy of administrative arrangements, including fiscal accountability
of grantee
1.

Initial Staff Information re

TENTATIVE OUTLINE OF

OPERATIONAL GRANT REVIEW AND APPROVAL PROCESS |

 

Operational Grant
Application Received

{ {

 

 

 

 

 

 

 

 

Vv v
Staff ‘Review Comnittpe
Members
N/
Ne Nie $

 

a. Pianning grant experience ————— | Review Committee Meet ing

b. Gross technical validity

c. Policy issues

d., Relationship to other
Federal programs

Review Committee Information &

Comments

Review Committee Guidance ——_—_—_——

(Prepared 2nd day by site team)

 

Comment

 

Y
—— I.
Ps

3.

Site Visit
(fwo days)

 

 

 

_—_ OS Guidance for Site Visit Team

Judgments re:

Concept of RMP
Cooperative Arrangements
Relationship of projects,
one to another and to the
total

Approximate magnitude of
support warranted
Quality of projects where
appropriate
 

In addition to application and ——~}> Review Committee Meeting SS Actions:
. s eens A *

vee satel i for , 1. Recommendations

. zona Consideration & Action © a. Approval

 

 

from applicant
2, Expert review re quality b. Approval with condi-

of project from outside tions

DRMP, where indicated; c. Deferral

may have necessitated d. Return for revision

technical site visit on Oy Disapproval

specific project(s) 2. Instructions to Staff
3. Further Staff information
4. Site visitor(s)

 

 

 

 

 

Vv

 

 

 

4 . .
In addition to above: —_—+ National Council Meeting ———> Actions: .
1. Review Committee recommenda- for 1. Recommendations
tions . or . . a. Approval
a t
2. Further Staff information a Consideration & Action b. Approval with condi-
per Committee instructions tions

ec. Deferral
d. Return for revision
e.- Disapproval

2, Instructions to Staff

January 1967
(Rerun April 17, 1967)