|
Results-Based Accountability™
(RBA) and Outcomes-Based Accountability™
(OBA) |
|
|
Home Publications
Papers you can read on line
Workshops Links to other sites
New
Stuff Mark's Page Feedback
FPSI:
1 July 2010 Results-Based AccountabilityTM (RBA) Masterclass Cardiff 22nd April 2010 Questions for Mark Friedman Delegates
were asked to discuss in groups one question which they would like to ask Mark
Friedman. The following questions will be sent to Mark who has agreed to
respond:
RBA is specifically designed so that it can be broadly and flexibly applied. I would always argue for understanding principles over rigid fidelity to RBA or any other model. Having said that, there are well thought out reasons for the conventions offered in RBA. For example, the difference between population and performance accountability reflects real differences in the nature of efforts to improve quality of life as opposed to delivering services. It is not a distinction you could set aside and still say you were using RBA. Other parts of RBA, such as the Population and Performance talk to action 7 questions are central to the thinking process of RBA/OBA, and are best preserved in their original form, with commentary or supplemental material shown in parentheses. Most other material can be supplemented, adjusted, reordered or rephrased as circumstances dictate. A famous music teacher was once quoted as saying that you should learn the rules of harmony so you can decide when not to use them.
See
the discussions of subpopulations on pages 39 and 99 of “Trying Hard Is Not
Good Enough.” One of the principle distinctions between population
accountability and performance accountability is the service/customer
relationship at the heart of all matters of service performance. Any time a
group is defined in terms of its receipt of a particular service, it is a
customer population and therefore part of performance accountability.
// Within performance
accountability, there are higher/lower or larger/smaller customer populations.
For example, the customer population of a particular service is smaller than the
customer population of the whole agency which is smaller than the customer
population of the service system to which the agency “belongs.”
// In population accountability, populations
and subpopulations can be defined in almost any way you like. In every case this
definition must involve a geographic area. So for example, we could consider the
well-being of the population of all people in Wales, all children in Wales, all
children in Cardiff, all children 0 to 5 in Cardiff, etc. RBA can be used for
whole populations or subpopulations depending on the circumstances. //
One place where subpopulation considerations are also important in RBA is
in the “Story behind the baseline.” If we are trying to understand the rate
of teenage pregnancy, or school failure, it may be useful to examine how these
problems are concentrated in certain areas or certain subpopulations so that we
can tailor our efforts to greatest effect. //
Another interesting part of the population / subpopulation discussion is
the fact that “target populations” are part of population accountability.
They represent the potential group from which actual customers are drawn, who
then become part of the customer population.
// The conclusion from all
this is that there are not strict
rules about how many levels there are in population and performance
accountability. It’s as many levels as are needed.
There
is a long history to confusion about “outcomes.” The first matter of
confusion has to do with language itself. What is meant by an “outcome” or a
“result.” I believe that the language discipline in RBA provides a template
for how to reach agreement on some form of common language. In it’s most basic
form, you only need agreement on how to label six ideas in order for people
across government and the voluntary sector to talk to each other in a
disciplined way. Those six ideas are presented on the Definitions slide of the
RBA 101 presentation and pages 19 and 20 of the book. The words I most often use
to label these ideas in Wales are (Results, Indicators, Performance Measures,
How much did we do? How well did we do it? Is anyone better off?)
// Having said that, you can use RBA as a diagnostic tool to see
if people have gotten ideas and words mixed up. It is not uncommon for the word
“outcome” to be attached to several different ideas. Or for a single idea to
have several different words associated with it. For example, in some of the
original work on ECM in England, they used a two tier outcome structure at the
population level, but they called the two level of outcomes by different names
(outcomes and aims). When you look at the content of these two differently named
things it turns out they are all quality of life statements and therefore they
are all population “results” or “outcomes.” // In this case ECM was proposing a two tier result or outcome
structure at the top. Two and sometimes even three tier systems are not
uncommon. But on page 53-54 of the book I show how you can often simplify the
structure by changing the third tier into indicators. //
It is likely that the sorting out you are thinking about has to do with
first making clear the distinction between population and performance
accountability and then finding a disciplined way of using language to label
important ideas like those six discussed above.
Again,
I am not familiar with the Welsh Performance Management Framework. Why don’t
you send it to me to look at. // It
is often important to get the auspices right under which a written review is
produced. It is always best if the review is requested by the relevant
authority. Where this can not be done, it is possible to create an analysis that
others can use to influence decisions in the right direction. //
Still another alternative is for a resident RBA expert in Wales to
analyse this framework and draft something for others to look at. I am very
committed to people building their own capacity to do this work.
// I have to be careful
about promising written comments on any complex document. Perhaps, after looking
at this, the questioner’s purposes could be met with a conference or web call.
This
is a complex question, but here is a simple answer. Most other frameworks (1) do
not clearly distinguish between population and performance accountability, (2)
often have a large and confused set of performance measures, in contrast to the
three plain English ones used in RBA (How much did we do? How well did we do it?
Is anyone better off?), and (3) many other frameworks are unwieldy,
paper-intensive processes that are not actually useful to the people who have to
fill out the forms, in contrast to the RBA 7 talk to action questions.
// All people who have invented these kind of frameworks, including me,
are essentially trying to do the same thing, that is provide tools to help make
the world better. So comparing frameworks is a matter of judging which of these
approaches works best for you. You might want to look at page 135 of “Trying
Hard Is Not Good Enough” where I discuss “Comparing Frameworks,” and
suggest that you judge alternatives by how well they measure up against some
simple criteria: “simple, common sense, plain language, minimum paper and
useful.” //
There are, of course, other differences between RBA and traditional
frameworks, including the use of baselines to judge progress, the use of targets
only when they are fair and useful, and the emphasis on no-cost and low-cost
ideas. But let me leave it at that for now.
This
is a question about the reliability of data. There are basically two ways to
know something in the social sciences. You can observe it (e.g. a person got a
job or didn’t, was re-admitted to hospital within 30 days or wasn’t) or you
can get some form of opinion from the customer, worker or third party. (e.g. the
young person thought the mentoring program helped him or her do better in
school, the older person thought the in-home attendant was kind, caring and
helpful, the teacher thought the child was making progress on social skills, the
parent reported better relations with his or her child etc. ) We sometimes refer
to reported opinion as “soft” data. And there are known problems associated
with interpreting customer satisfaction data. But this does not mean that it is
not important or does not have a place. My recommendation is always to use
“soft” data in conjunction with data based on observable facts.
This
answer may sound a little odd. But imagine the most important measure you could
conceivably have. Now you have two choices. You can create a
“pseudo-baseline” as we do in the Turn the Curve exercise, where we estimate
(1) where we are now on this measure, (2) what we think the history has been.
Has it been getting better or worse or about the same?, and (3) where do we
think it’s headed in the future if we don’t do something more or different
than what we’re doing now. This
working version of a baseline can be used to drive the basic discussions in
getting from talk to action in RBA, namely, What is the story behind the
baseline? Who are the partners with a role to play in doing better? What works
to do better? and What do we propose to actually do?
If for some reason you don’t want to create a pseudo or working
baseline, then you can simply reach a consensus on whether things are getting
better or worse. In this case the direction of the baseline is used to drive the
turn the curve discussion. There is a third somewhat more complex variation of
this presented on pages 128 and 129 of the RBA book.
// The most extreme variation would set data aside altogether
and ask What is the purpose of the service? How are we doing and why?
Partners…..What works….Action Plan. If this helps, then use it. But this
should make you curious about how to know with greater certainty the answer to
the “How are we doing?” question. This inevitably requires some form of
data. So create a Data Development Agenda and begin gathering the data you need
to manage and improve the service. Workers in the organization will generally be
more agreeable about gathering data if they can see from the beginning that it
is needed to answer a fundamental question about the effectiveness of their
work. Be careful not to gather more data than you need. Many data systems place
unnecessary burdens on workers and actually detract from service delivery. Data
systems should be useful to the line workers who generate the data. You might
want to look into case management software like that of RLG’s partner Social
Solutions. // The principle here is NOT to wait for data or any other
precondition to get started. Get started with what you have. Make informed
guesses about what you don’t have. Be deliberate about making the process
better and more complete over time. Hope
this helps. Mark Mark Friedman
|
