This paper summarises TLRP's present thinking on the nature of outputs of various
sorts. It is an evolving document, benefiting from the advice of
the Publications and Other Outputs Group (POOG) and further versions
may be expected as things evolve. The paper should be read in conjunction
with the Programme's long-established Communication and Impact Plan,
with its conception of impact as a collaborative, user-engaged process
which is 'interactive, iterative, constructive, distributed and
transformative'. This text can be located within the terms of that
plan, but has relatively practical concerns.
The paper seeks, in particular, to:
- set out some principles of a TLRP strategy for outputs;
- offer a framework for thinking about possible outputs;
- begin to clarify the nature and possibilities offered by
each form of output;
- identify priorities for Programme and project development,
bearing in mind suitability, effectiveness, available resources
- clarify the responsibilities of projects and the Programme
as a whole.
The paper may helpfully be read in conjunction with ESRC's excellent series of
booklets on achieving impact (available from External Relations).
- Developing a Media Strategy (Romesh Vaitilingam)
- Television and Radio: A Best Practice Guide (Ivor Gaber)
AN OVERALL STRATEGY - TARGETING AND COLLABORATION
A recent ESRC review of the views of centre and programme directors regarding
effective impact activities concluded in the following terms:
'In general, positive initiatives have been based on targeted activity that understands
and delivers for its intended audience, some examples being specialist
conferences for practitioners, magazines and briefings produced
for key audiences. Websites are frequently highlighted for suitability
in posting research papers. Particularly innovative and positive
experiences included use of art to illustrate issues, shows and
postcards. Writing pieces direct for the media have been successful
for some. Careful collaboration with suitable partners in activities,
sometimes with journalists and other specialist intermediaries,
using partners' networks, connections and resources can clearly
work well' (Alsop and Ham, 2002).
Every TLRP project, network or other activity has unique features. A differentiated
publication/output strategy is thus likely to be most effective
in maximising impact. We also want to fully benefit from the energy,
enthusiasm and specialist expertise of researcher/user teams. All
TLRP teams should therefore feel encouraged to publish and produce
outputs in whatever ways they collectively feel are most appropriate
for their key findings, circumstances and audiences. Let a thousand
flowers bloom! Indeed, all TLRP investments already have their own
impact and communication plans and are funded to fulfil these. These
plans catalogue an impressive range of ideas and commitments, and
many more are being developed as work unfolds. New initiatives may
be supported further by TLRP's scheme for supplementary funding
for user engagement, communication and impact.
Additionally however, there are significant benefits from the presentation of
degree of overall Programme coherence. Among these benefits are
the establishment of a quality expectation and brand recognition.
Increased effectiveness and economies of cost and effort should
also be possible through a degree of collective organisation for
impact. The Programme Office aims, for example, to develop a data-base
capable of very specific targeting for mail-shots to get network
and project findings to key users, journalists and other interested
individuals and organisations.
The Programme's publication and output strategy should thus encourage and support
targeted diversity, whilst also specifying minimal coordinated expectations
and maximising the benefits of collaboration.
EXISTING OR POTENTIAL OUTPUTS
The tables following provides overviews of the main project outputs which have,
so far, been considered within the Programme. There are a lot of
them, reflecting the rich range of ideas which are available. To
impose some sort of order, they have been located, very approximately,
within an organising matrix structured by three factors:
- the stage during each project when the output is most
likely to be used ('conducting research' or 'sharing knowledge');
- the main audience for the type of output (along a
public/user - academic/technical spectrum);
- the form of the output (electronic, paper, events,
Two general points may be made.
First, that user engagement is regarded as making a very significant contribution
at all stages of all projects.
Second, that the identification of 'key findings' is taken to be a crucial strategic
turning point in directing the outputs strategy for all projects.
Project teams will of course devise their own engagement and impact strategies,
producing a particular portfolio of outputs and reporting on this
annually. However, within the overall provision, the Programme expects
the following to be normally included. As appropriate, direct support
in achieving this will be provided.
- 'Gateway web pages'
- Appropriate user representation on Advisory Groups
- Media training of a nominated spokesperson
- Project website
- Use of the British Education Index
- Identifying 'key findings'
- Cross-Programme user events
- (soon) multimedia presentations of key findings
- Pro-active press strategies
- 'Briefings' for policy makers and others
- 'Improving Learning books' through Routledge
- Journal papers through Routeledge
- Participation in targeted academic conferences
The nature of each type of output, together with the Programme's expectations
and support, is described briefly in the text which follows. Where
TLRP has particular expectations, supplementary information is also
likely to be available.
'Public and User audiences'
The 'conducting research' stage
Forms of project output
1a Regard registration
2a Project newsletters
2b Articles in popular outlets
3a User engagement events
4a Reactive press strategies
1c Gateway web page
2c Articles in professional outlets
3b Advisory Group events
4b Media training
5a Project website
7a Inter-project activities
5b Website working papers
6a Conference and working papers
7b Thematic Group activities
5c British Education Index
7c Systematic reviews
Public and User Audiences
The 'sharing knowledge' stage
Forms of project output
11a Key findings
8a User e-dissemination
9a User dissemination
10a User events
11b User championing
9b Teaching materials
10b Cultural innovation (drama/art)
11c Pro-active press strategies
8b Multimedia presentation clips (soon)
10c Policy discussion and commentary
11d Press releases
8c Discussion forums
10d Scenario planning
11e Press interviews
8d e-Imroving learning books
9e Improving Learning books
10e Consensus conferences
13a Journal papers
14a Academic conferences
13b Academic & technical books
13c Final report to ESRC
CONDUCTING RESEARCH: PUBLIC AND USER AUDIENCES
1a. 'Regard' registration (required by ESRC)
Registration on this ESRC database is a requirement before the award is finalised.
As progress is made and publications come out, updating is required.
Updating is also requested following the end of the award.
1b. CERUK registration
This educational research database is maintained by NFER and all on-going Programme
investments should be registered on it. CERUK is increasingly used
by press and policy-makers in reviewing work in progress.
1c. Gateway web page (expected by TLRP)
This is a simple project description provided on the TLRP website and is a key
requirement for all networks, projects, research fellows, etc. The
Programme's standard template must be used. More complex and individually
designed 'developed' websites can be accessed through this summarising
2a. Project Newsletters
These provide effective dissemination to targeted users and have been very well
used by some teams. Cost can be significant and distribution systems
need consideration. Design challenges fall on project teams.
2b. Articles in popular outlets
Material for 'lay' audiences in mass circulation magazines etc (eg: Good Housekeeping).
This could be an extremely effective form of dissemination for those
with journalistic skills who can place a story.
2c. Articles in professional outlets
Material in particular 'professional' or 'trade' publications designed for a
specific audience group (eg: TES, THES, Management Today). A very
valuable way of targeting dissemination of information about projects
and/or findings. For example, the DfES publications for primary
and secondary teachers is distributed to every teacher in England.
3a. User engagement events
Projects are likely to want to hold events for directly participating users as
work unfolds, and there are many successful examples of this. Gradually,
as progress is made, a wider circle of user participants may be
drawn into such events.
3b. Advisory Group events (expected by TLRP)
It is clear that Advisory Group meetings can be effectively used to keep key
users informed and involved in the detail of project development.
They also, of course, fulfil many other purposes. All projects are
expected to have appropriate Advisory Groups, with significant user
representation. This should normally include some colleagues from
research sites (providing grounding) and some from national bodies
(offering potential impact leverage). It may also be helpful to
invite 'critical friends' allocated from the TLRP Steering Committee
to be members of the Advisory Group.
4a. Reactive press responses
Many colleagues no doubt get called by journalists from time to time in the hope
of quick comments on issues of the day. We may be able to develop
strategies for managing this, such as calling on known experts and
involving users. Over time, it may be possible for the TLRP office
to act as a liaison point between journalists and experts within
4b. Media Training (expected by TLRP)
As projects move towards their conclusion, TLRP will arrange for a spokesperson
from each project to receive ESRC media training at no cost to the
project. This should ensure that there is somebody who is equipped
to take on a higher profile in explaining the key findings of the
project, if called upon to do so, and the training should help in
developing the project's impact strategy. Colleagues should consider
who they wish to participate in the training. There is, of course,
no constraint on the number of people who actually fulfil the role
CONDUCTING RESEARCH: ACADEMIC AND TECHNICAL AUDIENCES
5a. Project websites (expected by TLRP)
This is specialised website which all projects are expected to establish and
maintain during the period of their funding and, as appropriate,
5b. Website working papers, etc
Working papers and other materials may be posted on project websites to maximise
access. However, where the material is appropriate, TLRP would prefer
deposition in the BEI (see below) with a direct link back to the
project website. This will ensure that materials remain in the public
domain even if the project website is eventually withdrawn.
5c. British Education Index (expected by TLRP)
This database provides access to a very wide range of 'grey' research literature.
It includes Educationline and depositions from many major
UK conferences, including BERA. All material is professional indexed
and will be maintained indefinitely. TLRP has a partnership agreement
with BEI and links can be provided with project websites. There
are also good links to Carfax journals. A paper on the details of
the partnership arrangements will be available in due course.
6a. Conference and Working papers
These are important forms of output and often build into more substantial later
works. Hard copies may be distributed more widely beyond those attending
any particular event, and the TLRP Programme Office may be able
to help with this. Once produced, the expectation is that such papers
should be made electronically available through the BEI with links
to the project website.
7a. Thematic Group activities
Thematic Group activity is expected to become increasingly important as the Programme
progresses and, in part, is likely to be targeted to maximise cross-Programme
impact in relation to contemporary issues. The participation of
project colleagues in this will be crucial.
7b. Inter-project activities
All forms of inter-project activity, whether for substantive, theoretical or
methodological purposes are to be encouraged. Sharing in this way
represents a form of internal output, helping to develop the Programme
as a whole. Additional funding is available to support such activity.
7c. Systematic reviews
All TLRP projects are based on a comprehensive review of literature. However,
some have gone further and participated, either directly or indirectly,
in systematic review work using EPPI or Campbell guidelines. This
may be a valuable way of strengthening the foundation of project
contributions and the review may form part of a project's portfolio
of outcomes. Colleagues in EPPI are engaging in valuable work on
how to assess the quality of products from a wide range of research
traditions and how to appropriately integrate such findings. It
is expected that TLRP will want to support such activities both
formatively and, if appropriate, as a way of contextualising the
outcomes of completed projects.
SHARING KNOWLEDGE: PUBLIC AND USER AUDIENCES
Key findings (expected by TLRP)
This topic is emphasised across the matrix because it is absolutely central to
the entire impact strategy of any project. Key findings are,
in part, an outcome of intellectual and empirical struggle to understand
the research, and in part, they represent a strategic presentation
of such findings in terms of contemporary issues. In any event,
they represent the main messages which are to be conveyed through
a project's impact strategy.
Project reports, required annually, ask for an indication of anticipated key
findings. These are likely to become progressively precise, so that
the final impact strategy can be clearly targeted. Briefings, multimedia
clips, pro-active press strategies, press releases and Gateway books
are seen as specific vehicles for key findings.
8a. User e-dissemination
Some of our key users (eg: the National College for School Leadership) have excellent
electronic dissemination systems which can be used to cost-effectively
reach very large numbers of targeted potential research users. This
is something well worth exploring.
8b. Multimedia presentation clips (soon, expected by TLRP)
The Programme intends to support the preparation of multimedia presentations
including short clips of video material designed to illustrate key
findings from each project in vivid ways. These presentations
will be accessible from the Programme website and made available
for other uses too. They may, for example, be incorporated within
academic presentations, used in teaching materials, or offered to
broadcast media to illustrate key points being made. The intention
is to create simple, visual interactive episodes which memorably
crystallise key issues and/or findings. Issues concerning the appropriate
use of images of children are being investigated.
Subject to final approval, a team from Bristol's InterActive project, with some
support from Cambridge, is expected to provide technical support
for this. However, editorial control must, of course, rest with
8c. Discussion Forums
TLRP is able to provide the technical support for any project which would like
to run a discussion forum in association with their website. However,
an appropriate member of a project team would need to stimulate
and manage the discussion.
8d. e-Improving Learning books
This is a standard output for gateway books published under the RoutledgeFalmer
imprint following our partnership agreement with Taylor & Francis.
No additional work is required of authors.
9a. User dissemination
This involves users offering their own internal communication systems: journals,
newsletters, bulletins, websites, etc, to disseminate project outcomes.
It is likely to be extremely valuable, particularly if high-leverage
national user organisations have been engaged and if dissemination
of project findings fulfils aspects of their own goals. The Programme
Team may be able to help in identifying opportunities for this (eg:
9b. Teaching materials
Teaching materials are a very important form of output and could have a significant
impact. They have been mentioned by a significant proportion of
the users with whom the Directors' Team have been in discussion.
Where teaching materials are produced, there is an expectation that TLRP badging
will be used by publishers on covers, etc. The TLRP office will
negotiate this with major publishers and with others at the request
of project teams. See below on intellectual property rights.
9c. Teacher training materials
This is a specific form of output, encouraged by the TTA and other agencies and
providers. An excellent form of output for a longer-term impact.
9d. Briefings (expected by TLRP)
These are simple four-page summaries of outcomes directed at policy-makers, journalists
and other interested users and intended as a effective means for
achieving rapid and highly targeted impact.
Text from projects will need to fit a template, and the TLRP office will coordinate
printing and distribution, using an agreed, targeted distribution
strategy. Suggested template: p1, TLRP banner, project title, paragraph
summary, bullet points of key findings; pp 2-3, details of findings;
p4, methodological and warrant information, contact details.
The number of Briefings deriving from a network/project/CDA/RTF might depend
on the nature of the key findings identified. From ESRC resources,
it is hoped to provide assistance from a journalist to support this
The targeted distribution strategy is dependent on effective integration of cross-Programme
user information, and work on this will be continuous.
Printing and distribution might be funded, in whole or in part, from central
These are seen as A5 size, 12/16/20/24 pages, overarching summaries from projects
aimed more at those who would appreciate more detail and discussion.
(cf: SEED Interchange series, 'Inside the Black Box' successes).
TLRP will provide design, print and distribution services for such
publications (but costs will need to be part-funded from project
9f. Improving Learning books (expected by TLRP)
A Programme series, directed at 'intelligent, interested, user audiences' has
been established to provide accessible overviews of projects and
to show the relevance of key findings. The series title is
Each book is expected to be relatively short (normally, up to 65,000 words) and
should be written in an accessible style. It should act as an overview
for the network or project as a whole and as a gateway to its more
detailed outputs. In particular, each book should aim to demonstrate
the combination of high quality research on topics of relevance
to policy and practice in teaching and learning. It is envisaged
that each network and project will contribute to this series, with
CDAs and RTFs doing so where appropriate. Additionally, contributions
from outside the Programme could be considered where they could
add value as a whole volume or as an element within a Programme
10a. User events
It is understood that networks and projects have already planned to hold a large
number of user events.
However, there is a place for high-profile, coordinated, cross-Programme events
which could draw in a significant number of users and generate publicity
for both individual networks/projects and the Programme as a whole.
The exact nature of such events needs a much more discussion, but
the first could possibly be based around the Phase 1 networks.
It is possible that some form or road show might be developed.
10b. Cultural innovations
This broad category encompasses a range of creative events, drama, art, tv, radio,
media, etc which might bring issues and findings to new groups of
users in interesting new ways. This could be hugely exciting and
is well worth future brainstorming, discussion and follow through.
Networks and projects may create highly divergent and engaging forms
At a Programme level, a Thematic Group on Transformation and Impact will be considering
this, among other issues, including the idea of using biographical
narratives to convey key findings across the lifecourse.
10c. Policy discussion and/or commentary
This output offers information and specialist advice to user organisations. The
benefit of good user-relationships is, potentially, to be trusted
to contribute to discussion of issues, policy formulation or evaluation
in specialist fields. The Programme Team will facilitate this using
their sectoral contacts when they can, but the expertise for membership
of delegations is likely to lie with networks, projects, etc.
10d. Scenario planning
This approach will be explored as a structured way of using project insights
to illuminate key long-term issues, and thus of supporting policy
making processes within user organisations. In some ways, scenario
planning is seen as enabling project teams to play to the analytic
strengths of academic work, rather than attempting to respond to
immediate policy imperatives.
10e. Consensus conference
This is an established approach in fields such as medicine - try it in 'google'.
It involves a focused conference event with systematic gathering
and interrogation of multiple expert opinions in relation to a particular
issue, and the drawing out of collective, best-judgement, evidence-informed
conclusions. Although this approach has not often been used in education
(as far as we are aware), it seems very appropriate for a field
which has to deal with great complexity and uncertainties. It is
therefore something which TLRP might try at an appropriate point.
11a. Key findings
As indicated above, these are intended to be the most significant intellectual
outcome of each research activity, and the organisational lynch-pin
of impact activities. Clarity about key findings is crucial to the
production of Briefings, Gateway books and pro-active press strategies.
Not easy to achieve such clarity though.
In due course, the accumulation of key findings from networks, projects, etc
will be used to build an account of the achievements of the Programme
as a whole. Through the search for any continuities or common issues,
we hope to be able to demonstrate added value and also add to our
11b. User championing
The involvement of research users has an enormous impact in offering credibility
to academic analyses during dissemination. If user engagement has
been continuous though a project's work, and user colleagues are
prepared to explain the work and advocate its findings, then this
is likely to be of considerable benefit in dealing with the press
or sharing findings with other users. Roles of project 'champions'
should be negotiated and nurtured over the long-term so that there
is a trusting relationship and excellent understanding of the research
and its relevance.
11c. Pro-active press strategies (expected by TLRP)
This concerns the managed promotion of key findings.
It is suggested that, when significant key findings have been identified, a pro-active
press strategy is developed, including the use of press releases
and briefing of key journalists. In the terms of the HEFCE-ESRC
contract regarding TLRP, copies of press releases should be sent
to HEFCE for information before going out (this is not for vetting).
In fact, we recommend a constructive approach to funders and government
agencies, preparing and informing them about imminent press initiatives.
Whilst maintaining our independence, it is not TLRP's policy to
attempt to wrong-foot agencies with whom we otherwise have cooperative
The Programme expects to be able to help with this. The Directors' Team will
develop journalistic links in their sector and the Office Team will
assist with the production and targeted distribution of press releases.
Media training will be offered to project representatives. ESRC
will fund a media fellow to advise on specific project issues.
11d. Press releases
Clearly a crucial part of any impact strategy, designed to communicate key findings
in relation to contemporary issues. See ESRC's pamphlet on Developing
a Media Strategy.
11e. Press interviews
An important skill, providing opportunities to communicate key findings in relation
to contemporary issues. See ESRC's pamphlet on Television and
SHARING KNOWLEDGE: ACADEMIC AND TECHNICAL AUDIENCES
This is an increasingly common medium for dissemination, and will be provided
as part of the TLRP partnership agreement with Taylor & Francis.
13a. Journal papers (expected by TLRP)
These are the traditional range of peer-refereed outputs. Such papers may be
more honed versions of working papers previously made available
on websites. If using Taylor and Francis journals, which are covered
by the TLRP/Routledge partnership agreement, then electronic versions
of the journal will be able to offer direct links to project websites.
Textual acknowledgement of TLRP/ESRC support is required within
all published papers, with logos included if publishers agree.
13b. Academic books
For academic audiences, these might include detailed theoretical, substantive
or methodological accounts and analyses, or other volumes.
There is an expectation that TLRP badging will be used by publishers on covers,
etc. The TLRP office will negotiate this with major publishers and
with others at the request of project teams. See below on intellectual
13c. Final report to ESRC (ESRC requirement)
Final reports are an important academic output, and will be posted on Regard.
They are due for submission three months after the end of funding.
14a. Academic conferences
(expected by TLRP)
Presentations at academic conferences are well established for most individual
networks and projects. Coordinated, targeted TLRP presentations
are being developed for BERA, AERA, EERA, EARLI, etc and other major
events to 2008/9. There is scope, in due course, for more sectoral
presentations (eg: SRHE) or for issue-based coordination of presentations.
Badging (required by TLRP)
Wherever possible the dual TLRP/ESRC logo should be used as a right-hand side
footer. Where this is really not possible (for example, in a journal
publication), a simple text acknowledgement of the Programme should
be included in an appropriate place. For example: 'Note 1: this
project is part of ESRC's Teaching and Learning Research Programme
(see www.TLRP.org )'.
It is understood that website design might require the logos to
be placed elsewhere too.
Some further guidelines on TLRP presentation are being produced by a professional
designer. They will be used for all cross-Programme outputs. Selected
elements will be shared with projects as soon as they become available.
Intellectual property rights
TLRP's concern for impact and sustainability through user-engagement could call
for innovative arrangements with both commercial and non-commercial
user organisations that can maximise and sustain impact.
According to the formal ESRC guidance, intellectual property rights arising from
ESRC awards should be 'clear from the outset'. They are normally
taken to be assigned to the institution receiving an award. However,
in social science fields where commercial values tend to be low,
institutions normally pass these to research teams. Even so, it
is prudent for research colleagues to agree protocols for the appropriate
attribution of authorship and recognition of all contributions made.
Agreement of principles should be reached early in the duration
of the project.
Regarding the distribution of any royalties or other income, ESRC again suggests
that 'the basis of sharing needs to be clear from the outset'. They
state their expectations as follows:
Whoever holds the intellectual property, it should be clear what the distribution
of any income might be. For income up to £10,000 the ESRC
assumes that this will normally rest with the principal investigators
and the research team, unless otherwise stated to the contrary.
For income between £10,000 and £50,000 the ESRC assumes
that the major part will accrue to the university or institution,
but the ESRC reserves the right to reclaim up to one third of the
total, up to the value of its original award. For income over £50,000
the ESRC requires that the major part of any income should accrue
to the university or institution, and again reserves the right to
reclaim up to one third of the total up to the value of its original
award (Funding Guidelines 19.2.2).
It will be nice if some colleagues have to worry about this issue - and some
commercial forms of high-impact, user-engaged activity (eg: teaching
materials) might well generate significant surpluses. Hopefully,
such profits may actually underpin sustainability, and thus improve
If any network or project thinks that more than £10,000 in surplus income
might be generated from their TLRP funded work, they should contact
ESRC for discussion.
Comments on this paper are very welcome and should be sent to Andrew