Outputs Portfolio –

INTRODUCTION

This paper summarises TLRP's present thinking on the nature of outputs of various sorts. It is an evolving document, benefiting from the advice of the Publications and Other Outputs Group (POOG) and further versions may be expected as things evolve. The paper should be read in conjunction with the Programme's long-established Communication and Impact Plan, with its conception of impact as a collaborative, user-engaged process which is 'interactive, iterative, constructive, distributed and transformative'. This text can be located within the terms of that plan, but has relatively practical concerns.

The paper seeks, in particular, to:

set out some principles of a TLRP strategy for outputs;

offer a framework for thinking about possible outputs;

begin to clarify the nature and possibilities offered by each form of output;

identify priorities for Programme and project development, bearing in mind suitability, effectiveness, available resources and obligations;

clarify the responsibilities of projects and the Programme as a whole.

The paper may helpfully be read in conjunction with ESRC's excellent series of booklets on achieving impact (available from External Relations). For example:

Developing a Media Strategy (Romesh Vaitilingam)

Television and Radio: A Best Practice Guide (Ivor Gaber)

 

AN OVERALL STRATEGY - TARGETING AND COLLABORATION

A recent ESRC review of the views of centre and programme directors regarding effective impact activities concluded in the following terms:

'In general, positive initiatives have been based on targeted activity that understands and delivers for its intended audience, some examples being specialist conferences for practitioners, magazines and briefings produced for key audiences. Websites are frequently highlighted for suitability in posting research papers. Particularly innovative and positive experiences included use of art to illustrate issues, shows and postcards. Writing pieces direct for the media have been successful for some. Careful collaboration with suitable partners in activities, sometimes with journalists and other specialist intermediaries, using partners' networks, connections and resources can clearly work well' (Alsop and Ham, 2002).

Every TLRP project, network or other activity has unique features. A differentiated publication/output strategy is thus likely to be most effective in maximising impact. We also want to fully benefit from the energy, enthusiasm and specialist expertise of researcher/user teams. All TLRP teams should therefore feel encouraged to publish and produce outputs in whatever ways they collectively feel are most appropriate for their key findings, circumstances and audiences. Let a thousand flowers bloom! Indeed, all TLRP investments already have their own impact and communication plans and are funded to fulfil these. These plans catalogue an impressive range of ideas and commitments, and many more are being developed as work unfolds. New initiatives may be supported further by TLRP's scheme for supplementary funding for user engagement, communication and impact.

Additionally however, there are significant benefits from the presentation of degree of overall Programme coherence. Among these benefits are the establishment of a quality expectation and brand recognition. Increased effectiveness and economies of cost and effort should also be possible through a degree of collective organisation for impact. The Programme Office aims, for example, to develop a data-base capable of very specific targeting for mail-shots to get network and project findings to key users, journalists and other interested individuals and organisations.

The Programme's publication and output strategy should thus encourage and support targeted diversity, whilst also specifying minimal coordinated expectations and maximising the benefits of collaboration.

EXISTING OR POTENTIAL OUTPUTS

The tables following provides overviews of the main project outputs which have, so far, been considered within the Programme. There are a lot of them, reflecting the rich range of ideas which are available. To impose some sort of order, they have been located, very approximately, within an organising matrix structured by three factors:

the stage during each project when the output is most likely to be used ('conducting research' or 'sharing knowledge');

the main audience for the type of output (along a public/user - academic/technical spectrum);

the form of the output (electronic, paper, events, media-related).

Two general points may be made.

First, that user engagement is regarded as making a very significant contribution at all stages of all projects.

Second, that the identification of 'key findings' is taken to be a crucial strategic turning point in directing the outputs strategy for all projects.

Project teams will of course devise their own engagement and impact strategies, producing a particular portfolio of outputs and reporting on this annually. However, within the overall provision, the Programme expects the following to be normally included. As appropriate, direct support in achieving this will be provided.

'Gateway web pages'

Appropriate user representation on Advisory Groups

Media training of a nominated spokesperson

Project website

Use of the British Education Index

Identifying 'key findings'

Cross-Programme user events

(soon) multimedia presentations of key findings

Pro-active press strategies

'Briefings' for policy makers and others

'Improving Learning books' through Routledge

Journal papers through Routeledge

Participation in targeted academic conferences

The nature of each type of output, together with the Programme's expectations and support, is described briefly in the text which follows. Where TLRP has particular expectations, supplementary information is also likely to be available.

CONDUCTING RESEARCH: PUBLIC AND USER AUDIENCES

1a. 'Regard' registration (required by ESRC)

Registration on this ESRC database is a requirement before the award is finalised. As progress is made and publications come out, updating is required. Updating is also requested following the end of the award.

1b. CERUK registration

This educational research database is maintained by NFER and all on-going Programme investments should be registered on it. CERUK is increasingly used by press and policy-makers in reviewing work in progress.

1c. Gateway web page (expected by TLRP)

This is a simple project description provided on the TLRP website and is a key requirement for all networks, projects, research fellows, etc. The Programme's standard template must be used. More complex and individually designed 'developed' websites can be accessed through this summarising page.

2a. Project Newsletters

These provide effective dissemination to targeted users and have been very well used by some teams. Cost can be significant and distribution systems need consideration. Design challenges fall on project teams.

2b. Articles in popular outlets

Material for 'lay' audiences in mass circulation magazines etc (eg: Good Housekeeping). This could be an extremely effective form of dissemination for those with journalistic skills who can place a story.

2c. Articles in professional outlets

Material in particular 'professional' or 'trade' publications designed for a specific audience group (eg: TES, THES, Management Today). A very valuable way of targeting dissemination of information about projects and/or findings. For example, the DfES publications for primary and secondary teachers is distributed to every teacher in England.

3a. User engagement events

Projects are likely to want to hold events for directly participating users as work unfolds, and there are many successful examples of this. Gradually, as progress is made, a wider circle of user participants may be drawn into such events.

3b. Advisory Group events (expected by TLRP)

It is clear that Advisory Group meetings can be effectively used to keep key users informed and involved in the detail of project development. They also, of course, fulfil many other purposes. All projects are expected to have appropriate Advisory Groups, with significant user representation. This should normally include some colleagues from research sites (providing grounding) and some from national bodies (offering potential impact leverage). It may also be helpful to invite 'critical friends' allocated from the TLRP Steering Committee to be members of the Advisory Group.

4a. Reactive press responses

Many colleagues no doubt get called by journalists from time to time in the hope of quick comments on issues of the day. We may be able to develop strategies for managing this, such as calling on known experts and involving users. Over time, it may be possible for the TLRP office to act as a liaison point between journalists and experts within the Programme.

4b. Media Training (expected by TLRP)

As projects move towards their conclusion, TLRP will arrange for a spokesperson from each project to receive ESRC media training at no cost to the project. This should ensure that there is somebody who is equipped to take on a higher profile in explaining the key findings of the project, if called upon to do so, and the training should help in developing the project's impact strategy. Colleagues should consider who they wish to participate in the training. There is, of course, no constraint on the number of people who actually fulfil the role of spokespersons.

CONDUCTING RESEARCH: ACADEMIC AND TECHNICAL AUDIENCES

5a. Project websites (expected by TLRP)

This is specialised website which all projects are expected to establish and maintain during the period of their funding and, as appropriate, thereafter.

5b. Website working papers, etc

Working papers and other materials may be posted on project websites to maximise access. However, where the material is appropriate, TLRP would prefer deposition in the BEI (see below) with a direct link back to the project website. This will ensure that materials remain in the public domain even if the project website is eventually withdrawn.

5c. British Education Index (expected by TLRP)

This database provides access to a very wide range of 'grey' research literature. It includes Educationline and depositions from many major UK conferences, including BERA. All material is professional indexed and will be maintained indefinitely. TLRP has a partnership agreement with BEI and links can be provided with project websites. There are also good links to Carfax journals. A paper on the details of the partnership arrangements will be available in due course.

6a. Conference and Working papers

These are important forms of output and often build into more substantial later works. Hard copies may be distributed more widely beyond those attending any particular event, and the TLRP Programme Office may be able to help with this. Once produced, the expectation is that such papers should be made electronically available through the BEI with links to the project website.

7a. Thematic Group activities

Thematic Group activity is expected to become increasingly important as the Programme progresses and, in part, is likely to be targeted to maximise cross-Programme impact in relation to contemporary issues. The participation of project colleagues in this will be crucial.

7b. Inter-project activities

All forms of inter-project activity, whether for substantive, theoretical or methodological purposes are to be encouraged. Sharing in this way represents a form of internal output, helping to develop the Programme as a whole. Additional funding is available to support such activity.

7c. Systematic reviews

All TLRP projects are based on a comprehensive review of literature. However, some have gone further and participated, either directly or indirectly, in systematic review work using EPPI or Campbell guidelines. This may be a valuable way of strengthening the foundation of project contributions and the review may form part of a project's portfolio of outcomes. Colleagues in EPPI are engaging in valuable work on how to assess the quality of products from a wide range of research traditions and how to appropriately integrate such findings. It is expected that TLRP will want to support such activities both formatively and, if appropriate, as a way of contextualising the outcomes of completed projects.

SHARING KNOWLEDGE: PUBLIC AND USER AUDIENCES

Key findings (expected by TLRP)

This topic is emphasised across the matrix because it is absolutely central to the entire impact strategy of any project. Key findings are, in part, an outcome of intellectual and empirical struggle to understand the research, and in part, they represent a strategic presentation of such findings in terms of contemporary issues. In any event, they represent the main messages which are to be conveyed through a project's impact strategy.

Project reports, required annually, ask for an indication of anticipated key findings. These are likely to become progressively precise, so that the final impact strategy can be clearly targeted. Briefings, multimedia clips, pro-active press strategies, press releases and Gateway books are seen as specific vehicles for key findings.

8a. User e-dissemination

Some of our key users (eg: the National College for School Leadership) have excellent electronic dissemination systems which can be used to cost-effectively reach very large numbers of targeted potential research users. This is something well worth exploring.

8b. Multimedia presentation clips (soon, expected by TLRP)

The Programme intends to support the preparation of multimedia presentations including short clips of video material designed to illustrate key findings from each project in vivid ways. These presentations will be accessible from the Programme website and made available for other uses too. They may, for example, be incorporated within academic presentations, used in teaching materials, or offered to broadcast media to illustrate key points being made. The intention is to create simple, visual interactive episodes which memorably crystallise key issues and/or findings. Issues concerning the appropriate use of images of children are being investigated.

Subject to final approval, a team from Bristol's InterActive project, with some support from Cambridge, is expected to provide technical support for this. However, editorial control must, of course, rest with project teams.

8c. Discussion Forums

TLRP is able to provide the technical support for any project which would like to run a discussion forum in association with their website. However, an appropriate member of a project team would need to stimulate and manage the discussion.

8d. e-Improving Learning books

This is a standard output for gateway books published under the RoutledgeFalmer imprint following our partnership agreement with Taylor & Francis. No additional work is required of authors.

9a. User dissemination

This involves users offering their own internal communication systems: journals, newsletters, bulletins, websites, etc, to disseminate project outcomes. It is likely to be extremely valuable, particularly if high-leverage national user organisations have been engaged and if dissemination of project findings fulfils aspects of their own goals. The Programme Team may be able to help in identifying opportunities for this (eg: ATL offer).

9b. Teaching materials

Teaching materials are a very important form of output and could have a significant impact. They have been mentioned by a significant proportion of the users with whom the Directors' Team have been in discussion.

Where teaching materials are produced, there is an expectation that TLRP badging will be used by publishers on covers, etc. The TLRP office will negotiate this with major publishers and with others at the request of project teams. See below on intellectual property rights.

9c. Teacher training materials

This is a specific form of output, encouraged by the TTA and other agencies and providers. An excellent form of output for a longer-term impact.

9d. Briefings (expected by TLRP)

These are simple four-page summaries of outcomes directed at policy-makers, journalists and other interested users and intended as a effective means for achieving rapid and highly targeted impact.

Text from projects will need to fit a template, and the TLRP office will coordinate printing and distribution, using an agreed, targeted distribution strategy. Suggested template: p1, TLRP banner, project title, paragraph summary, bullet points of key findings; pp 2-3, details of findings; p4, methodological and warrant information, contact details.

The number of Briefings deriving from a network/project/CDA/RTF might depend on the nature of the key findings identified. From ESRC resources, it is hoped to provide assistance from a journalist to support this work.

The targeted distribution strategy is dependent on effective integration of cross-Programme user information, and work on this will be continuous.

Printing and distribution might be funded, in whole or in part, from central TLRP resources.

9e. Pamphlets

These are seen as A5 size, 12/16/20/24 pages, overarching summaries from projects aimed more at those who would appreciate more detail and discussion. (cf: SEED Interchange series, 'Inside the Black Box' successes). TLRP will provide design, print and distribution services for such publications (but costs will need to be part-funded from project resources).

9f. Improving Learning books (expected by TLRP)

A Programme series, directed at 'intelligent, interested, user audiences' has been established to provide accessible overviews of projects and to show the relevance of key findings. The series title is 'Improving Learning'.

Each book is expected to be relatively short (normally, up to 65,000 words) and should be written in an accessible style. It should act as an overview for the network or project as a whole and as a gateway to its more detailed outputs. In particular, each book should aim to demonstrate the combination of high quality research on topics of relevance to policy and practice in teaching and learning. It is envisaged that each network and project will contribute to this series, with CDAs and RTFs doing so where appropriate. Additionally, contributions from outside the Programme could be considered where they could add value as a whole volume or as an element within a Programme book.

10a. User events

It is understood that networks and projects have already planned to hold a large number of user events.

However, there is a place for high-profile, coordinated, cross-Programme events which could draw in a significant number of users and generate publicity for both individual networks/projects and the Programme as a whole. The exact nature of such events needs a much more discussion, but the first could possibly be based around the Phase 1 networks.

It is possible that some form or road show might be developed.

10b. Cultural innovations

This broad category encompasses a range of creative events, drama, art, tv, radio, media, etc which might bring issues and findings to new groups of users in interesting new ways. This could be hugely exciting and is well worth future brainstorming, discussion and follow through. Networks and projects may create highly divergent and engaging forms of impact.

At a Programme level, a Thematic Group on Transformation and Impact will be considering this, among other issues, including the idea of using biographical narratives to convey key findings across the lifecourse.

10c. Policy discussion and/or commentary

This output offers information and specialist advice to user organisations. The benefit of good user-relationships is, potentially, to be trusted to contribute to discussion of issues, policy formulation or evaluation in specialist fields. The Programme Team will facilitate this using their sectoral contacts when they can, but the expertise for membership of delegations is likely to lie with networks, projects, etc.

10d. Scenario planning

This approach will be explored as a structured way of using project insights to illuminate key long-term issues, and thus of supporting policy making processes within user organisations. In some ways, scenario planning is seen as enabling project teams to play to the analytic strengths of academic work, rather than attempting to respond to immediate policy imperatives.

10e. Consensus conference

This is an established approach in fields such as medicine - try it in 'google'. It involves a focused conference event with systematic gathering and interrogation of multiple expert opinions in relation to a particular issue, and the drawing out of collective, best-judgement, evidence-informed conclusions. Although this approach has not often been used in education (as far as we are aware), it seems very appropriate for a field which has to deal with great complexity and uncertainties. It is therefore something which TLRP might try at an appropriate point.

11a. Key findings

As indicated above, these are intended to be the most significant intellectual outcome of each research activity, and the organisational lynch-pin of impact activities. Clarity about key findings is crucial to the production of Briefings, Gateway books and pro-active press strategies. Not easy to achieve such clarity though.

In due course, the accumulation of key findings from networks, projects, etc will be used to build an account of the achievements of the Programme as a whole. Through the search for any continuities or common issues, we hope to be able to demonstrate added value and also add to our collective impact.

11b. User championing

The involvement of research users has an enormous impact in offering credibility to academic analyses during dissemination. If user engagement has been continuous though a project's work, and user colleagues are prepared to explain the work and advocate its findings, then this is likely to be of considerable benefit in dealing with the press or sharing findings with other users. Roles of project 'champions' should be negotiated and nurtured over the long-term so that there is a trusting relationship and excellent understanding of the research and its relevance.

11c. Pro-active press strategies (expected by TLRP)

This concerns the managed promotion of key findings.

It is suggested that, when significant key findings have been identified, a pro-active press strategy is developed, including the use of press releases and briefing of key journalists. In the terms of the HEFCE-ESRC contract regarding TLRP, copies of press releases should be sent to HEFCE for information before going out (this is not for vetting). In fact, we recommend a constructive approach to funders and government agencies, preparing and informing them about imminent press initiatives. Whilst maintaining our independence, it is not TLRP's policy to attempt to wrong-foot agencies with whom we otherwise have cooperative working relationships.

The Programme expects to be able to help with this. The Directors' Team will develop journalistic links in their sector and the Office Team will assist with the production and targeted distribution of press releases. Media training will be offered to project representatives. ESRC will fund a media fellow to advise on specific project issues.

11d. Press releases

Clearly a crucial part of any impact strategy, designed to communicate key findings in relation to contemporary issues. See ESRC's pamphlet on Developing a Media Strategy.

11e. Press interviews

An important skill, providing opportunities to communicate key findings in relation to contemporary issues. See ESRC's pamphlet on Television and Radio.

SHARING KNOWLEDGE: ACADEMIC AND TECHNICAL AUDIENCES

12a. e-books

This is an increasingly common medium for dissemination, and will be provided as part of the TLRP partnership agreement with Taylor & Francis.

13a. Journal papers (expected by TLRP)

These are the traditional range of peer-refereed outputs. Such papers may be more honed versions of working papers previously made available on websites. If using Taylor and Francis journals, which are covered by the TLRP/Routledge partnership agreement, then electronic versions of the journal will be able to offer direct links to project websites. Textual acknowledgement of TLRP/ESRC support is required within all published papers, with logos included if publishers agree.

13b. Academic books

For academic audiences, these might include detailed theoretical, substantive or methodological accounts and analyses, or other volumes.

There is an expectation that TLRP badging will be used by publishers on covers, etc. The TLRP office will negotiate this with major publishers and with others at the request of project teams. See below on intellectual property rights.

13c. Final report to ESRC (ESRC requirement)

Final reports are an important academic output, and will be posted on Regard. They are due for submission three months after the end of funding.

14a. Academic conferences (expected by TLRP)

Presentations at academic conferences are well established for most individual networks and projects. Coordinated, targeted TLRP presentations are being developed for BERA, AERA, EERA, EARLI, etc and other major events to 2008/9. There is scope, in due course, for more sectoral presentations (eg: SRHE) or for issue-based coordination of presentations.

OTHER ISSUES

Badging (required by TLRP)

Wherever possible the dual TLRP/ESRC logo should be used as a right-hand side footer. Where this is really not possible (for example, in a journal publication), a simple text acknowledgement of the Programme should be included in an appropriate place. For example: 'Note 1: this project is part of ESRC's Teaching and Learning Research Programme (see www.TLRP.org )'. It is understood that website design might require the logos to be placed elsewhere too.

Some further guidelines on TLRP presentation are being produced by a professional designer. They will be used for all cross-Programme outputs. Selected elements will be shared with projects as soon as they become available.

Intellectual property rights

TLRP's concern for impact and sustainability through user-engagement could call for innovative arrangements with both commercial and non-commercial user organisations that can maximise and sustain impact.

According to the formal ESRC guidance, intellectual property rights arising from ESRC awards should be 'clear from the outset'. They are normally taken to be assigned to the institution receiving an award. However, in social science fields where commercial values tend to be low, institutions normally pass these to research teams. Even so, it is prudent for research colleagues to agree protocols for the appropriate attribution of authorship and recognition of all contributions made. Agreement of principles should be reached early in the duration of the project.

Regarding the distribution of any royalties or other income, ESRC again suggests that 'the basis of sharing needs to be clear from the outset'. They state their expectations as follows:

Whoever holds the intellectual property, it should be clear what the distribution of any income might be. For income up to £10,000 the ESRC assumes that this will normally rest with the principal investigators and the research team, unless otherwise stated to the contrary. For income between £10,000 and £50,000 the ESRC assumes that the major part will accrue to the university or institution, but the ESRC reserves the right to reclaim up to one third of the total, up to the value of its original award. For income over £50,000 the ESRC requires that the major part of any income should accrue to the university or institution, and again reserves the right to reclaim up to one third of the total up to the value of its original award (Funding Guidelines 19.2.2).

It will be nice if some colleagues have to worry about this issue - and some commercial forms of high-impact, user-engaged activity (eg: teaching materials) might well generate significant surpluses. Hopefully, such profits may actually underpin sustainability, and thus improve impact.

If any network or project thinks that more than £10,000 in surplus income might be generated from their TLRP funded work, they should contact ESRC for discussion.

Comments on this paper are very welcome and should be sent to Andrew Pollard: a.pollard@ioe.ac.uk