Home

Journal Score Cards

Base Score Card

The Base Score Card is a questionnaire on the transparency of the journal’s website with respect to the following aspects: Editorial information, Peer review, Governance and Workflow. Every aspect has four bullets and each bullet can be rated from 1 to 5. The average of these ratings is called the subscore for this aspect. The lowest subscore of these four aspects is called the Base Score of the journal.
As a consequence a journal with a high Base Score, has a web site that addresses the four critical journal aspects adequately. On the other hand, a journal with a low Base Score has at least one weak aspect, but may still be strong in other respects. In order to climb on the list a journal must address its weakest aspect.

1. Editorial information

  • Aims, scope, and expected readership of the journal are clearly specified on the journal’s website.
  • The names and affiliations of members of the editorial board are listed on the website.
  • The website of the journal indicates how open editorial correspondence and reviewer's comments are.
  • The role of members of the editorial board is explicated on the website including who will make final decisions about article acceptance (e.g. editor, associate/action editor).

2. Peer review

  • The website indicates whether all submissions are sent out for review.
  • Criteria used by reviewers to rate submissions are specified on the website.
  • The website indicates if authors have a say in suggesting names of (non-)desired reviewers.
  • The journal website allows ratings of papers and post-publication commentaries by the community.

3. Governance

  • The journal (publisher) has clear guidelines concerning sharing and availability of data for verification purposes.
  • The journal (publisher) makes copies of published articles available in trusted third-party repositories (e.g. PubMed Central) immediately upon publication.
  • The journal (publisher) takes care of inclusion of its articles in relevant indexing services.
  • Journal’s website highlights issues of publication ethics (e.g. plagiarism, retraction policy), conflicts of interest, and (if applicable) codes of conduct for research in life sciences and social sciences.

4. Workflow

  • The website provides a track & trace service enabling authors to follow the status of their submission (e.g. under review).
  • Published papers include information on dates of original submission and acceptance.
  • The journal (publisher) provides the articles with a Digital Object Identifier.
  • The journal discloses the past (yearly) number of submissions, publications, and rejection rates.

Additional question

Pop up box in case the first or second checkbox is ticked:

Valuation Score Card

The Valuation Score Card is a reality check, preferably based on experience as an author, editor or peer reviewer of the journal. It has four bullets that can be rated from 1 to 5 and the average of these ratings is called the Valuation Score of the journal.

Valuation

  • The editor of the journal is responsive.
  • The peer review of the journal has added value.
  • I would recommend scholars to submit their work to this journal.
  • I would deem this journal good value for money

(NB As of May 20, 2016, the first two questions replace the question ‘I would consider the peer revview process of this journal to be transparent’ of the old version)

Additional questions

I have published an article in this journal less than a year ago (If ‘Yes’, please tick the box)

Pop up question in case of ‘Yes’:

Publishing my article in this journal did cost

I am an editor or peer reviewer of this journal (If ‘Yes’, please tick the box)

In case of ‘Yes’ a text box with two questions pops up:

  1. How high is the publication fee of your journal? What do you think of it?
  2. Does your journal have third party revenues? If so, is there any risk of a potential conflict of interest?

The SWOT matrix

Together, Base Score and Valuation Score determine the following SWOT matrix:

  Valuation Score
> 3 ≤ 3
Base
Score		 > 3 Strong journal Threat to author:
Journal may not meet its promises
≤ 3 Opportunity to publisher:
Improve the journal's web site		 Weaker journal

Per journal this is depicted as follows.

Strong journal
SWOT matrix
 
 
 
 
 
Weaker journal
SWOT matrix
 
 
 
 
 
Opportunity to publisher
SWOT matrix
 
 
 
 
 
Threat to author
SWOT matrix
 
 
 
 

The life cycle of Journal Score Cards

To avoid framing only members of the academic community can publish Journal Score Cards. Therefore, only people with an academic email address can log in to the scoring section of QOAM. Usually, library staff publish Base Score Cards; authors, editors and peer reviewers publish Valuation Score Cards. Per person only one JSC may be published for a specific journal. In case several people publish a JSC for the same journal the subscores will be averaged. For calculating the scores the scorecards of the preceding two years are used. A month before a scorecard becomes ineffective its author will be notified of this and will be invited to update their JSC (provided they are still a member of the academic community). In case of no response the Journal Score Card will be open-archived in QOAM.

In order to reflect new developments of or new experiences with the journal, everybody may edit their JSC at any moment. Updated Journal Score Cards will be dealt with as newly published ones.

The Journal Score Card’s format will be evaluated periodically and possibly adapted to new insights. However, changes will not be abrupt and draft versions will be circulated for comment prior to installation. As JSCs are only effective for two years the migration period in which ‘old’ and ‘new’ subscores sit together is limited. Effectively the migration period could be shorter because everybody may renew their JSC at any time. Thus, the JSC may evolve organically.