Thankyou to everyone who responded in the consultation exercise.
All comments made on this site during the consultation period will be taken into account in developing the final reports to HEDIIP. Reports will be presented to HEDIIP at the end of June 2015 (the HECoS Scheme) and at the end of August 2015 (governance model and adoption plan). A second report is also scheduled for submission at the end of June, which will include proposals on a way forward for subject-based aggregation.
This site remains open for comment, but we do not guarantee to process comments made after the close of the consultation period.
Information about discipline, subject, and topic is one of the key building blocks of the Higher Education data and information landscape.
This information ought to have sector-wide and consistent approaches that are well-matched to academic practice, serve the interests of efficiency, improve the experience of learners and researchers, and provide better information for managers and policy-makers. The existing Joint Academic Coding System (JACS) is in need of major review because:
- the limit of the existing coding framework has been reached;
- changes and growth in JACS’ range of functions mean it is no longer consistently applied;
- it does not meet the needs of all of the key sector stakeholders.
There are opportunities to reduce data management and reporting burden and to enable emerging uses in student information, planning & analytics, resource management, marketing, etc. To learn more about the motivation for the review, consult the HEDIIP “Classifying subject of study” report.
To address these issues, the Higher Education Data and Information Improvement Programme (HEDIIP) includes a project to consult stakeholders in UK higher education and develop a new subject coding scheme that meets their needs. The project is progressing in two stages.
The final report from Stage 1 of this work, covering requirements and impact analysis, has been published on the HEDIIP web site. Team member Wilbert Kraan has also written a blog post: “Subject coding is changing from JACS3 to HECoS; here’s what’s different” as a quick introduction to the approach being proposed. A webinar recording is also available which explains the background and the proposed HECoS scheme.
Stage 2 consultation began on 23 February 2014, and proceeded through to the end of May 2015. Final reports will be prepared during June and July and published on the HEDIIP web site following acceptance by the Project Board.
About commenting:
All comments are moderated so there will be a delay between posting and public visibility. We are unlikely to reply to comments but may contact individuals for more information in some cases. Comments will be assumed to be assigned a Creative Commons Attribution licence unless you, as the commenter, indicate otherwise.
Higher Education Classification of Subjects - HECoS 
The introduction of a new subject identification scheme will create a substantial impact across the Higher Education sector. Before embarking on a change to a new scheme, there needs to be a detailed cost\benefit analysis, which is accepted by the sector, and
which clearly demonstrates that the benefits of a new scheme over the existing scheme significantly outweigh the costs.
This important question was raised at one of the workshops last July and it’s a pity that it remains unanswered. There must be a business case so:
What are the costs; how have they been estimated?
What are the benefits; how have they been quantified?
What alternatives have been similarly considered; what is the assessment leading to the decision that this is the best. Alternatives must always include Do nothing.
As we are a specialist institution, I am concerned about the acceptance criteria that the term must be used by 2 or more HEPs. Are we going to lose out on being able to detail some of our provision?
Amanda –
Thankyou for commenting. We recognise the issue you raise as an important one, and one for which we would like to hear creative approaches to dealing with the problem. How can we have a process for acceptance that avoids HECoS “exploding” to an unmanageable number of terms, while also ensuring that necessary detail is supported. I suspect there is also a tension between requirements arising from statistical uses, admissions, marketing, etc. This is very much a balancing act.
The current stage of the consultation process is a good opportunity for organisations such as Harper Adams to review the existing HECoS terms in relation to existing courses and to feed back to us where difficulties exist. It may be that some adjustment would resolve issues.
Cheers, Adam
(HECoS team)
I think there’s a different question: what benefits are there from describing [your] provision at a particular level of detail? Adam refers to supporting “necessary detail” but we haven’t talked much about necessary for what.
Let’s look at some examples. The proposed scheme carries over the distinction between metal crafts and silversmithing & goldsmithing. I take it that at least two institutions have used each of these. But can we point to any decisions that have been influenced by the difference? Is there anything that would have been different if the only choice available for these metalworkers, along with the clay and stone workers, glass, fabric and leather workers etc had been Crafts?
It may be easier to find an example for use of the distinctions between animal health, animal pathology, animal pharmacology, veterinary pharmacy and veterinary medicine (to pick a scattering of examples); it would be good to see it?
These are examples from the first and last pages of the list of hecos terms. You could do the same opening it at any page. The question is not is there a difference between animal health and animal pathology: clearly there is. Rather, what difference does it make classifying modules or courses with these distinctions? “So we know” is not enough of an answer. what has anyone ever done with this knowledge? What difference has it made?
These are not trivial questions. Coding complexity increases costs. It also increases the scope for differences of interpretation and simple error. Necessary detail does not mean necessary for some approach to inherent “truth”; it must be necessary to support decision making.
1) How do you expect institutions to handle the migration from 4 character (alpha) codes to 6 character (numeric)? Will you be providing a “mapping” table to initially migrate all JACS codes to HECoS codes? If you wont be providing a mapping, then the migration costs will be excessive (i.e. each institution will have to hire temps for a short period to type in all the data).
2) When is the cut-over likely to occur – August 1st 2016?
Gordon –
By way of pre-amble, it is worth noting that this project will make recommendations about what we think should happen on the basis of consultations. Final decisions will be made by HEDIIP in conjunction it’s sponsors, and the ways in which HESA, UCAS, and other bodies handle migration may differ, even within a collective adoption plan.
Re #1 – mapping
We recommend all courses/modules ARE manually recoded to avoid pollution from the existing scheme and some of the coding practices which it has led to. Earlier work has identified problems with JACS3 and data using it and our remit is to consult to path-find the solution. Recoding will require effort, but our perspective on this is that the adoption plan that we propose should allow it to happen in a manageable way, without emergency-response measures. Our view is that, while some HEIs may wish to hire temporary staff, it would be very important for them to be properly educated about the classification process and to have the time to undertake quality coding.
We are interested in views from the community on what support materials (or training) would be useful, bearing in mind that an automatic JACS3-HECoS mapping would have undesirable consequences. The adoption plan will make specific recommendations. (draft available via this site)
Re #2 – cutover
Our view is that 2016 is not a likely date, given the combination of a realistic adoption timetable and the time delay between our recommendations being delivered to HEDIIP (summer 2015) and decisions being taken in the HEDIIP governance process. We would like to encourage all stakeolders to respond with their perspectives on realistic adoption timelines.
Adam Cooper
(on behalf of the HEDIIP New Subject Coding Scheme Project)
Following the recent Webinar colleagues offered the following comments
1) How will the HECOS structure map to Cost Centres? Some suggestion at the webinar that it won’t, but the interrelation between the subjects taught and the money to support it seems utterly key.
2) HECOS is clearly a set of building blocks that will facilitate many different types of mapping. The key issue for data and information managers within Universities in the future could well be a proliferation of mappings. If one is used for subject areas, one used for research activity, another used somewhere else, substantial effort will be required to track, implement and test these on our own data. Has any thought been given to a central ‘holding place’ for at least the key mappings? How about integrating them into HEIDI so we can all get instant access?
3) On the above, obviously if the number of mappings proliferates beyond a certain point within the key areas of University activity (i.e. we have one mapping underpinning NSS, another underpinning UCAS activity, another underpinning Staff adapt, another under research subjects, etc.) the very point of a unified structure collapses. Key players should be very strongly encouraged not just to use the same building blocks, but the same mapping of them into aggregated subjects – then you can get proper join up. What must not happen is constant requests by external stakeholders (e.g. research Councils) to recast all of our data into their mapping – that way we could end up with a huge bureaucratic task creating many versions of the same data set along slightly different cuts.
4) Will the three main HESA returns and HESES share the same mappings?
5) Given the importance of time series, it is important that some sense of the relationship to the current JACS framework is understood. An official mapping of this would be really useful?
Andrew – thanks for your comments. Following your numbering…
1. HECoS is formally for expressing subject of study, i.e. the focus is on the subject matter a student engages with. During phase 1 we set out to gather requirements very broadly but it became apparent that trying to cover all scenarios would cause the scheme to be compromised. Hence HECoS is not designed to designate cost centres because the way subjects are grouped is not uniform across the sector, and is often partially arbitrary or the result of accidents of history. We would expect that each institution will have its own mapping.
2. Yes, this is a matter we are considering and one where consultation feedback will shape our recommendations. We recognise the issue, and have a presumption against proliferation since the point of HEDIIP is to improve on the current situation. We think there are limitations to mappings across different contexts: for example from research topics to subject of study of taught courses. A classification of “subject” must fulfil different requirements. Nevertheless, people will be interested in correlation, no matter whether it is lossy or imprecise, and the idea of pooling them for re-use is a good one. There may be a related governance question: would these have a seal of approval or just be a collection of what has been created?
3. Mappings take many forms, as you outline. I’d like to pick up on your references to aggregate statistics. This is a very important area and my previous notes of caution about mapping between different conceptions of subject do not apply if we are talking about aggregation of data where “subject of study” applies across all records. Part of our project is devoted to this issue; we are tasked with exploring the process by which the currently divergent approaches to aggregation by JACS code and different approaches to FT/PT and joint honours courses etc can be rationalised.
4. We cannot comment on that; it is a matter for how core sector bodies decide to proceed.
5. Yes, this is an important topic for several kinds of stakeholder. There are several possible approaches which we are considering, and the input from consultees is important in helping us to form recommendations. One option is to confine this to time series of aggregate statistics. I have my doubts about the utility of attempting this at a lower-level, especially for some subject areas. It might be possible to pre-define JACS/HECoS correlations, but an alternative would be to operate both JACS3 and HECoS in parallel for a transitional period, with new and harmonised aggregation rules (as per #3) for HECoS, and to compute de facto formulae that would allow approximately continuous statistics to be estimated.
Cheers, Adam Cooper
NSCS Project Manager
Do the governance arrangements set out in the KPMG report, The Blueprint for a New HE Data Landscape, published last Thursday, cut across elements of this consultation? In particular is the governance body set out in that report intended to be the same one as would be governing the new HECOS codes and structures?
Peter –
thanks for your comment.
Yes, there is a relationship, and our final governance model report will refer to the Blueprint when the current draft is revised to a final report. On the whole, we have focused on the functions required to govern the HECoS scheme and, in the draft, we deliberately did not identify a particular home. i.e. we’ve tried to work at different levels of the HEDIIP problem, to be parallel rather than “cut across”.
Cheers, Adam
HEDIIP NSCS Project Manager
Hi,
I realise I am commenting very late in your project, but I am concerned that Entrepreneurship, a subject area of higher education which has been developing into wider areas in recent years has not been included correctly in the proposed new numbering system.
In addition, I am surprised to have found that Enterprise Educators UK, representing entrepreneurship educators in UK (http://www.enterprise. ac.uk) has not been consulted during the project.
As a result Entrepreneurship has only been inclued as a sub sub-set of Business and Management under Business Studies, whereas I believe it should be at least a direct sub-set of Business and Management.
In addition, QAA has recognised that Entrepreneurship is not just a Business subject, but it is also studied in other areas, such as Computing, Creativity and Engineering. However your proposed new system does not recognise this.
I have commented on this matter on your consultation web site and request that you should take into account my concerns and those of my friends, who are also Enterprise Educators.
Best regards,
Nigel Adams
Programme Director BSc Business Enterprise
University of Buckingham
Hi,
I realise I am commenting very late in your project, but I am concerned that Entrepreneurship, a subject area of higher education which has been developing into wider areas in recent years has not been included correctly in the proposed new numbering system.
In addition, I am surprised to have found that Enterprise Educators UK, representing entrepreneurship educators in UK (http://www.enterprise. ac.uk) has not been consulted during the project.
As a result Entrepreneurship has only been inclued as a sub sub-set of Business and Management under Business Studies, whereas I believe it should be at least a direct sub-set of Business and Management.
In addition, QAA has recognised that Entrepreneurship is not just a Business subject, but it is also studied in other areas, such as Computing, Creativity and Engineering. However your proposed new system does not recognise this.
I have commented on this matter on your consultation web site and request that you should take into account my concerns and those of my friends, who are also Enterprise Educators.
Best regards,
Nigel Adams
Programme Director BSc Business Enterprise
University of Buckingham
As far as I can see there is no governance model proposed for the *grouping* of HECos codes. Despite its inflexibility, JACS at least has a centrally controlled hierarchy that provides a standard way of relating data from different sources. From a data analyst point of view the aggregation of subjects into larger units, standardised across datasets, is vital.
It is good that HEcOS allows more flexibility in creating more than one subject grouping, but without any governance I don’t think we will get the necessary consistency between datasets (e.g. UCAS, HESA). We’ve gone from no flexibility to too much. Are there plans already in progress to address this issue?
Also, can I suggest a name change from HECoS to “HECS”? It’s quicker to say (and type), unambiguous to pronounce, and there’s no need to remember where to put the lower case character.
1. As commented by others there still has been no attempt to justify the benefits to be gained from the proposed changes against the cost of implementation within the sector.
2. The newly proposed HECOS meets many of the stated requirements from the requirements gathering exercise in terms of a flat list with no inherent hierarchies that is extensible and future-proofed. It is beneficial to have the ability to create one or more separate groupings that can be modified over time, rather than as an in-built hierarchy within the codes.
3. It is good that there is a governance process to handle revisions to the code list – in particular newly proposed codes. Every effort should be made to ensure this list is manageable – there are concerns that the desired detail provided by additional codes may be hard to achieve in practice, particularly in research areas.
4. A major weakness is the absence of an ‘approved’ grouping of the codes that is commonly shared by various data collectors, and which can be used to compare different data sets for bench marking purposes. For example to report UCAS, HESA, RCUK, NSS data and League tables – ideally aligned to Cost centres. One of the aims of HEDIIP was to reduce the complexity in data collections and reduce the burden on HEI. This seems an ideal opportunity to demonstrate how this could be done by introducing a common approach. The governance group should approve the official grouping (hierarchy) as well as new codes. Other groupings can obviously be determined by stakeholders, but the suggestion would be that if they do this then the burden is on them to provide an automatic mechanism for transformation (or an open data template) for the sector that avoids each HEI having to construct its own.
5. As a starting point it is important that the governance group approve the mapping from the JACS3 list to the new ‘Grouping’ of HECOS codes (point 4), with a recommended HECOS code within that grouping. This helps promote consistency across the sector in managing the transition, and will reduce the time taken to migrate from old codes to new. Adam Cooper (May 1st re#1 mapping) commented that every course/module should be recoded but no appreciation seems to have been given to what a huge task this may be for each institution. The existence of a recommended mapping table for as many of the JACS3 codes as possible would significantly help reduce this burden. It would also be a useful exercise to attempt in justifying the cost/benefit analysis referred to earlier.
We are concerned that the proposed model will result in an unmanageable and ever-changing coding structure with a proliferation of codes- this will have substantial knock-on impact on the administrative costs of maintaining currency of data.
It is absolutely imperative that the onward use of data is considered in detail before this scheme is adopted- this is what has most impact on us as institutions. A clear hierarchy for publication purposes with an appropriate governance structure to manage future change is essential, as is consideration for adoption mapping across various agencies/exercises, e.g. UCAS, HESA, NSS, REF – if we don’t solve some of these issues up-front then little progress from JACS3 will have been made.
It is also essential that the ability to maintain benchmarking capabilities is maintained through an appropriate reporting hierarchy, and one that fosters backwards capability with JACS- we can’t afford to have a break in subject data given the substantial investment already made in JACS coding.
It is still not clear to us that a compelling case based on cost/benefit analysis has been made regarding the decision to proceed with this project. Substantial costs relating to software, staff time, changes to reporting, planning and benchmarking processes will need to be met by institutions to deal with the scope and scale of this change. I note in particular the suggestion above that “all courses/modules ARE manually recoded to avoid pollution from the existing scheme and some of the coding practices which it has led to” – this is totally impractical and unrealistic; where do you think the resource to achieve this is going to come from?
Following on from a question we had last year with regards to the HEDIIP subject coding review, we wanted to query how the new system will enable HEI’s to create new subject codes?
Previous question raised with HEDIIP in May – July 2014, emanating from a group of academics at NTU, as well as peers from other institutions:
“The issue is that Biomedical Science has become lost within B900 ‘other subjects allied to health’, which includes all other subjects like social studies and health studies.
This has implications for league table reporting and does not allow prospective students to see data specific to the Biomedical Science subject area. There are c.70 institutions who currently offer this provision.
It has previously been considered that this could be grouped into the Bioscience subject group, but again this is not felt to be appropriate.
Ultimately, a JACS code(s) that specifically relates to Biomedical science only is what is required.”
We were advised last year that this would form part of this years review, but would like an update please in terms of how and timescales? Many thanks.
Lisa
Thank you for your comments. We have added in a term for biomedical sciences in HECoS; please see http://www.ovod.net/tematres/vocab/index.php?tema=100265&/biomedical-sciences for details.
Our recommendations for governance include a suggested mechanism for adding in new terms. The project’s final recommendations in this area will be guided by comments from the consultation.
Alan Paull
(on behalf of the HEDIIP New Subject Coding Scheme Project)
HECoS appears to be an unnecessarily radical fix to something which, in our opinion, was not particularly broken. We agree that some review and restructuring of subject coding is required, but the proposed solution risks losing more than it can add.
In particular it risks a significant loss of comparability, both time series data and in likely inconsistent use of new codes – the proposed new structure is in some ways too flexible. Robust benchmarking is an essential component of our performance reporting and improvement planning, and at best it will be much more difficult, with much higher levels of uncertainty, than at present.
There is also likely to be a loss of ease of use and accessibility – a recognisable/memorable hierarchy is useful, both for sense checking and proof-reading and for use of codes and structured data by non-specialists, including both academics and organisations such as league table providers. The onward use of coded subject data would again, in our opinion, be more difficult and more prone to inconsistency and potential error.
The proposed new structure brings with it what may be a significant overhead both for implementation and for on-going maintenance. It will be time consuming to satisfy ourselves internally that we have used the right new codes, and much more time consuming to consult with colleagues elsewhere that we have taken compatible approaches as we attempt to maintain benchmarking. Allocating codes to any new subjects will also be more difficult. Systems issues also appear not to have been thought through.
We accept the arguments that have been made that JACS3 is no longer fit for purpose and can see that there are advantages to a flat structure. However, the proposed solution fails to take into account
– the cost to institutions of initial recoding (we note that the intention is not to provide a mapping)
– the cost to institutions in making any necessary changes to IT systems
– the cost to institutions of maintaining coding at this level of granularity, including not only inputting time but increased complexity around consulting with colleagues on appropriate coding
– the cost to institutions and sector arising from a decline in quality of management information,
the loss of continuity with prevous years’ data and the risk of inconsistent use across the sector in terms of benchmarking and monitoring
Many of these costs will not just be incurred by individual HEIs, of course, but a full range of external users including not only government and government agencies, but external league table providers. Any corners cut as a result of the added complexity the proposed system brings will have a direct impact on the quality of information about institutions appearing in the public domain.
We are concerned about the amount of work involved in updating systems, given that mapping existing codes will not be possible. The complexity of the coding structure is likely to generate a considerable amount of queries, and consequent delays, in implementation. It is unclear who would be able to provide accurate advice, guidance and possibly training to staff tasked with recoding.
The new coding system also introduces an additional risk to the quality of data, given the amount of effort already put into making sure existing JACS codes are accurate. Have the implications for the production of time-series analyses been fully considered? I notice you are currently reviewing various options but we hope you will recognise the importance of being able to properly carry out this type of analysis.
The University of Surrey is broadly in agreement with the detailed responses provided by Lucy Hodson on behalf of HESPA. We have a number of specific comments we would emphasise:
• We feel strongly the proposed list is too long and will reduce the ability to benchmark meaningfully.
• We are concerned about the lack of transparency, and therefore potentially consistency, regarding the aggregation of data by all the multiple stakeholders AND the mapping of the structures to other coding frames
• The processes for updating the codes appears bureaucratic and counter to the aims of stability.
• We question whether fully replacing the JACS codes is necessary to resolve the recognised issues; the proposal seems to be over engineered when considered in light of the objectives
• We are concerned at both the cost of implementation and the cost of the ongoing governance model.
• We are concerned that there could be a significant loss of historic trend reporting during and after the cut over period.
In common with many of the correspondents above, we are concerned about the additional workload which may accompany the implementation of HECoS – in particular if thousands of programme and module codes are to be manually re-keyed. While JACS has its limitations, it is relatively simple to understand and use, especially for colleagues whose day job is not subject classification. With the ongoing drive to improve data quality across HEPs and their agents, it is not clear how moving to a system of unrecognisable numeric codes will enhance this process, nor assist with historic analysis and sector wide benchmarking. We appreciate that HECoS has the advantage of extensibility and might eventually prove a more robust framework than JACS, but – especially without a comprehensive mapping methodology in place – it is difficult to see how the transition will be managed without significant additional burden on HEPs and negative short to medium term impact on the quality of subject information across the sector.
University of Warwick would like to echo all the points made in the HESPA response to the consultation, and in particular the following:
· There is a real danger of producing too many codes, which will make it operationally difficult to assign the “correct” code to a given course, as well as making any analysis which uses HECoS codes more difficult, if not impossible.
· We are concerned that the inherent property of HECoS to be able to create “multiple hierarchies”, will destroy the capability of HEIs and other agencies to compare and benchmark across the sector. In addition, new hierarchies run the risk of breaking time-series comparisons of, for example, NSS data or market trends data.
· Related to the above, we would like to see some reassurance that any standard hierarchy within the new coding frame will be adopted, at least, by all the main HE agencies. Also, that other users are transparent in the hierarchies applied in any analysis, so that HEIs have an opportunity to check appropriateness.
· There is potential for huge cost to HEIs in implementation – has a full cost-benefit analysis been undertaken?
We would also make the following points regarding implementation:
· There is a need for “mappings” to JACS, in order to allow HEIs to benchmark against historical data. However, we also see the argument that providing mappings runs the risk of existing errors in coding being propagated forward. We are unclear how these conflicting demands can both be met.
· The link to ATAS needs to be considered, since the current process for issuing a certificate relies on JACS codes.
· It is difficult to predict the effects on the availability and look of data and analysis with regard to market data until we know more about UCAS’ plans for data provision.
· Whatever implementation timescale is agreed, it needs to give HEIs sufficient opportunity to undertake what could be a relatively complex and time consuming exercise in recoding.
· With regard to adding new terms to the subject listing, it would be helpful to try and understand where the definition of ‘substantive’ will fall, and how these changes/additions might be communicated to appropriate colleagues within HEPs so that we would be aware of this, and so that there was an ability to comment on new proposals if necessary.
· It is welcome that there is still the option to attach multiple subjects to joint/triple courses if this is appropriate and there is no recommended change to the current process for registering this.
· Sufficient notice and detail will need to be provided to student record system vendors, to allow any changes to be implemented in good time.
The University of Salford also endorses the main thrust of the submission from HESPA, in particular the following points.
We are concerned about the volume of codes in the new system. We do not believe that more precise coding will lead to more accurate coding. Precision and accuracy are not the same and may be in conflict.
Section 4.2 of HEDIIP NSCS Structure and Candidate Scheme explains that a much simpler set of subject codes (between 50 and 150) was rejected because of fears that it might mean not being able to track a niche subject that could be of interest in the future. The unstated assumption is that including a code for a niche subject will enable future (retrospective) tracking: the presence of the code in the list will mean it has been used and used consistently. This assumption is shaky. If there have been no effects of choosing a particular code then there can be limited confidence about how it has been used (or avoided). Written guidance and training will have limited impact.
Raising the threshold for carrying a classification over from JACS to HECoS from use in two institutions would be a step towards reducing the volume of codes. However a more important test is the extent to which distinctions between codes have been used in the reporting. As HESA publishes information at only the level of principal subjects a better starting point could be the requirement to justify (via demonstration of use in practice) a need for finer classification in any area, and a challenge to over-classification in areas with very little activity.
The prospect of multiple hierarchies is unattractive. This already exists with JACS (for example JACS “levels”; newspaper league table subject groups) and generates confusion and misunderstanding. Multiplication of hierarchies can’t be prevented but should be discouraged by consistent, authoritative aggregations.
Stability of the coding system is valuable and there should not be over-frequent amendments to the code scheme. New terms and removal of obsolete ones should be considered together with the explicit objective of preventing growth of the overall number.
Costs and benefits need to be clearly defined before further expenditure is committed. The main benefit suggested for HEPs is the general adoption of HECoS by bodies they must report to; and simplification and reduction in the reporting required. At this point we would like to see explicit commitment from the relevant organisations to this.
Whilst the current coding system has many limitations it is important to extract some of the key benefits out of the system as we consider alternative coding frameworks. The proposed replacement system appears to be extensive with the potential for creating a significant increase in the number of entities. If this were to be the case it would be likely that operational staff could not track previous activity resulting in duplicate or overlapping records. The aggregation of the coding system into manageable hierarchies will require considerable oversight and discussion if a single structure is to be agreed. The failure to achieve this agreement would result in multiple coding systems again stymieing effective bench-marking and comparative analysis.
The data burden within institutions is well recognized and the aim of reducing this via the implementation of the new information landscape is to be applauded. However it is important that these benefits are not offset via the introduction of a revised system. There remain many areas of activity where institutions are required to present complex information sets which see no parallel within the private sector.
I am concerned, as are others, about the costs of implementing this new coding system against the potential benefits. This is a huge data mapping exercise for HEPs if no mapping document is to be provided. Furthermore, loosing ‘meaning’ that is currently in the JACS 3 coding, will make data validations more difficult and time consuming. Finally, I have concerns about the ensuring the reliability of time series data, especially if a mapping document does not exist.
We share the concerns raised above by other institutions. In particular about the amount of work involved in updating systems (both in terms of data entry and reporting from them) and the potential loss of comparable time series.
We note that one of the recognised shortcomings with JACS is the lack of consistency in its application, although this is as much about data input as coding structures. We are concerned that data quality may be no better under HECoS unless there is a very quick and simple way of finding appropriate HECoS codes. Some form of navigation to aid user input is promised and the success of HECoS could depend upon it. At Sussex we run over 1900 modules each year and in 2014/15 they included 350 not offered in 2013/14. To code from scratch this many modules with limited resources and probably in a short time frame is likely to be possible only by applying a variant of golf ball rules – if a good match isn’t found after a single search the closest code presented is used. This may be why generic codes featured so much in JACS?
The University of York has a number of concerns about these proposals which it views as an over-complex solution to something which, with some challenges at the margin, still works relatively well. We agree that some review and restructuring of subject coding is required in a limited number of areas, but the proposed solution risks losing more than it can add.
Our concerns centre on :
(a) – the scale of the coding frame and related issues about its complexity and the associated scope for error/misinterpretation in construction, use and understanding of the coding structures.
(b) – the ability to sustain medium to long-term benchmarking – this requires a commitment to work with institutions and agencies to ensure a readily understood and technically straightforward mapping solution from current datasets. We would support the notion in the Birmingham response that a major weakness is the absence of an ‘approved’ grouping/hierarchy of the codes, shared in common with the various data collectors, including league table compilers, which could be used to compare different data sets for benchmarking purposes, ideally aligned to cost centres.
(c) – the very significant cost of implementation of such an arrangement at a time when institutions across the sector are likely to be severely challenged financially and when support costs in particular are likely to come under close scrutiny. The costs of implementation are multiple and are well set out in both the response of the University of Oxford and in that of HESPA , whose recommendations we fully support.
(d) – the project having the potential, by virtue of its complexity, to undermine an approach that a number of institutions have been pursuing with some success in recent years, namely to demystify, and be more open and transparent in, the use of comparative data and the sharing of such data with academic colleagues
Whilst welcoming a review of JACS, we have some observations on the proposal.
THE EXISTING JACS SYSTEM –
There are 1551 separate JACS codes, yet a very common observation from academics attempting to select one for a new programme or module is, “I can’t find one that fits properly.” The fact is that this is not proof that JACS does not cover all subjects, but evidence that it is too detailed. Because there are 1551 codes, all slightly different, people required to do the coding are encouraged to find something that fits exactly. When they can’t do that, they often resort to selecting the 90 codes which are described as ‘not elsewhere classified’ and ‘To be used sparingly’. Any classification system that contains a subject listed as ‘others’ is not doing its job. And it is a fact that once a programme is coded, it is unlikely to have that coding reviewed.
We fully endorse the survey response quoted in the paper that ‘The value of detail in JACS codes is limited by the detail of the data we get back’. It was not until the eve of the NSS that we discovered many of the oddities in our programme JACS codes and went about correcting them. Since we have never seen JACS codes at module level used for any purpose, we have not given consideration of their coding much attention. Even programme JACS codes are never presented back to us at the level of ANNN.
HECoS –
Whilst recognising the value of the observation, in the paper ‘Classifying subject of study A roadmap to a new Joint Academic Coding System’, that a new system could … work effectively with specialist vocabularies in use in research and the world of medicine as well as relating it to the more general keywords that prospective students might use when undertaking a Google search for information we would suggest that a broader classification with more detail provided as meta data to the publishers of data is an option worth investigating.
Different institutions will always apply the same coding system differently and the more detail the coding system includes, the more likelihood of variation. For benchmarking purposes, therefore, we have to group a number of JACS codes together, even if we, as an institution, do not use them. The classic example is History, where to compare history provision across the county, it is necessary to select data for V1 History by period, V2 History by area and V3 History by topic.
A subject system that does not provide data amalgamated into larger subject groups is useless. It would result in small cohorts of students disappearing from published statistics entirely. There are 107 subjects at level 3 in the NSS. While we believe that some of these are too detailed and some not detailed enough, they do represent a broadly useful number of different subjects. The proposed HECoS system contains 1269 subjects at present. If these cannot be amalgamated, the detail will become unusable. They are grouped into 73 subject groups, which we think is too few for practical purposes, but which could form the basis of useful reporting groups.
If a coding system on the proposed scale is introduced it will be vital that a formal governance structure is implemented and effectively managed. Specifically this must have right of veto over aggregations imposed by League Tables and any other published cross sector comparisons. This of course is easily stated but may well be impossible to police.
I attended a Stage 2 consultation in December 2014. I liked the proposed approval system for new codes which included users ‘upvoting’ new codes. However, I’m concerned that there will be no improvement in consistency of application of terms. Having terms which have a hierarchical relationship to one another in the same flat list may lead to inconsistencies in application which then affect comparisons between HEPs. Also, coders may be confused about which terms to use, and will likely need new or amended systems to support internal data and HESA returns. These are my own opinions and not representative of my institution.
JACS-based subject data derived from HE sources is widely used within and outside the sector for IAG, especially in careers services and for pre-entry guidance.
An an IAG-oriented data specialist, the new subject system needs to meet the following criteria in order to avoid a situation where the new system cannot deliver the quality of information for guidance that JACS, for its flaws, was and is able to.
The new system needs to be:
– able to be aggregated. Many guidance queries are broad, and HE subject data is of great value externally to illuminate social, economic and policy issues (the STEM agenda being a prime example).
– back-compatible with JACS to allow the continuation of time series. It is not just individual institutions or groups of institutions who use and need this data; the regional and national level datasets are useful for social and economic tracking, and their outputs have value in delivery of IAG, where many forms of comparison are very useful.
– consistently applied across the sector. This goes without saying.
– *genuinely* flexible and extensible.
With employability so high up the agenda, it’s important that we produce a subject classification system that *enhances* information provision for empoyability and guidance. JACS is not perfect, and it has well-recognised flaws. But by and large, the sector works reasonably effectively with it.
Following on from the other comments we have some questions/concerns :
• Mapping/Definitions – will a mapping structure/table be provided for institutions to use to ensure consistency across the sector? There would be an impact on benchmarking exercises without a consistent approach.
• Costs – has thought been given to the additional costs which will be incurred from implementing the proposed new structure? This would include staff time re mapping from JACS and the time required to update systems (if the current systems have the capacity for the new structure) and possibly the need for new systems or system updates
• Trend analysis & HESA/HEIDI data – how will the impact of a new structure be managed regarding historical data?
• League tables – have the implications for league table reporting been considered?
We agree with the justification set out for the need for major reform of the JACS coding system and appreciate this consultation on the proposals developed thus far for a replacement. However, in line with the comments from other HEP’s on this webpage we think that the current proposals, and how they will be implemented, need further development in order for the proposed benefits to be realised.
One key theme is the need for more work on mappings between the JACS and HECoS, and the need for detail on how sector bodies intend themselves to use HECoS. We appreciate that this is a chicken and egg situation, but it’s essential that work on this is developed in parallel with the development of the new coding framework. Otherwise it’s impossible to judge whether or not HECoS and its implementation will truly result in the benefits desired for the sector, or indeed for a true cost/benefit analysis to be carried out.
In line with other institutions we note the considerable burden on institutions of implementing HECoS and the need for a formal governance structure to be defined together with firm proposals for how it will be implemented and effectively managed.
We share strongly the concerns expressed in the HESDA response to the consultation on the Review of Subject Coding. From our own perspective at the University of Worcester we would wish to emphasise the following points:
1 The cost of implementation would be considerable ands the advantages of the costs involved are not immediately obvious – sector wide and at individual institution level. This is especially relevant for student record systems which may not be able to support the new structure without expensive adaptation.
2 The framework as proposed is likely to produce too many codes.
3 We are concerned that the end of meaningful codes would lead to situations where many colleagues who are not immersed in the coding structures find the whole system of reporting less transparent and useful.
4 If the proposed system is to win support far more investment of time analysing the costs and benefits of its introduction is required.
5 We are all encouraged to benchmark – this may become more challenging should the system as proposed be implemented.
This response is in the context of being generally supportive of the view that a change from JACS is required, but with some significant concerns about how this is proceeding.
The principle that there is not a permanent fixed hierarchy set in stone within the individual codings could be, for the long term use of the new coding system, a positive aspect. However the notion that there will be no central reference hierarchy for the system, whilst it may have theoretical attractions, is in practice likely to lead to a number of difficulties. There is further the potential for chaotic results as different externally devised hierarchies attempt to represent the complexity of the sector’s provision. This may be alleviated if external hierarchies are operating only as variants on a sector owned hierarchy.
The lack of any initial hierarchy also leads to another concern which is in the practicality of moving to the new system. The statement that all recoding should be undertaken afresh manually, with no element of translation from JACS to HECOS, in my view shows a worrying lack of appreciation of the scale and cost of this task, on top of the other costs of the change This applies both to individual institutions and in summation across the sector.
This then leads to the final point about whether there is any (however ball park in nature) attempt to quantify the costs which institutions and the sector as a whole will incur.
Response from the University of Edinburgh:
We would like to support and echo, amongst others Ray Lashley’s response (27 May under “Subject Scheme”). The number of codes proposed is very large because stakeholders want to monitor niche subjects. It may be reasonable to monitor niche subjects at module level; and indeed a list of 1,000 codes might be required for that task. Courses however, vary widely between the specialised and the general. A specialised course can be defined well by picking two or three codes from a list of 1,000. A more general course may also contain that niche content but it is hidden by the coding – at course level. Do stakeholders wish to identify pockets of niche delivery defined as a specific course (programme of study), or all pockets of niche delivery? If the latter, surely that aim is defined only at module level.
We think it is insufficiently clear until one looks at the list of codes suggested, that the hierarchy is being flattened, and not that the lowest level of detail is being provided. In other words, ‘Physics’ is still an option; one doesn’t have to choose between ‘Theoretical Physics’ and ‘Applied Physics’. Many of the options available will have hierarchical relationships with each other and this has implications for user choice and comparability.
We also support comments regarding the scale of the administrative task. Our student record currently holds circa 900 distinct undergraduate and postgraduate course titles (and multiple qualification aims associated with many of them). The expectation seems to be that we should code all of these afresh without referring to their existing JACS coding; that is a considerable task merely for courses, never mind modules. It also invites discontinuity in trend reporting; others have raised this issue and we would agree that disruption to time series reliability is likely.
We agree with others that in principle anonymous 6 digit codes will increase the likelihood of miscoding or at the very least take longer to work with. We are not sure that the extra aggregate administrative cost to the sector of using the proposed coding has been assessed alongside alternatives. That concern relates to ongoing costs as well as set-up costs.
A core argument is that JACS has run out of space; in some cases it is difficult to fit new detail required into the existing hierarchical ‘space’. Does it necessarily follow that the convention of three numerals and a letter is insufficient? One could suggest a half-way house, for instance the form “123H” to replace“H123”, with the numerals becoming non-hierarchical but the letter representing a subject area in the broadest sense. Shouldn’t four character options be considered? Switching to six characters suggests a degree of IT upheaval alongside the administrative complexity; both seem out of proportion to the problems.
Finally; how many stakeholders require the proposed level of detail? This is one aspect of the cost benefit assessment which we think is generally unclear. If much detail is required in particular subject areas or for particular purposes, should this dictate the general purpose coding?
I can only echo many of the comments already raised, particularly concerns around quantifying the cost/benefit.
From a purely data analysis/BI perspective, the ability to apply a hierarchy to subject codes is vital. In the absence of a sector-wide hierarchy each institution/body will be left to devise their own inconsistent hierarchies, which will pose significant challenges around effective comparison and benchmarking as others have already highlighted.
Aside from the initial transitional workload, I also have concerns around the ongoing management and potential future proliferation of codes for similar/duplicated subjects. With no recognised structure, the entire set of codes would need to be considered to identify the best fit for a specific purpose. Imaginary example: IT Management could already exist under a similar name (Information Technology Management, Computer Management, Computing Management, etc). Only by inspecting the entire list would this be revealed, whereas the current hierarchy (however imperfect it may be) at least allows for easier search and maintenance of a subset of the subject list.
We would broadly support the views expressed by HESPA and many others that the costs appear to be excessive, both in the transition to the new structure and in its future operation. We are keen to promote transparency and the sharing of data across the sector to aid benchmarking and to improve our understanding of comparative performance across providers, for which simple aggregation of courses by subject is essential.
At the consultation event at SROC in March it seemed clear that classifying modules was reasonably straightforward but that classifying courses whose modules covered a range of related topics frequently fell foul of the parameters set out in the classification. Many participants said that they and their colleagues would be tempted to put a course in the ‘other’ category if it did not match precisely the definition in the coding structure. More subjects on the list did not appear to resolve the problem.
We are also concerned that the continued evolution of the subject list may prove problematic in operation; initial validation, updating subject lists and review of previous years’ coding against revised lists. We would welcome clarification of the impact of revised subject coding of courses on NSS, KIS, SLC links and other users of course information.