Abstract

The main focus of the study was to analyse knowledge-related Attained learning outcomes gaps detection and resolution of the BS graduate programme. The research design of the study was confirmatory mixed methods. During phase 1, data were collected through Google forms from seven hundred and twenty-two graduates studying in four public sector universities of Punjab. While, in phase II, ninety-six faculty members were selected for the interview schedule to confirm the results of phase-I. The claim of BS graduates was that they attained the maximum level of knowledge-related learning outcomes. While faculty members marked that BS graduates attained medium level. One sample t-test was used and found a significant gap between Anticipated and Attained levels of knowledge. The study suggested to the HEC of Pakistan, concerning the provision of the NQF document should officially include in the course outlines of the graduate programme.

Key Words

Knowledge-related Learning Outcomes, Gaps Detection, Gaps Resolution of BS Graduates

Introduction

Learning outcomes are the statements, which help the learners, parents, teachers and assessors to work for the expectations and to clarify the programme's objectives (ECDVT, 2017). Proitz (2015) also said that learning outcomes played a significant role like cutting-edge in the development of education.

At the end of the twentieth century, a remarkable development in education took place due to assessments for measuring outcomes in the local education system (Kellaghan & Greeny, 2019).  An analysis of learning outcomes shows inflexible controls placed on educators (Proitz, 2015). Whereas, Hargreaves and Moore (2000) said that outcome-based education grasps advanced perspective for reordering, and understanding knowledge to the assistance of learners.

In the current situation, learning outcomes are considered a key success of any programme. They are performing best for giving a clear assessment of what is likely to be attained at the end of a programme (Nusche, 2008).

Bologna process in 1999, the enhanced significance of learning outcomes, which is why they seem as the basic pillar of educational improvements. Moreover, learning outcomes are used as a useful scheme for flexibility, and effectiveness in learning methods (Adam & Expert, 2008). 

In 2015, Pakistan develop its national qualifications framework (NQF) to follow the Bologna process, like many other countries 

have followed in the world to develop their national qualifications framework. Learning outcomes are clearly defined in NQF. It has defined eight levels of education including the BS programme. Every level has its own learning outcomes, these are divided into three constituents: knowledge, skills and competence (NQF, 2015).

Achievement of an anticipated ability by a student after completing an academic programme is denoted as knowledge, which also includes realities, key values, philosophies and practices (Kennedy, 2006). This achievement can be measured through learning outcomes, which play a vital role in refining the excellence and efficiency of educational programmes. Researchers have also turned the focus to this domain as it has become a significant area of research. (Kuh, & Ewell, 2010). For example, Aryanti and Adhariani (2020) conducted a study to analyse the insights of accounting learners and the prospects of the owners to investigate the need for bridging the gap. As a result, a belief gap between learners' awareness and that of the owners' prospects was found by researchers.

In a nutshell, the majority of the studies on learning outcomes were conducted in the West. After reviewing related literature, two studies on learning outcomes have been found in the Pakistani context. Firstly, Huma and Mahboob (2020) compared learning outcomes suggested by Pakistan Qualifications Framework (2009) and (NQF), 2015. Secondly, Shah et al., (2020) juxtaposed the learning outcomes of B.Ed. and MA Education graduates without consulting NQF learning outcomes. 

Omaish et al., (2022) also conducted a study to know knowledge gaps exist in students. Results of the study show that substantial knowledge gaps exist between maximum students. 

The maximum studies highlighted a significant gap between the anticipated and the attained level of learning outcomes.  This phenomenon generated a need to investigate the same type of achievement gap in Pakistani graduates.  Moreover, the previous studies in the Pakistani context focused on comparing and matching learning outcomes leaving the (NQF) unquoted. No one study has focused on the Attained learning outcomes at the BS level. These facts generated the need to evaluate the attained level of learning outcomes of BS graduates to find the gap between anticipated and attained outcomes. Therefore, the current study was led to examine the gap between the anticipated and attained knowledge level of BS graduates according to the NQF of Pakistan.

The Procedure of the Study

An explanatory sequential design was employed in the present study. According to Creswell and Plano (2014), explanatory sequential design is employed to validate quantitative data through qualitative data. This study also validated quantitative findings through qualitative findings. That was why the explanatory sequential design was the most suitable design to reach the results.




Population and Sample 

All the registered BS graduates of all the public universities of Punjab (Pakistan) were the population of the study. A multistage sampling technique was employed for the selection of the sample. In the first stage, four public sector universities were selected purposively. Three disciplines were selected from each university (natural sciences, social sciences and languages). In the second stage, six departments, Chemistry and Physics, Education and History, and Urdu and English were selected from each university. In the third stage, thirty (30) BS graduates were selected conveniently (180 from each university, 720 were total graduates. Four faculty members were selected conveniently from each department (24 from each university, 96 in total).

 A total of 816 respondents were the sample of the study (720 BS graduates and 96 faculty members).




Delimitation of the Study

BS programmes of public sector universities of

Punjab (Pakistan) were the delimitation of the current research. 




Phases of the Study 

There were two phases of the study.




Instrument of Phase-I (Quantitative)

Data were collected by using a self-developed questionnaire from the NQF document. The questionnaire was comprised of 14 items. 




Data collection 

Due to the prevailing COVID-19 virus,

universities were closed. That is why the researcher could not collect the data physically. So, a survey was changed to Google forms. The research tool was circulated in graduates' WhatsApp groups for data collection and a total of 1027 responses were received.

Results Phase-I

 The detail of the results is given below.




 

Table 1. Attained Level of Knowledge-related Learning Outcome

Statements	SA	A	SA+A	UND	DA	SDA	SDA+DA	M	Level
1. professional work for graduates	16.1	63.0	79.1	8.4	7.5	5.1	12.6	3.78	High
2. basics concepts of research to graduates	18.4	63.5	81.9	7.4	7.0	3.7	10.7	3.86	High
3.  innovative ideas of research for graduates	17.8	55.9	73.7	11.8	9.5	5.0	14.5	3.72	High
4. research methods for graduates	18.2	58.2	76.4	10.3	9.3	4.0	13.3	3.77	High
5. use of technology for graduates	24.2	53.8	78.0	8.8	9.3	3.8	13.3	3.85	High
6. concepts in one or more disciplines to graduates	17.2	59.1	76.3	11.7	9.0	3.0	12.0	3.79	High
7. analysis of new ideas for graduates	21.1	56.9	78.0	11.2	6.9	3.9	10.8	3.84	High
8. new contribution to the field of research	19.4	53.7	73.1	12.2	10.7	4.1	14.8	3.74	High
9. Urdu language proficiency for graduates	18.1	46.6	64.7	13.5	15.7	6.0	21.7	3.55	Medium
10. English language proficiency for graduates	23.6	56.2	79.8	7.7	8.7	3.9	12.6	3.87	High
11. logical reasoning to solve problems for graduates	21.1	57.5	78.6	11.7	6.4	3.2	9.6	3.87	High
12. critical expression to solve the problems of graduates	17.0	54.5	71.5	14.1	10.8	3.5	14.3	3.71	High
13. reflective thinking to solve the problems to graduates	15.6	59.3	74.9	14.1	7.8	3.2	11.0	3.76	High
14. Solve the difficult problems for graduates	19.6	58.3	77.9	10.6	8.3	3.2	11.5	3.83	High
The overall percentage of Knowledge level	19.1	56.9	76	11	19.1	4	12.1	3.78	High

 




The data in table 1 show, BS graduates attained high knowledge levels like: 'knowledge of professional work', 'basics concepts and innovative ideas of research', 'research methods', 'use of technology' 'concepts in one or more disciplines', 'analysis of new ideas', 'new contribution in the field of research’, ‘English language proficiency’, ‘logical reasoning to solve problem’, ‘critical expression to solve problem’, ‘ reflective thinking to solve problem’, ‘solve the difficult problems’.




 

Table 2. The gap between Anticipated and Attained Knowledge Level

Learning Outcomes	Level	M	Gap	SD	t	P
Knowledge of
1.  professional work for graduates	Attained	3.78	-1.23	.975	-40.27	.000
	Anticipated	5.00
2.  basics concepts of research to graduates	Attained	3.86	-1.14	.923	-39.61	.000
	Anticipated	5.00
3. innovative ideas of research for graduates	Attained	3.72	-1.28	1.024	-40.06	.000
	Anticipated	5.00
4.  research methods for graduates	Attained	3.77	-1.23	.982	-40.00	.000
	Anticipated	5.00
5. use of technology for graduates	Attained	3.85	-1.15	1.013	-36.27	.000
	Anticipated	5.00
6. concepts in one or more disciplines to graduates	Attained	3.79	-1.21	.935	-41.62	.000
	Anticipated	5.00
7.  analysis of new ideas for graduates	Attained	3.84	-1.16	.963	-38.47	.000
	Anticipated	5.00
8.  new contribution to the field of research	Attained	3.74	-1.27	1.021	-39.69	.000
	Anticipated	5.00
9.  Urdu language proficiency for graduates	Attained	3.55	-1.45	1.134	-40.94	.000
	Anticipated	5.00
10.  English language proficiency for graduates	Attained	3.87	-1.13	.997	-36.38	.000
	Anticipated	5.00
11.  logical reasoning to solve the problems of graduates	Attained	3.87	-1.13	.926	-39.11	.000
	Anticipated	5.00
12.  critical expression to solve the problem for graduates	Attained	3.71	-1.29	.987	-41.94	.000
	Anticipated	5.00
13.  reflective thinking to solve the problem for graduates	Attained	3.76	-1.24	.918	-43.22	.000
	Anticipated	5.00
14. solve the difficult problems for graduates	Attained	3.83	-1.17	.945	-39.75	.000
	Anticipated	5.00

 




Table 2 indicates that a one-sample t-test was applied. There exists a significant gap between

attained and anticipated knowledge level.




 

Table 3. Correlation between Perceived and CGPA

Variable	Achievement
Knowledge	R	0.03
	p-value	0.419

P < 0.05

 




Table 3 shows the Pearson correlation. The results showed no relationship between the perceived and actual achievement (CGPA).

 

Knowledge-Related Attained Learning Outcomes

It was found, a high knowledge level of learning outcomes was attained by BS graduates in all indicators including: 'knowledge of professional work', 'basics concepts and innovative ideas of research', 'research methods', 'use of technology' 'concepts in one or more discipline', 'analysis of new ideas', 'new contribution in the field of research', 'English language proficiency', 'logical reasoning to solve a problem', 'critical expression to solve a problem', ' reflective thinking to solve a problem', 'solve the difficult problems'.

 

Achievement Gaps

It was found that the attained knowledge level

was lesser than their anticipated level by BS graduates.

 

Relationship Between Reported and (CGPA)

The findings of the quantitative data showed that there is no significant relationship between stated and actual attained learning outcomes (CGPA).

Phase-2 (Qualitative Phase)

Instrument of the Study

On the basis of quantitative findings, a semi-structured interview schedule was developed. It was comprised of 4 questions. The first three of them were related to quantitative findings while the fourth one was to ask for suggestions from faculty members for the resolution of gaps. 




 

Table 4. Achievement Gaps its Reasons and the Resolution

Sub-Theme	Graduates Response	Faculty Members Response	Reasons	Suggestions	f
Attained level	Graduates claim: Attained a high level of knowledge	Teachers claim: graduate Attained medium-level knowledge	Reasons to achieve the low level: Lack of self-dedication in their studies	Teachers recommended that students there is a need to develop self-dedication in students	(N= 80, 83%)
			Students’ low level of self-motivation	Teachers should enhance students' motivation by appreciating their work and offering rewards	(N=80, 83%)
			students lack in sense of responsibility towards their studies	Teachers should use the project method of teaching to incarnate a sense of responsibility among students	(N=80, 83%)
			Weaker basic knowledge of students	Needs to brush up on their basic knowledge	(N=80, 83%)
			Lack of prior knowledge (Intermediate  level)	Teachers should strengthen students’ prior knowledge by relating their current lecture to the prior one. Universities may revise their admission policies and introduce a relatively strict criterion to admit high achievers only	(N=80, 83%)
			Different levels of education (elementary secondary and higher) lack vertical alignment	The curriculum needs to be revised to ensure the vertical alignment between different levels of education	(N=78, 81%)
			Outdated curriculum	The curriculum needs to revise according to the contemporary needs of the students learning.	(N=84, 87%)
			The existing curriculum does not meet the national and international standards	The curriculum should develop according to the international and national standards	(N=80, 83%)
			lack of  independent learning ability among students	Students should be urged to set their own learning goals and deadlines. And try to accomplish them accordingly.	(N=80, 83%)
			Irrelevant teaching methods	Relevant teaching methods should be used to make students better understand the study concepts.	(N=81, 84%)
			Communication gap due to language barrier (English as a medium of instruction)	i. teachers should communicate according to the level of students. ii. Students should try to enhance their competence in English communication.	(N=75, 78%)
			Lack of competent teaching faculty	Universities should appoint competent teachers as well as should try to enhance their capacity through the faculty development programme.	(N= 79, 82%)
			unfair assessment	Needs to ensure fair assessment	(N=80, 83%)
			Lack of good reading habits among students	The culture of book reading should be resumed and encouraged and the university administration should take solid steps to initiate it.	(N=82, 85%)
			Inadequate use of modern technology in the classroom	University management should ensure the availability and adequate use of modern technology by the teacher's regular classroom teaching.	(N=82, 85%)
			Selective reading by the students in the semester system	The question paper should be constructed in such a way that it must cover the whole course outline.	(N=67, 69%)
			Students' do not concentrate properly on their studies	There is a dire need to analyse the situation, trace out the underline factors and find the relevant solutions.	(N=70, 72%)
			The communication gap between teachers and students due to the language barrier	During their lesson, the instructor should use simple words while communicating with the students.	(N=75, 78%)
			Shortage of teachers who keep their knowledge up to date with the new demands	Teachers must be up to date with the most recent field-specific knowledge.	(N=80, 83%)

 

Table 4 shows the perception of BS graduates, the responses of faculty members’ motives of conflict and propositions.

 




Findings of Qualitative Data: Phase-2

Faculty Members’ Perspective

Faculty members opposed the statements of BS graduates due to some essential reasons. These reasons include 'lack of self-dedication, self-motivation, 'basic knowledge, 'lack of vertical alignment of different levels of education, 'outdated curriculum, independent learning ability, 'communication, competent teaching faculty, 'fair assessment, 'inadequate use of modern technology in the classroom, 'selective reading by the students in semester system, 'students' do not concentration properly on their studies, 'communication gap between teachers and students due to language barrier, 'shortage of teachers who keep their knowledge up to date with the new demands.

Discussion

The outcomes-based method is entirely student-centred and concentrates on what they really know and are capable of. Focusing more on student learning outcomes is a paradigm change in educational theory and practice that goes beyond simple changes to established structures and procedures (Tam, 2014). The establishment of the Bologna Process in 1999 was the first significant step toward the worldwide advancement of outcome-based education and it sparked a total paradigm change away from the conventional teacher-centred approach and toward outcome-based learning (Huisman et al., 2012). 

The Bologna Process was implemented by over 100 nations by the year 2015 in order to modernise their higher education systems in accordance with global norms. Similar efforts have been made in Pakistan over the past 20 years to improve the standard of instruction at all educational levels. Moreover, the quality of higher education is a specific priority for the Higher Education Commission of Pakistan, especially, since its formation. HEC is also pursuing a variety of actions consistently and progressively to improve the quality of higher education. (Akhtar et al., 2011).

The Higher Education Commission of Pakistan has engaged in a further major step by developing the NQF. Its creation began in 2009, and it was released in 2015. The goals of NQF are to place a strong emphasis on evaluating learners' necessary qualifications and preparing candidates for national and worldwide standards of qualifications. The NQF emphasizes the value of smart and effective learning. For this purpose, it presented learning outcomes i.e., knowledge, skills and competence (Yahya, 2016).

The goal of the present study was to evaluate the graduate program's knowledge-related learning outcomes. The findings of the study revealed that BS graduates got a high knowledge level.

These findings are based on BS graduates' self-evaluations of their learning outcomes. The high degree of learning outcomes attained by BS graduates may be attributable to the self-assessment rating scales' drawback, which is that respondents frequently overrate their accomplishments (Karnilowicz, 2012). However, several studies have indicated a high degree of accomplishment and/or satisfaction in various educational programmes. For instance, the research of Zhu et al., (2018) supported this stance. Whereas, the findings of the comparison of the Attained and Anticipated levels of knowledge for BS graduates revealed that the Attained level of these outcomes was lower than the predicted level. The accomplishment gap in learning outcomes in terms of knowledge endangers an organization's overall success. The insufficiency of knowledge among people to satisfy workplace expectations is referred to as this accomplishment gap (McGuinness & Ortiz 2016). Knowledge gaps are the differences between an organization's current skills and the competencies it seeks to achieve its goals, according to the American Society for Training and Development (ASTD) (Alshare, & Sewailem 2018). 

According to research on learning outcomes, there are three key causes for the knowledge gaps. First, it exposes flaws in the educational system that do not equip fresh graduates with fundamental information, abilities, and competence (Hobson et al., 2014). Secondly, these inadequacies, become barriers for an organisation to keep up with the rapid pace of technological advancement (Chen et al., 2015). Thirdly, they drastically reduce the opportunity for training inside an organisation that would have helped students advance their knowledge, skills, and competency (Olson, 2015). The results of the current research are authenticated by the results of Aryanti and Adhariani (2020). The study analyses accounting students' perspectives and employers' expectations regarding the abilities and expertise required of accounting graduates in Indonesia. The study came to the conclusion that there is a knowledge and competence gap between how students see themselves and what employers demand of accounting graduates. Above all the studies may be different: firstly, these studies were carried out outside of Pakistan, notably in established European nations that accepted the Bologna Process and built and executed their (NQF) at the start of the twenty-first century. By doing this, they changed their conventional teacher-centred to contemporary outcome-based educational systems, whereas, Pakistan adopted its national credentials system in 2015 and is still attempting to apply it properly. Secondly, the research that produced the varied results was executed at the graduate level, whereas the current study was done with BS graduates.

Conclusions

BS graduates perceived that they attained an optimal level of knowledge-related attained learning outcomes. But faculty members specified that BS graduates had attained average levels. Hence, it was concluded that BS graduates' did not attain a high level.

Faculty members proposed numerous actions to achieve a high level. Likewise: ‘teachers recommended that students must be self-dedicated, motivated in their task through appreciating their work and offering rewards, enhance their competence in English communication and ‘teachers should use project method of teaching to teach students, communicate according to the level of students ‘universities revised their admission policies, curriculum and furthermore, appoint competent teachers also related institutions should try to make vertical alignment between different levels of education.

References

• Adam, S. (2004). Using learning outcomes. In Report for United Kingdom Bologna Seminar. 1-2.
• Akhtar, M. M. S., Rafi, M. S., Ahmed, S., & Rauf, M. (2011). Quality in higher education: Issues and current practices. Journal of Elementary Education. 21(1), 43-51
• Alshare, K., & Sewailem, M. F. (2018). A Gap Analysis of Business Students' Skills in the 21st Century: A Case Study of Qatar. Academy of Educational Leadership Journal, 22(1), 1-22.
• Aryanti, C., & Adhariani, D. (2020). Students' perceptions and expectation gap on the skills and knowledge of accounting graduates. The Journal of Asian Finance, Economics, and Business, 7(9), 649-657.
• Chen, J., Neubaum, D. O., Reilly, R. R., & Lynn, G. S. (2014). The relationship between team autonomy and new product development performance under different levels of technological turbulence⋆. Journal of Operations Management, 33-34(1), 83–96.
• Creswell, J. W. (2014). A concise introduction to mixed methods research. SAGE publications.
• Hargreaves, A., & Moore, S. (2000). Educational Outcomes, Modern and Postmodern Interpretations: Response to Smyth and Dow. British Journal of Sociology of Education, 21(1), 27–42.
• Hobson, C. J., Strupeck, D., Griffin, A., Szostek, J., & Rominger, A. S. (2014). Teaching MBA Students Teamwork And Team Leadership Skills: An Empirical Evaluation Of A Classroom Educational Program. American Journal of Business Education (AJBE), 7(3), 191.
• Huisman, J., Adelman, C., Hsieh, C. C., Shams, F., & Wilkins, S. (2012). The Bologna process and its impact in the European Higher Education Area and beyond. In The SAGE handbook of international higher education. 81-100. Thousand Oaks: Sage Publications.
• Huma, Z., Baseer, N., & Mahboob, U. (2020). Cross Matching Learning Outcomes of Postgraduate Basic Medical Sciences to the Qualification Framework of Pakistan. Advances in Basic Medical Sciences, 4(1), 4- 10
• Karnilowicz, W. (2012). A comparison of self- assessment and tutor assessment of undergraduate psychology students. Social Behaviour and Personality, 40(4), 591- 604.
• Kellaghan, T., & Greaney, V. (2019). The Role of School-Based Assessment.
• Kennedy, D. (2006). Writing and using learning outcomes: a practical guide. University College Cork.
• Kuh, G. D., & Ewell, P. T. (2010). The state of learning outcomes assessment in the United States. Higher education management and policy, 22(1), 1-20.
• McGuinness, S., & Ortiz, L. (2016). Skill gaps in the workplace: measurement, determinants and impacts. Industrial Relations Journal, 47(3), 253–278
• Nusche, D. (2008). Assessment of learning outcomes in higher education: A comparative review of selected practices.
• Olson, M. P. (2015). A multilateral approach to bridging the global skills gap. Cornell HR Review. 1-5.
• Omaish, A. H., Sennuo, A., Alymany, G., Abdullah, M. U., AlNakib, S., Divan, A., & Dionigi, F. (2022). Knowledge gaps amongst students entering higher education in the non-regime North of Syria: Causes and possible solutions. International Journal of Educational Research Open, 3, 100129.
• Proitz, T. S. (2015). Learning outcomes as a key concept in policy documentsthroughout policy changes. Scandinavian Journal of Educational Research, 59(3), 275-296.
• Shah, A. A., Syeda, Z. F., & Shahzadi, U. (2020). Assessment of Higher Education Learning Outcomes of University Graduates. Global Educational Studies Review, 5(1), 72-83.
• Tam, M. (2014). Outcomes-based approach to quality assessment and curriculum improvement in higher education. Quality Assurance in Education, 22(2), 158–168.
• Yahya, S. (2016). National qualifications framework for higher education (proposal).
• Zhu, J., Chen, J., McNeill, N., Zheng, T., Liu, Q., Chen, B., & Cai, J. (2018). Mapping engineering students’ learning outcomes from international experiences: designing an instrument to measure attainment of knowledge, skills, and attitudes. IEEE Transactions on Education, 62(2), 108- 118.