Research methods for undergraduate delivery : Evaluation of problem-based learning

Research methods are considered a complex and difficult subject to teach and there is limited data on innovative ways to teach it to undergraduate public health professionals. Although problembased learning (PBL) is a common approach in various disciplines, little is published on its use in developing research skills within the South African context. This paper outlines the delivery of an undergraduate research methods course with an evaluation of the problem based learning approach with respect to student experience and competence. A cross sectional descriptive case study was conducted among the students of Environmental Health within the Faculty of Health Sciences at the Durban University of Technology. All students enrolled for 2015 were invited to participate. Questionnaires were administered after each section and at the end of the Research Methods course. Participants were required to recall how they experienced different sections of the course and to self-reflect on perceived competency post-delivery. Participants reported that they benefitted from the PBL approach and indicated that independent learning helped to develop skills that they could transfer to their professional environment. They enjoyed group interactions and private consultations with the instructor, but expressed difficulty with the literature review, choosing and summarising relevant information and statistical concepts and terminology. The final proposal was a ‘light bulb’ moment and participants successfully converted their research problem into a proposal. Problem-based learning may be used as a successful strategy for skills development in research methods courses.


Introduction
Many consider research methods courses a complex domain that confounds students and academics alike (Lehti & Lehtinen, 2005).Instructors have the unenviable task of transferring the complexities of these concepts to students who are completely new to the research process (Earley, 2014).However, in spite of significant interest in promoting research skills among postgraduate students, there is limited data for undergraduate students (Lombard & Kloppers, 2015).The national development plan (National The current format of Research methods 4 in the Environmental Health programme actively engages students in participative exercises.The course journeys tentatively, outcome by outcome, into the research process by providing students with practice in each competency.The assessments are iterative and cumulative, leading to a research proposal at the end of the course that is prepared using a postgraduate template.My primary objective in teaching the research methods course has been to develop student's critical thinking and practical skills in research.It is all about the "what, why, when, where, who, how and so what" questions.Simplifying a complex research problem to a few questions facilitates learning; i.e. what is the problem, why is it important to investigate, when do you want to do the project, where is the proposed study site, who are your participants/sample, how will you do it (methods) and what is the proposed outcome/importance?However, what exactly are students learning?Are the students going through the motions just to pass the course or do a few reach the 'light bulb' moment of understanding the concept of research?
Although PBL is a common approach in various disciplines, there is little published on its use in developing research skills, particularly in the South African context.This paper outlines the overall delivery of a PBL undergraduate research methods course with an evaluation of student experience and competence.

Method
A cross sectional descriptive case study of the Research methods 4 undergraduate course in the Bachelor of Technology: Environmental Health Programme within the Faculty of Health Sciences at the Durban University of Technology (DUT) was carried out.All students enrolled in 2015 were invited to participate (n=40).Participation was voluntary and all participants signed informed consent (n=37).The DUT Institutional Research Ethics Committee approved the study.
Students were directed with respect to a topic selection by allocation of a Sustainable Development Goal (SDG) (United Nations Development Programme, 2017) at random.The student was tasked with finding an area of interest within the given SDG.A student, for example, allocated SDG3, chose a topic related to the use and availability of water and sanitation in relation to health outcomes within an informal settlement.SDG3 relates to ensuring healthy lives and promoting wellbeing for all at all ages.It was mandatory that all topics were related to the field of environmental health.With guidance from the instructor, all the proposed topics and aims were discussed and finalised in a one-on-one session.The topics were deconstructed throughout the course.The student was required to prepare a literature review, methodology, ethics documents and data collection instruments.The goal was that each assignment would lead up to the final proposal.The course outline, with the various learning activities and associated assessments, is presented in Table 1.Apart from five formal assessments, small learning activities (as denoted in Table 1) were given after each section to facilitate understanding.In addition, the instructor was available for individual consultations and evaluation of drafts.Additional resources related to each section were provided in an online classroom.

Data collection
Questionnaires were piloted to ensure content validity and reliability.The instructor was not present during questionnaire administration and was blinded to student identifiers.An openended questionnaire was administered to all participants at the end of each section (Table 1) to evaluate their perceptions and understanding of that section.A summative questionnaire was administered at the end of the course.Participants were required to recall how they experienced different sections of the course.Perceived competence with research skills was also evaluated.The summative questionnaire was developed with consideration of the learning outcomes and competencies as described in the course syllabus.Questions included student reflections on learning and understanding of research concepts.This included the development of effective search strategies and skills in order to identify and locate appropriate sources for articulating research topics; preparing a literature review with annotated bibliography; engaging with quantitative and qualitative methodologies; ethics in research; statistics and data analysis; understanding of the research process; proposal development; and, overall experience and perception of the course.Data from open-ended questions was content analysed for emerging themes.Students were required to rate each assessment for difficulty from 1-5 (where 1=easy; 2=moderately easy, 3=average, 4=difficult and 5=most difficult) and a mean score was calculated from student ratings.Summary statistics (mean, median, range) for student assessments were calculated.STATA (version 12) was used for quantitative data analysis.

Session
Learning outcomes Teaching and learning activities* Assessment

The research process
The student should be able to define basic research concepts and explore various research designs; The student should be able to develop and frame ideas or questions into aims and objectives; The student acquire skills to search for scientifically valid literature using various databases; The student will be familiar with reading scientific literature and extracting information; The student should be able to write a literature review and reference appropriately.
On first contact student was given a Sustainable Development Goal (SDG) that was allocated at random.Training on the use of online databases was provided by the library.
Each student submitted a 10-page literature review related to their topic.A list of references was included and prepared according to the Harvard format as per institutional guidelines.Assessed using a rubric.

Methodology
The student should be able to evaluate ethical issues as pertaining to different research scenarios; The student is able to understand the challenges and advantages of qualitative and quantitative methodologies; The student learns the basic principles of qualitative data analysis; The student should be able to acquire basic skills to set up and conduct interviews and focus groups; and The student acquires skills to formulate questionnaires and focus group questions.
Instructor used a single study topic to teach the various topics on methodology.
Students were given a group activity to design a questionnaire on a given topic.This was peer reviewed and critically evaluated by other groups in the class.

Data analysis
The student understands how to deal with research data; The student understands basic statistics terminology; The student should be able to distinguish and apply descriptive and some inferential statistics; and The student is able to use Excel to do graphs and basic statistical analysis.*Contact sessions with the instructor was held for each session and online readings and resources were provided

Results
Of the 40 students enrolled for the course, 37 completed both questionnaires.Most participants were female (65%) with 81% African and 19% (7) Indian.Approximately 73% were part time students.Themes were identified from the collective answers provided for open-ended questions (Table 2).Participants reported that the section on preparing a literature review helped them to understand how to access and deconstruct peer reviewed literature.They acknowledged that the classwork exercises facilitated learning.Participants valued the personal development and the skills transfer such as independent learning scientific writing and reading during this section as the following student quotation suggests: …being able to work on my own and reading up on topics that were related to my study was challenging but helped me to grow.Doing the final literature review was proof of how much knowledge I had gained pertaining to my study topic (Participant 3).
Some participants reported that referencing and scientific writing was challenging and they found difficulty with selecting relevant information with respect to their topics.
Most participants enjoyed the section on methodology.A general methodology plan was provided with guiding subheadings that were structured and simple to follow.In addition, participants were handed a group-work task to formulate a questionnaire on a given topic, which was subsequently peer reviewed and discussed in class.Participants enjoyed these teaching strategies.Solving the problem or 'puzzle' through a well-developed methodology enhanced understanding and participant competence.Participant 4 described her experience with this section: The given questions or breakdown related to methodology gave me clarity about my topic…it was more practical and made me understand all the steps required to conduct a study (Participant 4).
Although some participants enjoyed the opportunity to apply learning, others found it challenging to customise the theoretical concepts of methodology to their respective topics.Participants requested additional learning time for methodology with both good and poor examples of methodology to review in class.Many participants reported that the skills learnt during the section on data analysis and statistics was very useful in teaching them how to analyse data and use MS Excel, but they had difficulty with the statistical discourse in terms of understanding and applying terminology.Many participants could not cope with the pace and it was evident that additional time should be allocated to this section as illustrated by the excerpt below: need to allocate more time..All the information at once was overwhelming and confusing.We need a theory class before the practical session as most of us aren't familiar with terms and the use of Excel (Participant 29).
The proposal writing and dissemination session was both challenging and rewarding for participants.Competence in research was developed as they began to identify the links with topic, aims and methodology.As one student explained: ..now I am able to link my objectives with the methodology that I need for my study, it made me understand that in future, we will be able to write our very own proposal and budget going forward (Participant 36).
In addition, most of the work was already done in previous assessments and they could concentrate on building the final proposal.Individual supervision in this phase was beneficial.Participants enjoyed learning from dissemination of research by oral presentations.Watching their peers present their own projects further enhanced their own understanding of multifaceted research.Participants developed various skills during this session, including resource utilisation and self-awareness.Additional student responses related to each session are included in Table 2.The post-course survey required participants to evaluate aspects of the PBL approach, their perception of skills learned, and overall experience of the course using a 4-point Likert scale (Figure 1).While the majority of participants agreed that the teaching and learning strategies were successful, 30% of participants did not like being directed towards a research topic based on a SDG goal.Participants' perception of skills acquired during the course is presented in Figure 2. Most participants felt competent with aspects of methodology and ethics but 69% did not understand all the concepts of statistics.In addition, some reported a limited perceived competence in searching for relevant literature (47%), evaluating useful sources (25%), and writing a literature review (36%).It was interesting to note that even though 86% agreed that they were able to turn a problem into a research question, only 32% were confident with preparing aims for a particular topic.Figure 3 shows overall evaluation of student experience of the course.Most participants enjoyed the teaching style (66%) and instructor facilitated learning (97%).It was encouraging to note that all participants understood how the various pieces fit together to form a research proposal, and many indicated that they may consider enrolling for a Master's degree in the future.Participants' rating of assessment difficulty and actual mean scores are presented in Table 3 (ranging from 1-5 in terms of difficulty).Participants rated the assessments for the literature review and the statistics/test to be most difficult (3.8 and 3.6 respectively).This was reinforced by the actual mean scores from student assessments with the literature review (57.1) and statistics test (54.8)having the lowest means when compared with other assessments.Exercises in class taught me to use Excel.
Reading material and online resources for methodology section was adequate.
The classwork on questionnaire development was helpful.
I learnt from the class activity on focus groups.
The one-on-one interaction with the instructor on the lit.review was beneficial.
I found the exercise on summarising a journal article useful.
I preferred being given an SDG to direct my research.
Exercises in class taught me to use Excel.
Reading material and online resources for methodology section was adequate.
The classwork on questionnaire development was helpful.
I learnt from the class activity on focus groups.
The one-on-one interaction with the instructor on the lit.review was beneficial.
I found the exercise on summarising a journal article useful.
I preferred being given an SDG to direct my research.I completed the classwork with ease.I understood the concepts in statistics.
I know the difference between quantitative and qualitative methods.
I am able to prepare informed consent for a research project.
I understand the importance of ethical research.
I know how to construct a reference list.
I know how to reference in-text.
I understand the concept of plagiarism.
I am competent at writing scientifically for a literature review.
I know how to use information from journal articles to write a literature review.
I know how to evaluate if literature sources are useful or not.
I had difficulty finding journal articles relevant to my topic.
It was the first time I read a journal article.
I can prepare aims for a particular topic.
I know how to turn a problem into a research question.
I completed the classwork with ease.I understood the concepts in statistics.
I know the difference between quantitative and qualitative methods.
I am able to prepare informed consent for a research project.
I understand the importance of ethical research.
I know how to construct a reference list.
I know how to reference in-text.
I understand the concept of plagiarism.
I am competent at writing scientifically for a literature review.
I know how to use information from journal articles to write a literature review.
I know how to evaluate if literature sources are useful or not.
I had difficulty finding journal articles relevant to my topic.
It was the first time I read a journal article.
I can prepare aims for a particular topic.
I know how to turn a problem into a research question.I enjoyed the teaching style used in this course where I was learning through doing.
The assessments were relevant and contributed to my learning.
I prefer the examination method rather than continuous assessment.
I may enrol for a Master's degree in future.
I understand how all the pieces I learnt throughout the year fit together for a research proposal.
The instructor facilitated my learning.
I am able to prepare a research proposal on a given topic.
I would have liked to actually do my proposed project.
I enjoyed the teaching style used in this course where I was learning through doing.
The assessments were relevant and contributed to my learning.
I prefer the examination method rather than continuous assessment.
I may enrol for a Master's degree in future.
I understand how all the pieces I learnt throughout the year fit together for a research proposal.
The instructor facilitated my learning.
I am able to prepare a research proposal on a given topic.*Ratings from 1-5 where 1 = easy and 5 = very difficult

Discussion
Research methods is a difficult course to teach at the undergraduate level and requires that the instructor reach into the teaching and learning toolkit to find innovative strategies to impart knowledge and skills related to a complex discourse.Although PBL is a common approach in various disciplines, there has been little published on its use in developing undergraduate research skills.This study evaluated student experience and competence using PBL as a teaching paradigm for a research methods undergraduate course.Participants generally benefitted from the PBL approach and felt that independent and critical learning helped develop skills that they could transfer to their professional environment.Participants reported difficulty with statistical concepts and terminology although they enjoyed the practical component.
The final proposal was a 'light bulb' moment and participants successfully converted their problems into proposals.
The Research methods 4 course is skills based to enable students to be intelligent users of research and to effectively design, implement and interpret projects in their professional lives.However, in my experience of teaching Research methods 4 over 10 years, it would appear that most students dread the course and find it abstract and difficult.A facilitator needs to address these concerns while ensuring that the course is relevant, current and broadly focused enough to hold the interest of students specialising in the various fields of environmental health, while also stimulating independent learning and an interest in research.This is a daunting prospect for the instructor, particularly as it involves supervising the progress of over 40 completely different research project proposals from novice students.
Different methods of delivery for research methods have been explored within the context of the Faculty of Health Sciences at DUT.The first strategy was that all students across various disciplines were combined and academics with appropriate expertise were then assigned a particular topic e.g.qualitative research.Unfortunately, this met with limited success as the students failed to connect research content with their discipline areas.In the subsequent model, each student was required to design, execute and report on a mini research project within a very short space of time (about 10 months).The format of delivery and assessment was then modified within our programme to ensure that students are equipped with the necessary tools and skills to identify a problem and locate practical solutions in a systematic way.The research proposal is the final outcome, so that students learn each new research related skill, and then test their abilities by fitting together the pieces of a jigsaw puzzle into one coherent proposal.
As far back as 1975, research methods instructors have recognised that "being able to do research is a skill consisting of both knowledge and practical decision-making competence" (Earley, 2014).A review of literature on various prescribed approaches to teaching research methods showed that international universities have significant emphasis on training students to conduct research (Wagner et al., 2011).There are various methods employed by universities to transfer research method skills to students but there is no obvious 'one size fits all' strategy.While current research teachers have access to expansive literature explaining research methods, the jury is still out on the most effective teaching and learning styles of concepts (Wagner et al., 2011).The PBL approach may be well suited for delivery of research methods.During PBL, the students themselves describe the context, develop and construct the problem given minimal direction.This strategy encourages the "how to learn" as well as decision-making about "what to choose" (Carlisle & Ibbotson, 2005).Students are engaged with using their research question or problem to unpack each component of the proposal writing process.They journey tentatively, outcome by outcome, into the research process by formulating a research question; conducting a literature review; devising a methodology to address the research question; data analysis; dissemination; and, proposal writing.Formative exercises are given after each section to evaluate understanding.The final product or 'solution' is a scientifically rigorous research proposal.Such instructional strategies challenge students to solve 'authentic' problems and acquire complex knowledge in information-rich settings based on the assumption that having students construct their own solutions leads to the most effective learning experience.Problem-based learning also develops 'soft skills' such as scheduling and planning of tasks, organising and summarising information, and reading and analysing literature (Kirschner, Sweller & Clark, 2006).
In this study, the initial contact session, where the participants were directed with respect to developing their topics and aims, was useful in fostering an early 'supervisory' relationship between instructor and student.This allowed for an ongoing collaboration where the student was comfortable with 'exposing their flaws' to the instructor.I reinforced the notion that the research question and design should only be as complicated as is necessary to answer the research question.Participants were introduced to peer reviewed literature and database searching which proved useful in understanding their own topics.The realisation that every journal article was 'someone's actual research project' was discussed in class.Participants found that peer assessment of actual examples of literature reviews were very helpful.They wanted examples of 'bad' literature reviews so that they could learn from them.Developing a critical balanced literature review was a difficult task for participants.This is an essential skill as Tashakkori and Teddlie (2003) reported that students develop an appreciation for the changing landscape of knowledge construction and are able to see how a hypothesis is supported in one study but may be refuted in another.
Competence was developed by guiding participants to develop the methodology in a stepwise method.Students were encouraged to engage with other proposals and dissertations that showcase various research constructs which enabled them to connect theory with reality.It was interesting to note that participants once again applied this learning to their professional work environments.While undergraduate students should understand basic statistics for a given set of data, they should not be expected to derive statistical formulae.In general, students are more receptive to a discussion of concepts with relatable examples.Authors agree that statistics may dazzle students with jargon and rhetoric and tend to emphasise data analysis rather than the process (Denham, 1997;Ryan et al., 2014).In this study, theory and practical was directly linked in the methodology section, and the participants enjoyed learning MS Excel with the use of practical data analysis examples.Apart from skills development, the participants also felt that the session challenged their ability to evaluate information or data critically.However, participants had difficulty with complex statistical discourse and jargon, finding it hard to keep up with the terminology, concepts, problems and practicals at the same time.Participants suggested that we improve this section by including additional time and offering pre-stats lectures that introduce terminology in preparation for the statistics session.The section on proposal writing and dissemination was the 'light bulb' moment for participants, where they were able to link concepts together to form a rigorous research proposal.
PBL was used to organise the curricular content around problem scenarios and the student was guided to adopt a systematic, student-centred enquiry process.This learning process may be described as a "construction of knowledge" through the use of disciplined inquiry and the use of PBL in teaching research methods enables instructors to deliver the constructivist philosophy to successfully assist students to create and understand knowledge (Kemp, 2011).An understanding of the constructivist underpinnings of PBL enables instructors to reflect on the goals of teaching and pedagogical strategies to promote learning within the context of courses such as Research methods.One cannot expect that undergraduate research students will understand the entire research discourse after a year's instruction, but we may guide them towards solving a single problem using a single strategy and provide the skills-set required to investigate other problems using similar resources.Other studies have reported that a successful course in research methods at the undergraduate level may foster lifelong learning, professional development and personal growth and further develop selfconfidence and self-esteem.In addition, it may inspire them to pursue postgraduate studies with more confidence (Dohm & Cummings, 2002;Waite & Davis, 2006;Crooks, Castleden, and Meerveld, 2010;Lombard & Kloppers, 2015).
South African HEIs need to strategise around building a strong research base that is underpinned by developing critical thinking and problem solving skills.At the same time, there is a need to be responsive and relevant to the current problems faced by community.Problem based learning provides the opportunity to combine these goals to facilitate learning in a subject that has been reported to be complex and difficult to both facilitate and understand.Delivery of PBL may vary from the 'pure' PBL (presentation of the problem with no prior lectures or presentation of other materials) model where students work in small groups with a facilitator, to a 'hybrid' model where students are supported by lectures and greater facilitator intervention (Read, 2010).The 'hybrid' model was adopted in this context and the learning outcomes were achieved successfully.Flexibility and consideration of context, resources and prior learning is imperative in using the PBL approach to teach undergraduate research methods.

Conclusion
Research methods are a pivotal course for undergraduate health science students, as they are required to address a myriad of community health issues in their professional lives.Empowering students with skills to identify a problem and to look for plausible solutions in a systematic and scientific manner strengthen the students' ability to work, think and practice independently.This evaluation showed that the PBL was generally successful as students felt that they used self-directed learning with instructor facilitation and supervision to reach their goal of developing independent research proposals.In addition, they recognised that they would be able to use these skills in the workplace and some were motivated to pursue postgraduate work.Problem-based learning requires significant commitment from both the student and the facilitator but it has the potential to build research capacity and support the postgraduate initiative in South African institutions of higher learning.

Figure 1 :Figure 1 :
Figure 1: Responses of Research methods 4 participants regarding the teaching and learning

Figure 2 :
Figure 2: Participants' perception of skills development during the Research methods course (n = 37) I would have liked to actually do my proposed project.

Table 2 :
Participant responses to open-ended questions after each completed section

Table 3 :
Participants' ratings and achievement in Research Methods (n = 37)