Create AI-sensitive exams
One of the most discussed questions when it comes to AI in education is, how to create examinations in a world where AI has been shown to pass advanced examinations, such as entrance exams.
Especially during unsupervised, text-based examinations, such as a Bachelor's- or Master's Thesis or written assignments, feelings or uncertainty are growing.
On this page, we'd like to give you general recommendations on how to create AI-sensitive examinations.
During the second part, we'd like to explore how existing exams can be adjusted, so that AI will not be able to replace the student's proof of the acquired competences. The recommendations on this page are taken from the writings of Prof. Dr. Albert Busch, whom we'd like to thank for the permission to publish them.
1. Subjective Questions: Ask questions that require subjective answers and a personal interpretation or analysis. Avoid questions with clear factual answers that can be easily looked up in texts or databases. For example, you could ask about outlooks or discussions of a research topic, interpretations and competing interpretations, or creative solutions to topics or texts.
2. Essays and assignments: Ask your students to write essays or assignments that require profound analysis, argumentation and independent classification and trade-off of research positions. Such exercises usually require critical thinking and individuality, that can hardly be replicated by AI.
Tip: The E-Examination room offers the possibility to write essays and assignments in a controlled environment on a PC. Additionally, there will no problems with readability due to different handwritings.
3. Group discussions: Organise group discussions, where students have to interact and discuss with each other in real time. This requires critical thinking and reacting to the contributions of others, which is diffcult for AI to do.
4. Practical applications: Create examinations, that are based on real applications. This could entail the analysis of a real case study, practical problem solving or the creation of creative projects, that require a deep understanding of the topic.
5. Open questions and discussions: Give your students the possibility to pose their own questions and partake on open discussions, where they can freely exchange their thoughts and ideas. This promotes critical thinking and the development of resoning skills.
These measures obviously require additional work and the further development of existing procedures, and they could render exam preparations more complex. Nonetheless, they could contribute to a successful evaluation of real understanding and critical thinking.
Now, we'd like to propose possible solutions to differnt kinds of examinations.
Assignments can be coupled with oral examinations. This might entail a change in the mode of examination in the examination regulations (Prüfungsordnung), for instance, if a former solely written exam will be converted to a written exam with oral evaluations.
But even without a change in the examination regulations, more AI-sensitive exams can be achieved with these tweakments:
1. Clear Question and Originality: Make sure, that the students select a clear research question, or that they select a topic that requires originality and a unique thought process. The research question should not be answerable by looking up databases or reading single texts.
2. Use of specific and empiric research questions: Encourage research questions that require critical reflection and not only factual knowledge.
3. Own Empiricism: Encourage own empirical projects that answer specific research questions.
4. Comprehensive literature research: Do pose great attention to a thorough and systematic literature research. The students should search and critically evaluate relevant sources and build their work upon well-founded information.
5. Specify research reflection: For example, specify certain research positions or literature that must be applied to given texts, authors, genres, narrative styles, objects, procedures, epochs, historical events and constellations, etc. in the context of the seminar paper
6. Independent argumentation: If possible, set the task in such a way that students must be able to develop and defend their own arguments in order to answer it. The seminar paper should not only be a compilation of information (AI can usually do this better today), but should also contain its own critical analysis and argumentation.
7. Unique perspectives and interpretations: Wherever possible, encourage students to present self-developed and professionally validated perspectives or interpretations of the topic. This requires creative thinking and the ability to develop ideas that go beyond what is found in the existing literature
8. References to discussions in the seminar: If the seminar has had discussions on relevant topics or papers during the semester, ask the students to take these discussions into account in their work and to refer to them.
9. Peer-Feedback: Implement peer feedback sessions where students share their work at the draft stage with their peers and receive constructive feedback
10. Reflection section for critical examination of sources and research discussion: Ask students to critically question and discuss the credibility and relevance of the sources they use in a reflection section of the paper. This also shows that they are able to distinguish high-quality information from inferior, no longer tenable or outdated information.
11. Plagiarism check: Carry out a thorough plagiarism check (see above for software) to ensure that the submitted work does not contain any unmarked text passages.
12. Presentation and defense: If you can organize the time, have the students present and defend their seminar paper or their own basic lines of argumentation for it (e.g. in a subsequent office hour or a scheduled appointment). This will allow you to ask questions and assess the students' actual knowledge and understanding.
By implementing such measures, you can ensure that term papers require individuality, research-based critical thinking and a thorough exploration of the chosen topic, making it more difficult to create them through AI.
1. Individual Presentations: For example, use individualized questions for the presentation to ensure that students also create their presentations individually. This prevents them from relying on ready-made presentations (or presentations already created by other people). In large events, it also makes it possible to distribute presentations among groups so that the individual parts remain clearly visible.
2. Own research: Provide presentation tasks that require you to base presentations on your own research, understanding and reflection.
3. Use current literature: Emphasize the need to use current and relevant literature and research in their presentations to ensure that they are using current information and that their presentations are up to date.
4. Sources: Ask students to properly cite all sources they use in their presentation and discuss them critically if necessary.
5. Originality of arguments: Formulate requirements that make it necessary for students to develop their own arguments and not simply reproduce information. They should show that they are able to develop their own points of view and classify, weigh and defend their own interpretations.
6. Discussion and interaction: After the presentation (fixed time limit), questions and discussions with the other seminar participants should be allowed. This also allows teachers to assess students' actual understanding and reasoning skills.
7. Peer-Feedback: Facilitate feedback rounds in which students evaluate the presentations of their fellow students and give constructive feedback.
8. Assessment criteria: Continue to clarify in advance the criteria according to which the presentations will be assessed, (e.g. technical adequacy, adherence to time, own positioning and understanding...) in order to convey clear expectations to the students regarding the performance requirement.
9. Plagiarism check: If necessary, carry out a plagiarism check to ensure that the presentations do not contain any unmarked text passages or graphics.
A portfolio in a seminar setting is used to document a student's work, progress and achievements over a period of time or on a specific topic. What can be done here to ensure that a portfolio is authentic and representative of the student's achievements and learning process and is based on originality, authenticity and non-copyable own achievements?
1. Clear requirements: Provide clear requirements for the portfolio and define the goals and expectations. This may include the type of material to be included in the portfolio as well as the structural design and scope of the portfolio.
2. Reflection and analysis: Encourage students to add reflective texts or comments to each item in the portfolio. This allows students to document and analyze their thoughts and learning process.
3. Authentic work: Make sure that the work contained in the portfolio is authentic and was created by the students themselves. Check the originality of the work by means of plagiarism checks (see above for software) or other suitable means
4. Show progress and development: The portfolio should document the student's development and knowledge of a subject area or issue over time.This can be achieved by including earlier drafts, sketches or work samples in conjunction with the final work.
5. Self-assessment: Encourage students to assess themselves and reflect on their strengths, weaknesses and learning goals in the portfolio. This promotes personal responsibility for learning.
6. Diverse content: The portfolio should include different types of content, such as written work, presentations, projects, research findings, sketches, reflections and more to show the breadth of the learning process.
7. Accompanying explanations: Ask students to provide an accompanying explanation for each item in the portfolio. They should explain why they have chosen certain pieces of work and how these contribute to their learning objectives and academic progress.
8. Assessment criteria: Clarify in advance the criteria according to which the portfolio will be assessed. This gives students clear expectations and makes the assessment easier.
9. Peer feedback: Implement peer feedback sessions where students evaluate their peers' portfolios and provide constructive feedback.
10. Progress monitoring and conversations: Where possible, have regular conversations with students about their progress to ensure that they are actually working on compiling and reflecting on their portfolio and not relying on unauthorized help.
Designing language tests that are difficult for AI to master requires the integration of tasks that require human skills and understanding that are difficult for AI to replicate. Depending on the competence to be tested, the following designs, for example, can help:
1. Oral examinations with open questions: If the examination format is suitable, ask oral questions that require a personal, spontaneous answer from the students. Avoid pure knowledge questions that can easily be answered from previous notes or knowledge from books.
2. Interactive conversations: Conduct exams where students have to interact in a real conversation situation. This can be a role-play exercise, for example, where they have to react to unexpected turns of events.
3. Authentic text analysis: Ask students to analyze complex texts, articles or poems and write interpretations, arguments or reviews. These assignments require a deep understanding of the texts and the ability to critically analyze them.
4. Creative writing exercises: Have students write creative texts such as stories, poems or essays. These assignments require originality and creative thinking
5. Practical applications: Require students to give oral or written instructions or explanations of practical tasks or processes. This can test the ability to communicate clearly and solve problems.
6. Critical discussions: Organize discussion groups or debates on controversial topics and assess students' ability to present their arguments and respond to counter-arguments.
7. Linguistic creativity: Ask students to invent new words or terms or to use metaphors and puns in their native language. These tasks test the ability to use language creatively.
8. Individual presentations: Have students create and deliver presentations on complex topics that require them to demonstrate their knowledge of the research and understanding of the topic in the foreign language.
9. Communication in different contexts: Check the students' ability to express themselves appropriately in different communication situations and text types such as formal presentations, informal conversations or written texts at the desired level of competence.
10. Authentic conversations: Create tasks that require students to have real conversations with people from different cultural backgrounds to assess intercultural communication skills.
11. Evaluate idioms and cultural nuances: Ask students to identify and explain idioms, cultural nuances or humorous expressions in texts.
12. Phonetic tasks: Have the students solve tasks on phonetics and the correct pronunciation of words or phrases in presence.
13. Text production under time pressure: Require students to produce tasks such as summaries, comments or translations in a limited amount of time to test their ability to process language quickly.
If you have specific experience with examination formats for a possible good practice scenario, please contact us at elearning@uni-goettingen.de.