STUDENT EVALUATION, ASSESSMENT, & TESTING

Question 1

WHAT IS AN ASSESSMENT

Answer 1

• ASSESSING MEANS EVALUATING THE QUALITY OR ABILITY OF SOMEONE OR SOMETHING
• INSTRUCTORS CONTINUOUSLY ASSESS A LEARNER’S PERFORMANCE TO PROVIDE GUIDANCE, SUGGESTIONS FOR IMPROVEMENT, & POSITIVE REINFORCEMENT

Question 2

WHAT IS THE PURPOSE OF AN ASSESSMENT?

Answer 2

• A GOOD ASSESSMENT PROVIDES PRACTICAL & SPECIFIC FEEDBACK TO LEARNERS; THIS INCLUDES DIRECTION & GUIDANCE INDICATING HOW THEY MAY RAISE THEIR LEVEL OR PERFORMANCE
• A WELL DESIGNED & EFFECTIVE ASSESSMENT PROVIDES AN OPPORTUNITY FOR SELF-EVALUATION THAT ENHANCES AERONAUTICAL DECISION-MAKING & JUDGMENT SKILLS
• IT ALSO HIGHLIGHTS AREAS IN WHICH A LEARNER’S PERFORMANCE IS INCORRECT OR INADEQUATE, IT HELPS THE INSTRUCTOR SEE WHERE MORE EMPHASIS IS NEEDED

A PROPER ASSESSMENT..

1.) CONTRIBUTES TO LEARNING
2.) REVEALS THE QUALITY OF INSTRUCTION
3.) HIGHLIGHTS STRONG & WEAK AREAS
4.) INDICATES IF THE LEARNING OBJECTIVES HAVE BEEN MET

Question 3

WHAT ARE THE 8 CHARACTERISTICS OF AN EFFECTIVE ASSESSMENT? (COCOAS TF)

Answer 3

1.) COMPREHENSIVE: COVERS BOTH STRENGTHS & WEAKNESSES & NOT LONG

2.) OBJECTIVE: NEEDS TO BE HONEST & MUST BE BASED ON THE FACTS OF THE PERFORMANCE AS THEY WERE; SHOULD NOT REFLECT THE PERSONAL OPINIONS OF THE INSTRUCTOR

3.) CONSTRUCTIVE: BENEFITS THE LEARNER; WHEN IDENTIFYING A MISTAKE OR WEAKNESS, THE INSTRUCTOR NEEDS TO GIVE POSITIVE GUIDANCE FOR CORRECTION

4.) ORGANIZED: NEEDS TO BE LOGICAL & MAKE SENSE TO THE LEARNER

5.) ACCEPTABLE: THE LEARNER MUST ACCEPT THE INSTRUCTOR TO ACCEPT THE CRITIQUE

6.) SPECIFIC: GUIDANCE SHOULD BE CLEAR & CONCISE

7.) THOUGHTFUL: IT SHOULD NOT RIDICULE OR ANGER THE LEARNER

8.) FLEXIBLE: FITS THE LEARNER & THE OCCASION & ALLOWS FOR VARIABLES

Question 4

WHAT ARE THE 2 TYPES OF ASSESSMENTS?

Answer 4

1.) TRADITIONAL ASSESSMENT: EMPHASIZES STRUCTURED TESTING WITH DEFINED ANSWERS, OFFERING A CLEAR MEASURE OF RETAINED KNOWLEDGE

2.) AUTHENTIC: EMPHASIZES PERFORMANCE IN REALISTIC SITUATIONS, SHOWING HOW KNOWLEDGE IS APPLIED IN PRACTICE

Question 5

WHAT IS A TRADITIONAL ASSESSMENT & WHAT DOES IT CONSIST OF?

Answer 5

• TRADITIONAL ASSESSMENTS GENERALLY REFER TO WRITTEN TESTS WITH LIMITED CHOICES, SUCH AS MULTIPLE-CHOICE, MATCHING, OR FILL-IN-THE-BLANK
• THEY CAN HELP ASSESS A LEARNER’S PROGRESS WITHIN A COURSE, BUT OFFER LITTLE OPPORTUNITY TO EVALUATE CRITICAL THINKING SKILLS

TRADITIONAL ASSESSMENTS:

1.) REQUIRE LEARNERS TO DEMONSTRATE THE ROTE & UNDERSTANDING LEVELS OF LEARNING
2.) ARE INSTRUCTOR-CENTERED. THEY BENEFIT THE INSTRUCTOR BY BEING EASY TO GIVE & GRADE

Question 6

WHAT ARE CHARACTERISTICS OF A GOOD WRITTEN ASSESSMENT (TEST) - TRADITIONAL ASSESSMENT? (6 - CRUDO V)

Answer 6

1.) COMPREHENSIVENESS: SAMPLES A GOOD CROSS-SECTION OF ALL THE KNOWLEDGE BEING MEASURED

2.) RELIABILITY: YIELDS CONSISTENT RESULTS EACH TIME THE TEST IS ADMINISTERED

3.) USABILITY: EASY TO GIVE, EASY TO READ, & EASILY GRADED

4.) DISCRIMINATION: SMALL DIFFERENCES IN ACHIEVEMENT CAN BE MEASURED BETWEEN LEARNERS

5.) OBJECTIVITY: THE TEST CAN BE GRADED CONSISTENTLY; SELECTION-TYPE TEST ITEMS ARE MUCH EASIER TO GRADE OBJECTIVELY

6.) VALIDITY: A TEST IS VALID WHEN IT ACCURATELY MEASURES THE LESSON OBJECTIVES, MAKING IT THE MOST IMPORTANT FACTOR IN TEST EVALUATION

Question 7

WHAT IS AN AUTHENTIC ASSESSMENT & WHAT DOES IT CONSIST OF?

Answer 7

• AUTHENTIC ASSESSMENTS FOCUS ON REAL-WORLD TASKS, ALLOWING LEARNERS TO GENERATE SOLUTIONS IN “AUTHENTIC” SETTINGS RATHER THAN SELECTING PREDETERMINED RESPONSES
• BY INSTRUCTORS USING OPEN-ENDED QUESTIONS & ESTABLISHED PERFORMANCE CRITERIA, AUTHENTIC ASSESSMENT FOCUSES ON THE LEARNING PROCESS, ENHANCES THE DEVELOPMENT OF REAL-WORLD SKILLS, ENCOURAGES HIGHER ORDER THINKING SKILLS & TEACHES LEARNERS TO ASSESS THEIR OWN WORK & PERFORMANCE

AUTHENTIC ASSESSMENTS:

1.) REQUIRE LEARNERS TO DEMONSTRATE THE APPLICATION & CORRELATION LEVELS OF LEARNING
2.) ARE LEARNER-CENTERED. THEY HELP THE LEARNER BY DOUBLING AS LEARNING EXPERIENCES

Question 8

WHAT IS A LEARNER-CENTERED ASSESSMENT & WHAT ARE THE 4 STEP SERIES OF OPEN-ENDED QUESTIONS?

Answer 8

• LEARNER-CENTERED ASSESSMENT IS THE USE OF OPEN-ENDED QUESTIONS IN A COLLABORATIVE CRITIQUE (FOCUSED ON LEARNING RATHER THAN GRADING)

4 STEP SERIES OF OPEN ENDED QUESTIONS:

1.) REPLAY: THE INSTRUCTORS ASKS THE LEARNER TO VERBALLY REPLAY THE FLIGHT OR PROCEDURE
- WHILE THE LEARNER SPEAKS, THE INSTRUCTOR LISTENS FOR AREAS WHERE THE ACCOUNT DOES NOT SEEM ACCURATE

2.) RECONSTRUCT: THE RECONSTRUCTION STAGE ENCOURAGES LEARNING BY IDENTIFYING THE KEY THINGS THAT THE LEARNER WOULD HAVE, COULD HAVE, OR SHOULD HAVE DONE DIFFERENTLY DURING THE FLIGHT OR PROCEDURE

3.) REFLECT: INSIGHTS COME FROM INVESTING PERCEPTIONS & EXPERIENCES WITH MEANING, REQUIRING REFLECTION ON THE EVENTS
- EX.. WHAT WAS THE MOST IMPORTANT THING YOU LEARNED TODAY?

4.) REDIRECT: FINAL STEP IS TO HELP THE LEARNER RELATE LESSONS LEARNED IN THIS SESSION TO OTHER EXPERIENCES & CONSIDER HOW THEY MIGHT HELP IN FUTURE SESSIONS
- QUESTIONS MIGHT INCLUDE.. HOW DOES THIS EXPERIENCE RELATE TO PREVIOUS LESSONS?

Question 9

WHAT ARE MANEUVER OR PROCEDURE “GRADES” (4)

Answer 9

1. DESCRIBE: THE LEARNER CAN DESCRIBE THE PHYSICAL CHARACTERISTICS OF THE MANEUVER BUT NEEDS ASSISTANCE TO EXECUTE THE MANEUVER OR PROCEDURE SUCCESSFULLY

2.) EXPLAIN: LEARNER CAN EXPLAIN THE MANEUVER;S UNDERLYING CONCEPTS, PRINCIPLES, & PROCEDURES BUT STILL NEEDS ASSISTANCE IN EXECUTING THE MANEUVER OR PROCEDURE SUCCESSFULLY

3.) PRACTICE: LEARNER CAN PLAN & EXECUTE THE MANEUVERS WITH COACHING & ASSISTANCE TO CORRECT DEVIATIONS & ERRORS

4.) PERFORM: LEARNER CAN PLAN & EXECUTE THE MANEUVER TO CERTIFICATION STANDARDS WITHOUT ASSISTANCE OR COACHING. LEARNER CAN IDENTIFY & CORRECT ERRORS & DEVIATIONS

Question 10

HOW SHOULD INSTRUCTORS ASSESS RISK MANAGEMENT SKILLS?

Answer 10

1.) EXPLAIN: THE LEARNER CAN VERBALLY IDENTIFY, DESCRIBE, & UNDERSTAND THE RISKS INHERENT IN THE FLIGHT SCENARIO, BUT NEEDS TO BE PROMPTED TO IDENTIFY RISKS & MAKE DECISIONS

2.) PRACTICE: THE LEARNER IS ABLE TO IDENTIFY, UNDERSTAND, & APPLY SRM PRINCIPLES TO THE ACTUAL FLIGHT SITUATION. THE LEARNER IS AN ACTIVE DECISION MAKER

3.) MANAGE-DECIDE: THE LEARNER CAN CORRECTLY GATHER THE MOST IMPORTANT DATA AVAILABLE BOTH INSIDE & OUTSIDE THE FLIGHT DECK, IDENTIFY POSSIBLE COURSES OF ACTION, EVALUATE THE RISK INHERENT IN EACH COURSE OF ACTION & MAKE THE APPROPRIATE DECISION
- INSTRUCTOR INTERVENTION IS NOT REQUIRED FOR THE SAFE COMPLETION OF THE FLIGHT

Question 11

HOW DO YOU CHOOSE AN EFFECTIVE ASSESSMENT? (DLED)

Answer 11

1.) DETERMINE LEVEL-OF-LEARNING OBJECTIVES: THE TRAINING OBJECTIVES SHOULD MEASURE ONE OF THE LEARNING LEVELS OF THE COGNITIVE, AFFECTIVE, OR PSYCHOMOTOR DOMAINS
- EX.. IF ROTE KNOWLEDGE IS BEING EVALUATED, INSTRUCTORS SHOULD REFERENCE THE LOWEST ;EVEL OF THE COGNITIVE DOMAIN

2.) LIST INDICATORS OF DESIRED BEHAVIORS: LIST SAMPLES OF BEHAVIORS THAT INDICATE THE OBJECTIVE’S ACHIEVEMENT
- EX.. THE GENERAL STEPS TO COMPLETE A NAVIGATION LOG CAN BE LISTED IF THE ASSESSMENT INVOLVES CROSS-COUNTRY FLIGHT PLANNING

3.) ESTABLISH CRITERION (PERFORMANCE-BASED) OBJECTIVES: THE INSTRUCTORS INCLUDES LIMITING PARAMETERS (CRITERIA) FOR THE LISTED BEHAVIORS; CRITERION OBJECTIVES STATE THE CONDITIONS UNDER WHICH THE BEHAVIOR MUST BE PERFORMED & THE CRITERIA TO BE MET
- EX.. AS A CONDITION, THE INSTRUCTOR MAY LIMIT THE LEARNER’S USE OF ELECTRONIC DEVICES DURING CROSS-COUNTRY FLIGHT PLANNING

4.) DEVELOP CRITERION-REFERENCED TEST ITEMS: THE OBJECTIVES SHOULD SERVE AS A REFERENCE FOR DEVELOPING TEST QUESTIONS
- EX.. PRACTICAL TESTS ARE CRITERION-REFERENCED TESTS BECAUSE THEY HAVE ESTABLISHED PERFORMANCE STANDARDS. THE CRITERION (PERFORMANCE-BASED) OBJECTIVES ARE CONTAINED IN THE ACS

Question 12

WHAT IS A CRITIQUE?

Answer 12

• A CRITIQUE IS A DETAILED ANALYSIS OF A SPECIFIC PERFORMANCE; IT COVERS STRENGTHS & WEAKNESSES & PROVIDES SUGGESTIONS FOR IMPROVEMENT
• IT CAN BE ORAL, WRITTEN, OR BOTH

Question 13

WHAT ARE THE 7 DIFFERENT TYPES OF CRITIQUES? (WILIS IS)

Answer 13

1.) WRITTEN CRITIQUE: CRITIQUES IN A WRITTEN FORMAT. THEY CAN HELP THE INSTRUCTOR COVER ALL AREAS THAT WERE NOTICED DURING THE LESSON. THE LEARNER CAN GET A COPY TO REFERENCE LATER

2.) INSTRUCTOR/LEARNER CRITIQUE: THE INSTRUCTOR LEADS A GROUP DISCUSSION IN WHICH CLASS MEMBERS ARE INVITED TO OFFER CRITICISM OF A PERFORMANCE. THIS METHOD SHOULD BE CONTROLLED CAREFULLY

3.) LEARNER-LED CRITIQUE: THE INSTRUCTORS ASKS A LEARNER TO LEAD THE ASSESSMENT. IF THE LEARNER IS INEXPERIENCED, THESE CRITIQUES MAY NOT BE EFFICIENT

4.) INSTRUCTOR-LED CRITIQUE: AN INSTRUCTOR CRITIQUES A LEARNER’S PERFORMANCE TO IMPROVE SKILL, PROFICIENCY, & LEARNING. IT MAY BE CONDUCTED PRIVATELY OR BEFORE AN ENTIRE CLASS

5.) SMALL-GROUP CRITIQUE: A CLASS IS DIVIDED INTO SMALL GROUPS, & EACH GROUP IS ASSIGNED A SPECIFIC AREA TO ANALYZE. THE GROUPS THEN PRESENT THEIR FINDINGS TO THE CLASS

6.) INDIVIDUAL LEARNER CRITIQUE BY ANOTHER LEARNER: THE INSTRUCTOR REQUIRES ANOTHER LEARNER TO PRESENT A CRITIQUE. A VARIATION IS FOR THE INSTRUCTOR TO ASK SEVERAL LEARNERS ABOUT THE PERFORMANCE

7.) SELF-CRITIQUE: A LEARNER SELF-ASSESSES THEIR OWN PERFORMANCE

Question 14

WHAT ARE THE 2 TYPES OF ORAL ASSESSMENTS?

Answer 14

1.) OBJECTIVE QUESTIONS: THESE QUESTIONS HAVE ONLY ONE CORRECT & TYPICALLY ADDRESS THE SPECIFICS OF WHO, WHAT, WHEN, & WHERE

2.) OPEN-ENDED QUESTIONS: THESE QUESTIONS IMPLICITLY ASK FOR COMPLETION (E.G., WHY OR HOW). THEY REQUIRE LEARNERS TO ANALYZE SITUATIONS, SOLVE PROBLEMS, & REACH CONCLUSIONS

Question 15

WHAT ARE THE CHARACTERISTICS OF AN EFFECTIVE QUESTION? (CABBP)

Answer 15

CHARACTERISTICS OF AN EFFECTIVE QUESTION:

1.) CENTER ON ONLY ONE IDEA
2.) APPLY TO THE SUBJECT OF INSTRUCTION
3.) BE BRIEF & CONCISE, BUT ALSO CLEAR & DEFINITE
4.) BE ADAPTED TO THE ABILITY, EXPERIENCE, & STAGE OF TRAINING OF THE LEARNERS
5.) PRESENT A CHALLENGE TO THE LEARNERS

Question 16

WHAT ARE THE 7 TYPES OF QUESTIONS SHOULD YOU AVOID IN AN ORAL ASSESSMENT? (PITOT BY)

Answer 16

1.) PUZZLING: QUESTIONS WITH MANY PARTS (PUZZLE PIECES) THAT MUST BE ASSEMBLED BEFORE THEY CAN BE UNDERSTOOD
EX.. “WHAT IS THE FIRST ACTION YOU SHOULD TAKE IF A CONVENTIONAL GEAR AIRPLANE WITH A WEAK RIGHT BRAKE IS SWERVING LEFT IN A RIGHT CROSSWIND FURING A FULL FLAP, POWER-ON WHEEL LANDING?”

2.) IRRELEVANT: QUESTIONS THAT ARE NOT RELATED TO THE LEARNING OBJECTIVE
EX.. “HOW DO YOU DO A COMPRESSION CHECK?”

3.) TOSS-UP: QUESTIONS WITH TWO VIABLE OPTIONS
EX.. “IN AN EMERGENCY, SHOULD YOU SQUAWK 7700 OR PICK A LANDING SPOT?”

4.) OVERSIZE: QUESTIONS THAT ARE TOO BROAD
EX.. “WHAT SHOULD YOU DO BEFORE A CROSS-COUNTRY FLIGHT?”

5.) TRICK: QUESTIONS THAT ARE DESIGNED TO MISLEAD THE LEARNER PURPOSELY
EX.. A QUESTION IN WHICH THE RESPONSE OPTIONS ARE “A.) CLASS B AIRSPACE” & “B.) CLASS A AIRSPACE”

6.) BEWILDERING: QUESTIONS THAT ARE HARD TO READ OR UNDERSTAND

7.) “YES” OR “NO”: QUESTIONS THAT REQUIRE A SHORT ANSWER. INSTRUCTORS SHOULD USE OPEN-ENDED QUESTIONS, TYPICALLY BEGINNING WITH “HOW” OR “WHY”

Question 17

HOW SHOULD INSTRUCTORS RESPOND TO LEARNER QUESTIONS? (4)

Answer 17

WHEN ANSWERING A LEARNER’S QUESTION, INSTRUCTORS SHOULD:

1.) ENSURE THE QUESTION IS CLEARLY UNDERSTOOD
2.) DISPLAY INTEREST IN THE QUESTION
3.) BE CONCISE BUT CLEAR IN THE ANSWER
4.) DETERMINE IF THE LEARNER UNDERSTANDS THE EXPLANATION

Question 18

HOW SHOULD AN INSTRUCTOR RESPOND TO A LEARNER’S QUESTION THAT IS TOO ADVANCED FOR THE PARTICULAR LESSON?

Answer 18

IF A LEARNER’S QUESTION IS TOO ADVANCED FOR THE PARTICULAR LESSON, THE INSTRUCTOR SHOULD:

1.) STATE THAT THE QUESTION WAS GOOD
2.) EXPLAIN THAT THE ANSWER WOULD COMPLICATE THE TASK AT HAND
3.) ADVISE THE LEARNER TO REINTRODUCE THE QUESTION LATER

OCCASIONALLY, LEARNERS ASK QUESTIONS THAT THE INSTRUCTOR CANNOT ANSWER. THE INSTRUCTOR SHOULD ADMIT TO NOT KNOWING THE ANSWER & OFFER TO HELP THE LEARNER LOOK IT UP

Question 19

HOW SHOULD INSTRUCTORS ASSESS PILOTING ABILITY, PARTICULARLY DEMONSTRATED ABILITY?

Answer 19

• THE ASSESSMENT OF AERONAUTICAL SKILLS IS BASED ON ESTABLISHED PERFORMANCE STANDARDS, SUITABLY MODIFIED TO FIT THE LEARNER’S EXPERIENCE LEVEL & STAGE OF TRAINING

INSTRUCTORS SHOULD VERIFY THAT LEARNERS MEET THE PROFICIENCY REQUIREMENTS BEFORE SENDING THEM ON:

1.) FIRST SOLO & SOLO CROSS-COUNTRY FLIGHTS
2.) PRACTICAL TESTS & PROFICIENCY CHECKS
3.) STAGE CHECKS & END-OF-COURSE TESTS (14 CFR PART 141)

Question 20

WHAT ARE THE IMPORTANCE OF POSTFLIGHT EVALUATIONS?

Answer 20

• FLIGHT INSTRUCTORS SHOULD KEEP LEARNERS INFORMED OF THEIR PROGRESS
• THIS MAY BE DONE AS EACH PROCEDURE OR MANEUVER IS COMPLETED OR SUMMARIZED DURING POSTFLIGHT BRIEFINGS
• FLIGHT INSTRUCTORS SHOULD NOTE STRENGTHS & WEAKNESSES NOTICED DURING A LESSON
• WRITTEN NOTES ARE HELPFUL TO ENSURE ALL AREAS ARE COVERED & CAN BE GIVEN TO THE LEARNER FOR LATER REFERENCE

Question 21

HOW SHOULD INSTRUCTORS CORRECT LEARNER ERROR FOR THE DIFFERENT TYPES OF ERRORS? DISC)

Answer 21

1.) DECISION-BASED ERRORS: WHEN SAFETY PERMITS, ITS’ OFTEN BETTER TO LET LEARNERS PARTIALLY CONTINUE INTO AN ERROR INSTEAD OF IMMEDIATELY CORRECTING THEIR MISTAKE
- THIS TEACHES LEARNERS TO CORRECT MISTAKES INDEPENDENTLY & ENHANCES PROBLEM-SOLVING SKILLS

2.) INSTRUCTOR DEMONSTRATIONS: INSTRUCTORS SHOULD NEVER FEEL GUILTY ABOUT TAKING THE CONTROLS FROM A LEARNER TO DEMONSTRATE A MANEUVER
- WHILE IT IS GOOD TO BE CONCERNED, INSTRUCTORS SHOULD NOT SKIP THIS STEP BECAUSE THEY FEEL THE LEARNER’S TIME OR MONEY IS WASTED

3.) SKILL-BASED ERRORS: INSTRUCTORS MUST PREVENT THE DEVELOPMENT OF BAD HABITS. IF A PERFORMANCE ERROR IS NOTICED, THE INSTRUCTOR SHOULD PROVIDE IMMEDIATE FEEDBACK
- THE INSTRUCTOR MAY NEED TO STOP THE LEARNER & REPEAT THE DEMONSTRATION

4.) CONFIRMING UNDERSTANDING: LEARNERS MAU BE ABLE TO PERFORM A MANEUVER OR PROCEDURE CORRECTLY WITHOUT FULLY GRASPING THE UNDERLYING PRINCIPLES
- IF AN INSTRUCTOR SUSPECTS THIS, THE LEARNER CAN BE ASKED TO VARY THEIR PERFORMANCE SLIGHTLY; WITHOUT A STRONG FOUNDATION, LEARNERS WILL STRUGGLE